Skip to Main Content
 

Global Search Box

 
 
 
 

Files

File List


Zhou-dissertation.pdf (1.58 MB)

ETD Abstract Container

Abstract Header

Density Estimation in Kernel Exponential Families: Methods and Their Sensitivities

Zhou, Chenxi
http://rave.ohiolink.edu/etdc/view?acc_num=osu1661103199026584

Abstract Details

2022, Doctor of Philosophy, Ohio State University, Statistics.
This dissertation is concerned with the nonparametric density estimation problem in a kernel exponential family, which is an exponential family induced by a reproducing kernel Hilbert space (RKHS). The corresponding density estimation problem can be formulated as a convex minimization problem over a RKHS or a subset of it. The loss functionals we focus on are the negative log-likelihood (NLL) loss functional and the score matching (SM) loss functional. We propose a new density estimator called the early stopping SM density estimator, which is obtained by applying the gradient descent algorithm to minimizing the SM loss functional and terminating the algorithm early to regularize. We investigate various statistical properties of this density estimator. We also compare this early stopping SM density estimator with the penalized SM density estimator that has been studied in the literature and address their similarities and differences. In addition, we propose an algorithm to compute the penalized maximum likelihood (ML) density estimator that is obtained by minimizing the penalized NLL loss functional. We empirically compare the penalized and early stopping SM density estimators with the penalized ML density estimator and find out that when there is a small amount of regularization (corresponding to small values of the penalty parameter or large values of the number of iterations), the regularized SM density estimates contain a bump or become a spike at the isolated observation, but the penalized ML density estimates do not. Moreover, if we remove the isolated observation, the resulting regularized SM density estimates do not contain a bump or a spike when the regularization is small. We attempt to explain why this happens. Observations above motivate us to study the sensitivities of different density estimators to the presence of an additional observation. We extend the definition of the influence function by allowing its input to be function-valued statistical functionals. We study various properties of this extended influence functions of ML and SM density projections in finite-dimensional and kernel exponential families, and empirically demonstrate that regularized SM density estimators in a kernel exponential family are more sensitive to the presence of an additional observation than the penalized ML density estimator when the amount of regularization is small.
Vincent Vu (Advisor)
Sebastian Kurtek (Committee Member)
Yoonkyung Lee (Committee Member)
202 p.
Statistics
density estimation; exponential family; maximum likelihood estimation; score matching estimation; regularization; sensitivity analysis; influence function

Recommended Citations

Citations

  • Zhou, C. (2022). Density Estimation in Kernel Exponential Families: Methods and Their Sensitivities [Doctoral dissertation, Ohio State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=osu1661103199026584

    APA Style (7th edition)

  • Zhou, Chenxi. Density Estimation in Kernel Exponential Families: Methods and Their Sensitivities. 2022. Ohio State University, Doctoral dissertation. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=osu1661103199026584.

    MLA Style (8th edition)

  • Zhou, Chenxi. "Density Estimation in Kernel Exponential Families: Methods and Their Sensitivities." Doctoral dissertation, Ohio State University, 2022. http://rave.ohiolink.edu/etdc/view?acc_num=osu1661103199026584

    Chicago Manual of Style (17th edition)

osu1661103199026584
371
© 2022, all rights reserved.
This open access ETD is published by The Ohio State University and OhioLINK.