Language：English   Publishing type：Research paper (scientific journal)   Publisher：ELSEVIER SCIENCE BV  

In this paper, bootstrap prediction is adapted to resolve some problems in small sample datasets. The bootstrap predictive distribution is obtained by applying Breiman's bagging to the plug-in distribution with the maximum likelihood estimator. The effectiveness of bootstrap prediction has previously been shown, but some problems may arise when bootstrap prediction is constructed in small sample datasets. In this paper, Bayesian bootstrap is used to resolve the problems. The effectiveness of Bayesian bootstrap prediction is confirmed by some examples. These days, analysis of small sample data is quite important in various fields. In this paper, some datasets are analyzed in such a situation. For real datasets, it is shown that plug-in prediction and bootstrap prediction provide very poor prediction when the sample size is close to the dimension of parameter while Bayesian bootstrap prediction provides stable prediction. (C) 2009 Elsevier B.V. All rights reserved.

DOI： 10.1016/j.jspi.2009.06.007

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：MIT PRESS  

The correlation matrix is a fundamental statistic that used in many fields. For example, GroupLens, a collaborative filtering system, uses the correlation between users for predictive purposes. Since the correlation is a natural similarity measure between users, the correlation matrix may be used as the Gram matrix in kernel methods. However, the estimated correlation matrix sometimes has a serious defect: although the correlation matrix is originally positive semidefinite, the estimated one may not be positive semidefinite when not all ratings are observed. To obtain a positive semidefinite correlation matrix, the nearest correlation matrix problem has recently been studied in the fields of numerical analysis and optimization. However, statistical properties are not explicitly used in such studies. To obtain a positive semidefinite correlation matrix, we assume an approximate model. By using the model, an estimate is obtained as the optimal point of an optimization problem formulated with information on the variances of the estimated correlation coefficients. The problem is solved by a convex quadratic semidefinite program. A penalized likelihood approach is also examined. The MovieLens data set is used to test our approach.

DOI： 10.1162/neco.2009.04-08-765

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：MIT PRESS  

Density estimation plays an important and fundamental role in pattern recognition, machine learning, and statistics. In this article, we develop a parametric approach to univariate (or low-dimensional) density estimation based on semidefinite programming (SDP). Our density model is expressed as the product of a nonnegative polynomial and a base density such as normal distribution, exponential distribution, and uniform distribution. When the base density is specified, the maximum likelihood estimation of the polynomial is formulated as a variant of SDP that is solved in polynomial time with the interior point methods. Since the base density typically contains just one or two parameters, computation of the maximum likelihood estimate reduces to a one- or two-dimensional easy optimization problem with this use of SDP. Thus, the rigorous maximum likelihood estimate can be computed in our approach. Furthermore, such conditions as symmetry and unimodality of the density function can be easily handled within this framework. AIC is used to choose the best model. Through applications to several instances, we demonstrate flexibility of the model and performance of the proposed procedure. Combination with a mixture approach is also presented. The proposed approach has possible other applications beyond density estimation. This point is clarified through an application to the maximum likelihood estimation of the intensity function of a nonstationary Poisson process.

DOI： 10.1162/neco.2006.18.11.2777

Web of Science

researchmap

Publishing type：Research paper (scientific journal)  

We consider a statistical prediction problem under misspecified models. In a sense, Bayesian prediction is an optimal prediction method when an assumed model is true. Bootstrap prediction is obtained by applying Breiman's 'bagging' method to a plug-in prediction. Bootstrap prediction can be considered to be an approximation to the Bayesian prediction under the assumption that the model is true. However, in applications, there are frequently deviations from the assumed model. In this paper, both prediction methods arc compared by using the Kullback-Leibler loss under the assumption that the model does not contain the true distribution. We show that bootstrap prediction is asymptotically more effective than Bayesian prediction under misspecified models. © 2005 ISI/BS.

DOI： 10.3150/bj/1126126768

Scopus

researchmap

Publishing type：Research paper (scientific journal)  

Ensemble learning has recently been intensively studied in the field of machine learning. 'Bagging' is a method of ensemble learning and uses bootstrap data to construct various predictors. The required prediction is then obtained by averaging the predictors. Harris proposed using this technique with the parametric bootstrap predictive distribution to construct predictive distributions, and showed that the parametric bootstrap predictive distribution gives asymptotically better prediction than a plug-in distribution with the maximum likelihood estimator. In this paper, we investigate nonparametric bootstrap predictive distributions. The nonparametric bootstrap predictive distribution is precisely that obtained by applying bagging to the statistical prediction problem. We show that the nonparametric bootstrap predictive distribution gives predictions asymptotically as good as the parametric bootstrap predictive distribution. © 2005 ISI/BS.

DOI： 10.3150/bj/1116340296

Scopus

researchmap

Publishing type：Research paper (scientific journal)  

We investigate bootstrapping and Bayesian methods for prediction. The observations and the variable being predicted are distributed according to different distributions. Many important problems can be formulated in this setting. This type of prediction problem appears when we deal with a Poisson process. Regression problems can also be formulated in this setting. First, we show that bootstrap predictive distributions are equivalent to Bayesian predictive distributions in the second-order expansion when some conditions are satisfied. Next, the performance of predictive distributions is compared with that of a plug-in distribution with an estimator. The accuracy of prediction is evaluated by using the Kullback-Leibler divergence. Finally, we give some examples.

DOI： 10.1111/j.1467-9469.2004.02_127.x

Scopus

researchmap

Publishing type：Research paper (scientific journal)  

The stability of propagating precisely timed spikes from the viewpoint of spike timing dependent synaptic plasticity (STDP) was investigated. A phenomenon similar to stochastic resonance with respect to optimal level of background noise for learning was present on STDP. It was found that the noise can be related to learning by STDP and also supports the possibility of temporal spike coding in the brain.

Scopus

researchmap