1. Asymptotics of Bayesian estimation
for nested models
under misspecification
Nozomi Miya
Tota Suko
Goki Yasuda
Toshiyasu Matsushima
Waseda University
October 29, 2012
2. Introduction ~ background ~ No.1
Subject : Sequential prediction in the framework of
statistical decision theory
e.g.) universal lossless source coding, time series prediction
Our interest is the estimation of the source distribution
under the Bayesian principle (Bayesian estimation).
We analyze the asymptotic properties of the (expected)
cumulative logarithmic loss.
e.g.) the redundancy in source
~ our viewpoint ~ coding
• The employed model class is single or multiple.
• The source distribution does or does not belong to the
model class.
3. No.2
[Clarke & Barron,1990] [Takeuchi & Barron,1998]
[Gotoh & Matsushima,1998] Our study
Introduction ~ previous and our studies ~
5. No.4
Problem Setting ~ loss functions ~
We adopt the cumulative logarithmic loss
The cumulative logarithmic loss is defined as
The expected cumulative logarithmic loss is defined as
the redundancy of universal lossless source coding
(1)
(2)
7. Problem Setting ~ reformulating loss functions ~ No.6
In Bayesian estimation, loss functions are
reformulated as follows.
The loss function is redefined as
The risk function is defined as
The Bayes risk function is defined as
(3)
(4)
(5)
18. Conclusion No.17
We analyzed the asymptotic properties of the
expected cumulative logarithmic loss in
Bayesian estimation.
• If does not belong to , the
asymptotic loss per symbol asymptotically
goes to not 0 but KL-distance with .
• If belongs to , our results reduces to
the previous studies by Clarke and Barron and
Gotoh et al.
25. Appendix No.25
Outline of the proof.
• The equation can be separated by
• The proof of the Theorem1 is separated by the part A
and the part B.
Part A’ Part B’
(26)
26. Appendix No.26
Proof of the part A
• The next lemma contributes greatly to deriving the
Theorem1.
• This lemma shows that posterior distribution
asymptotically concentrates at the true model.
(27)
Lemma [Gotoh et al., 1998]