SlideShare uma empresa Scribd logo
1 de 28
Baixar para ler offline
Asymptotics of Bayesian estimation
for nested models
under misspecification
Nozomi Miya
Tota Suko
Goki Yasuda
Toshiyasu Matsushima
Waseda University
October 29, 2012
Introduction ~ background ~ No.1
 Subject : Sequential prediction in the framework of
statistical decision theory
e.g.) universal lossless source coding, time series prediction
 Our interest is the estimation of the source distribution
under the Bayesian principle (Bayesian estimation).
 We analyze the asymptotic properties of the (expected)
cumulative logarithmic loss.
e.g.) the redundancy in source
~ our viewpoint ~ coding
• The employed model class is single or multiple.
• The source distribution does or does not belong to the
model class.
No.2
[Clarke & Barron,1990] [Takeuchi & Barron,1998]
[Gotoh & Matsushima,1998] Our study
Introduction ~ previous and our studies ~
Problem Setting ~ sequential prediction ~ No.3
No.4
Problem Setting ~ loss functions ~
 We adopt the cumulative logarithmic loss
The cumulative logarithmic loss is defined as
The expected cumulative logarithmic loss is defined as
the redundancy of universal lossless source coding
(1)
(2)
Problem Setting ~ Bayesian estimation ~ No.5
~ given ~
Problem Setting ~ reformulating loss functions ~ No.6
 In Bayesian estimation, loss functions are
reformulated as follows.
The loss function is redefined as
The risk function is defined as
The Bayes risk function is defined as
(3)
(4)
(5)
Problem Setting ~ Bayesian optimal solution ~ No.7
(6)
(7)
Problem Setting ~ aim of our study ~ No.8
Results ~ main theorem ~ No.9
Under some conditions, it follows that
Main Theorem
(8)
Results ~ discussion ~ No.10
No.11
Results ~ outline of the proof ~
Conditions
Results ~ outline of the proof ~ No.12
Part A
Part B
(9)
Results ~ outline of the proof ~ No.13
Lemma
(10)
Results ~ outline of the proof ~ No.14
(11)
(12)
Results ~ outline of the proof ~ No.15
(13)
Results ~ comparison to the method of Gotoh et al. ~ No.16
Conclusion No.17
 We analyzed the asymptotic properties of the
expected cumulative logarithmic loss in
Bayesian estimation.
• If does not belong to , the
asymptotic loss per symbol asymptotically
goes to not 0 but KL-distance with .
• If belongs to , our results reduces to
the previous studies by Clarke and Barron and
Gotoh et al.
Appendix No.18
(14)
(15)
(16)
(17)
Appendix No.19
(20)
(19)
(18)
Appendix No.21
(21)
(22)
Appendix No.22
(23)
(24)
Appendix No.23
Theorem [Gotoh et al.,1998]
(25)
Appendix No.24
Appendix No.25
 Outline of the proof.
• The equation can be separated by
• The proof of the Theorem1 is separated by the part A
and the part B.
Part A’ Part B’
(26)
Appendix No.26
 Proof of the part A
• The next lemma contributes greatly to deriving the
Theorem1.
• This lemma shows that posterior distribution
asymptotically concentrates at the true model.
(27)
Lemma [Gotoh et al., 1998]
Appendix No.27
(28)
(29)
Appendix No.28
(30)

Mais conteúdo relacionado

Destaque

What is bayesian statistics and how is it different?
What is bayesian statistics and how is it different?What is bayesian statistics and how is it different?
What is bayesian statistics and how is it different?Wayne Lee
 
Introduction to Bayesian Statistics
Introduction to Bayesian StatisticsIntroduction to Bayesian Statistics
Introduction to Bayesian StatisticsPhilipp Singer
 
Bayesian statistics using r intro
Bayesian statistics using r   introBayesian statistics using r   intro
Bayesian statistics using r introBayesLaplace1
 
Bayesian statistical concepts
Bayesian statistical conceptsBayesian statistical concepts
Bayesian statistical conceptsAlexander Etz
 
Chapter 3 maximum likelihood and bayesian estimation-fix
Chapter 3   maximum likelihood and bayesian estimation-fixChapter 3   maximum likelihood and bayesian estimation-fix
Chapter 3 maximum likelihood and bayesian estimation-fixjelli123
 
An introduction to Bayesian Statistics using Python
An introduction to Bayesian Statistics using PythonAn introduction to Bayesian Statistics using Python
An introduction to Bayesian Statistics using Pythonfreshdatabos
 
An introduction to bayesian statistics
An introduction to bayesian statisticsAn introduction to bayesian statistics
An introduction to bayesian statisticsJohn Tyndall
 
Bayes theorem explained
Bayes theorem explainedBayes theorem explained
Bayes theorem explainedDaniel Ross
 
3 recursive bayesian estimation
3  recursive bayesian estimation3  recursive bayesian estimation
3 recursive bayesian estimationSolo Hermelin
 
Contingency approach
Contingency approachContingency approach
Contingency approachLee Morley
 

Destaque (12)

Bayesian intro
Bayesian introBayesian intro
Bayesian intro
 
What is bayesian statistics and how is it different?
What is bayesian statistics and how is it different?What is bayesian statistics and how is it different?
What is bayesian statistics and how is it different?
 
Introduction to Bayesian Statistics
Introduction to Bayesian StatisticsIntroduction to Bayesian Statistics
Introduction to Bayesian Statistics
 
Bayesian statistics using r intro
Bayesian statistics using r   introBayesian statistics using r   intro
Bayesian statistics using r intro
 
Bayesian statistical concepts
Bayesian statistical conceptsBayesian statistical concepts
Bayesian statistical concepts
 
Chapter 3 maximum likelihood and bayesian estimation-fix
Chapter 3   maximum likelihood and bayesian estimation-fixChapter 3   maximum likelihood and bayesian estimation-fix
Chapter 3 maximum likelihood and bayesian estimation-fix
 
An introduction to Bayesian Statistics using Python
An introduction to Bayesian Statistics using PythonAn introduction to Bayesian Statistics using Python
An introduction to Bayesian Statistics using Python
 
An introduction to bayesian statistics
An introduction to bayesian statisticsAn introduction to bayesian statistics
An introduction to bayesian statistics
 
Bayes theorem explained
Bayes theorem explainedBayes theorem explained
Bayes theorem explained
 
3 recursive bayesian estimation
3  recursive bayesian estimation3  recursive bayesian estimation
3 recursive bayesian estimation
 
Contingency approach
Contingency approachContingency approach
Contingency approach
 
Slideshare ppt
Slideshare pptSlideshare ppt
Slideshare ppt
 

Mais de matsushimalab (20)

ma99992012id537
ma99992012id537ma99992012id537
ma99992012id537
 
ma112011id535
ma112011id535ma112011id535
ma112011id535
 
ma99992011id513
ma99992011id513ma99992011id513
ma99992011id513
 
ma99992010id512
ma99992010id512ma99992010id512
ma99992010id512
 
ma99992010id510
ma99992010id510ma99992010id510
ma99992010id510
 
ma99992011id509
ma99992011id509ma99992011id509
ma99992011id509
 
ma99992011id508
ma99992011id508ma99992011id508
ma99992011id508
 
ma99992010id497
ma99992010id497ma99992010id497
ma99992010id497
 
ma99992010id464
ma99992010id464ma99992010id464
ma99992010id464
 
ma92010id461
ma92010id461ma92010id461
ma92010id461
 
ma112010id450
ma112010id450ma112010id450
ma112010id450
 
ma112009id434
ma112009id434ma112009id434
ma112009id434
 
ma52009id421
ma52009id421ma52009id421
ma52009id421
 
ma52009id420
ma52009id420ma52009id420
ma52009id420
 
ma52009id419
ma52009id419ma52009id419
ma52009id419
 
ma112007id414
ma112007id414ma112007id414
ma112007id414
 
ma112007id413
ma112007id413ma112007id413
ma112007id413
 
ma22006id411
ma22006id411ma22006id411
ma22006id411
 
ma112006id337
ma112006id337ma112006id337
ma112006id337
 
ma92007id395
ma92007id395ma92007id395
ma92007id395
 

ma12012id536