Rice University

This course is the continuation of STAT 581.

The course sequence covers the measure-theoretic foundations of
probability.

Open to qualified undergraduates.

[Syllabus] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]

Last update June 24, 2006

Time and Place

Class:

- M 10-11 HZ 118

- W 10-11 HZ 118
- W 11-12 HZ 118

- F 11-12 HZ 118 (if announced)

Dr. Rudolf Riedi

Duncan Hall 2082, 713 / 348 3020

Office Hours: Monday 3-4 pm

or by appointment

[Syllabus] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]

Convergence Concepts and Comparisons

Almost sure

In probability

Lp convergence

Laws of Large Numbers

Sums of Independent Random Variables

Weak and Strong versions of the LLN

Convergence in Distribution: Concepts and Comparisons

Weak and vague convergence.

Scheffe's lemma: convergence in total variation

Skorohod: almost sure convergence; Delta Method

Characteristic functions and Central Limit Theorem

Martingales

Conditional Expectation and the Radon-Nykodim Theorem

Upcrossings and convergence of martingales

Almost sure

In probability

Lp convergence

Laws of Large Numbers

Sums of Independent Random Variables

Weak and Strong versions of the LLN

Convergence in Distribution: Concepts and Comparisons

Weak and vague convergence.

Scheffe's lemma: convergence in total variation

Skorohod: almost sure convergence; Delta Method

Characteristic functions and Central Limit Theorem

Martingales

Conditional Expectation and the Radon-Nykodim Theorem

Upcrossings and convergence of martingales

Sidney Resnick"A Probability Path"

The course will closely follow this book; it is available at the campus bookstore.

Standard references and further suggested reading on Probability Theory

- A. Papoulis, `Probability, Random Variables, and Stochastic Processes'
- W. Davenport, `Probability and Random Processes'
- W. Feller, `An Introduction to Probability Theory and Its Applications'
- P. Billingsley, `Probability and Measure'

[Syllabus] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]

20% Homework

20% Quiz (Februay 16; on Convergence Concepts)

30% First EXAM (March 27: on sums of independent r.v. and basics on convergence in distribution)

30% Second EXAM (April: on Advanced material on Convergence in Distribution and on Martingales)

Details on tested material: [Knowledge Milestones]

[Syllabus] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]

**Classes** / Reading

Class contents and reading assignment will be posted here.

Check the schedule of 2005 for an idea on the course contents and progression.

Topics covered

reading (Resnick)

January 11 Orientation, Review Stat 581: proba spaces, expectation

January 13 Review Stat 581: convergence properties of expectations, Borel-Cantelli

January 16 No Class (Marting Luther King)

January 18 VI. Convergence Concepts

Almost sure, in probability; examples

Section 6.1 (p167)

January 20 a.s. implies in probability

p169-172

January 23 Cauchy criterium i.p.

p173-178

January 25 I Subsequences in the context of "a.s." and "i.P." convergence

Section 6.5, p179-180

January 25 II Convergence in Lp, relation to "a.s." and "i.P."

Section 6.5, p180-182

January 27 (for 2/6)

Uniform integrability

Section 6.5, p182-184

January 30 Uniform integrability and absolute continuity

Section 6.5, p184-186

February 1 I

Conv Lp <=> conv i.p. & u.i.

p190-194

February 1 II

Inequalities. Review a.s versus i.p.

p186-189

February 3 (for 3/20)

VII. Laws of Large Numbers.

First versions and general WLLN

p204-208

February 6

no class (made up for on Jan 27)

February 8 I

HW set 1; proof WLLN

p 208

February 8 II

Sums of indep r.v.: Levy's theorem

p209-213

February 13

Skorohod's inequality, Kolmogorov's convergence criterium

February 15 I

SLLN: Kronecker's lemma, Records

214-216

February 15 II

Homework discussion

February 17 (Friday)

Quiz on chapter 6 (Convergence concepts); Problem session

February 20 Kolmogorov's SLLN 219-222 February 22 I Kolmogorov's three series theorem 226-230 February 22 II VIII. Convergence in Distribution. Basics 247-248 February 27 Distribution functions on dense sets 248-251 March 1 I Vague and weak convergence, relation to convergence in probability 248-251 March 1 II moved to Friday March 3

March 3 (for 3/1)

Geometric distr

Scheffe's lemma and convergence in total variation

252-255

March 6 proof Scheffe's lemma

252-255

March 8 I

Skorohod's theorem

258-260

March 8 II

Continuous mapping, Delta Method

261-263

March 13

Spring Break

March 15

Spring Break

March 20

no class, made up for on Feb 3

handout Exam 1

March 22 I

Discussion Practice Test

March 22 II

Set theory: boundary and continuity of indicator

263-267

March 27

Exam 1 due

March 27

Portmanteau theorem and applications

263-267

March 29 I

Slutsky's theorem

268-271

March 29 II

Convergence to Types

274-279

April 3

Extreme Value Distributions

274-279

April 5 I

Discussion Exam 1

April 5 II

IX. Characteristic Function

Convolution, Simple Properties

293-297

April 10

Expansions of char fct, char fct of the Normal distribution

297-301

April 12 I

Uniqueness and Continuity of characteristic functions

302-305

April 12 II

CLT

312-314

April 14 (for 4/26)

X. Conditional Expectation: Basics

339-342

April 17

Conditional Expectation: simple properties

344-347

April 19 I

Conditional expectation as L2-projection, L2-martingale convergence

348-349 + standard refs

April 19 II

More on martingales: up-crossing and convergence of positive martingales

standard refs

April 19

Hand out exam 2, due last day of class

April 24

Review

April 26

No Class, made up for on 4/21

April 27, Thursday

last day of class, exam 2 due

**Homework**

(tex-source and solutions restricted to Rice University)

Homework sheet | Due date (in class) | Solutions |

Problem Set 1 [pdf] [ps] [tex] | February 8 |
Solution 1 [pdf] [ps] [tex] |

Problem Set 2 [pdf] [ps] [tex] | February 13 |
Solution 2 [pdf] [ps] [tex] |

Problem Set 3 [pdf] [ps] [tex] | March 3 |
Solution 3 [pdf] [ps] [tex] |

Problem Set 4 [pdf] [ps] [tex] | March 8 |
Solution 4 [pdf] [ps] [tex] |

Practice Exam Set 5 [pdf] [ps] [tex] | not graded |
Solution 5 [pdf] [ps] [tex] |

Problem Set 6 [pdf] [ps] [tex] | April 10 |
Solution 6 posted April 10 [pdf] [ps] [tex] |

Problem Set 7 [pdf] [ps] [tex] | April 17 |
Solution 7 posted April 17 [pdf] [ps] [tex] |

Problem Set 8 [pdf] [ps] [tex] | not graded |
Solution 8 posted April 21 [pdf] [ps] [tex] |

Homework is due at the beginning of class on the due date. After the due date, but before solutions are handed out, homework can be turned in for 50% credit. In this case, please slip your homework under the instructors's office door, or bring it to class. After solutions are handed out, 0% credit will be issued. You are encouraged to work in groups for homeworks but you will hand in your own solution which you are expected to understand.

Quiz |
February 17, 2006, in class |
lecture notes allowed |

Test 1 (30%) [pdf] [ps] | Due: March 27 |
Take home, 150+30 minutes, (open notes) |

Test 2 (30%) [pdf] | Due: last day of class |
Take home, 3 hours (open one book +lecture notes) |

Honor System

Homework:

Homework are "open-discussion". This means the following:

Collaboration for homework is encouraged. Any source of information is admissible. However, each student hands in her/his own homework which expresses his/her own understanding of the solution. Simple copying from others does not qualify as "collaboration".

Tests:

The term "open notes" means that look-up in any self-compiled hand-written source of knowledge is permitted.

The term "open books" means that look-up in any passive source of knowledge is permitted.

No help of any kind is allowed. For instance, no communication is admissible which involves any intelligent entity ---human or artifical--- or any active source which is able to respond to questions .

**Knowledge Milestones aquired in this course
**

- Convergence Concepts and Comparisons
- Be able to formulate and verify in simple cases

- Almost sure Convergence
- Convergence in probability
- Lp convergence
- Uniform integrability
- Cauchy criterium for all convergence concepts

- Know the relations between these concepts
- order the types of convergence in strength (e.g.: Lp => Lq
=> i.p. for p>q>0)

- equivalence in certain special cases

- monotone sequences (with proof): a.s. <=> i.P.
- sums of indep. r.v.: a.s. <=> i.P.

- order the types of convergence in strength (e.g.: Lp => Lq
=> i.p. for p>q>0)

- Be able to apply the Borel-Cantelli Lemmas

- to prove or disprove almost sure convergence

- uniform integrability

- Know sufficient and/or necessary conditions

- sufficient: Crystal ball, domination
- necessary: unif. bounded first abs. moments (with proof)
- equivalent: unif. abs. cont. + unif bounded 1st abs.
moments (without proof)

- for Lp convergence

- equivalent: conv i.P. + |X_n|^p is u.i.

- equivalent: conv i.P. + |X_n|^p is u.i.
- compute criteria for uniform integrability for simple
families

- Gaussian
- Uniform
- Poisson
- Exponential
- Geometric

- Know sufficient and/or necessary conditions
- Know the statement and proof and be able to apply:

- the first continuity theorem for a.s. conv and for conv. i. P.

- Know the statement of the Schwartz inequality

- Be able to formulate and verify in simple cases

- Laws of Large Numbers
- WLLN: Know the statement of

- L2-version (with proof)

- Khinchin

- Know the two conditions which are sufficient and necessary

- L2-version (with proof)
- SLLN: Know the sufficient and necessary condition for the SLLN

- WLLN: Know the statement of

- Sums of Independent Random Variables
- Know the statements of theorems

- Levy's theorem

- Kolmogorov's convergence criterium (with proof, you can use
Levy's thm without proving it)

- Kolmogorov's three series theorem
- Kolmogorov's two series theorem for positive r.v.

- Levy's theorem
- Be able to apply these theorems for simple familiies of r.v.
- Gaussian
- Uniform
- Poisson
- Exponential
- Geometric
- etc

- Know the statements of theorems

- Convergence in Distribution: Concepts and Comparisons
- Be able to formulate and verify in simple cases
- Vague convergence
- Weak convergence
- Proper convergence
- Convergence in total variation

- Know the relation between these concepts
- order them by strength

- know equivalence in special cases

- e.g. vague<=>weak for positive r.v. (with proof)

- e.g. vague<=>weak for proper limits (with proof)

- e.g. vague<=>weak for positive r.v. (with proof)

- order them by strength
- ...and the relation to convergence i.P.

- convergence i.P. implies conv. in distribution (with proof)
- conv. in distribution to a constant implies conv i.P. (with proof)

- Know simple facts:
- whether such limits are unique
- Distribution functions are determined uniquely on dense set
- the points of dis-continuity of d.f. are countable (with proof)

- Scheffe's lemma on conv in total variation
- Know the statement
- be able to verify conv in total variation in simple cases using Scheffe

- Be able to formulate and verify in simple cases

- Convergence in Distribution: Advanced material
- Know the following statements and be able to apply in simple
situations
- Skorohod's theorem

- second continuity theorem for conv. in distr. (with proof)

- Portmanteau theorem (with proof)
- Delta method
- Slutsky's theorem
- Convergence to types
- LIndeberg's Central Limit Theorem

- Know simple properties of characteristic functions
- boundedness, continuity, etc (with proof)
- ...in the context of sums of independent r.v. and convolution
- ...in the context of types of distributions

- Uniqueness and Continuity theorems for Characteristic functions
- Relation to moments
- Explicit form for simple distributions such as Normal,
exponential, Poisson

- Know the three basic forms of extreme value distributions
- Central Limit Theorem

- Know the statement for i.i.d. variables (with proof using Uniqueness and Continuity of char fct))
- Know the Lindeberg-Feller sufficient condition for independent
r.v.

- Know the following statements and be able to apply in simple
situations

- Martingales
- Know the two defining properties of Conditional Expectation
- Know the simple properies of Conditional Expectation in the case E[
X | Y,Z ]

- Additivity in X

- special cases:

- X,Y,Z mutually independent: E[ X | Y,Z] = E[X]
- E[ g(Y,Z)*W | Y,Z ] = g(Y,Z) E[ W | Y,Z]

- Iterated conditional expectation E[ E[ U | V,W ] | V ] = E[ U |
V ]

- Interpretation as a projection in the L2-case

- Additivity in X
- Be able to compute conditional expectations in simple cases
- using simple properties
- by explicit computation via conditional probabilities

- ...for discrete and for jointly continuous r.v.

- Martingales
- Know the definition of a martingale

- Know the statement of the Martingale-convergence-theorem

- Know the definition of a martingale

Any student with a documented disability needing academic adjustments or accommodations is requested to speak with me during the first two weeks of class. All discussions will remain confidential. Students with disabilities should also contact Disabled Student Services in the Ley Student Center.

February 17, 2005.