Rice University

This course is the continuation of STAT 581.

The course sequence covers
the measure-theoretic foundations of probability.

Open to qualified
undergraduates.

[Syllabus] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]

Test 2 due Wednesday, April 25 (30%) [pdf]

Time and Place

Class:

- MWF 10 - 10:50 HZ120

- Will be announced each. If announced they take place in the same class room and during one of the following times:
- WF 9 - 9:50 HZ120

Dr. Rudolf Riedi

Duncan Hall 2082, 713 / 348 3020

Office Hours: Monday 3-4 pm

or by appointment

[Syllabus] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]

Convergence Concepts and
Comparisons

Almost sure

In probability

Lp convergence

Laws of Large Numbers

Sums of Independent Random Variables

Weak and Strong versions of the LLN

Convergence in Distribution: Concepts and Comparisons

Weak and vague convergence.

Scheffe's lemma: convergence in total variation

Skorohod: almost sure convergence; Delta Method

Characteristic functions and Central Limit Theorem

Martingales

Conditional Expectation and the Radon-Nykodim Theorem

Upcrossings and convergence of martingales

Almost sure

In probability

Lp convergence

Laws of Large Numbers

Sums of Independent Random Variables

Weak and Strong versions of the LLN

Convergence in Distribution: Concepts and Comparisons

Weak and vague convergence.

Scheffe's lemma: convergence in total variation

Skorohod: almost sure convergence; Delta Method

Characteristic functions and Central Limit Theorem

Martingales

Conditional Expectation and the Radon-Nykodim Theorem

Upcrossings and convergence of martingales

Sidney Resnick"A Probability Path"

The course will closely follow this book; it is available at the campus bookstore.

Standard references and further suggested reading on Probability Theory

- A. Papoulis, `Probability, Random Variables, and Stochastic Processes'
- W. Davenport, `Probability and Random Processes'
- W. Feller, `An Introduction to Probability Theory and Its Applications'
- P. Billingsley, `Probability and Measure'

[Syllabus] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]

20% Homework

20% Quiz (mid Februay; on Convergence Concepts)

30% First EXAM (end March: on sums of independent r.v. and basics on convergence in distribution)

30% Second EXAM (April: on Advanced material on Convergence in Distribution and on Martingales)

Details on tested material: [Knowledge Milestones]

[Syllabus] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]

**Classes** / Reading

Class contents and reading assignment will be posted here.

Check the schedule of 2006 for an idea on the course contents and progression.

Topics covered reading (Resnick) January 8 no class January 10 Orientation, Review Stat 581: proba spaces, expectation January 12 Review Stat 581: convergence properties of expectations, Borel-Cantelli January 16 No Class (Marting Luther King) January 17 cancelled due to severe weather January 19 I VI. Convergence Concepts

Almost sure, in probability; examplesSection 6.1 (p167) January 19 II (Make-up for January 17) a.s. implies in probability, Cauchy criterium i.p. p169-172, January 22 Cauchy i.P. implies a subsequence converges a.s. p171-174 January 24 Subsequences ("a.s." and "i.P."), 1st Continuity thm, Lebesgue DCT p174-178, p175 January 26 [9-10] Convergence in Lp, relation to "a.s." and "i.P." Section 6.5, p180-182 January 26 [10-11] Uniform integrability Section 6.5, p182-184 January 29 Uniform integrability and moment conditions Section 6.5, p184-186 January 31 Conv L1 <=> conv i.p. & u.i. p190-194 February 2, [9-10] Absolute Continuity & bounded 1st moments <=> u.i. February 2, [10-11] Conv Lp <=> conv i.P. & |X_n|^p is u.i. Inequalities. p186-189 February 5 VII. LLN. Tail equivalence and L2 version of WLLN p203-204 February 7 simple sufficient and the exact equivalent conditions of the WLLN p 205-208 February 9 Sums of indep r.v.: Levy's theorem p209-213 February 12, [9:15] Quiz on chapter 6 (Convergence concepts) February 12, [10-11] Skorohod's inequality, Kolmogorov's convergence criterium February 14 Kronecker's lemma, Records 214-216 February 16 [9:30 - 10:50] Kolmogorov's SLLN 219-222 February 19 Kolmogorov's three series theorem 226-230 February 21 VIII. Convergence in Distribution. Basics, Dense sets 247-248, 248-251 February 23 Vague and weak convergence, relation to convergence in probability 248-251 February 26

Geometric and exponential distr

Scheffe's lemma and convergence in total variation

252-255

Feb 28 Skorohod's theorem

258-260

March 2 [9:30-11]

Continuous mapping, Delta Method

261-263

March 5

Spring Break

March 7

Spring Break

March 9

Spring Break

March 12

no class, made up for on Feb 16 and March 2

March 14

Portmanteau

263-268 March 16

Slutsky's theorem

268-271

March 19

Convergence to Types

274-279

March 21

Extreme Value Distributions

274-279

March 23

Extreme Value Distr. (proof)

Convolution

March 26

IX. Characteristic Function

Simple Properties

293-297

March 28

Expansions of char fct, char fct of the Normal distribution

297-301

March 30

Uniqueness of characteristic functions

302-305

April 2

Continuity of char fct and CLT

312-314

April 4

X. Conditional Expectation: Basics

339-342

April 6

Recess

April 9

Conditional Expectation: simple properties

344-347

April 11

Advanced properties

348-349 + standard refs

April 13

More on martingales: up-crossing and convergence of positive martingales

standard refs

April 16

Hand out exam 2, due last day of class

April 16

Conditional Exp for continuous r.v.

April 18

L2-martingale convergence

April 20

Review

April 23

No Class, made up for on 1/26

April 25

no class, made up for on 2/2

last day of class, exam 2 due

**Homework**

(tex-source and solutions restricted
to Rice University)

Homework sheet | Due date (in class) | Solutions |

Problem Set 1 [pdf] [ps] [tex] | Jan 31, 2007 | posted Feb 5 [pdf] [ps] [tex] |

Problem Set 2 [pdf] [ps] [tex] | Feb 7, 2007 |
posted Feb 7 [pdf] [ps] [tex] |

Problem Set 3 [pdf] [ps] [tex] | Feb 19, 2007 |
posted Feb 19 [pdf] [ps] [tex] |

Problem Set 4 [pdf] [ps] [tex] | Feb 28, 2007 |
posted March 1 [pdf] [ps] [tex] |

Practice Exam Set 5 [pdf] [ps] [tex] | March 16, 2007 |
posted March 6 [pdf] [ps] [tex] |

Problem Set 6 [pdf] [ps] [tex] | March 28 |
posted April 4 [pdf] [ps] [tex] |

Problem Set 7 [pdf] [ps] [tex] | April 4 |
posted April 11 [pdf] [ps] [tex] |

Problem Set 8 Practice Test 2[pdf] [ps] [tex] | due April 11 |
Solution 8 posted April 15 [pdf] [ps] [tex] |

Homework is due at the beginning of class on the due date. After the due date, but before solutions are handed out, homework can be turned in for 50% credit. In this case, please slip your homework under the instructors's office door, or bring it to class. After solutions are handed out, 0% credit will be issued. You are encouraged to work in groups for homeworks but you will hand in your own solution which you are expected to understand.

Quiz |
February 12, in class 9:15 | lecture notes allowed |

Test 1 (30%) [pdf] | Due: March 23 |
Take home, 150 minutes, (open notes) |

Test 2 (30%) [pdf] | Due: last day of class |
Take home, 3 hours (open one book +lecture notes) |

Honor System

Homework:

Homework are "open-discussion". This means the following:

Collaboration for homework is encouraged. Any source of information is admissible. However, each student hands in her/his own homework which expresses his/her own understanding of the solution. Simple copying from others does not qualify as "collaboration".

Tests:

The term "open notes" means that look-up in any self-compiled hand-written source of knowledge is permitted.

The term "open books" means that look-up in any passive source of knowledge is permitted.

No help of any kind is allowed. For instance, no communication is admissible which involves any intelligent entity ---human or artifical--- or any active source which is able to respond to questions .

**Knowledge Milestones aquired in this
course**

- Convergence Concepts and Comparisons
- Be able to formulate and verify in simple cases

- Almost sure Convergence
- Convergence in probability
- Lp convergence
- Uniform integrability
- Cauchy criterium for all convergence concepts

- Know the relations between these concepts
- order the types of convergence in strength (e.g.: Lp => Lq =>
i.p. for p>q>0)

- equivalence in certain special cases

- monotone sequences (with proof): a.s. <=> i.P.
- sums of indep. r.v.: a.s. <=> i.P.

- order the types of convergence in strength (e.g.: Lp => Lq =>
i.p. for p>q>0)

- Be able to apply the Borel-Cantelli Lemmas

- to prove or disprove almost sure convergence

- uniform integrability

- Know sufficient and/or necessary conditions

- sufficient: Crystal ball, domination
- necessary: unif. bounded first abs. moments (with proof)
- equivalent: unif. abs. cont. + unif bounded 1st abs. moments
(without proof)

- for Lp convergence

- equivalent: conv i.P. + |X_n|^p is u.i.

- equivalent: conv i.P. + |X_n|^p is u.i.
- compute criteria for uniform integrability for simple families

- Gaussian
- Uniform
- Poisson
- Exponential
- Geometric

- Know sufficient and/or necessary conditions
- Know the statement and proof and be able to apply:

- the first continuity theorem for a.s. conv and for conv. i. P.

- Know the statement of the Schwartz inequality

- Be able to formulate and verify in simple cases

- Laws of Large Numbers
- WLLN: Know the statement of

- L2-version (with proof)

- Khinchin

- Know the two conditions which are sufficient and necessary

- L2-version (with proof)
- SLLN: Know the sufficient and necessary condition for the SLLN

- WLLN: Know the statement of

- Sums of Independent Random Variables
- Know the statements of theorems

- Levy's theorem

- Kolmogorov's convergence criterium (with proof, you can use Levy's thm
without proving it)

- Kolmogorov's three series theorem
- Kolmogorov's two series theorem for positive r.v.

- Levy's theorem
- Be able to apply these theorems for simple familiies of r.v.
- Gaussian
- Uniform
- Poisson
- Exponential
- Geometric
- etc

- Know the statements of theorems

- Convergence in Distribution: Concepts and Comparisons
- Be able to formulate and verify in simple cases
- Vague convergence
- Weak convergence
- Proper convergence
- Convergence in total variation

- Know the relation between these concepts
- order them by strength

- know equivalence in special cases

- e.g. vague<=>weak for positive r.v. (with proof)

- e.g. vague<=>weak for proper limits (with proof)

- e.g. vague<=>weak for positive r.v. (with proof)

- order them by strength
- ...and the relation to convergence i.P.

- convergence i.P. implies conv. in distribution (with proof)
- conv. in distribution to a constant implies conv i.P. (with proof)

- Know simple facts:
- whether such limits are unique
- Distribution functions are determined uniquely on dense set
- the points of dis-continuity of d.f. are countable (with proof)

- Scheffe's lemma on conv in total variation
- Know the statement
- be able to verify conv in total variation in simple cases using Scheffe

- Be able to formulate and verify in simple cases

- Convergence in Distribution: Advanced material
- Know the following statements and be able to apply in simple situations
- Skorohod's theorem

- second continuity theorem for conv. in distr. (with proof)

- Portmanteau theorem (with proof)
- Delta method
- Slutsky's theorem
- Convergence to types
- LIndeberg's Central Limit Theorem

- Know simple properties of characteristic functions
- boundedness, continuity, etc (with proof)
- ...in the context of sums of independent r.v. and convolution
- ...in the context of types of distributions

- Uniqueness and Continuity theorems for Characteristic functions
- Relation to moments
- Explicit form for simple distributions such as Normal, exponential,
Poisson

- Know the three basic forms of extreme value distributions
- Central Limit Theorem

- Know the statement for i.i.d. variables (with proof using Uniqueness and Continuity of char fct))
- Know the Lindeberg-Feller sufficient condition for independent
r.v.

- Know the following statements and be able to apply in simple situations

- Martingales
- Know the two defining properties of Conditional Expectation
- Know the simple properies of Conditional Expectation in the case E[ X |
Y,Z ]

- Additivity in X

- special cases:

- X,Y,Z mutually independent: E[ X | Y,Z] = E[X]
- E[ g(Y,Z)*W | Y,Z ] = g(Y,Z) E[ W | Y,Z]

- Iterated conditional expectation E[ E[ U | V,W ] | V ] = E[ U | V
]

- Interpretation as a projection in the L2-case

- Additivity in X
- Be able to compute conditional expectations in simple cases
- using simple properties
- by explicit computation via conditional probabilities

- ...for discrete and for jointly continuous r.v.

- Martingales
- Know the definition of a martingale

- Know the statement of the Martingale-convergence-theorem

- Know the definition of a martingale

Any student with a documented disability needing academic adjustments or accommodations is requested to speak with me during the first two weeks of class. All discussions will remain confidential. Students with disabilities should also contact Disabled Student Services in the Ley Student Center.

December 12, 2006.