Rice University

*This course covers the theory of some of the most frequently used
stochastic processes in application: discrete and continuous time Markov chains, Poisson and renewal processes, and Brownian motion*

[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests]

**Instructor**

Dr. Rudolf Riedi

Duncan Hall 2082, 713 / 348 3020,

Office Hours: W 4-6pm (DH 2082), or by appointment

Mike Wakin

Tuesday Thursday 10:50 - 12:05 am, Room HB 427 in Herman Brown Hall

For current updates check the official Rice page

**Outline and suggested topics**

- Preliminaries (incl. probability generating functions, simple random walks and branching processes, stopping times, Wald equality)
- QUIZ (one sheet or two pages of personal notes)
- Discrete-time Markov chains (joint distributions, transition probability, irreducibility, absorbing, transient and recurrent states, invariant measures, stationarity and equilibrium, ergodicity (time averages, mixing))
- Poisson Processes (including thinned and marked Poisson processes)
- Midterm EXAM (open-notes, closed-books)
- Renewal Processes (more on Poisson processes, renewal theorem)
- Continuous-time Markov chains (Holding times, Chapman-Kolmogorov and Consistency, backward and forward equations)
- Brownian motion (construction via mid-points, self-similarity and scaling, Brownian bridges)
- Second EXAM (open-notes)

**Textbook**

Further suggested readingS. Resnick, `Adventures in Stochastic Processes'.

The course will closely follow this book; it is available at the campus bookstore. Also, Amazon quotes the prize $69.95

- Ross, `Introduction to Probability Models'
- A. Papoulis, `Probability, Random Variables, and Stochastic Processes'
- W. Davenport, `Probability and Random Processes'
- W. Feller, `An Introduction to Probability Theory and Its Applications'
- P. Billingsley, `Probability and Measure'

**Grading**

15% QUIZ

30% Midterm EXAM

30% Last EXAM

15% Homework

10% Notes and participation in class

[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests]

**Classes and Reading assignments**

This doubles as a calendar for the course.

Covered material | Reading: Resnick (2002) | |

August 26 | Orientation, history | |

August 28 | Basic review probability, moments | 1-8 |

September 2 | Probability generating function | 9-17 |

September 4 | Simple branching process | 18-26 |

September 9 | Continuity, simple random walk | 27-29, 33-39 |

September 11 | First Hit, stopping times, Wald | 47-48 |

September 16 | Markov Chains basics | 60-72 |

September 18 | Quiz | in class |

September 23 | Chapman-Kolmogorov, Accessible states | 72-79 |

September 25 | Classes, Closed sets, Dissection | 80-84 |

September 30 | Transience, Recurrence, Periodicity | 85-91 |

October 2 | Canonical Decomposition, Examples | 92-101 |

October 7 | Invariant measures and stationary distributions | 116-122 |

October 9 | Time Averages | 122-132 |

October 14 | Recess | |

October 16 | Limiting Distributions, Ergodicity | 122-136 |

October 21 | Poisson processes: basics, transformation | 300-303, 308-312 |

October 23 | Poisson processes: marking, thinning; Laplace functional | 316-320; 333-336 |

Up to here | Material for Test 1 | |

October 28 | Poisson processes: conditioning on the number of points, Records | 337-348 |

October 30 | Renewal processes: basics | 174-176,185 |

November 3 | Hand-out 1st Exam | due Nov 10 noon |

November 4 | Convolution, Renewal function | 176-184,186-187 |

November 6 | Renewal equation, Poisson | 197-205;211,182 |

November 10 | Test 1 due | |

November 11 | Limiting theorems | 189-191,212-217,224-227, 237 |

November 13 | Stationary renewal sequences | 214-215,224-229 |

November 17 | Markov processes: def, examples | Rao (Probab. Th.), pp 114-135 |

November 18 | Continuous time Markov chains | 367-376 |

November 20 | Forward and Backward Equations | 382-391 |

November 25 | Brownian motion, simple properties | 482-484,504-507,494-499 |

Up to here | Material for test 2 | |

November 27 | Thanksgiving | |

November 28 | Hand-out 2nd Exam | due last day of class midnight |

December 2 | Pathwise construction, Brownian Bridge | 489-493, 524-530 |

December 4 | no class (make up Nov 17) | |

December 5 | test 2 due |

[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests]

**Homework**

(tex-source and solutions restricted to Rice University)

This
file is needed to latex the source.

Homework sheet | Due date (in class) | Solutions |

Problem Set 1 [ps] [pdf] [tex] | Sept 11, 2003 | [ps] [pdf] [tex] |

Problem Set 2 [ps] [pdf] [tex] | Sept 16, 2003 | Problems 5+7: [ps] [pdf] [tex] Problem 6: [pdf] |

Problem Set 3 [ps] [pdf] [tex] | Oct 16, 2003 | [ps] [pdf] [tex] |

Problem Set 4 [ps] [pdf] [tex] | Oct 23, 2003 | [ps] [pdf] [tex] |

Problem Set 5 [ps] [pdf] [tex] | Nov 18, 2003 | [ps] [pdf] [tex] |

Problem Set 6 [ps] [pdf] [tex] | Nov 25 | [ps] [pdf] [tex] |

Homework is due at the beginning of class on the due date. After the due date, but before solutions are handed out, homework can be turned in for 50% credit. In this case, please slip your homework under the instructors's office door, or bring it to class. After solutions are handed out, 0% credit will be issued. You are encouraged to work in groups for homeworks but you will hand in your own solution which you are expected to understand.

[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests]

Quiz (15% towards the grade) | September 18. | In class, 30 min, (open: only two hand-written pages ), preliminaries |

Test 1 (30%) | download: [ps] [pdf] | Take home, 3 hours, (open notes), discrete MC and Poisson |

Test 2 (30%) | Handed out: Late November. To be scheduled | Take home, 4 hours (open books), continuous MC and Brown |

Any student with a documented disability needing academic adjustments or accommodations is requested to speak with me during the first two weeks of class. All discussions will remain confidential. Students with disabilities should also contact Disabled Student Services in the Ley Student Center.

Aug 1, 2003.