Applied Stochastic Processes

 STAT 552, Fall 2005

 Rice University

This course covers the theory of some of the stochastic processes used most frequentlyin application:
discrete and continuous time Markov chains, Poisson and renewal processes, and Brownian motion

[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]


Dr. Rudolf Riedi
Duncan Hall 2082, 713 / 348 3020,
Jong Soo Lee
Time and Place        

        According to the official Rice web page.
        Tuesday Thursday   9:25-10:40 am, Room RH 305 in Rayzor Hall

Problem sessions (weekly)

        Thursday from 5:30-6:30 PM, Duncan Hall 1046

Office hours:

    Monday 5:15 pm  (DH 2082), or by appointment

Outline and suggested topics


The course will closely follow this book; it is available at the campus bookstore.
Also, Amazon quotes currently (4/2005) a prize reduction by 14%
Further suggested reading


20%  QUIZ
30%  Midterm EXAM
30%  Last EXAM
20%  Homework

[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]

Classes and Reading assignments

This doubles as a calendar for the course.
See also the course schedule from Fall 2003

Covered material Reading: Resnick (2002)
August 23 Orientation, overview

August 25
Review Fubini & convergence, Probability generating function (pgf), moments 1-8
August 30
pgf, Simple branching process 9-17, 18-20
September 1 Extinction, Continuity, Simple random walk
21-26, 27-29, 33-34
September 6 First Hit at 1, stopping times, Wald's identity 34-39, 44-45, 47-48
September 8 Markov Chains basics 60-72
September 13 Chapman-Kolmogorov, Accessible states 72-79
September 15 Classes, Closed sets, Dissection Principle 80-84
September 20 Quiz in class
September 22 Class Cancelled due to Hurricane Rita

September 27 Transience, Recurrence, Periodicity, Canonical Decomposition 85-99
September 29 Finite closed classes, Invariant Distributions
October 4 Time averages, limiting distributions
October 6 Ergodicity. Make-up quiz
October 11 Recess
October 13 Discussing Quiz. Poisson processes: basics, transformation. Marking
October 18 Poisson processes: marking, thinning; Laplace functional 316-320; 333-336
October 20 Poisson processes: general construction 337-346
October 25 Order statistics and records; Compound Poisson 346-349; 330-331
Up to here Material for Test 1
October 27 Renewal processes: basics, Convolution 174-176, 176-184
November 1 Renewal function, Renewal equation 186-187, 197-205
November 3 exponential inter-arrival times, Poisson process 211,182
November 3 Hand-out 1st Exam due Nov 11 noon
November 8 Limiting theorems 189-191,212-217,224-227, 237
November 10 Stationary renewal sequences 214-215,224-229
November 15 Markov processes: def, cond. probabilities Rao (Probab. Th.), pp 114-135
November 17 Continuous time Markov chains 367-376
November 22 Forward and Backward Equations 382-391
Up to here Material for test 2
November 22 Hand-out 2nd Exam due last day of class midnight
November 24 Thanksgiving
November 29 Review Renewal processes, continuous MC
December 1 Brownian motion, simple properties 482-484,504-507,494-499
December 2 test 2 due

[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]

(tex-source and solutions restricted to Rice University)
This file is needed to latex the source.

Homework sheet Due date (in class) Solutions
Problem Set 1[ps] [pdf] [tex] Sept 8, 2005 posted Sept 20 [ps] [pdf] [tex]
Problem Set 2 [ps] [pdf] [tex] Sept 15, 2005 posted Sept 20
Problems 5+7: [ps] [pdf] [tex]
Problem 6: [pdf]
Problem Set 3 [ps] [pdf] [tex] Oct 4, 2005 (grace period due to Rita) posted Oct 4 [ps] [pdf] [tex]
Problem Set 4 [ps] [pdf] [tex] Oct 6, 2005 (grace period due to Rita) posted Oct 6 [ps] [pdf] [tex]
Problem Set 5 [ps] [pdf] [tex] Oct 20, 2005 posted Oct 26 [ps] [pdf] [tex]
Problem Set 6 [ps] [pdf] [tex] Oct 27 posted Nov 3 [ps] [pdf] [tex]
Problem Set 7 [ps] [pdf] [tex] Nov 3 posted Nov 3 [ps] [pdf] [tex]
Problem Set 8 [ps] [pdf] [tex] Nov 22 posted Nov 22 [ps] [pdf] [tex]
Problem Set 9 [ps] [pdf] [tex] Nov 29 posted Nov 29 [ps] [pdf] [tex]


Late Homework Policy

Homework is due at the beginning of class on the due date. After the due date, but before solutions are handed out, homework can be turned in for 50% credit. In this case, please slip your homework under the instructors's office door, or bring it to class. After solutions are handed out, 0% credit will be issued. You are encouraged to work in groups for homeworks but you will hand in your own solution which you are expected to understand.

[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]


Quiz (15% towards the grade) Sept 20 In class, 60 min, (open: only two hand-written pages ). On preliminaries
Test 1 (30%) Due Nov 11 noon. Take home, 3 hours, On discrete Markov chains and Poisson Point Processes. Open notes: Lecture notes, solutions and all personal notes allowed.
Test 2 (30%) Handed out: November 22nd. Due last day of class Dec 2nd. Take home, 3.5 hours. Open books. On Renewal Processes, and continuous MC. Download: [ps] [pdf]

[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]

Knowledge Milestones aquired in this course

Any student with a documented disability needing academic adjustments or accommodations is requested to speak with me during the first two weeks of class. All discussions will remain confidential. Students with disabilities should also contact Disabled Student Services in the Ley Student Center.

April 29, 2005Dr. Rudolf Riedi