Applied Stochastic Processes

 STAT 552, Fall 2006

 Rice University

This course covers the theory of some of the stochastic processes used most frequentlyin application:
discrete and continuous time Markov chains, Poisson and renewal processes, and Brownian motion


Test 2 due last day of class

Solution 9 available

Last update Dec 1. 2006

[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]


Dr. Rudolf Riedi
Duncan Hall 2082, 713 / 348 3020,
Email for appointments
Teaching Assistant        
Alireza Keshavarz-Haddad
Time and Place        
TR 09:25AM - 10:40AM HZ 119  (Hermann Brown)     
see also Time (official Rice page)
Location (Rice map)

Office hours:

    Monday 1-2 pm pm  (DH 2082), or by appointment

Outline and suggested topics


The course will closely follow this book; it is available at the campus bookstore.
Also, Amazon quotes currently (4/2005) a prize reduction by 14%
Further suggested reading


15%  QUIZ
30%  Midterm EXAM
30%  Last EXAM
25%  Homework

[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]

Classes and Reading assignments

This doubles as a calendar for the course.
See also the course schedule from Fall 2005
   Covered material Reading: Resnick
(Edition 2002)
August 29
August 31
Probability generating function (pgf), 1-17,
September 5 Pgf cont., Simple Branching process, 18-20, 21-26,
September 7
Extinction, Simple random walk
September 12 First Hit at 1, stopping times, Wald's identity

Up to here : material for QUIZ

34-39, 44-45, 47-48
September 14 Markov Chains basics, Chapman-Kolmogorov, Accessible States, Classes, Closed sets 60-79
September 19 REVIEW random walk, MC
September 21 Quiz hand out. Take home, Due Friday Sept 22
Transience, Recurrence, Periodicity

September 26 Canonical Decomposition 91-99
September 28 Finite closed classes 100-116
October 3
Stationary Distributions for recurrent chains 116-122
October 5
Time averages
October 10 Limiting distributions
October 12 Renewal processes: basics, Convolution 174-176, 176-184
October 17 Recess
October 19 Renewal function, Renewal equation 186-187, 197-205
Start to here Material for Test 1
October 24
exponential inter-arrival times, Poisson process on the line 211,182
October 26
Limiting theorems 189-191,212-217,224-227, 237
October 31
Review MC, discussion HW 6
November 2
Hand-out 1st Exam due Nov 10 noon
November 2 HW 6; Elementary renewal thm, Stationarity
November 7 Stationary renewal sequences, Blackwell 214-215,224-229
November 9 Poisson processes: basics, transformation. 300-318
November 14 Discussion Test 1;
Poisson processes: marking, queuing

316-320; 333-336
November 16 Poisson processes: thinning; Laplace functional 337-346
November 21
general construction, order statistics and records; Compound Poisson 346-349; 330-331
November 23 Thanksgiving
November 28
Hand-out test 2. Material = from Recess to here due last day of class 11:59
November 28 General Markov processes: def, cond. probabilities, marginals and consistency, Poisson Point Process on line Rao (Probab. Th.), pp 114-135
November 30 Review Renewal processes, Poisson Point Processes
December 5 Ex Markov: Poisson Renewal Process, Brownian motion 367-376
December 7 Continuous time Markov chains, Birth-Death process 382-391
December 8
Last day of class, test 2 due

[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]

(tex-source and solutions restricted to Rice University)
This file is needed to latex the source.

Homework sheet Due date (in class) Solutions
Problem Set 1 [ps] [pdf] [tex] Sept 7, 2006 posted Sept 12 [ps] [pdf] [tex]
Problem Set 2 [ps] [pdf] [tex] Sept 14, 2006 posted Sept 21 [ps] [pdf] [tex]
Problem Set 3 [ps] [pdf] [tex] Oct 3, 2006 posted Oct 3 [ps] [pdf] [tex]
Problem Set 4 [ps] [pdf] [tex] Oct 10, 2006 posted Oct 12 [ps] [pdf] [tex]
Problem Set 5 [ps] [pdf] [tex] Oct 19, 2006 posted Oct 23 [ps] [pdf] [tex]
Problem Set 6 [ps] [pdf] [tex] voluntary, practice test discussed in class
Problem Set 7 [ps] [pdf] [tex] Nov 16 posted Nov 18 [ps] [pdf] [tex]
Problem Set 8 [ps] [pdf] [tex] Nov 21 posted Nov 22 [ps] [pdf] [tex]
Problem Set 9 [ps] [pdf] [tex] voluntary, practice test discussed in class [ps] [pdf] [tex]


Late Homework Policy

Homework is due at the beginning of class on the due date. After the due date, but before solutions are handed out, homework can be turned in for 50% credit. In this case, please slip your homework under the instructors's office door, or bring it to class. After solutions are handed out, 0% credit will be issued. You are encouraged to work in groups for homeworks but you will hand in your own solution which you are expected to understand.

[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]


Quiz (15% towards the grade) Due Sept 22 In class, 60 min,
(open: only two hand-written pages )
Test 1 (30%) Due Nov 10 noon
Take home, 3 hours,

closed book, open notes open solution handouts
on everything up to Renewal equation

Test 2 (30%) Due last day of class Take home, 3 hours, open everything

on Renewal and Point processes

[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]

Knowledge Milestones aquired in this course

Any student with a documented disability needing academic adjustments or accommodations is requested to speak with me during the first two weeks of class. All discussions will remain confidential. Students with disabilities should also contact Disabled Student Services in the Ley Student Center.

June 29, 2006 Dr. Rudolf Riedi