Rice University
This course covers the theory of some of the stochastic processes used
most frequentlyin application:
discrete and continuous time Markov chains, Poisson and renewal processes,
and Brownian motion
Instructor
Dr. Rudolf RiediAssistant
Duncan Hall 2082, 713 / 348 3020,
Jong Soo LeeTime and Place
According to the official Rice web page.
Tuesday Thursday 9:2510:40 am, Room RH 305 in Rayzor Hall
Monday 5:15 pm (DH 2082), or by appointment
Outline and suggested topics
Textbook
Further suggested reading
 S. Resnick, `Adventures in Stochastic Processes'.
The course will closely follow this book; it is available at the campus bookstore.
Also, Amazon quotes currently (4/2005) a prize reduction by 14%
Grading
20% QUIZ
30% Midterm EXAM
30% Last EXAM
20% Homework
[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]
Classes and Reading assignments
This doubles as a calendar for the course.See also the course schedule from Fall 2003
Covered material  Reading: Resnick (2002)  
August 23  Orientation, overview 

August 25 
Review Fubini & convergence, Probability generating function (pgf), moments  18 
August 30 
pgf, Simple branching process  917, 1820 
September 1  Extinction, Continuity, Simple random walk 
2126, 2729, 3334 
September 6  First Hit at 1, stopping times, Wald's identity  3439, 4445, 4748 
September 8  Markov Chains basics  6072 
September 13  ChapmanKolmogorov, Accessible states  7279 
September 15  Classes, Closed sets, Dissection Principle  8084 
September 20  Quiz  in class 
September 22  Class Cancelled due to Hurricane
Rita 

September 27  Transience, Recurrence, Periodicity, Canonical Decomposition  8599 
September 29  Finite closed classes, Invariant
Distributions 
100118 
October 4  Time averages, limiting distributions 
118124 
October 6  Ergodicity. Makeup quiz 
124132 
October 11  Recess  
October 13  Discussing Quiz. Poisson processes: basics, transformation.
Marking 
300318 
October 18  Poisson processes: marking, thinning; Laplace functional  316320; 333336 
October 20  Poisson processes: general construction  337346 
October 25  Order statistics and records; Compound Poisson  346349; 330331 
Up to here  Material for Test 1  
October 27  Renewal processes: basics, Convolution  174176, 176184 
November 1  Renewal function, Renewal equation  186187, 197205 
November 3  exponential interarrival times, Poisson process  211,182 
November 3  Handout 1st Exam  due Nov 11 noon 
November 8  Limiting theorems  189191,212217,224227, 237 
November 10  Stationary renewal sequences  214215,224229 
November 15  Markov processes: def, cond. probabilities  Rao (Probab. Th.), pp 114135 
November 17  Continuous time Markov chains  367376 
November 22  Forward and Backward Equations  382391 
Up to here  Material for test 2  
November 22  Handout 2nd Exam  due last day of class midnight 
November 24  Thanksgiving  
November 29  Review Renewal processes, continuous MC  
December 1  Brownian motion, simple properties  482484,504507,494499 
December 2  test 2 due 
[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]
Homework
(texsource and solutions restricted to Rice University)
This
file is needed to latex the source.
Homework sheet  Due date (in class)  Solutions 
Problem Set 1[ps] [pdf] [tex]  Sept 8, 2005  posted Sept 20 [ps] [pdf] [tex] 
Problem Set 2 [ps] [pdf] [tex]  Sept 15, 2005  posted Sept 20 Problems 5+7: [ps] [pdf] [tex] Problem 6: [pdf] 
Problem Set 3 [ps] [pdf] [tex]  Oct 4, 2005 (grace period due to Rita)  posted Oct 4 [ps] [pdf] [tex] 
Problem Set 4 [ps] [pdf] [tex]  Oct 6, 2005 (grace period due to Rita)  posted Oct 6 [ps] [pdf] [tex] 
Problem Set 5 [ps] [pdf] [tex]  Oct 20, 2005  posted Oct 26 [ps] [pdf] [tex] 
Problem Set 6 [ps] [pdf] [tex]  Oct 27  posted Nov 3 [ps] [pdf] [tex] 
Problem Set 7 [ps] [pdf] [tex]  Nov 3  posted Nov 3 [ps] [pdf] [tex] 
Problem Set 8 [ps] [pdf] [tex]  Nov 22  posted Nov 22 [ps] [pdf] [tex] 
Problem Set 9 [ps] [pdf] [tex]  Nov 29  posted Nov 29 [ps] [pdf] [tex] 
Homework is due at the beginning of class on the due date. After the due date, but before solutions are handed out, homework can be turned in for 50% credit. In this case, please slip your homework under the instructors's office door, or bring it to class. After solutions are handed out, 0% credit will be issued. You are encouraged to work in groups for homeworks but you will hand in your own solution which you are expected to understand.
[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests][Knowledge Milestones]
Quiz (15% towards the grade)  Sept 20  In class, 60 min, (open: only two handwritten pages ). On preliminaries 
Test 1 (30%)  Due Nov 11 noon.  Take home, 3 hours, On discrete Markov chains and Poisson Point Processes. Open notes: Lecture notes, solutions and all personal notes allowed. 
Test 2 (30%)  Handed out: November 22nd. Due last day of class Dec 2nd.  Take home, 3.5 hours. Open books. On Renewal Processes, and
continuous MC. Download: [ps]
[pdf] 
Knowledge Milestones aquired in this course