## Introduction to Random Processes

ELEC 533, Fall 2003

Rice University

This course covers the basic concepts of probability theory and random processes.
Targeted at first year graduate students it introduces concepts at an appropriately rigorous level and discusses applications through examples and homework, such as to Digital Communication Systems. The syllabus covers elementary probability theory, random variables, limiting theorems such as the Law of Large Numbers, the Central Limit Theorem, and Martingales, as well as Gaussian, Markovian and Renewal Processes.

[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests]

Instructor

Dr. Rudolf Riedi
Duncan Hall 2082, 713 / 348 3020,
Office Hours: M 3-4, Tu 4-5pm or by appointment
Assistants
Rahul Chawathe
Time and Place
Monday 11:00-12:00, Wednesday 8:45 - 10:00 am, when Monday class cancelled : Friday 11:00-12:00
AL (Abercrombie Lab) 126

Outline

Review of Basic Probability Theory (incl. conditional probability)
QUIZ (one sheet or two pages of personal notes)
Random Vectors and Sequences (joint distributions, limiting laws)
Midterm EXAM (open-notes, closed-books)
Random Processes (wide sense stationarity, Poisson, Markov, Wiener processes)
Mean Square Estimation and Calculus (spectral properties, KLT)
Second EXAM (open-notes)

Textbook

• H. Stark and J. Woods, `Probability, Random Processes, and Estimation Theory for Engineers'.

• The course will closely follow this book; it is available at the campus bookstore.
• E. Wong and B. Hajek, `Stochastic Processes in Engineering Systems'
Standard references on Probability Theory
• A. Papoulis, `Probability, Random Variables, and Stochastic Processes'
• W. Davenport, `Probability and Random Processes'
• W. Feller, `An Introduction to Probability Theory and Its Applications'
• P. Billingsley, `Probability and Measure'

Stark & Woods, Wond & Hayek, and Papoulis are on reserve at Fondren Library

15%  QUIZ
30%  Midterm EXAM
30%  Last EXAM
15%  Homework
10%  Notes and participation in class

Classes

This doubles as a calendar for the course. Note that a * indicates a Monday lecture.
To get an idea what will be discussed during the whole course and what is likely to be covered in the quiz and tests please have a look at last years course schedule
 Covered material Reading: Stark&Woods (2002) August 27 Orientation, history August 29 Probability space: basics Review combinatorics: 24-31 September 3 Probability space: discrete, continuous 1-24 September 5 Random variable, CDF, pdf 58-68 September 10 Conditional Probability, Bayes, Independence 68-80 September 12 Functions of one r.v., expectation 116-134, 169-175 September 15 Moments, Independent experiments pp 192-196, 32-44 September 17 Joint distributions, Marginals, Independent r.v. pp 88-99 Up to here: Material for Quiz September 22 Functions of two r.v., Sums and Products pp 134-152 September 24 Covariance, Stieltjes integral September 26 QUIZ September 29 NO CLASS, moved to Oct 3 October 1 generalized pdf (Dirac), conditional CDF 75-80, 80-88 October 3 conditional pdf, E[Y|X]: rules 103-108 October 6 E[Y|X]: several variables 183-192 October 8 MMSE, E[Y|X] as a projection, Gaussian estimation 552-561 October 13 RECESS, moved to Oct 17 October 15 Characteristic function 216-225 October 17 Joint char fct, joint Gaussian 277- 280, 281 (also: 269-277) October 20 Inequalities, Convergence of functions pp 205-210, 375-376 October 22 Convergence of random variables pp 376-382 October 27 Limit theorems pp 383-387, 225-230, 214-216 October 29 Limit theorems Up to here: Material for TEST 1 November 5 Random Processes: Basics pp 401-407 November 7 Consistency, Stationarity November 12 Renewal Processes: Basics pp 408-416 November 14 Poisson Process November 19 Consistency: Gaussian Processes pp 418-421 November 21 Consistency: Markov (Chapman-Kolmogorov) pp 421-430 November 24 Spectral density 348-354 November 26 Cross-correlation and -spectrum, Mean square continuity 348-354, pp 487-490 November 28 Thanksgiving From Test 1 to here: Material for Test 2 December 1 Mean square calculus pp 487-497 December 3 More of Spectral density, White Noise, Karhunen-Loewe

Homework
(tex-source and solutions restricted to Rice University)
This file is needed to latex the source.

 Homework sheet Due date (in class) Solutions Problem Set 1 [ps] [pdf] [tex] Sept 10, 2003 [ps] [pdf] [tex] Problem Set 2 [ps] [pdf] [tex] Sept 17, 2003 [ps] [pdf] [tex] Problem Set 3 [ps] [pdf] [tex] Sept 24, 2003 [ps] [pdf] [tex] Problem Set 4 [ps] [pdf] [tex] Oct 8, 2003 [ps] [pdf] [tex] Problem Set 5 [ps] [pdf] [tex] Oct 15, 2003 [ps] [pdf] [tex] Problem Set 6 [ps] [pdf] [tex] Oct 22, 2003 [ps] [pdf] [tex] Problem Set 7 [ps] [pdf] [tex] Oct 29, 2003 [ps] [pdf] [tex] Problem Set 8 [ps] [pdf] [tex] Nov 21, 2003 [ps] [pdf] [tex] Problem Set 9 [ps] [pdf] [tex] for practice only, not graded handed out with problem set [ps] [pdf] [tex]

Late Homework Policy

Homework is due at the beginning of class on the due date. After the due date, but before solutions are handed out, homework can be turned in for 50% credit. In this case, please slip your homework under the instructors's office door, or bring it to class. After solutions are handed out, 0% credit will be issued. You are encouraged to work in groups for homeworks but you will hand in your own solution which you are expected to understand.

Tests

 Quiz (15% towards the grade) Sept 26, 11:00-11:40, AL 126. In class, 30 min, (open: only two hand-written pages ) Test 1 (30%) Handed out: Mid October. To be scheduled Take home, 3 hours, (open notes) Test 2 (30%) Handed out: Late November. To be scheduled Take home, 4 hours (open books)

Any student with a documented disability needing academic adjustments or accommodations is requested to speak with me during the first two weeks of class. All discussions will remain confidential. Students with disabilities should also contact Disabled Student Services in the Ley Student Center.
August 1, 2003.  Dr. Rudolf Riedi