Introduction to Random Processes

ELEC 533, Fall 2003

Rice University

This course covers the basic concepts of probability theory and random processes.
Targeted at first year graduate students it introduces concepts at an appropriately rigorous level and discusses applications through examples and homework, such as to Digital Communication Systems. The syllabus covers elementary probability theory, random variables, limiting theorems such as the Law of Large Numbers, the Central Limit Theorem, and Martingales, as well as Gaussian, Markovian and Renewal Processes.


[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests]

Instructor

Dr. Rudolf Riedi
Duncan Hall 2082, 713 / 348 3020,
Office Hours: M 3-4, Tu 4-5pm or by appointment
Assistants
Rahul Chawathe
Alireza Keshavarz-Haddad
Time and Place
Monday 11:00-12:00, Wednesday 8:45 - 10:00 am, when Monday class cancelled : Friday 11:00-12:00
AL (Abercrombie Lab) 126


Outline
 

Review of Basic Probability Theory (incl. conditional probability)
QUIZ (one sheet or two pages of personal notes)
Random Vectors and Sequences (joint distributions, limiting laws)
Midterm EXAM (open-notes, closed-books)
Random Processes (wide sense stationarity, Poisson, Markov, Wiener processes)
Mean Square Estimation and Calculus (spectral properties, KLT)
Second EXAM (open-notes)


Textbook

  • H. Stark and J. Woods, `Probability, Random Processes, and Estimation Theory for Engineers'.

    • The course will closely follow this book; it is available at the campus bookstore.
    Further suggested reading Standard references on Probability Theory


    Stark & Woods, Wond & Hayek, and Papoulis are on reserve at Fondren Library

    Grading

    15%  QUIZ
    30%  Midterm EXAM
    30%  Last EXAM
    15%  Homework
    10%  Notes and participation in class

    [Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests]

    Classes
     

    This doubles as a calendar for the course. Note that a * indicates a Monday lecture. 
    To get an idea what will be discussed during the whole course and what is likely to be covered in the quiz and tests please have a look at last years course schedule

    		 Covered material Reading: Stark&Woods (2002)
    August 27 Orientation, history
    August 29 Probability space: basics Review combinatorics: 24-31
    September 3 Probability space: discrete, continuous 1-24
    September 5 Random variable, CDF, pdf 58-68
    September 10 Conditional Probability, Bayes, Independence 68-80
    September 12 Functions of one r.v., expectation 116-134, 169-175
    September 15 Moments, Independent experiments pp 192-196, 32-44
    September 17 Joint distributions, Marginals, Independent r.v. pp 88-99
    Up to here: Material for Quiz
    September 22 Functions of two r.v., Sums and Products pp 134-152
    September 24 Covariance, Stieltjes integral
    September 26 QUIZ
    September 29 NO CLASS, moved to Oct 3
    October 1 generalized pdf (Dirac), conditional CDF 75-80, 80-88
    October 3 conditional pdf, E[Y|X]: rules 103-108
    October 6 E[Y|X]: several variables 183-192
    October 8 MMSE, E[Y|X] as a projection, Gaussian estimation 552-561
    October 13 RECESS, moved to Oct 17
    October 15 Characteristic function 216-225
    October 17 Joint char fct, joint Gaussian 277- 280, 281 (also: 269-277)
    October 20 Inequalities, Convergence of functions pp 205-210, 375-376
    October 22 Convergence of random variables pp 376-382
    October 27 Limit theorems pp 383-387, 225-230, 214-216
    October 29 Limit theorems
    Up to here: Material for TEST 1
    November 5 Random Processes: Basics pp 401-407
    November 7 Consistency, Stationarity
    November 12 Renewal Processes: Basics pp 408-416
    November 14 Poisson Process
    November 19 Consistency: Gaussian Processes pp 418-421
    November 21 Consistency: Markov (Chapman-Kolmogorov) pp 421-430
    November 24 Spectral density 348-354
    November 26 Cross-correlation and -spectrum, Mean square continuity 348-354, pp 487-490
    November 28 Thanksgiving
    From Test 1 to here: Material for Test 2
    December 1 Mean square calculus pp 487-497
    December 3 More of Spectral density, White Noise, Karhunen-Loewe

    [Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests]

    Homework
    (tex-source and solutions restricted to Rice University)
    This file is needed to latex the source.
     
    Homework sheet Due date (in class) Solutions
    Problem Set 1 [ps] [pdf] [tex] Sept 10, 2003 [ps] [pdf] [tex]
    Problem Set 2 [ps] [pdf] [tex] Sept 17, 2003 [ps] [pdf] [tex]
    Problem Set 3 [ps] [pdf] [tex] Sept 24, 2003 [ps] [pdf] [tex]
    Problem Set 4 [ps] [pdf] [tex] Oct 8, 2003 [ps] [pdf] [tex]
    Problem Set 5 [ps] [pdf] [tex] Oct 15, 2003 [ps] [pdf] [tex]
    Problem Set 6 [ps] [pdf] [tex] Oct 22, 2003 [ps] [pdf] [tex]
    Problem Set 7 [ps] [pdf] [tex] Oct 29, 2003 [ps] [pdf] [tex]
    Problem Set 8 [ps] [pdf] [tex] Nov 21, 2003 [ps] [pdf] [tex]
    Problem Set 9 [ps] [pdf] [tex] for practice only, not graded handed out with problem set
    [ps] [pdf] [tex]

     

    Late Homework Policy
     

    Homework is due at the beginning of class on the due date. After the due date, but before solutions are handed out, homework can be turned in for 50% credit. In this case, please slip your homework under the instructors's office door, or bring it to class. After solutions are handed out, 0% credit will be issued. You are encouraged to work in groups for homeworks but you will hand in your own solution which you are expected to understand.

    [Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests]

    Tests
     
    Quiz (15% towards the grade) Sept 26, 11:00-11:40, AL 126. In class, 30 min, (open: only two hand-written pages )
    Test 1 (30%) Handed out: Mid October. To be scheduled Take home, 3 hours, (open notes)
    Test 2 (30%) Handed out: Late November. To be scheduled Take home, 4 hours (open books)

    Any student with a documented disability needing academic adjustments or accommodations is requested to speak with me during the first two weeks of class. All discussions will remain confidential. Students with disabilities should also contact Disabled Student Services in the Ley Student Center.
    August 1, 2003.  Dr. Rudolf Riedi