Introduction to Random Processes

 ELEC 533, Fall 2001

 Rice University

This course covers the basic concepts of probability theory and random processes
at a fairly rigorous level and discusses applications such as to Digital Communication Systems.


[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests]

Instructor

Dr. Rudolf Riedi
Duncan Hall 2025, 713 / 348 3020,
Office Hours: Tu 4-6pm and W 10-12am (DH 2025), or by appointment
Assistants
Rui Castro
Duncan Hall 2122, 713 / 348 2821
Office Hours: Th 3-5pm (DH 2122), or by appointment
Shriram Sarvotham
Duncan Hall 2120, 713 / 348 2600
Office Hours Th 5-7pm (DH 2120), or by appointment


Time and Place

 
Wednesday Friday   9:00 - 10:15 am, AL (Abercrombie Lab) 126
For current updates check the official Rice page

Outline
 

Review of Basic Probability Theory (incl. conditional probability)
QUIZ (one sheet or two pages of personal notes)
Random Vectors and Sequences (joint distributions, limiting laws)
Midterm EXAM (open-notes, closed-books)
Random Processes (wide sense stationarity, Poisson, Markov, Wiener processes)
Signal Detection and Parameter Estimation (spectral properties, KLT)
Second EXAM (open-notes)

 

 

Textbook

  • H. Stark and J. Woods, `Probability, Random Processes, and Estimation Theory for Engineers'.

    • The course will closely follow this book; it is available at the campus bookstore.
    Further suggested reading Standard references on Probability Theory


    Stark & Woods, Wond & Hayek, and Papoulis are on reserve at Fondren Library

    Grading

    15%  QUIZ
    30%  Midterm EXAM
    30%  Last EXAM
    15%  Homework
    10%  Notes and participation in class

    [Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests]

    Classes
     

    This doubles as a calendar for the course. Note that a * indicates a Monday lecture. 
    To get an idea what will be discussed during the whole course and what is likely to be covered in the quiz and tests please have a look at last years course schedule
    Covered material Reading: Stark&Woods (2002)
    August 29 Orientation, history
    September 5 Probability space pp 1-11mid
    September 7 Discrete and Continuous Proba Spaces pp 11-15
    September 12 Borel sets, Conditional Prob., Bayes pp 16-24; Combinatorics 24-32
    September 14 Independence, Random variables, CDF pp 58-80
    September 19 pdf, expectation, functions of one r.v. pp 169-171, 116-134
    September 21 Expect. (examples), E[g(X)], variance pp 172-175, 192-196
    September 26 Bernoulli trials, Joint distributions, Marginals, Independent r.v. pp 32-44, 88-99
    Up to here: Material for Quiz
    September 28 Functions of two r.v., Sums and Products, Covariance pp 134-152
    October 3 Joint Gaussian Pair, Stieltjes integral pp 100-102, 80-88
    October 5 Conditional distribution and expectation pp 103-108, 183-192
    October 8 (Monday 9:30-10) Quiz
    October 10 Discussion of Quiz, E[Y|X]: continuous pp 183-192
    October 12 E[Y|X]: projection, MMSE; Gaussian linear prediction  pp 552-555, 556-558
    October 15 (for your information) Fallbreak
    October 17 (!) Characteristic function pp 216-225
    October 19 Multivariate Gaussian, Covariance, Inequalities pp 269-277, 205-210
    October 24 Conv. of functions (pointwise, uniform ,L2) pp 375-376
    October 26 Convergence of r.v. (as, ms, ip, D) pp 377-383
    * October 29 (instead Nov 2) Comparing convergence, Martingales pp 225-230, 383-387
    From beginning to here: Material for Test 1
    October 31 Discussion of Homework and Convergence
    November 7 Limit theorems (Martingales and LLN, CLT, Chernoff bound) 383-387, 225-230, 214-216
    November 9 Random Processes, basics, examples 401-407
    November 14 Auto-correlation, Stationarity
    November 16 Renewal processes, Poisson 408-414
    * November 19 Consistency: Gaussian Processes, Markov 418-421, 421-430
    November 21 Chapman-Kolmogorov, Spectral density 429-430, 348-354
    November 23 Thanksgiving
    November 28 Gauss Markov, homework 362-365, 421-423, 429-430
    November 30 Auto-corr of wss Markov; Mean square calculus 430, Ex 9.1-4 p 565; pp 487-497
    From Test 1 (Nov 7) to here: Material for Test 2
    December 5 Discussion of homework and test 1
    December 7 Linear Systems with random input, White Noise

    [Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests]

    Homework
    (tex-source and solutions restricted to Rice University)
    This file is needed to latex the source.
     
    Homework sheet Due date (in class) Solutions
    Problem Set 1 [ps] [pdf] [tex] Sept 14, 2001 handed out Sept 21 [ps] [pdf] [tex]
    Problem Set 2 [ps] [pdf] [tex] Sept 21, 2001 handed out Sept 28 [ps] [pdf] [tex]
    Problem Set 3 [ps] [pdf] [tex] Sept 28, 2001 handed out Oct 3 [ps] [pdf] [tex]
    Problem Set 4 [ps] [pdf] [tex] Oct 3, 2001 handed out Oct 5 [ps] [pdf] [tex]
    Problem Set 5 [ps] [pdf] [tex] Oct 19, 2001 handed out Oct 26 [ps] [pdf] [tex]
    Problem Set 6 [ps] [pdf] [tex] Oct 26, 2001 handed out Oct 31 [ps] [pdf] [tex]
    Problem Set 7 [ps] [pdf] [tex] Oct 31, 2001 handed out Nov 2 [ps] [pdf] [tex]
    Problem Set 8 [ps] [pdf] [tex] Nov 21, 2001 handed out Nov 28 [ps] [pdf] [tex]
    Problem Set 9 [ps] [pdf] [tex] Nov 30, 2001 handed out Nov 30 [ps] [pdf] [tex]

     

    Late Homework Policy
     

    Homework is due at the beginning of class on the due date. After the due date, but before solutions are handed out, homework can be turned in for 50% credit. In this case, please slip your homework under the door of DH 2025, or DH 2121, or bring it to class. After solutions are handed out, 0% credit will be issued. You are encouraged to work in groups for homeworks but you will hand in your own solution which you are expected to understand.

    [Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests]

    Tests
     
    Quiz (15% towards the grade) October 8 In class, 30 min, (open: only two hand-written pages )
    Test 1 (30%) Handed out: October 31. Due: November 14. Take home, 3 hours, (open notes)
    Test 2 (30%) Handed out: November 30. Due December 7 Take home, 4 hours (open books)

    Any student with a documented disability needing academic adjustments or accommodations is requested to speak with me during the first two weeks of class. All discussions will remain confidential. Students with disabilities should also contact Disabled Student Services in the Ley Student Center.
    May 17, 2001.  Dr. Rudolf Riedi