Introduction to Random Processes

ELEC 533, Fall 2002

Rice University


This course covers the basic concepts of probability theory and random processes
at a fairly rigorous level and discusses applications such as to Digital Communication Systems.



[Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests]

Instructor

Dr. Rudolf Riedi
Duncan Hall 2025, 713 / 348 3020,
Office Hours: Tu 4-6pm and W 10-12am (DH 2025), or by appointment
Assistants
Bhaskar Nallapureddy
Duncan Hall DH1040, 713 / 348 3579,
Office Hours: MW 4:30pm-5:30pm, or by appointment
Mohammed Ahamed Khan
Duncan Hall DH2113, 713 / 348 2776,
Office Hours: TTh 10:30-11:30am, or by appointment
Time and Place of Classes
Wednesday Friday   8:45 - 10:00 am, AL (Abercrombie Lab) 126
For current updates check the official Rice page



Outline
 

Review of Basic Probability Theory (incl. conditional probability)
QUIZ (one sheet or two pages of personal notes)
Random Vectors and Sequences (joint distributions, limiting laws)
Midterm EXAM (open-notes, closed-books)
Random Processes (wide sense stationarity, Poisson, Markov, Wiener processes)
Signal Detection and Parameter Estimation (spectral properties, KLT)
Second EXAM (open-notes)



Textbook

  • H. Stark and J. Woods, `Probability, Random Processes, and Estimation Theory for Engineers'.

  • The course will closely follow this book; it is available at the campus bookstore.
    Further suggested reading Standard references on Probability Theory


    Stark & Woods, Wond & Hayek, and Papoulis are on reserve at Fondren Library


    Grading

    15%  QUIZ
    30%  Midterm EXAM
    30%  Last EXAM
    15%  Homework
    10%  Notes and participation in class

    [Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests]



    Classes
     

    This doubles as a calendar for the course. Note that a * indicates a Monday lecture. 
    To get an idea what will be discussed during the whole course and what is likely to be covered in the quiz and tests please have a look at last years course schedule

    Covered material Reading: Stark&Woods (2002)
    August 28 Orientation, history
    August 30 Probability space pp 1-15, Combinatorics: 24-31
    September 4 Sigma-Algebra, Borel sets, continuous spaces read pp 16-24
    September 6 Bayes, Independence, Random variables pp 58-68
    September 9 (instead Wed) CDF, density (pdf), functions of one r.v. pp 68-80, 129
    September 13 HW 1, expectation, E[g(X)] pp 116-134, 169-175
    September 18 Moments, Independent experiments pp 192-196, 32-44
    September 20 Joint distributions, Marginals, Independent r.v. pp 88-99
    Up to here: Material for Quiz

    September 25 Functions of two r.v., Sums and Products, Covariance pp 134-152
    September 27 Stieltjes integral, generalized density (Dirac)
    October 2 Conditional density and expectation E[Y|X] pp 80-88, 103-108
    October 4 E[Y|X]: rules, several variables pp 183-192
    October 9 Characteristic function, Cumulants pp 216-225
    October 11 Multivariate Distributions, Covariance pp 269-277
    October 16 Jointly Gaussian: distribution and estimation pp 269-277, 556-561
    October 18 Convergence of functions and random variables pp 375-379
    October 23 Inequalities, Comparison of Convergence pp 205-210, 375-379
    October 25 Comparing convergence, Martingales pp 225-230, 383-387
    Monday October 28 Limit theorems (Martingales and LLN) pp 383-387, 225-230, 214-216
    October 30
    Limit theorems (CLT, Chernoff, LDP)

    From beginning to here: Material for Test 1


    November 1 NO CLASS
    November 6 Random Processes, basics, examples pp 401-407
    November 8 Auto-correlation, Stationarity pp
    Monday November 11 (instead 13) Renewal processes, Poisson pp 408-414
    November 13 NO CLASS

    November 15 NO CLASS
    Monday November 18 (instead 15) Poisson properties pp 408-416
    November 20 Consistency: Gaussian Processes, Markov pp 418-421, 421-430
    November 22 Chapman-Kolmogorov, pp 429-430
    Monday November 25 (Thanksgiving) Spectral density 348-354
    November 27 Mean square calculus pp 487-497
    November 29 Thanksgiving
    From Test 1 to here: Material for Test 2
    December 4 Spectral density, White Noise
    December 6 Linear Systems, Gauss-Markov, Karhunen-Loewe


    [Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests]


    Homework
    (tex-source and solutions restricted to Rice University)
    This file is needed to latex the source.
     
    Homework sheet Due date (in class) Solutions
    Problem Set 1 [ps] [pdf] [tex] Sept 6, 2002 handed out Sept 13 [ps] [pdf] [tex]
    Problem Set 2 [ps] [pdf] [tex] Sept 13, 2002 handed out Sept 20 [ps] [pdf] [tex]
    Problem Set 3 [ps] [pdf] [tex] Sept 20, 2002 handed out Sept 27 [ps] [pdf] [tex]
    Problem Set 4 [ps] [pdf] [tex] Sept 27, 2002 handed out Sept 27 [ps] [pdf] [tex]
    Problem Set 5 [ps] [pdf] [tex] Oct 11, 2002 handed out Oct 18 [ps] [pdf] [tex]
    Problem Set 6 [ps] [pdf] [tex] Oct 18, 2002 handed out Oct 25 [ps] [pdf] [tex]
    Problem Set 7 [ps] [pdf] [tex] Oct 25, 2002 handed out Oct 30 [ps] [pdf] [tex]
    Problem Set 8 [ps] [pdf] [tex] Oct 30, 2002 handed out Oct 30 [ps] [pdf] [tex]
    Problem Set 8b [ps] [pdf] [tex] not mandatory handed out Oct 30 with problem set
    Problem Set 9a [ps] [pdf] [tex] Nov 18, 2002 handed out Nov 22 [ps] [pdf] [tex]
    Problem Set 10 [ps] [pdf] [tex] Nov 27, 2002 handed out Nov 27 [ps] [pdf] [tex]

     

    Late Homework Policy
     

    Homework is due at the beginning of class on the due date. After the due date, but before solutions are handed out, homework can be turned in for 50% credit. In this case, please slip your homework under the door of DH 2025, or DH 2121, or bring it to class. After solutions are handed out, 0% credit will be issued. You are encouraged to work in groups for homeworks but you will hand in your own solution which you are expected to understand.

    [Outline] [Textbooks] [Grading] [Reading assignment] [Homework problems and solutions] [Tests]


    Tests
     
    Quiz (15% towards the grade) Mo, Sept 30, 9:20-9:50 am In class, 30 min, (open: only two hand-written pages )
    Test 1 (30%) Handed out: Mid October. Take home, 3 hours, (open notes)
    Test 2 (30%) Handed out: Late November. Due Dec 6 Take home, 3 1/2 hours (open books)


    Any student with a documented disability needing academic adjustments or accommodations is requested to speak with me during the first two weeks of class. All discussions will remain confidential. Students with disabilities should also contact Disabled Student Services in the Ley Student Center.
    May 1, 2002.  Dr. Rudolf Riedi