next up previous
Next: Relation of Covariance and Up: Theory: Covariance & Correlation Previous: Review of Mathematical Expectation.

Covariance and Correlation.

Now consider a pair of r.v.'s (X,Y) with joint density tex2html_wrap_inline817 which may be either continuous or discrete (or a mixture of discrete and continuous). As we shall deal almost exclusively with continuous random variables in time series applications, we will implicitly assume that r.v.'s are continuous, but the corresponding formulae for discrete r.v.'s can be obtained by replace sums by integrals. The covariance between X and Y (or the covariance of X and Y; the appropriate preposition is not entirely fixed) is defined to be


Useful facts are collected in the next result.


Proof. Part (i) is easy:


The first equation in part (ii) is trivial (plug in Y = X in the definition tex2html_wrap_inline853 . For the second equation, one can find the result in Hogg & Craig in the section on Expectations of Functions of Random Variables, but it is not explicitly stated, so


Part (iii) is an exercise in Hogg & Craig, so we give its proof here, but after proving the remaining parts of the proposition. It is basically the Cauchy-Schwarz inequality in one guise.

For part (iv), similarly to the proof of part (ii) of Proposition 2,


Part (v) is similar:


Part (vi) is already proved in Hogg & Craig.

Now we are ready to give the proof of part (iii). Let t be a real variable, then by (i) of Proposition 2,


where the last equation follows by some ``algebra'' based on (ii), (v) above and (ii) of Proposition 2. Writing this in the form


we have a function of t this is a quadratic polynomial which is always nonnegative, so the discriminant is nonpositive, i.e.


from which it follows that


Now if we have equality, i.e.


then it follows that there is a value of t for which Q(t) = 0, i.e. tex2html_wrap_inline863 = 0 for some t, and hence by (i) of Proposition 2, tX + Y = c with probability 1 where C is a constant, and thus Y = - tX + c which proves the claim.

It follows from (i), (iv), and (v) of the last proposition that


The correlation or correlation coefficient is defined as


From part (iii), we have the Correlation Inequality:




From part (iii) of Proposition 3, we only know that tex2html_wrap_inline885 = tex2html_wrap_inline889 implies Y = aX + b with probability 1, but one can check that a < 0 implies tex2html_wrap_inline885 < 0 and similarly for a > 0. We also have for any constants a, b, c, and d, with a > 0 and c > 0,


This follows by application of (iv) and (v) of Proposition 3 and part (ii) of Proposition 2.

Given a sample of (X,Y) data, say tex2html_wrap_inline919 , tex2html_wrap_inline921 , tex2html_wrap_inline627 , tex2html_wrap_inline925 , we can consider the sample covariance and correlation defined by


Of course, in the above tex2html_wrap_inline927 and tex2html_wrap_inline929 are the sample mean and variance of the X sample tex2html_wrap_inline623 , tex2html_wrap_inline625 , tex2html_wrap_inline627 , tex2html_wrap_inline629 , and tex2html_wrap_inline941 and tex2html_wrap_inline943 are the sample mean and variance of the Y sample tex2html_wrap_inline947 , tex2html_wrap_inline949 , tex2html_wrap_inline627 , tex2html_wrap_inline953 . Note that we use the form (2) of the sample variance here, although this is not entirely standardized.

next up previous
Next: Relation of Covariance and Up: Theory: Covariance & Correlation Previous: Review of Mathematical Expectation.

Dennis Cox
Tue Jan 21 09:20:27 CST 1997