It is known that ρ , the pearson correlation, is a measure for the linear dependence of two random variables say X, Y. But can't you say just transform X and Y such that we have, ρ X , Y ( f ( X ) , g ( Y ) )where f, g are non-linear functions such that it measures other kinds of dependce (take for example f ( s ) = g ( s ) = s 2 for quadratic dependence).

Question

It is known that   ρ , the pearson correlation, is a measure for the linear dependence of two random variables say X, Y. But can&#039;t you say just transform X and Y such that we have,      ρ          X      ,      Y        (  f  (  X  )  ,  g  (  Y  )  )where f, g are non-linear functions such that it measures other kinds of dependce (take for example   f  (  s  )  =  g  (  s  )  =      s    2   for quadratic dependence).

Ahmed Kirby · Accepted Answer

Step 1Consider the vector space       S    =            L        2    (  Ω  ,      F    ,      P    ) of square-integrable random variables. Here, you can introduce the inner-product  (  X  ∣  Y  )  =      ∫    Ω    X  Y  d      P    . .Suppose now that X and Y are two elements in S and we further assume they are standarised (a.k.a. studentised), in other words       E    (  X  )  =  0 and       V    (  X  )  =  1  , , similar for Y. Under these circumstances,      c    o    r    r    (  X  ,  Y  )  =  (  X  ∣  Y  )  .Interpretation. We know from elementary linear algebra that the inner-product between two vectors is the cosine of the angle they form (in the plane generated by both). So,       c    o    r    r    (  X  ,  Y  )  =  0 means orthogonal in S and is equivalent to   θ  ∈  {  0  ,  π  }  , , where   θ is the angle formed by X and Y. Of course, θ=0  θ  =  0 means that X and Y point in the same direction while   θ  =  π means that they point in opposite directions, in either case, X and Y are colinear, and this reduces to   Y  =  a  X  +  b in the general setting (i.e. not necessarily standarised).What people often confuse is that       c    o    r    r    (  X  ,  Y  )  =  0 should signify   Y  =  f  (  X  ) for some measurable function f. In fact,   Y  =  f  (  X  ) if and only if   Y  ∈            S        X   where             S        X   is the subspace of S generated by all bounded measurable images of X, it can be shown that             S        X    =            L        2    (  Ω  ,      X          −      1        (            B              R        )  ,      P    )  . . This is not the same as the span of X, the span of X is   ⟨  X  ⟩  =      R    X  =  {  a  X  ∣  a  ∈      R    }  . . In this linear algebra language, the correlation is then the orthogonal projection operator onto the span of X. That is            p      r              ⟨      X      ⟩        (  Y  )  =      c    o    r    r    (  X  ,  Y  )  X  .Furthermore, the function Z↦E(Z∣X)  Z  ↦      E    (  Z  ∣  X  ) from       S    →            S        X   is the orthogonal projection from S onto             S        X    ,, which is the right object when we want functional dependency. Explictly,            p      r                                S                X              (  Z  )  =      E    (  Z  ∣  X  )  .As you can tell, the mathematical framework is crystal clear. If you want to show that Y is not functionally dependent of X you have to show that   Y  ⊥            S        X   which is the same as       E    (  Y  ∣  X  )  =  0. . (Actually, the condition &quot;Y is functionally dependent on X&quot; signifies   Y  ∈            S        X   , so its negation really is   Y  ∉            S        X   and not   Y  ⊥            S        X    , , this latter relation has no English translation, or not an obvious one, it is akin to &quot;Nothing of Y can be functionally dependent on X.&quot;) If you show that       c    o    r    r    (  X  ,  Y  )  =  0  , the you are just showing that   Y  ⊥  ⟨  X  ⟩  , , but as already mentioned above, many functions   f  (  X  ) are orthogonal to X.

hisyhauttaq84w · Accepted Answer

Step 1As you note, Pearson&#039;s correlation coefficient reflects linear dependence of random variables. It is invariant to changes in scale and location, i.e.  ρ  (  X  ,  Y  )  =  ρ  (  a  +  b  X  ,  c  +  d  Y  )for constants a, b, c, d and   b  ,  d  &amp;gt;  0 . It is not invariant to more general nonlinear transformations.An alternative measure of dependence is mutual information. Intuitively, this captures how different a joint distribution of two random variables is from the product of their marginals, and thus reflects a broader notion of dependence (not just linear dependence). Moreover, mutual information is invariant to bijective transformations, i.e.  I  (  X  ;  Y  )  =  I  (  f  (  X  )  ;  g  (  Y  )  )for bijections f, g.

It is known that ρ , the pearson correlation, is a measure for the linear dep

Answered question

Answer & Explanation

New Questions in Inferential Statistics