Closed-form formula for E x [ max ( u ⊤ x , 0 ) max ( v ⊤ x , 0 ) ] where u,v are fixed vectors in R d and x is uniform on the sphereIs there a closed-form formula for f ( u , v ) := E x [ max ( u ⊤ x , 0 ) max ( v ⊤ x , 0 ) ]?

Question

Closed-form formula for       E          x        [  max  (      u    ⊤    x  ,  0  )  max  (      v    ⊤    x  ,  0  )  ] where u,v are fixed vectors in             R        d   and x is uniform on the sphereIs there a closed-form formula for   f  (  u  ,  v  )  :=            E              x        [  max  (      u    ⊤    x  ,  0  )  max  (      v    ⊤    x  ,  0  )  ]?

Larissa Hart · Accepted Answer

Step 1Proved that   f  (  u  ,  v  )  =  ‖  u  ‖  ‖  v  ‖  ϕ  (                    u        ⊤            v              ‖      u      ‖      ‖      v      ‖        ), where   ϕ  :  [  −  1  ,  1  ]  →      R   is defined by   ϕ  (  t  )  =                    π        t        +        2                  1          −                      t            2                          +        2        t        arctan        ⁡        (                  t                      1            −                          t              2                                      )                    4        (        d        +        2        )        π            .We&#039;re in the uv-plane and   t  ∈  [  −  1  ,  1  ] is nothing but the height a poin   P  =  (      1    −          t      2        ,  t  ) on the unit circle in this plane. The angle pointint the positive v axis ray OP is given by   θ  =  arctan  ⁡  (  t      /        1    −          t      2        ). By inspecting the geometry of the situation (basically drawing a diagram on a piece of paper...), it is evident that we can rewrite   θ  =  arccos  ⁡  (  t  ). Thus, we can rewrite the function   ϕ asStep 2Now is the time for the harvest.The second line in the above display is nothing but the main ingredient in so-called arc-cosine kernels: these is the kernel for the Gaussian process which emerges when one looks at infinite-width limit of neural networks with ReLU activations, width the weights initialized at random. More precisely,   f  ∼  (  1      /    (  d  +  2  )  )      K          R      e      L      U      .Noting that, for large d, (1) The individual coordinates of a point sampled uniformy at random the unit sphere in             R        d   is roughly distributed as       N    (  0  ,  1      /    d  ), and (2) The pairwise correlations of the coordinates is approximately zero, we could have expected f(u,v) to be approximately equal to             E              z      ∼              N            (      0      ,      1              /            d      )        [  max  (      u    ⊤    z  ,  0  )  max  (      v    ⊤    z  ,  0  )  ]  =  (  1      /    d  )      K          R      e      L      U        (  u  ,  v  )

Julius Frey · Accepted Answer

Step 1We can look at normalized vector:  f  (  u  ,  v  )  =      |        |    u      |        |    ⋅      |        |    v      |        |    ⋅  f            (            u                  |                    |            u              |                    |              ,      v                  |                    |            v              |                    |                        )      , so let us assume that u and v are normalized. Then we can write   v  =  ρ  u  +      1    −          ρ      2            u    2   with   ρ  =  (      u    T    v  )  ∈  [  −  1  ,  1  ] and       u    2   unitary orthogonal to u, and then complete   (  u  ,      u    2    ) into an orthogonal base of             R        d  , and look at coordinates   (      x    1    ,  .  .  .  ,      x    d    ) in that base.Initial remarks:- the volume       V    k   of the unit ball       B    k   in             R        k   is       V    k    =            π              k                  /                2                    Γ      (      k              /            2      +      1      )      - the density of   (  R  ,  θ  ) s.t.   (  R  cos  ⁡  (  θ  )  ,  R  sin  ⁡  (  θ  )  ) is uniformly distributed on the unit disk is       R    π    d  R  d  θ                    f        (        u        ,        v        )                            =                                            ∫                                                B                  d                                                              max          (                      x            1                    ,          0          )          max                                    (                                ρ                      x            1                    +          (          1          −          ρ          )                      x            2                    ,          0                                    )                                                          d                              x                1                            .              .              .              d                              x                d                                                    V              d                                                                        =                                            ∫              0              1                                                                          ∫                                  −                  π                                                  π                                                      max            (            r            cos            ⁡            (            θ            )            ,            0            )            max            (            ρ            r            cos            ⁡            (            θ            )            +                          1              −                              ρ                2                                      r            sin            ⁡            (            θ            )            ,            0            )            ⋅            r            d            θ            ⋅                                          (                1                −                                  r                  2                                                  )                                      d                                          /                                        2                    −                    1                                                                    V                                      d                    −                    2                                                                              V                d                                      d            r                                                            =                                            ∫                              −                π                                            π                                              max          (          cos          ⁡          (          θ          )          ,          0          )          max          (          ρ          cos          ⁡          (          θ          )          +                      1            −                          ρ              2                                sin          ⁡          (          θ          )          ,          0          )          d          θ          ⋅                                                    ∫                0                1                                                    r              3                        (            1            −                          r              2                                      )                              d                                  /                                2                −                1                                      d            r            ⋅                          d                              2                π                                                        where   (  1  −      r    2        )          (      d      −      2      )              /            2            V          d      −      2       stems from the integration over       x    3    ,  .  .  .  ,      x    d   with   (      x    1    ,  .  .  .  ,      x    d    )  ∈      B    d   and       x    1    2    +      x    2    2    =      r    2  , and on the last line we use our formula for       V    d   and       V          d      −      2      .Let us compute the two integrals. The second one is simpler:                    ∫        0        1                    r      3        (    1    −          r      2              )              d                  /                2        −        1              d    r    =                  [              −                  (        1        −                  r          2                          )                      d                          /                        2                                                (                          2        +        d                  r          2                                      )                                      d        (        d        +        2        )                                      ]                    0      1        =          2              d        (        d        +        2        )            Step 2In the first integral, the integrand is non zero iff   θ  ∈            [        −      π    2    ,      π    2              ]       and   ρ  cos  ⁡  (  θ  )  +      1    −          ρ      2        sin  ⁡  (  θ  ), so iff   θ  ∈            [        0  ,      π    2              ]       or   −      Arctan              (            ρ          1      −              ρ        2                        )        ≤  θ  ≤  0. Let us denote       θ    0    =  −      Arctan              (            ρ          1      −              ρ        2                        )      Thus the integral is                                                         ∫                                                θ                  0                                                                              π                  2                                                              ρ          cos          ⁡          (          θ                      )            2                    +                      1            −                          ρ              2                                cos          ⁡          (          θ          )          sin          ⁡          (          θ          )          d          θ                                    =                              2                          1              −                              ρ                2                                      cos            ⁡            (                          θ              0                                      )              2                        +            ρ            (            π            −            2                          θ              0                        −            sin            ⁡            (            2                          θ              0                        )            )                    4                                                  =                              ρ            π            +            2                          1              −                              ρ                2                                      +            2            ρ                          Arctan                                                      (                                                    ρ                              1                −                                  ρ                  2                                                                                    )                                              4                    To conclude,       with     ρ  =            (              u        T            v      )              (              u        T            u      )      (              v        T            v      )        ,    f  (  u  ,  v  )  =      (          u      T        u    )    (          v      T        v    )    ⋅            ρ      π      +      2              1        −                  ρ          2                    +      2      ρ              Arctan                              (                            ρ                  1          −                      ρ            2                                                )                            4      (      d      +      2      )      π        .

Is there a closed-form formula for f(u,v):=E_x[max(u^⊤ x,0)max(v^⊤x,0)]?

Open question

Answer & Explanation

New Questions in High school geometry