Entropy of geometric random variable?I am wondering how to derive the entropy of a geometric random variable? Or where I can find some proof/derivation? I tried to search online, but seems not much resources is available.Here is the probability density function of geometric distribution: ( 1 − p ) k − 1 pHere is the entropy of a geometric distribution: − ( 1 − p ) log 2 ⁡ ( 1 − p ) − p log 2 ⁡ p p Where p is the probability for the event to occur during each single experiment.

Question

Entropy of geometric random variable?I am wondering how to derive the entropy of a geometric random variable? Or where I can find some proof/derivation? I tried to search online, but seems not much resources is available.Here is the probability density function of geometric distribution:   (  1  −  p      )          k      −      1          pHere is the entropy of a geometric distribution:             −      (      1      −      p      )              log        2            ⁡      (      1      −      p      )      −      p              log        2            ⁡      p        p  Where p is the probability for the event to occur during each single experiment.

Dobricap · Accepted Answer

Step 1Assume   P  (  X  =  k  )  =  (  1  −  p      )          k      −      1        p, where   k  ∈      Z          +      , then the entropy isStep 2                    E        n        t        r        o        p        y        (        X        )                            =                  ∑                      k            =            1                                +            ∞                          −        (        1        −        p                  )                      k            −            1                          p        ⋅                  log                      2                          ⁡                  (          (          1          −          p                      )                          k              −              1                                p          )                                                  =        −        p        ⋅                  log                      2                          ⁡                  (          p          )                          ∑                      k            =            1                                +            ∞                          (        1        −        p                  )                      k            −            1                          −        p        ⋅                  log                      2                          ⁡                  (          1          −          p          )                          ∑                      k            =            1                                +            ∞                          (        k        −        1        )        (        1        −        p                  )                      k            −            1                                                            =        −                  log                      2                          ⁡                  (          p          )                −                              (            1            −            p            )                          log                              2                                      ⁡                          (              1              −              p              )                                p

Mattie Monroe · Accepted Answer

Step 1It may be faster to derive the above entropy using the memoryless property of the Geometric distribution. Let Y be an auxiliary binary random variable such that   Y  =  0 indicates   X  &amp;gt;  1. Then   H  (  X  )  =      H    B    (  p  )      /    p follows from the following:                    H        (        X        ,        Y        )                            =        H        (        X                  |                Y        )        +        H        (        Y        )                                          =        H        (        Y                  |                X        )        +        H        (        X        )            Step 2Note that                     H        (        X                  |                Y        )                            =        H        (        X                  |                Y        =        1        )        P        (        Y        =        1        )        +        H        (        X                  |                Y        =        0        )        P        (        Y        =        0        )        =        H        (        X                  |                Y        =        0        )        (        1        −        p        )        =        (        1        −        p        )        H        (        X        )                            H        (        Y                  |                X        )                            =        0                            H        (        Y        )                            =                  H          B                (        p        )        ,            where the memoryless property indicates   H  (  X      |    Y  =  0  )  =  H  (  X  ), and       H    B    (  p  ) is the binary entropy function.

Answered question

Answer & Explanation

New Questions in High school geometry