Information theory -- Computer Science-Mathematics connections -- info.uvt.ro

Entropy

$0\log _{2}0=0$ convention

Connects with: Calculus

In computing entropy, the convention that $0\log _{2}0=0$ convention is made due to the fact that $\lim _{x\to 0}x\log _{2}x=0,$ as a continuity argument.

Proof:

$\lim _{x\to 0}x\log _{2}x=\lim _{x\to 0}{\frac {-\log _{2}{\frac {1}{x}}}{\frac {1}{x}}}$.

L'Hôpital's rule may be applied, since $\lim _{x\to 0}-\log _{2}{\frac {1}{x}}=-\infty$ and $\lim _{x\to 0}{\frac {1}{x}}=\infty$.

Thus, $\lim _{x\to 0}{\frac {-\log _{2}{\frac {1}{x}}}{\frac {1}{x}}}=\lim _{x\to 0}{\frac {(-\log _{2}{\frac {1}{x}})'}{({\frac {1}{x}})'}}$ $=\lim _{x\to 0}-{\frac {1}{\ln 2}}{\frac {(\ln {\frac {1}{x}})'}{({\frac {1}{x}})'}}$ $=\lim _{x\to 0}-{\frac {1}{\ln 2}}{\frac {x({\frac {1}{x}})'}{({\frac {1}{x}})'}}$ $=\lim _{x\to 0}-{\frac {1}{\ln 2}}x=0.$