Relation to other distances

The total variation distance is related to the Kullback–Leibler divergence by Pinsker’s inequality:

${\displaystyle \delta (P,Q)\leq {\sqrt {{\frac {1}{2}}D_{\mathrm {KL} }(P\parallel Q)}}.}$

One also has the following inequality, due to Bretagnolle and Huber^[2] (see, also, Tsybakov^[3]), which has the advantage of providing a non-vacuous bound even when ${\displaystyle D_{\mathrm {KL} }(P\parallel Q)>2}$:

${\displaystyle \delta (P,Q)\leq {\sqrt {1-e^{-D_{\mathrm {KL} }(P\parallel Q)}}}.}$

The total variation distance is half of the L¹ distance between the probability functions: on discrete domains this is the distance between probability mass functions^[4] ${\displaystyle \delta (P,Q)={\frac {1}{2}}\sum _{x}|P(x)-Q(x)|}$, The relationship holds more generally as well:^[5] ${\displaystyle \delta (P,Q)={\frac {1}{2}}\int |p(x)-q(x)|\mathrm {d} x}$ when the distributions have standard probability density functions p and q, or the analogous distance between Radon-Nikodym derivatives with any common dominating measure.

The total variation distance is related to the Hellinger distance ${\displaystyle H(P,Q)}$ as follows:^[6]

${\displaystyle H^{2}(P,Q)\leq \delta (P,Q)\leq {\sqrt {2}}H(P,Q).}$

These inequalities follow immediately from the inequalities between the 1-norm and the 2-norm.

Connection to transportation theory

The total variation distance (or half the norm) arises as the optimal transportation cost, when the cost function is ${\displaystyle c(x,y)={\mathbf {1} }_{x\neq y}}$, that is,

${\displaystyle {\frac {1}{2}}\|P-Q\|_{1}=\delta (P,Q)=\inf\{\mathbb {P} (X\neq Y):{\text{Law}}(X)=P,{\text{Law}}(Y)=Q\}=\inf _{\pi }\operatorname {E} _{\pi }[{\mathbf {1} }_{x\neq y}],}$

where the expectation is taken with respect to the probability measure ${\displaystyle \pi }$ on the space where ${\displaystyle (x,y)}$ lives, and the infimum is taken over all such ${\displaystyle \pi }$ with marginals ${\displaystyle P}$ and ${\displaystyle Q}$, respectively.^[7]