probability theory - Expected value of visits in a state of a discrete Markov chain

Let

• $X=(X_n)_{n\in\mathbb N_0}$ be a Markov chain with values in a at most countable Polish space $E$ and $\mathcal E$ be the Borel $\sigma$-algebra on $E$


• $(\operatorname P_x)_{x\in E}$ be the distributions of $X$


• $N(y)=\sum_{n\in\mathbb N_0}1_{\left\{X_n=y\right\}}$ be the number of visits of $X$ in $y\in E$

Clearly,

$$\operatorname E_x[N(y)]=\sum_{n\in\mathbb N_0}\operatorname P_x[X_n=y]\;.$$

I've read that it holds

$$\operatorname E_x[N(y)]=\sum_{k\in\mathbb N}\operatorname P_x[N(y)\ge k]\;,$$ but I don't understand why this is true. Is it a typo and what's really meant is "=" instead of "\ge"?

Answer

Here's a formula with uses in lots of places: If $N$ is a non-negative integer-valued random variable, then $E[N] =\sum_{k=1}^\infty P[N\ge k]$. To see this write

$$\sum_{k=1}^\infty P[N\ge k]=\sum_{k=1}^\infty \sum_{j=k}^\infty P[N=j]=\sum_{j=1}^\infty \sum_{k=1}^j P[N=j]=\sum_{j=1}^\infty j\cdot P[N=j]=E[N]$$