real analysis - Summation Symbol: Changing the Order

I have some questions regarding the order of the summation signs (I have tried things out and also read the wikipedia page, nevertheless some questions remained unanswered):

Original 1. wikipedia says that:

$$\sum_{k=1}^m a_k \sum_{\color{red}{k}=1}^n b_l = \sum_{k=1}^m \sum_{l=1}^n a_k b_l$$

does not necessarily hold. What would be a concrete example for that?

Edited 1. wikipedia says that:

$$\sum_{k=1}^m a_k \sum_{\color{red}{l}=1}^n b_l = \sum_{k=1}^m \sum_{l=1}^n a_k b_l$$

does not necessarily hold. What would be a concrete example for that?

2.As far as I see generally it holds that:

$$\sum_{j=1}^m \sum_{i=1}^n a_ib_j = \sum_{i=1}^n \sum_{j=1}^m a_ib_j$$

why is that? It is not due to the property, that multiplication is commutative, is it?

3.What about infinite series, when does:
$$\sum_{k=1}^{\infty}\sum_{l=1}^{\infty} a_kb_l = \sum_{k=1}^{\infty}a_k \sum_{l=1}^{\infty}b_l$$ hold?
And does here too $$\sum_{k=1}^{\infty}\sum_{l=1}^{\infty} a_kb_l = \sum_{l=1}^{\infty}\sum_{k=1}^{\infty} a_kb_l$$ hold?

Thanks

Answer

For the *original first question where $l = k$, let $m=n=2$, $a_1=b_1=1$, and $a_2=b_2=2$; then

$$\sum_{k=1}^2a_k\sum_{k=1}^2b_k=\sum_{k=1}^2a_k(1+2)=1\cdot3+2\cdot3=9\;,$$

but $$\sum_{k=1}^2\sum_{k=1}^2a_kb_k=\sum_{k=1}^2(1^2+2^2)=5+5=10\;.$$

For the second question, imagine arranging the terms $a_ib_j$ in an $n\times m$ array:

$$\begin{array}{ccccc|c} a_1b_1&a_1b_2&a_1b_3&\dots&a_1b_m&\sum_{j=1}^ma_1b_j\\ a_2b_1&a_2b_2&a_2b_3&\dots&a_2b_m&\sum_{j=1}^ma_2b_j\\ a_3b_1&a_3b_2&a_3b_3&\dots&a_3b_m&\sum_{j=1}^ma_3b_j\\ \vdots&\vdots&\vdots&&\vdots&\vdots\\ a_nb_1&a_nb_2&a_nb_3&\dots&a_nb_m&\sum_{j=1}^ma_nb_j\\ \hline \sum_{i=1}^na_ib_1&\sum_{i=1}^na_ib_2&\sum_{i=1}^na_ib_3&\dots&\sum_{i=1}^na_ib_m \end{array}$$

For each $j=1,\dots,m$, $\sum_{i=1}^na_ib_j$ is the sum of the entries in column $j$, and for each $i=1,\dots,n$, $\sum_{j=1}^ma_ib_j$ is the sum of the entries in row $i$. Thus,

$$\begin{align*} \sum_{j=1}^m\sum_{i=1}^na_ib_j&=\sum_{j=1}^m\text{sum of column }j\\ &=\sum_{i=1}^n\text{sum of row }i\\ &=\sum_{i=1}^n\sum_{j=1}^ma_ib_j\;. \end{align*}$$

For infinite double series the situation is a bit more complicated, since an infinite series need not converge. However, it is at least true that if either of

$$\sum_{j=1}^m\sum_{i=1}^n|a_ib_j|\quad\text{and}\quad\sum_{i=1}^n\sum_{j=1}^m|a_ib_j|$$

converges, then the series without the absolute values converge and are equal. This PDF has much more information on double sequences and series.