Taken from Real Analysis and Probability by R.M. Dudley.

For a sequence of $n$ repeated, independent trials of an experiment, some probability distributions and variables converge as $n$ tends to infinity. In proving such limit theorems, it is useful to be able to construct a probability space on which a sequence of independent random variables is defined in a natural way; specifically, as coordinates for a countable Cartesian product.

The Cartesian product of finitely many $\sigma$-finite measure spaces gives a $\sigma$-finite measure space. For example, Cartesian products of Lesbesgue measure on the line give Lesbesgue measure on finite-dimensional Euclidean spaces. But suppose we take a measure space $\{0,1\}$ with two points each having measure $1$$\mu(\{0\})=1=\mu(\{1\})$, and form a countable Cartesian product of copies of this space, so that the measure of any countable product of sets equals the product of their measures. Then we would get an uncountable space in which all singletons have measure $1$, giving the measure usually called counting measure. An uncountable set with counting measure is not a $\sigma$-finite space, although in this example it was a countable product of finite measure spaces. By contrast, the the countable product of probability measures will again be a probability space. Here are some definitions.

For each $n=1,2,\dots$ let $(\Omega_n,S_n,P_n)$ be a probability space. Let $\Omega$ be the Cartesian product $\displaystyle \prod_{n\geq 1}\Omega_n$, that is, the set of all sequences $\{\omega_n\}_{n\geq 1}$ with $\omega_n\in\Omega_n$ for all $n$. Let $\pi_n$ be the natural projection of $\Omega$ onto $\Omega_n$ for each $n$$\pi_n\left(\{\omega_m\}_{m\geq 1}\right)=\omega_n$ for all $n$. Let $S$ be the smallest $\sigma$-algebra of subsets of $\Omega$ such that for all $m$$\pi_m$ is measurable from $(\Omega,S)$ to $(\Omega_m,S_m)$. In other words, $S$ is the smallest $\sigma$-algebra containing all sets $\pi_n^{-1}(A)$ for all $n$ and all $A\in S_n$.

Let $\mathcal{R}$ be the collection of all sets $\displaystyle \prod_{n\geq 1}A_n\subset\Omega$ where $A_n\in \mathcal{S}_n$ for all $n$ and $A_m=\Omega_m$ except for at most finitely many values of $n$. Elements of $\mathcal{R}$ will be called rectangles. Now recall the notion of semiring. $\mathcal{R}$ has this property.

### Definition

For any set $X$, a collection $\mathcal{D}\subset 2^X$ is called a semiring if $\emptyset\in\mathcal{D}$ and for any $A$ and $B$ in $\mathcal{D}$, we have $A\cap B\in\mathcal{D}$ and $\displaystyle A\backslash B=\bigcup_{j\geq 1}C_j$ for some finite $n$ and disjoint $C_j\in\mathcal{D}$.

## Proposition

The collection $\mathcal{R}$ if rectangles in the infinite product $\Omega$ is a semiring. The algebra $\mathcal{A}$ generated by $\mathcal{R}$ is the collection of finite disjoint unions of elements of $\mathcal{R}$.

Proof : If $C$ and $D$ are any two rectangles, then clearly $C\cap D$ is a rectangle ($\checkmark$). In a product of two spaces, the collection of rectangles is a semiring (p.95, Proposition 3.2.2). Specifically, a difference of two rectangles is a union of two disjoint rectangles:

$\displaystyle (A\times B)\backslash (E\times F)=((A\backslash E)\times B)\cup ((A\cap E)\times(B\backslash F))$.

It follows by induction that in any finite Cartesian product, any difference $C\backslash D$ of rectangles is a finite disjoint union of rectangles. Thus $\mathcal{R}$ is a semiring. We have $\Omega\in\mathcal{R}$, so the ring generated by $\mathcal{R}$ is an algebra (p.96, Proposition 3.2.3). Since every algebra is a ring, $\mathcal{A}$ is the algebra generated by $\mathcal{R}$. By Proposition 3.2.3, $\mathcal{A}$ consists of all finite disjoint unions of elements of $\mathcal{R}$ $\bullet$

Now for $A=\prod_n A_n\in\mathcal{R}$, let $P(A):=\prod_nP_n(A_n)$. The product converges since all but finitely many factors are $1$. Here is the main theorem to be proved in the rest of this section:

## Theorem: Existence Theorem for Infinite Product Probabilities

$P$ on $\mathcal{R}$ extends uniquely to a (countably additive) probability measure on $\mathcal{S}$.

Proof : For each $A\in\mathcal{A}$, write $A$ as a finite disjoint union of sets in $\mathcal{R}$, say

$\displaystyle A=\bigcup_{r=1}^N B_r$,

and define

$\displaystyle P(A):=\sum_{r=1}^NP(B_r)$.

Let us first show that $P$ is well-defined and finitely additive on finite disjoint unions. Each $B_r$ is a product of sets $A_{rn}\in\mathcal{S}_n$ with $A_{rn}=\Omega_n$ for all $n\geq n(r)$ for some finite $n(r)$. Let $m$ be the maximum of the $n(r)$ for $r=1,\dots,N$. Then since all the $A_{rn}$ equal $\Omega_n$ for $n\geq m$, properties of $P$ on such sets are equivalent to properties of the finite product measure on $\Omega_1\times\cdots\times\Omega_m$. To show that $P$ is well-defined, if a set of two different finite disjoint union of sets in $\mathcal{R}$, we can take the maximum of the values of $m$ for the two unions and still have a finite product. So $P$ is well-defined and finitely additive on $\mathcal{A}$ by the finite product measure theorem (p.139 , Theorem 4.4.6).

If $P$ us countably additive on $\mathcal{A}$, then it has a unique countably additive extension to $\mathcal{S}$ by the Carathéodory Extension Theorem. So it’s enough to prove countable additivity on $\mathcal{A}$. Equivalently, if $A_j\in \mathcal{A}$, $A_1\supset A_2\supset\cdots$, and $\cap_j A_j=\emptyset$ we want to prove $P(A_j)\downarrow 0$ ($P$ is continuous at $\emptyset$” — p.86, Theorem 3.1.1). In other words, if $A_j$ is a decreasing sequence of sets in $\mathcal{A}$ and for some $\varepsilon>0$$P(A_j)\geq\varepsilon$ for all $j$, we must show $\cap_jA_j\neq\emptyset$.

Let $P^{(0)}L=P$ on $\mathcal{A}$. For each $n\geq 1$, let

$\displaystyle \Omega^{(n)}:=\prod_{m>n}\Omega_m$.

Let $\mathcal{A}^{(n)}$ and $P^{(n)}$ be defined on $\Omega^{(n)}$ just as $\mathcal{A}$ and $P$ were on $\Omega$. For each $E\subset \Omega$ and $x_i\in\Omega_i$$i=1,\dots,n$, let

$\displaystyle E^{(n)}(x_1,\dots,x_n):=\left\{\{x_m\}_{m>n}\in\Omega^{(n)}:x=\{x_i\}_{i\geq 1}\in E\right\}$.

For a set $A$ in a product space $X\times Y$ and $x\in X$, let $A_x:=\{y\in Y:(x,y)\in A\}$. If $A$ is in a product $\sigma$-algebra $\mathcal{S}\otimes\mathcal{T}$ then $A_x\in\mathcal{T}$ (due to the proof of Theorem 4.4.3, p. 135). For any $E\in\mathcal{A}$ there is an $N$ large enough so that

$\displaystyle E=F\times\prod_{n>N}\Omega_n$ for some $\displaystyle F\subset \prod_{n\leq N}\Omega$.

Since $E$ is a finite union of rectangles with this property, take the maximum of the values of $NN$ for the rectangles. Then

$\displaystyle F=\bigcup_{k=1}^m F_k$

where

$\displaystyle F_k=\prod_{i=1}^NF_{ki}$

for some $F_{ki}\in\mathcal{S}_i$$i=1,\dots,N$$k=1,\dots,m$. Now for any $n, and $x_i\in\Omega_i$$i=1,\dots,n$$E^{(n)}(x_1,\dots,x_n)=G\times \Omega^{(N)}$ where $G$ is the union of those sets

$\displaystyle \prod_{i=n+1}^NF_{ki}$ such that $x_i\in F_{ki}$ for all $i=1,\dots,n$.

Thus $E^{(n)}(x_1,\dots,x_n)\in \mathcal{A}^{(n)}$, so $P^{(n)}$ of it is defined. Then by the Tonelli-Fubini theorem in

$\displaystyle\Omega_1\times\cdots\times\Omega_n\times\prod_{i=n+1}^N\Omega_i$,

we have

$\displaystyle P(E)=\int P^{(n)}\left(E^{(n)}(x_1,\dots,x_n)\right)\,\prod_{j=1}^ndP_j(x_j)$.

For $\varepsilon$ with $P(A_j)\geq\varepsilon$ for all $j$, let

$F_j:=\left\{x_1\in\Omega_1:P^{(1)}\left(A_j^{(1)}(x_1)\right)>\varepsilon/2\right\}$.

For each $j$, apply the formula for $P$ to $E=A_j$$n=1$. Then

$\displaystyle\varepsilon\leq P(A_j)=\int P^{(1)}\left(A_j^{(1)}(x_1)\right)\,dP_1(x_1)$

$\displaystyle \left(\int_{F_j}+\int_{\Omega_1\backslash F_j}\right)P^{(1)}\left(A_j^{(1)}(x_1)\right)\,dP_1(x_1)$

$\leq P_1(F_j)+\varepsilon/2$.

Thus $P_1(F_j)\geq\varepsilon/2$ for all $j$. As $j$ increases, the sets $A_j$ decrease; thus, so do the $A_j^{(1)}$ and the $F_j$.

Since $P_1$ is countably additive,

$\displaystyle P_1\left(\bigcap_{j}F_j\right)\geq \varepsilon/2>0$

by monotone convergence of indicator functions, so $\cap_j F_j\neq\emptyset$. Take any $y\in\cap_j F_j$. Let $f_j(y,x):=P^{(2)}\left(A_j^{(2)}(y,x)\right)$ and $G_j:=\{x_2\in\Omega_2:f_j(y_1,x_2)>\varepsilon/4\}$. Then $G_j$ decreases as $j$ increases,

$\displaystyle \varepsilon/2    for al $j$,

and $P_2(G_j)>\varepsilon/4$ for all $j$, so the intersection of all the $G_j$ is non-empty in $\Omega_2$ and we can choose $y_2$ in it.

Inductively, by the same argument there are $y_n\in\Omega_n$ for all $n$ such that $P^{(n)}\left(y_1,\cdots,y_n\right)\geq\varepsilon/2^n$ for all $j$ and $n$. Let $y:=\{y_n\}_{n\geq 1}\in\Omega$. To prove that $y\in A_j$ for each $j$, choose $n$ large enough (depending on $j$) so that for all $x_1,\dots,x_n$$A_j^{(n)}(x_1,\dots,x_n)=\emptyset$ or $\Omega^{(n)}$. This is possible since $A_j\in\mathcal{A}$. Then $A_j^{(n)}(y_1,\dots,y_n)=\Omega^{(n)}$, so $y\in A_j$. Hence $\cap_j A_j\neq\emptyset$ $\bullet$

Actually, this theorem holds for arbitrary (not necessarily countable) products of probability spaces. The proof needs no major change, since each set in the $\sigma$-algebra $\mathcal{S}$ depends only on countably many coordinates. In other words, given a product $\prod_{i\in I}\Omega)i$, where $I$ is a possibly uncountable index set, for each set $A\in\mathcal{S}$ there is a countable subset $J$ of $I$ and a set $B\subset\prod_{i\in J}\Omega_i$ such that $A=B\times\prod_{i\not\in J}\Omega_j$.