Lecture Note: Econ 703 (week 1)

Date: 2023.08.29

This class focus on modes of convergence.

• almost sure convergence

  • defined by $P(lim)$

• convergence in probability

  • defined by $lim(P)$

• convergence in distribution/week convergence

  • defined by CDF or probability in set $P(X_n\in A)$

Modes of Convergence ​

Probability Space ​

Definition a probability space by $(\mathrm{\Omega },\mathcal{F},\mathcal{P})$, and $X_n,\mathrm{\Omega }\to \mathcal{R}$

Sure convergence ​

almost sure convergence ​

Define as $X_n\stackrel{a.s.}{\to }X$.

Suppose we have a random variable $X_n$ and $X$, then almost sure convergence means:

Note: strongest one, and defined on probability is nearly the same everywhere.

Convergence in probability ​

Define $X_n\stackrel{p}{\to }X$ as $X_n$ converges in probability to $X$, and the key is the probability of limit.

Properties ​

• Convergence in probability → Convergence in distribution

• Convergence in distribution → Convergence in probability when X = c (a constant)

• [continuous mapping theorem] for any continuous function $g$:

  $$X_n\stackrel{p}{\to }X,g(X_n)\stackrel{p}{\to }g(X)$$

• For $X_n\stackrel{p}{\to }X,Y_n\stackrel{p}{\to }Y$, we can have $X_n+Y_n\stackrel{p}{\to }X+Y$

  • in some special cases, it applies for conv in distribution
  • if $X_n,Y_n$ are independent, we have $X_n-Y_n\stackrel{p}{\to }X-Y$

Convergence in distribution ​

Other names can be converge weakly/converge in law.

Properties ​

• Portmanteau lemma

  following two arguments are same:

• Continuous mapping theorem

  for any continuous function $g$:

  $$X_n\stackrel{d}{\to }X,g(X_n)\stackrel{d}{\to }g(X)$$

Other: Convergence in mean ​

Define $X_n$ converges in $r^{th}$ mean towards the random variable $X$ if and only if:

So some special cases are:

• when $r=1$, we say $X_n$ converges in mean to $X$
• when $r=2$, we say $X_n$ converges in mean square to $X$

Theorem from Marginal Joint distribution ​

For $X_n\stackrel{p}{\to }X,Y_n\stackrel{p}{\to }Y$, we have marginal joint distribution $(X_n,Y_n)\stackrel{p}{\to }(X,Y)$

But when it comes to converges in distribution, we need extra assumptions:

• For $X_n\stackrel{p}{\to }X,Y_n\stackrel{p}{\to }C$, we can have $(X_n,Y_n)\stackrel{d}{\to }(X,C)$

Stochastic Order Notation ​

Big O: stochastic boundedness ​

$X_n=O(r_n)$ $|\frac{X_n}{r_n}|<M\in C$ iff

which means $X_n$ is stochastically bounded.

Small o: convergence in probability ​

$X_n=o(r_n)$ iff

for every positive $ϵ$, which means that $r_n$ increase much faster than $X_n$ in any time.

Theorems ​

• $o_p(1)+o_p(1)=o_p(1)$
• $o_p(1)+O_p(1)=O_p(1)$
• $o_p(1)O_p(1)=o_p(1)$

LLNs ​

This section covers several variants of LLM(Law of Large numbers).

Here we use notation as follows:

• $\{X_i\}$ is a series of random vars $\{X_1,X_2,...X_n\}$
• Denote $\mathbb{E}$ as mean of var, and denote $var(X_i)$ as variance
• i.i.d: identical and independent distribution
• Different convergence: $\stackrel{d}{\to },\stackrel{p}{\to }$

Theorem(1713) ​

Given $\{X_i\}\stackrel{i.i.d}{\to }Bernoulli(p)$, then $\overline{X_n}\stackrel{d}{\to }p$

• Note: a special case for LLN.

Forms of Theorems ​

when we wanna find a specific theorem, we should define:

Assumptions

• independent vars
• allows distribution not change

Conclusions

• convergence in p (weak) or in a.s. (strong)
• shape of tails (fat/not fat)

Theorem (pre Chebyshev weak LLN) ​

For $\{X_i\}$ all independent (not i.i.d), we have $\mathbb{E}(X-i)=\mu <\mathrm{\infty }$ and $var(X_i)\le M$ for all $i$, then there exists $0<M<\mathrm{\infty }$ such that:

• Assumption about variance can be relaxed to: $\frac{1}{n^2}\sum _{i}var(X_i)\le o(1)$
• Note: when random var $X_i$ is not unusual (infinite variance), averaging converges to expectation.

Proof

• Given $\mathbb{E}(X_i)=\mu$, we know $\mathbb{E}(\bar{X})=\mu$

• use property: $P(|\overline{X_n}-\mathbb{E}(X_n)|\ge ϵ)\le \frac{var(\overline{X_n})}{{ϵ}^2}$

  from a bounded variance, we can know:

  $$P(|\overline{X_n}-\mathbb{E}(X_n)|\ge ϵ)\le \frac{var(\overline{X_n})}{{ϵ}^2}\le \frac{1}{{ϵ}^2}\frac{M}{n}$$

• According to definition of convergence in p, when $n\to \mathrm{\infty }$, we ensure that $\underset{n\to \mathrm{\infty }}{lim}P(|\overline{X_n}-\mathbb{E}(X_n)|\ge ϵ)\le \underset{n\to \mathrm{\infty }}{lim}\frac{1}{{ϵ}^2}\frac{M}{n}=0$

Theorem (Kolmogorov's 2nd strong LLN) ​

Given $\{X_i\}$ are i.i.d, then

iff $\mathbb{E}(X_i)$ exists and equals to $\mu$ for all i

• for i.i.d sequence, not all finite variance for all vars, no a.s. convergence

CLTs ​

This section covers several variants of CLT(Central Limit Theorem).

From LLN we know:

Now we wanna know the "shape" of such convergence, which is about asymptotic distribution/density.

An intuitive approach is to add a scaler $f(n)$ to enlarge the item:

CLT implies that for a special scaler:

we have a magical property that we can have a special distribution $Z$:

Theorem (Lindeberg-Lévy CLT) ​

Given:

• $\{X_i\}$ are i.i.d
• $\mathbb{E}(X_i)=\mu$ and $var(X_i)={\sigma }^2$

then we have:

• Note: a special scaler results in a special distribution, and it is robust for all random vars

Theorem (Cramér–Wold, vector-form) ​

The above theorem can be easily generalized to vector form.

Following are equal:

• ${\mathbb{X}}_n\stackrel{d}{\to }\mathbb{X}$
• ${\lambda }^{\prime }{\mathbb{X}}_n\stackrel{d}{\to }{\lambda }^{\prime }\mathbb{X}$ for all $\lambda \in R^k$

So ${\lambda }^{\prime }{\mathbb{X}}_n$ is the linear combination of the vector of random vars.

Theorem (multi-var form) ​

• $\{X_i\}$ are i.i.d
• $\mathbb{E}(X_i)=\mu$ and $var(X_i)=v$

then we have:

Theorem (Berry-Esseen) ​

let $J_n(t)=P(\sqrt{n}(\frac{\overline{X_n}-\mu }{\sigma })\le t)$, which is our targeted CDF.

let $\{X_i\}$ be i.i.d with finite 3rd moment, then there exists constant $c$ such that:

• Note: our target CDF is bounded, and generally we can find a small