Skip to content

Lecture Note: Econ 703 (week 1)

🕒 Published at:

Date: 2023.08.29

This class focus on modes of convergence.

  • almost sure convergence

    • defined by
  • convergence in probability

    • defined by
  • convergence in distribution/week convergence

    • defined by CDF or probability in set

Modes of Convergence

Probability Space

Definition a probability space by , and

Sure convergence

almost sure convergence

Define as .

Suppose we have a random variable and , then almost sure convergence means:

Note: strongest one, and defined on probability is nearly the same everywhere.

Convergence in probability

Define as converges in probability to , and the key is the probability of limit.


  • Convergence in probability → Convergence in distribution

  • Convergence in distribution → Convergence in probability when X = c (a constant)

  • [continuous mapping theorem] for any continuous function :

  • For , we can have

    • in some special cases, it applies for conv in distribution
    • if are independent, we have

Convergence in distribution

Other names can be converge weakly/converge in law.


  • Portmanteau lemma

    following two arguments are same:

  • Continuous mapping theorem

    for any continuous function :

Other: Convergence in mean

Define converges in mean towards the random variable if and only if:

So some special cases are:

  • when , we say converges in mean to
  • when , we say converges in mean square to

Theorem from Marginal Joint distribution

For , we have marginal joint distribution

But when it comes to converges in distribution, we need extra assumptions:

  • For , we can have

Stochastic Order Notation

Big O: stochastic boundedness


which means is stochastically bounded.

Small o: convergence in probability


for every positive , which means that increase much faster than in any time.



This section covers several variants of LLM(Law of Large numbers).

Here we use notation as follows:

  • is a series of random vars
  • Denote as mean of var, and denote as variance
  • i.i.d: identical and independent distribution
  • Different convergence:


Given , then

  • Note: a special case for LLN.

Forms of Theorems

when we wanna find a specific theorem, we should define:


  • independent vars
  • allows distribution not change


  • convergence in p (weak) or in a.s. (strong)
  • shape of tails (fat/not fat)

Theorem (pre Chebyshev weak LLN)

For all independent (not i.i.d), we have and for all , then there exists such that:

  • Assumption about variance can be relaxed to:
  • Note: when random var is not unusual (infinite variance), averaging converges to expectation.


  • Given , we know

  • use property:

    from a bounded variance, we can know:

  • According to definition of convergence in p, when , we ensure that

Theorem (Kolmogorov's 2nd strong LLN)

Given are i.i.d, then

iff exists and equals to for all i

  • for i.i.d sequence, not all finite variance for all vars, no a.s. convergence


This section covers several variants of CLT(Central Limit Theorem).

From LLN we know:

Now we wanna know the "shape" of such convergence, which is about asymptotic distribution/density.

An intuitive approach is to add a scaler to enlarge the item:

CLT implies that for a special scaler:

we have a magical property that we can have a special distribution :

Theorem (Lindeberg-Lévy CLT)


  • are i.i.d
  • and

then we have:

  • Note: a special scaler results in a special distribution, and it is robust for all random vars

Theorem (Cramér–Wold, vector-form)

The above theorem can be easily generalized to vector form.

Following are equal:

  • for all

So is the linear combination of the vector of random vars.

Theorem (multi-var form)

  • are i.i.d
  • and

then we have:

Theorem (Berry-Esseen)

let , which is our targeted CDF.

let be i.i.d with finite 3rd moment, then there exists constant such that:

  • Note: our target CDF is bounded, and generally we can find a small