Home  ›  Conditional Expectation One Discrete One Continuous

Conditional Expectation One Discrete One Continuous

Written By Friday, 14 October 2022 Add Comment Edit

The conditional expectation (or conditional expected value, or conditional mean) is the expected value of a random variable, computed with respect to a conditional probability distribution.

Table of Contents

Table of contents

  1. A pragmatic approach

  2. Definition

  3. Conditional expectation of a discrete random variable

  4. Conditional expectation of a continuous random variable

  5. Conditional expectation in general

  6. Properties of conditional expectation

  7. Law of iterated expectations

  8. Solved exercises

    1. Exercise 1

    2. Exercise 2

    3. Exercise 3

As in the case of the expected value, a completely rigorous definition of the conditional expectation requires a complicated mathematical apparatus.

To make things simpler, we do not give a completely rigorous definition in this lecture. We rather give an informal definition and we show how the conditional expectation can be computed.

In particular, we discuss how to calculate the conditional expected value of a random variable X when we observe the realization of another random variable Y , that is, when we receive the information that Y=y .

The following informal definition is very similar to our previous definition of the expected value.

Definition Let X and Y be two random variables. The conditional expectation of X given Y=y is the weighted average of the values that X can take on, where each possible value is weighted by its respective conditional probability (conditional on the information that Y=y ).

The expectation of a random variable X conditional on Y=y is denoted by [eq1]

We start with the case in which X and Y are two discrete random variables and, considered together, they form a discrete random vector.

The formula for the conditional mean of X given Y=y is a straightforward implementation of the above informal definition: the weights of the average are given by the conditional probability mass function of X .

Definition Let X and Y be two discrete random variables. Let R_X be the support of X and let [eq2] be the conditional probability mass function of X given Y=y . The conditional expectation of X given Y=y is [eq3] provided that [eq4]

If you do not understand the symbol $sum_{xin R_{X}}$ and the finiteness condition above (absolute summability) go back to the lecture on the Expected value, where they are explained.

Example Let the support of the random vector [eq5] be [eq6] and its joint probability mass function be [eq7] Let us compute the conditional probability mass function of X given $Y=0$ . The marginal probability mass function of Y evaluated at $y=0$ is [eq8] The support of X is [eq9] Thus, the conditional probability mass function of X given $Y=0$ is [eq10] The conditional expectation of X given $Y=0$ is [eq11]

Let us now tackle the case in which X and Y are continuous random variables, forming a continuous random vector.

The formula for the conditional mean of X given Y=y involves an integral, which can be thought of as the limiting case of the summation [eq12] found in the discrete case above.

Definition Let X and Y be two continuous random variables. Let R_X be the support of X and let [eq13] be the conditional probability density function of X given Y=y . The conditional expectation of X given Y=y is [eq14] provided that [eq15]

If you do not understand why an integration is required and why the finiteness condition above (absolute integrability) is imposed, you can find an explanation in the lecture entitled Expected value.

Example Let the support of the random vector [eq16] be [eq17] and its joint probability density function be [eq18] Let us compute the conditional probability density function of X given $Y=2$ . The support of Y is [eq19] When [eq20] , the marginal probability density function of Y is 0 ; when [eq21] , the marginal probability density function is [eq22] Thus, the marginal probability density function of Y is [eq23] When evaluated at $y=2$ , it is [eq24] The support of X is [eq25] Thus, the conditional probability density function of X given $Y=2$ is [eq26] The conditional expected value of X given $Y=2$ is [eq27]

The general formula for the conditional expectation of X given Y=y does not require that the two variables form a discrete or a continuous random vector, but it is applicable to any random vector.

Definition Let [eq28] be the conditional distribution function of X given Y=y . The conditional expectation of X given Y=y is [eq29] where the integral is a Riemann-Stieltjes integral and the expected value exists and is well-defined only as long as the integral is well-defined.

The above formula follows the same logic of the formula for the expected value [eq30] with the only difference that the unconditional distribution function [eq31] has now been replaced with the conditional distribution function [eq32] .

If you are puzzled by these formulae, you can go back to the lecture on the Expected value, which provides an intuitive introduction to the Riemann-Stieltjes integral.

From the above sections, it should be clear that the conditional expectation is computed exactly as the expected value, with the only difference that probabilities and probability densities are replaced by conditional probabilities and conditional probability densities.

Therefore, the properties enjoyed by the expected value, such as linearity, are also enjoyed by the conditional expectation.

Before knowing the realization of Y , the conditional expectation of X given Y is unknown and can itself be regarded as a random variable. We denote it by [eq33] .

In other words, [eq34] is a random variable such that its realization equals [eq35] when $y$ is the realization of Y .

This random variable satisfies a very important property, known as law of iterated expectations (or tower property): [eq36]

Proof

For discrete random variables this is proved as follows: [eq37] For continuous random variables the proof is analogous: [eq38]

Below you can find some exercises with explained solutions.

Exercise 1

Let [eq39] be a discrete random vector with support [eq40] and joint probability mass function [eq41]

What is the conditional expectation of X given $Y=2$ ?

Solution

Let us compute the conditional probability mass function of X given $Y=2$ . The marginal probability mass function of Y evaluated at $y=2$ is [eq42] The support of X is [eq43] Thus, the conditional probability mass function of X given $Y=2$ is [eq44] The conditional expectation of X given $Y=2$ is [eq45]

Exercise 2

Suppose that [eq39] is a continuous random vector with support [eq47] and joint probability density function [eq48]

Compute the expected value of Y conditional on $X=1$ .

Solution

We first need to compute the conditional probability density function of Y given $X=1$ , by using the formula [eq49] Note that, by using indicator functions, we can write [eq50] The marginal probability density function [eq51] is obtained by marginalizing the joint density: [eq52] When evaluated at $x=1$ , it is [eq53] Furthermore, [eq54] Thus, the conditional probability density function of Y given $X=1$ is [eq55] The conditional expectation of Y given $X=1$ is [eq56]

Exercise 3

Let X and Y be two random variables.

Remember that the variance of X can be computed as [eq57]

In a similar manner, the conditional variance of X , given Y=y , can be defined as [eq58]

Use the law of iterated expectations to prove that [eq59]

Solution

Please cite as:

Taboga, Marco (2021). "Conditional expectation", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-probability/conditional-expectation.

stephensonthation.blogspot.com

Source: https://www.statlect.com/fundamentals-of-probability/conditional-expectation

0 Response to "Conditional Expectation One Discrete One Continuous"

Post a Comment

Subscribe to: Post Comments (Atom)

Iklan Atas Artikel

Iklan Tengah Artikel 1

Iklan Tengah Artikel 2

Iklan Bawah Artikel