técnicas de reducción de varianza
TRANSCRIPT
-
8/18/2019 Técnicas de reducción de varianza
1/49
Reduction variance techniques
Chapter 9 from Ross (2013)
Freddy Hernández BarajasUniversidad Nacional de Colombia
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
http://find/
-
8/18/2019 Técnicas de reducción de varianza
2/49
Introduction
Suppose as usual that we wish to estimate θ. Then the standardsimulation algorithm is:
1 Generate U1, U2, . . . , Un.
2
Estimate θ with θ̂n = n
j =1 X j /n where X j = h(U j )3 Approximate 100(1 − α)% confidence intervals are given by
θ̂n − z α/2
σ̂n√ n
, θ̂n + z α/2σ̂n√ n
where σ̂n is the usual estimate of Var (X ) based on X 1,X 2, . . . ,X n.
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
http://find/
-
8/18/2019 Técnicas de reducción de varianza
3/49
Introduction
We would like IC to be small, but sometimes this is difficult toachieve. This may be because Var (X ) is too large, or too muchcomputational effort is required to simulate each X j so that n isnecessarily small, or some combination of the two.
There are a number of things we can do:
Develop a good simulation algorithm.
Program carefully to minimize execution time.
Program carefully to minimize storage requirements. Forexample we do not need to store all the X j ’s: we only need to
keep track of
X j and
X 2 j .
Decrease the variability of the simulation output that we use to
estimate θ. The techniques used to do this are usually calledvariance reduction techniques.
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
http://goforward/http://find/http://goback/
-
8/18/2019 Técnicas de reducción de varianza
4/49
Variance reduction techniques
1 Antithetic variables.
2 Control variates.3 Conditioning.
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
http://find/
-
8/18/2019 Técnicas de reducción de varianza
5/49
1. Antithetic variables
Suppose we are interested in using simulation to estimate θ = E [X ]and suppose we have generated X 1 and X 2, identically distributedrandom variables having mean θ. Then
Var
X 1 + X 2
2
=
1
4 (Var (X 1) + Var (X 2) + 2Cov (X 1,X 2))
Hence it would be adventageous if X 1 and X 2 rather than being
independent were negatively correlated.
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
http://find/http://goback/
-
8/18/2019 Técnicas de reducción de varianza
6/49
. . . continuation
To see how we migth arrange for X 1 and X 2 to be negativelycorrelated, suppose that X 1 is a function of m random numbers
X 1 = h(U 1,U 2, . . . ,U m)
where U 1,U 2, . . . ,U m are m independent random numbers.
If U ∼ U (0, 1) then 1 − U is also distribuited as U (0, 1) and
X 2 = h(1 − U 1, 1 − U 2, . . . , 1 − U m)
has the same distribution as X 1 and it is negatively correlated withX 1.
What type of function could be h(·)? Consult page 156.
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
http://find/
-
8/18/2019 Técnicas de reducción de varianza
7/49
Example 9d
Suppose we were interested in using simulation to estimate
θ = E [e u ] = 1
0
e x dx
It is apropiate the use of the antithetic variable 1 − u ?We will compare the variance without and with antithetic variablesto decide.
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
http://find/
-
8/18/2019 Técnicas de reducción de varianza
8/49
. . . continuation
First
Second
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
http://find/
-
8/18/2019 Técnicas de reducción de varianza
9/49
. . . continuation
We see that the use of independent variables results in a variance of
whereas the use of the antithetic variables U and 1 − U gives avariance of
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
http://find/
-
8/18/2019 Técnicas de reducción de varianza
10/49
. . . continuation
Without antithetic variables
g
-
8/18/2019 Técnicas de reducción de varianza
11/49
. . . continuation
Using antithetic variables
x2
-
8/18/2019 Técnicas de reducción de varianza
12/49
. . . continuation
Comparing the two approaches
0 200 400 600 800 1000
1.5
1.6
1.7
1.8
1.9
2.0
Without antithetic variables
Iterations
M e a n e v o l u t i o n
True integral value
0 200 400 600 800 1000
1.5
1.6
1.7
1.8
1.9
2.0
With antithetic variable
Iterations
M e a n e v o l u t i o n
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
ti ti
http://find/
-
8/18/2019 Técnicas de reducción de varianza
13/49
. . . continuation
f
-
8/18/2019 Técnicas de reducción de varianza
14/49
. . . continuation
aprox
-
8/18/2019 Técnicas de reducción de varianza
15/49
. . . continuation
n
-
8/18/2019 Técnicas de reducción de varianza
16/49
. . . continuation
mean(without.anti)
## [1] 1.718942
sd(without.anti)
## [1] 0.01525596
mean(with.anti)
## [1] 1.718274
sd(with.anti)
## [1] 0.002755573
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
Example 1
http://find/
-
8/18/2019 Técnicas de reducción de varianza
17/49
Example 1
Estimate the integral ∞
0 g (x ) dx = ∞
0 log(1 + x 2)e −x dx .
To solve the integral we can use the change variable t = 1 − e −x toensure that f (
·) is a monotonic function.
f (t ) = log(1 + log2(1 − t ))
and the integral to solve is
1
0 f (t ) = log(1 + log2(1 − t ))dt .
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
continuation
http://find/
-
8/18/2019 Técnicas de reducción de varianza
18/49
. . . continuation
0 5 10 15
0 .
0 0
0 .
0 5
0 .
1 0
0 .
1 5
0 .
2 0
0 .
2 5
x
g ( x )
0.0 0.2 0.4 0.6 0.8 1.0
0 .
0
0 .
5
1 .
0
1 .
5
2 .
0
2 .
5
3 .
0
x
f ( x )
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
continuation
http://find/
-
8/18/2019 Técnicas de reducción de varianza
19/49
. . . continuation
f
-
8/18/2019 Técnicas de reducción de varianza
20/49
. . . continuation
aprox
-
8/18/2019 Técnicas de reducción de varianza
21/49
. . . continuation
n
-
8/18/2019 Técnicas de reducción de varianza
22/49
Example 2
Estimate V = π/4
0
π/40 x
2y 2 sin(x + y ) ln(x + y )dxdy .
If we use x = πu 1/4, y = πu 2/4 the integral limits are 0 and 1.Note that f (u 1, u 2) is monotonic in both variables, for u 1 ∈ [0, 1]and u 2 ∈ [0, 1] .
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
. . . continuation
http://find/
-
8/18/2019 Técnicas de reducción de varianza
23/49
. . . continuation
Not using antithetic variables
g
-
8/18/2019 Técnicas de reducción de varianza
24/49
Using antithetic variables
g
-
8/18/2019 Técnicas de reducción de varianza
25/49
Comparing results.
0 200 400 600 800 1000
0.000
0.002
0.004
0.006
0.008
0.010
Iterations
M e a n e v o l u t i o n
Without a.v.
With a.v.
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
Exercise from Rizzo (2008)
http://find/http://find/
-
8/18/2019 Técnicas de reducción de varianza
26/49
( )
Use Monte Carlo integration with antithetic variables to estimate
10
e −x
1 + x 2dx ,
and find the approximate reduction in variance as a percentage of the variance without variance reduction.
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
2. Control variates
http://find/
-
8/18/2019 Técnicas de reducción de varianza
27/49
Suppose that we want to use simulation to estimate Θ = E [X ],where X is the output of a simulation. Now suppose that for someother output variable Y, the expected value of Y is known-say,E [Y ] = µY .
Then for any constant c , the quantity
X + c (Y − µY )
is also an unbiased estimator of Θ.
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
. . . continuation
http://find/http://goback/
-
8/18/2019 Técnicas de reducción de varianza
28/49
To determine the best value of c , note that
Simple calculus now shows that the above is minimized whenc = c , where
c = −Cov (X ,Y )Var (Y )
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
. . . continuation
http://find/
-
8/18/2019 Técnicas de reducción de varianza
29/49
and for this value the variance of the estimator is
The quantity Y is called a control variate for the simulationestimator X .
The quantity X + c (Y − µY ) is called the controlledestimator.
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
. . . continuation
http://find/
-
8/18/2019 Técnicas de reducción de varianza
30/49
Upon dividing the last Equation by Var (X ), we obtain that
is the correlation between X and Y . Hence, the variance reductionobtained in using the control variate Y is 100 Corr 2(X ,Y ) percent.
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
. . . continuation
http://find/http://goback/
-
8/18/2019 Técnicas de reducción de varianza
31/49
The quantities Cov (X ,Y ) and Var (Y ) are usually not known inadvance and must be estimated from the simulated data. If nsimulation runs are performed, and the output data X i ,Y i , withi = 1, . . . , n, result, then using the estimators
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
. . . continuation
http://find/
-
8/18/2019 Técnicas de reducción de varianza
32/49
we can approximate c by ĉ , where
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
Example 9h
http://find/http://goback/
-
8/18/2019 Técnicas de reducción de varianza
33/49
As in Example 9d, suppose we were interested in using simulation to
compute Θ = E [e U ]. Here, a natural variate to use as a control isthe random number U .
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
. . . continuation
http://find/
-
8/18/2019 Técnicas de reducción de varianza
34/49
Because Var (U ) = 1/12 it follows that
where the above used, from Example 9d, that Var (e U ) = 0.2420.Hence, in this case, the use of the control variate U can lead to avariance reduction of up to 98.4 percent.
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
. . . continuation
http://find/
-
8/18/2019 Técnicas de reducción de varianza
35/49
We want to estimate E [e U ] until its standard deviation d
-
8/18/2019 Técnicas de reducción de varianza
36/49
The results are
i
## [1] 2429
mean( x )
## [1] 1.731676
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
. . . continuation
http://goforward/http://find/http://goback/
-
8/18/2019 Técnicas de reducción de varianza
37/49
Using control variate.
U
-
8/18/2019 Técnicas de reducción de varianza
38/49
The results are
i
## [1] 43
mean( Z )
## [1] 1.721574
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
. . . continuation
http://goforward/http://find/http://goback/
-
8/18/2019 Técnicas de reducción de varianza
39/49
Resuming
0 500 1000 1500 2000 2500
1.5
1.6
1.7
1.8
1.9
2.0
Without control variate
Iterations
x M e a n
0 10 20 30 40
1.5
1.6
1.7
1.8
1.9
2.0
With control variate
Iterations
Z M e a n
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
3. Variance reduction by Conditioning
http://goforward/http://find/http://goback/
-
8/18/2019 Técnicas de reducción de varianza
40/49
Suppose we are interested in performing a simulation study so as to
ascertain the value of Θ = E [X ], where X is an output variable of asimulation run. Also, suppose there is a second variable Y , suchthat E [X
|Y ] is known and takes on a value that can be determined
from the simulation run. Since
E [E [X | Y ]] = E [X ] = Θ
it follows that E [X
|Y ] is also an unbiased estimator of Θ.
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
. . . continuation
http://find/
-
8/18/2019 Técnicas de reducción de varianza
41/49
Recall the conditional variance formula proved in Section 2.10 of Chapter 2.
Var (X ) = E [Var (X | Y )] + Var (E [X | Y ])
Since both terms on the right are nonnegative,
Var (X ) ≥ Var (E [X | Y ])
it follows that as an estimator of Θ, E [X | Y ] is superior to the(raw) estimator X .
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
Example 9k
http://goforward/http://find/http://goback/
-
8/18/2019 Técnicas de reducción de varianza
42/49
Show how we can estimate π by determining how often a randomlychosen point in the square of area 4 centered around the origin falls
within the inscribed circle of radius 1. Specifically, if we letV i = 2U i − 1, where U i , i = 1, 2, are random numbers, and set
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
. . . continuation
http://goforward/http://find/http://goback/
-
8/18/2019 Técnicas de reducción de varianza
43/49
The estimators I
and (1 −V 2
1 )
12
have mean π/4.
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
. . . continuation
http://goforward/http://find/http://goback/
-
8/18/2019 Técnicas de reducción de varianza
44/49
Since
it follows that V 21 and U 2 have the same distribution, so we cansimplify by using the estimator (1 − U 2) 12 where U ∼ U (0, 1).What are the variances of I and (1 − U 2) 12 ?
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
. . . continuation
http://find/
-
8/18/2019 Técnicas de reducción de varianza
45/49
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
. . . continuation
http://find/
-
8/18/2019 Técnicas de reducción de varianza
46/49
We can illustrate the example in R:
n
-
8/18/2019 Técnicas de reducción de varianza
47/49
0 100 200 300 400 500
0.5
0.6
0.7
0.8
0.9
1.0
1.1
1.2
Mean evolution
Iterations
M e a
n
True value
Without conditioningWith conditioning
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
. . . continuation
http://find/http://find/http://goback/
-
8/18/2019 Técnicas de reducción de varianza
48/49
We can obtain the empirical variances of I and (1 − U 2
)
12
.
var(I)
## [1] 0.1638918
var(Z)
## [1] 0.04519196
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
Homework
http://find/
-
8/18/2019 Técnicas de reducción de varianza
49/49
Do exercises.
Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques
http://find/