# PhD Statistics Homework HW #1

Problem 1. Suppose that X is a normal random variable with variance 1 and unknown mean θ. It is

desired to guess the valu

PhD Statistics Homework HW #1

Problem 1. Suppose that X is a normal random variable with variance 1 and unknown mean θ. It is

desired to guess the value of unknown mean θ. Since the experimenter feels the loss is roughly like

square error (d − θ)2 when the true θ is small but is like squared relative error (θ−1d − 1)2 when |θ| is

large, he or she chooses loss function (θ − d)2/(1 + θ2) to reflect this behavior.

(a) Specify S, Ω, D, and L (i.e., the sample space, the set of all possible distribution functions, the

decision space, and the loss function).

(b) Determine and plot on the same graph the risk function of the 6 procedures δi defined by

δ1(X) = X; δ2(X) = (1 + X)/2; δ3(X) = X/2;

δ4(X) = 2X; δ5(X) = 0; δ6(X) = 1;

[You can save time by working (e) first but may find it easier to work (b) first. Your calculation

will be made simpler if you first compute the risk function of a general procedure of the form

δ(X) = a + bX. A check: Rδ4 (θ) = (θ

2 + 4)/(1 + θ2).]

(c) From these calculations, can you assert that any of these six procedures is inadmissible?

(d) On the basis of the risk functions, if one of these 6 procedures must be used, which procedure

would you use, and why? (Note: Don’t consult any references in answering this. Later you

will find out the precise meaning of your present intuition.)

(e) Suppose X is replaced by the vector (X1, . . . ,Xn) of iid normal N(θ, 1) random variables. The

procedures corresponding to δ1,δ2,δ3,δ6 are

δ1,n(X1, . . . ,Xn) = X̄n; δ2,n(X1, . . . ,Xn) =

X̄n + n

−1

1 + n−1

;

δ3,n(X1, . . . ,Xn) =

√

n X̄n

1 +

√

n

; δ6,n(X1, . . . ,Xn) = 1.

Compute the risk functions of these four procedures, and plot graphs of these four risk func-

tions (or, rather, of nRδi,n to make the results comparable to those of part (b)) for n large

(e.g., for n = 10, 000). [Use the fact that X̄n is N(θ,n

−1) distributed. Again, you may find

it is easier first to find (1 + θ2)−1Eθ(a + bX̄n −θ)2 for general a,b.]

(f) If n is large, which of the four procedures of part (e) would you use, and why?

(Your answer to this last may differ from the answer to part (d) for the case n = 1; does it?)

(g) Suppose the statistician decides to restrict consideration to procedures δa,b,n = a + bX̄n of

the form mentioned at the end of (e). He or she is concerned about the behavior of the risk

function when |θ| is large. Show that the risk function approaches 0 as |θ|→∞ if and only if

1

b = 1. In addition, among procedures with b = 1, show that the choice a = 0 gives uniformly

smallest risk function.

[This justification of the procedure δ1,n = X̄n under the restriction to procedures of the form

δa,b,n will seem more sensible to many people than a justification in terms of the “unbiased-

ness” criterion to be discussed later].

(h) Show that the procedure δ6,n, defined by δ6,n(X1, . . . ,Xn) ≡ 1, is admissible for each n. [Hints:

how can another procedure δ′ satisfy Rδ′ (θ) ≤ Rδ6,n (θ) when θ = 1? ]

Problem 2. Assume that we observe a binomial random variable X with parameter (n,θ), i.e.,

the probability mass function of X is given by P(X = i) =

(

n

i

)

θi(1−θ)n−i for i = 0, 1, . . . ,n, where

n ≥ 1 is a known integer and 0 ≤ θ ≤ 1 is unknown. Consider the problem of estimating θ under

the so-called “absolute deviation” loss function defined by L(θ,d) = |θ −d|.

(a) Specify S, Ω,D, and L (i.e., the sample space, the set of all possible distribution functions, the

decision space, and the loss function).

(b) When n = 20, graph and compare the risk functions of the following three procedures:

δ1(X) =

X

n

, δ2(X) =

1

7

, and δ3(X) = 1.

Note that the risk functions may not have simple expressions, and it will be OK to use some

computer software to plot the risk functions.

(c) Show that for any given integer n ≥ 1, the procedure δ2(X) = 17 is admissible. [Hints: how

can another procedure δ′ satisfy Rδ′ (θ) ≤ Rδ2 (θ) when θ =

1

7

? ]

(d) Show that when n = 2, the procedure δ3(X) = 1 is admissible.

Remarks: The proof of (d) is much more challenging than that in (c), since 1

7

is inside the domain Ω

but 1 is on the boundary! If your proofs did not distinct such difference, you likely made a mistake.

Also parts (c) and (d) suggest that an admissible estimator may not be appealing. Of course,

it is clear that inadmissible estimators are definitely not desirable.

In Part (b), the following R code can be used to plot the risk functions. Please make sure you

understand these R codes. For more information about the free statistical software R, please see

the website

n= 20;

theta <- seq(0,1,0.0001); R1 <- 0; for (i in 0:n){ R1 <- R1+choose(n,i)*(theta^i)*((1-theta)^(n-i))*abs(i/n - theta); } R2 <- abs(1/7 - theta); R3 <- abs(1 - theta); plot(theta, R1,"l", ylab="Risk Function", ylim=c(0,1)); lines(theta, R2, col="red"); lines(theta, R3, col="blue") 2