Extra Problems for Chapter 10

T.M. Cover and J.A. Thomas

1. Gaussian noise with time varying mean. Consider a additive noise channel , where the input to the channel is constrained to have power P. The noise , i.e., it has a Gaussian distribution with mean and variance N. What is the capacity of the channel if
   1. for all i?
   2. for all i and is known to both sender and receiver?
   3. , i.e., has a normal distribution with mean 0 and variance ?
2. Multipath Gaussian channel. Consider a Gaussian noise channel of power contraint P, where the signal takes two different paths and the received noisy signals are added together at the antenna.
   
   1. Find the capacity of this channel if and are jointly normal with covariance matrix .
   2. What is the capacity for , , ?
3. Parallel Gaussian channels

   Consider the following parallel Gaussian channel

   

   where and are independent Gaussian random variables and . We wish to allocate power to the two parallel channels. Let and be fixed. Consider a total cost constraint where is the power allocated to the channel and is the cost per unit power in that channel. Thus and can be chosen subject to the cost constraint .

   1. For what value of does the channel stop acting like a single channel and start acting like a pair of channels?
   2. Evaluate the capacity and find that achieve capacity for and .
4. Gaussian mutual information. Suppose that (X,Y,Z) are jointly Gaussian and that forms a Markov chain. Let X and Y have correlation coefficient and let Y and Z have correlation coefficient . Find I(X;Z).
5. Robust decoding. Consider an additive noise channel whose output Y is given by

   

   where the channel input X is average power limited,

   

   and the noise process is iid with marginal distribution (not necessarily Gaussian) of power N,

   

   1. Show that the channel capacity, , is lower bounded by where

      

      i.e., the capacity corresponding to white Gaussian noise.

   2. Decoding the received vector to the codeword that is closest to it in Euclidean distance is in general sub-optimal, if the noise is non-Gaussian. Show, however, that the rate is achievable even if one insists on performing nearest neighbor decoding (minimum Euclidean distance decoding) rather than the optimal maximum-likelihood or joint typicality decoding (with respect to the true noise distribution).
   3. Extend the result to the case where the noise is not iid but is stationary and ergodic with power N.

   Hint for b and c: Consider a size random codebook whose codewords are drawn independently of each other according to a uniform distribution over the n dimensional sphere of radius .

   • Using a symmetry argument show that, conditioned on the noise vector, the ensemble average probability of error depends on the noise vector only via its Euclidean norm .
   • Use a geometric argument to show that this dependence is monotonic.
   • Given a rate choose some N' > N such that

     

     Compare the case where the noise is iid to the case at hand.

   • Conclude the proof using the fact that the above ensemble of codebooks can achieve the capacity of the Gaussian channel (no need to prove that).
6. Time varying channel. A train pulls out of the station at constant velocity. The received signal energy thus falls off with time as . The total received signal at time i is

   

   where are i.i.d. . The transmitter constraint for block length n is

   

   Using Fano's inequality, show that the capacity C is equal to zero for this channel.

7. Feedback capacity for n=2. Let Find the maximum of with and without feedback given a trace (power) constraint
8. Additive noise channel. Consider the channel Y=X+Z, where X is the transmitted signal with power constraint P, Z is independent additive noise, and Y is the received signal. Let

   

   where Thus Z has a mixture distribution which is the mixture of a Gaussian distribution and a degenerate distribution with mass 1 at 0.

   1. What is the capacity of this channel?
   2. How would you signal in such a manner as to achieve capacity?
9. Discrete input continuous output channel. Let , , and let Y=X+Z, where Z is uniform over the interval [0,a], a>1, and Z is independent of X.
   1. Calculate

      

   2. Now calculate I(X;Y) the other way by

      

   3. Calculate the capacity of this channel by maximizing over p
10. The capacity of photographic film. Here is a problem with a nice answer that takes a little time. We're interested in the capacity of photographic film. The film consists of silver iodide crystals, Poisson distributed, with a density of particles per square inch. The film is illuminated without knowledge of the position of the silver iodide particles. It is then developed and the receiver sees only the silver iodide particles that have been illuminated. It is assumed that light incident on a cell exposes the grain if it is there and otherwise results in a blank response. Silver iodide particles that are not illuminated and vacant portions of the film remain blank. The question is, ``What is the capacity of this film?''

    We make the following assumptions. We grid the film very finely into cells of area dA. It is assumed that there is at most one silver iodide particle per cell and that no silver iodide particle is intersected by the cell boundaries. Thus, the film can be considered to be a large number of parallel binary asymmetric channels with crossover probability .

    By calculating the capacity of this binary asymmetric channel to first order in dA (making the necessary approximations) one can calculate the capacity of the film in bits per square inch. It is, of course, proportional to . The question is what is the multiplicative constant?

    The answer would be bits per unit area if both illuminator and receiver knew the positions of the crystals.

11. Output power constraint. Consider an additive white Gaussian noise channel with an expected output power constraint P. Thus Y=X+Z, , Z is independent of X, and . Find the channel capacity.
12. Exponential noise channels. Consider an additive noise channel , where is i.i.d. exponentially distributed noise with mean . Assume that we have a mean constraint on the signal, i.e., . Show that the capacity of such a channel is .
13. Fading channel.
    Consider an additive noise fading channel

    

    

    where Z is additive noise, V is a random variable representing fading, and Z and V are independent of each other and of X. Argue that knowledge of the fading factor V improves capacity by showing

    

Latex file

Postscript file