Cramer Rao Lower Bound (CRLB) for Vector Parameter Estimation

CRLB for scalar parameter estimation was discussed in previous posts. The same concept is extended to vector parameter estimation. Consider a set of deterministic parameters \(\mathbb{\theta}=[ \theta_1, \theta_2, ..., \theta_p]^{T} \) that we wish to estimate. The estimate is denoted in vector form as,\(\mathbb{\hat{\theta}} = [ \hat{\theta_1}, \hat{\theta_2}, ..., \hat{\theta_p} ]^{T} \). Assume that the estimate is unbiased \(E[\hat{\theta}] = \theta \). Covariance Matrix For the scalar parameter estimation, the variance of the estimate was considered. For vector parameter estimation, the covariance of the vector of estimates are considered. The covariance matrix for the vector of estimates is given by $$ C_{\hat{\theta}} =var ( \hat{\theta} )  = E \left[  ( \hat{\theta} - \theta )( \hat{\theta} - \theta )^T \right] $$ For example, if \(A,B\) and C are the unknown parameters to be estimated, then the  covariance matrix for the parameter vector \( \theta = [A,B,C]^T \) is given by $$  C_{\hat{\theta}} = \left[  \begin{matrix} var(\hat{A}) & cov(\hat{A},\hat{B}) & cov(\hat{A},\hat{C}) \\  cov(\hat{B},\hat{A}) & var(\hat{B}) & cov(\hat{B},\hat{C}) \\  cov(\hat{C},\hat{A}) & cov(\hat{C},\hat{B}) & var(\hat{C}) \end{matrix} \right]  $$ Fisher Information Matrix For the scalar parameter estimation, Fisher Information was considered. Same concept is extended for the vector case and is called the Fisher Information Matrix \(I(\theta)\). The ijth element of the Fisher Information Matrix \(I(\theta)\) (evaluated at the true values of the parameter vector) is given by $$ [I(\theta)] _{ij} = -E \left[  \frac{\delta^2}{\delta \theta_i \delta \theta_j} ln p(x;\theta) \right]  \; \; i,j =1,2,3,…,p$$ CRLB Matrix Under the same regularity condition (as that of the scalar parameter estimation case), $$ E […]