What parameter estimate is considered consistent, unbiased, effective?

1) Consistent assessment

Wealthy an estimate in mathematical statistics is a point estimate that converges in probability to the parameter being estimated.

Definitions

· Let-- sampling from a distribution depending on the parameter. Then the estimate is called consistent if

according to probability at.

Otherwise, the assessment is called invalid.

· An estimate is said to be strongly consistent if

almost probably at.

Properties

· From the properties of convergences random variables we have that a strongly consistent estimate is always consistent. The converse, generally speaking, is not true.

• · The sample mean is a consistent estimate of the mathematical expectation X i .
• · The periodogram is an unbiased but inconsistent estimate of spectral density.
• 2) Unbiased estimate

An unbiased estimate in mathematical statistics is a point estimate whose mathematical expectation is equal to the estimated parameter.

Definition

Let be a sample from a distribution depending on the parameter. Then the estimate is called unbiased if

Otherwise, the estimate is called biased, and the random variable is called its bias.

Sample average

is an unbiased estimate of the mathematical expectation of X i, since if

· Let random variables X i have finite variance DX i = ? 2. Let's build estimates

Sample variance,

Corrected sample variance.

Then is the parameter estimate biased and S 2 unbiased? 2.

3) Effective assessment

Current version (not tested)

Definition

An estimate of a parameter is called an efficient estimate in a class if the inequality for any holds for any other estimate.

Unbiased estimates play a special role in mathematical statistics. If an unbiased estimator is an effective estimator in the class of unbiased ones, then such statistics are usually called simply effective.

An effective estimate in a class where is a certain function exists and is unique up to values ​​on the set, the probability of being in which is zero ().

An estimate of a parameter is called effective if for it the Cramer-Rao inequality becomes an equality. Thus, the inequality can be used to prove that the variance of a given estimate is the smallest possible, that is, that this estimate is in some sense better than all others.

In mathematical statistics, the Cramer-Ramo inequality (in honor of Harald Cramer and K.R. Rao) is an inequality that, under certain conditions on the statistical model, gives a lower bound for the variance of the estimate of an unknown parameter, expressing it in terms of Fisher information.

One of the main requirements when constructing estimates is to obtain estimates with minimum variance or minimum dispersion (if they exist). In this regard, the concept of effective estimates was introduced in mathematical statistics,

In relation to biased estimates of a signal parameter, the estimate is called effective if the average value of the squared deviation of the estimate from the true value of the estimated parameter I does not exceed the average value of the squared deviation of any other estimate y, i.e., the inequality is satisfied

For an unbiased estimator, the dispersion of the estimator is the same as its variance; therefore, the effective unbiased estimator is defined as the estimator with the minimum variance.

S. Rao and Cramer independently obtained expressions for the lower bounds of the conditional variances and dispersion of estimates, which are the dispersion and dispersion of effective estimates, provided that they exist for the given parameters.

Let us present the derivation of this expression, assuming that the necessary assumptions are valid.

We present the estimate of parameter y in abbreviated form where X is a multidimensional sample from the implementation over a time interval

Let's average the expression

for all possible values ​​of a multidimensional sample X, which is described by a conditional probability density. Taking into account the known relation for the derivative of the natural logarithm after averaging, we obtain

Due to the normalization property of the probability density, the last term in (1.3.3) is equal to zero. The integral of the first term represents the average value of the estimate

Taking the latter into account, the averaged value can be written in the form

The left side of this expression is the average of the product of two random variables with finite values ​​of the first two moments. Under these conditions, the Bunyakovsky-Schwartz inequality, known from mathematical statistics, is valid for random variables

which turns into equality if the random variables are related by a deterministic dependence. Taking (1.3.6) into account, from expression (1.3.5) we can obtain

For unbiased and constant-biased estimators, the estimator variance satisfies the Rao-Kramer inequality

It should be noted that in all relationships, averaging is performed over a multidimensional sample of observed data X (with continuous processing - over all possible implementations of a

derivatives are taken at the point of the true value of the estimated parameter.

The equal sign in expressions (1.3.7) and (1-3.8) is achieved only for effective estimates.

In relation to expression (1.3.7), we consider the conditions under which the inequality turns into equality, i.e., the parameter estimate is an effective biased estimate. According to (1.3.6), this requires that the cross-correlation coefficient between be equal to one, i.e. so that these random functions are related by a deterministic linear relationship.

Indeed, let us represent the derivative of the logarithm of the likelihood function in the form

where is a function that does not depend on the estimate of y and the sample of observed data, but may depend on the estimated parameter. When substituting (1.3.5) and (1.3.9) into inequality (1.3.7), it turns into equality. However, representation of the derivative of the logarithm of the likelihood function in the form (1.3.9) is possible if the sufficiency condition (1.2.9) is satisfied to estimate y, from which it follows that

and therefore, if the derivative of the logarithm of the likelihood ratio depends linearly on the sufficient estimate, then the proportionality coefficient does not depend on the sample

Thus, for the existence of a biased effective estimate, two conditions must be met: the estimate must be sufficient (1.2.9) and relation (1.3.9) must be satisfied. Similar restrictions are imposed on the existence of effective unbiased estimates, under which in expression (1.3.8) the inequality sign turns into equality.

The expression obtained above for the lower bound of the dispersion of the biased estimate is also valid for the lower bound of the dispersion of the biased estimate, since i.e.

The last inequality turns into equality if, in addition to the condition of sufficiency of the estimate, the following relation is true:

where has the same meaning as in expression (1.3.9).

Formula (1.3.10) is derived similarly to (1.3.7), if in the original expression (1.3.2) instead of considering

From the nature of conditions (1.2.9) and (1.3.9) it is clear that effective estimates exist only in very specific cases. It should also be noted that an effective estimate necessarily belongs to the class of sufficient estimates, while a sufficient estimate is not necessarily effective.

Analysis of the expression for the variance of the effective mixed estimator 1.3.7) shows that there may be biased estimators that provide less estimator variance than unbiased ones. To do this, it is necessary that the derivative of the offset have a negative value and be close to unity in absolute value at the point of the true value of the parameter.

Since in most cases the average square of the resulting estimation error (dispersion) is of interest, it makes sense to talk about the average square of the estimation error, which for any estimate is bounded from below:

In this case, for effective estimates there is an equal sign.

It is easy to show that relations (1.3.10) and (1.3.12) coincide if conditions (1.3.11) and (1.3.9) are satisfied, respectively. Indeed, substituting the values ​​expressed through functions into the numerator and denominator (1.3.10) we obtain (1.3.12).

Using the properties of effective estimates discussed above, we will clarify their definition. We will call an estimate y effective if either conditions (1.2.9) and (1.3.11) are satisfied for it, or if it has a dispersion for a given bias

or scattering

or with zero bias this estimate has variance

Note that the characteristics of the effective estimate (1.3.13) - (1.3.15) can also be calculated for those parameters for which there is no effective estimate. In this case, values ​​(1.3.13) - (1.3.15) determine the lower limit (unattainable) for the corresponding assessment characteristics.

To compare real estimates with effective ones in mathematical statistics, the concept of relative efficiency of estimates has been introduced, representing the ratio of the mean square deviation of the effective estimate relative to the true value of the parameter to the mean square deviation of the real estimate relative to the true value of the parameter:

Here y is the real estimate, the effectiveness of which is equal to the effective estimate.

From the definition of the variance of the effective estimate (1.3.1) it is clear that the relative efficiency of the estimate varies within

In addition to the concept of effective estimates, there is the concept of asymptotically effective estimates. It is assumed that for a sufficiently long observation time or an unlimited increase in the signal-to-noise ratio, the limiting value of the relative efficiency of the real estimate is equal to unity. This means that with an asymptotically efficient estimate, the variance of the estimate for a given bias is determined by expression (1.3.13), and in the absence of a bias, by expression (1.3.15).

probabilities, which has the property that as the number of observations increases, the probability of deviations of the estimate from the estimated parameter by an amount exceeding a certain given number tends to zero. More precisely: let X 1 , X 2 ,......, X n - independent observational results, the distribution of which depends on the unknown parameter θ, and for each n function Tn = Tn(X 1,..., X n) is an estimate of θ constructed from the first n observations, then the sequence of estimates ( Tn) is called consistent if n→ ∞ for every arbitrary number ε > 0 and any admissible value of θ

(i.e. Tn converges to θ in probability). For example, any unbiased estimate (See Unbiased estimate) Tn parameter θ (or estimate with ETn→ 0), the dispersion of which tends to zero with increasing n, is S. o. parameter θ due to Chebyshev’s inequality

So, the sample mean

Consistency, which is a desirable characteristic of any statistical estimate, relates only to the asymptotic properties of the estimate and weakly characterizes the quality of the estimate for a finite sample size in practical problems. There are criteria that allow you to choose from among all possible S. o. some parameter is the one that has the necessary qualities. See Statistical Estimates.

The concept of S. o. was first proposed by the English mathematician R. Fisher (1922).

Lit.: Kramer G., Mathematical methods of statistics, trans. from English. M., 1975; Rao S. R., Linear statistical methods and their applications, trans. from English. M., 1968.

A. V. Prokhorov.

Great Soviet Encyclopedia. - M.: Soviet Encyclopedia. 1969-1978 .

See what “Substantial assessment” is in other dictionaries:

In mathematical statistics, this is a point estimate that converges in probability to the parameter being estimated. Contents 1 Definitions 2 Properties 3 ... Wikipedia

A shortened version of the term “consistent sequence of evaluations ... Mathematical Encyclopedia

- ... Wikipedia

A function of random variables used to estimate unknown theoretical parameters. probability distributions. Methods of the theory of O. s. serve as the basis for modern error theory; usually the unknown parameters are measured physical... ... Mathematical Encyclopedia

ASSESSMENT CONSISTENT- STATISTICAL ASSESSMENT… Sociology: Encyclopedia

A concept that extends the idea of ​​efficient estimation to large samples. An unambiguous definition of A. e. O. does not have. For example, in the classic option we are talking about asymptotic. the effectiveness of the assessment in a suitably selected class of assessments. Exactly,… … Mathematical Encyclopedia

Supereffective estimate, a generally accepted abbreviation of the term supereffective (supereffective) sequence of estimates, used in relation to a consistent sequence of asymptotically normal estimates of an unknown parameter, to paradise ... Mathematical Encyclopedia

- (probit model, English probit) used in various fields (econometrics, toxicology, etc.) statistical (nonlinear) model and method of analyzing the dependence of qualitative (primarily binary) variables on a set ... ... Wikipedia

The sample (empirical) mean is an approximation of the theoretical mean of the distribution based on a sample from it. Definition Let a sample from a probability distribution defined on some probability space... ... Wikipedia

Statistical estimates are statistics that are used to estimate unknown parameters of distributions of a random variable. For example, if these are independent random variables with a given normal distribution, then there will be... ... Wikipedia

Books

• A simple, positive semidefinite estimator of the asymptotic covariance matrix, consistent in the presence of heteroscedasticity and autocorrelation, Whitney Newey. The work of Whitney K. Newey and Kenneth D. West is one of the most cited and widely known papers in economics due to its broad scope.…

• Let texvc not found; See math/README - help with setup.): X_1,\ldots, X_n,\ldots- sample for distribution depending on parameter Unable to parse expression (Executable file texvc not found; See math/README for setup help.): \theta \in \Theta. Then the estimate Unable to parse expression (Executable file texvc not found; See math/README for setup help.): \hat(\theta) \equiv \hat(\theta)(X_1,\ldots,X_n) is called wealthy if

Unable to parse expression (Executable file texvc by probability at Unable to parse expression (Executable file texvc .

Otherwise, the assessment is called invalid.

• Grade Unable to parse expression (Executable file texvc not found; See math/README for setup help.): \hat(\theta) called very wealthy, If

Unable to parse expression (Executable file texvc not found; See math/README for setup help.): \hat(\theta) \to \theta,\quad \forall \theta\in \Theta almost certainly at Unable to parse expression (Executable file texvc not found; See math/README - help with setup.): n \to \infty .

In practice, it is not possible to “see” convergence “almost certainly”, since the samples are finite. Thus, for applied statistics it is sufficient to require the consistency of the assessment. Moreover, assessments that would be reasonable, but not very wealthy, “in life” are very rare. The law of large numbers for identically distributed and independent quantities with a finite first moment is also fulfilled in a strengthened version; all extreme order statistics also converge due to monotonicity, not only in probability, but almost certainly.

Sign

• If the estimate converges to the true value of the parameter "in the mean square" or if the estimate is asymptotically unbiased and its variance tends to zero, then such an estimate will be consistent.

Properties

• From the properties of convergence of random variables we have that a strongly consistent estimate is always consistent. The converse, generally speaking, is not true.
• Since the dispersion of consistent estimates tends to zero, often at a rate of the order of 1/n, consistent estimates are compared with each other by the asymptotic dispersion of the random variable Unable to parse expression (Executable file texvc not found; See math/README for setup help.): \sqrt (n) (\hat(\theta)-\theta)(the asymptotic mathematical expectation of this value is zero).

Related concepts

• The score is called super wealthy, if the variance of the random variable Unable to parse expression (Executable file texvc not found; See math/README for setup help.): n (\hat(\theta)-\theta) tends to a finite value. That is, the rate of convergence of the estimate to the true value is significantly higher than that of a consistent estimate. For example, estimates of regression parameters of cointegrated time series turn out to be super consistent.

Examples

• Sample mean Unable to parse expression (Executable file texvc not found; See math/README for help with setting up.): \bar(X) = \frac(1)(n) \sum\limits_(i=1)^n X_i is a strongly consistent estimate of the mathematical expectation Unable to parse expression (Executable file texvc not found; See math/README - help with setup.): X_i .
• The periodogram is an unbiased but inconsistent estimate of spectral density.

see also

Write a review about the article "Sound Assessment"

Excerpt characterizing a meaningful assessment

Isidora’s sincere, deeply sad story deadened our children’s hearts with pain, without even giving time to wake up... It seemed that there was no limit to the inhuman torment inflicted by the callous souls of the ugly executioners on this amazing and courageous woman!.. I was sincerely afraid and anxious, only just thinking about what awaited us at the end of her amazing story!..
I looked at Stella - my warlike friend was frightenedly huddling close to Anna, not taking her shocked, widened eyes off Isidora... Apparently, even she - so brave and not giving up - was stunned by human cruelty.
Yes, Stella and I probably saw more than other children at 5-10 years old. We already knew what loss was, we knew what pain meant... But we still had to go through a lot to understand even a small part of what Isidora felt now!.. And I only hoped that I would never have to experience this to really experience...
I looked in fascination at this beautiful, brave, amazingly gifted woman, unable to hide the sorrowful tears welling up in my eyes... How did “people” dare to call themselves PEOPLE, doing this to her?! How did the Earth even tolerate such criminal abomination, allowing itself to be trampled on without opening up its depths?!
Isidora was still far away from us, in her deeply wounding memories, and I honestly didn’t want her to continue telling the story any further... Her story tormented my childhood soul, forcing me to die a hundred times from indignation and pain. I wasn't ready for this. I didn’t know how to protect myself from such atrocity... And it seemed that if this whole heart-tearing story didn’t stop right away, I would simply die without waiting for its end. It was too cruel and beyond my normal childhood understanding...
But Isidora, as if nothing had happened, continued to talk further, and we had no choice but to plunge with her again into her distorted, but so high and pure, unlived earthly LIFE...
I woke up very late the next morning. Apparently the peace that the North gave me with its touch warmed my tormented heart, allowing me to relax a little, so that I could meet the new day with my head held high, no matter what this day brings me... Anna still did not answer - apparently Karaffa firmly decided not to allow us to communicate until I break down, or until he has some great need for it.
Isolated from my sweet girl, but knowing that she was nearby, I tried to come up with different, wonderful ways to communicate with her, although in my heart I knew very well that I would not be able to find anything. Caraffa had his own reliable plan, which he was not going to change, in accordance with my desire. Rather, it’s the other way around - the more I wanted to see Anna, the longer he was going to keep her locked up, not allowing the meeting. Anna changed, becoming very confident and strong, which scared me a little, because, knowing her stubborn fatherly character, I could only imagine how far she could go in her stubbornness... I so wanted her to live!.. So that Caraffa’s executioner did not encroach on her fragile life, which had not even had time to fully bloom!.. So that my girl still has only the future...

Selected characteristics. Wealthy,

At the beginning of the course, concepts such as classical and statistical probability were considered.

If classical probability is a theoretical characteristic that can be determined without resorting to experience, then statistical probability can only be determined from the results of an experiment. With a larger number of experiments, the value W(A) can serve as an estimate for the probability P(A). Suffice it to recall the classic experiments of Buffon and Pearson. Similar analogies can be continued further. For example, for a theoretical characteristic M(x) such an analogy would be - average:

= i f i / n ,

for variance D(x) the empirical analogue would be statistical variance:

S 2 (x) = (x i - ) 2 f i/n .

Empirical characteristics, S 2 (x) ,W(A) are parameter estimates M(x) ,D(x) ,P(A) . In cases where empirical characteristics are determined on the basis of a large number of experiments, using them as theoretical parameters will not lead to significant errors in the study, but in cases where the number of experiments is limited, the error in replacement will be significant. Therefore, three requirements are imposed on empirical characteristics that are estimates of theoretical parameters:

estimates must be consistent, unbiased, and efficient.

An estimate is called consistent if the probability of its deviation from the estimated parameter by an amount less than an arbitrarily small positive number tends to unity with an unlimited increase in the number of observations n, those.

P(| - | < ) = 1

Where - some parameter of the general population,

/ - evaluation of this parameter. Most estimates of various numerical parameters meet these requirements. However, this requirement alone is not enough. It is necessary that they are also undisplaced.

An estimate is called unbiased if the mathematical expectation of this estimate is equal to the estimated parameter:

M ( / ) = .

An example of a consistent and unbiased estimate of systematic expectation is the arithmetic mean:

M() = .

An example of a consistent and biased estimate is

dispersion:

M ( S 2 (x)) = [ (n – 1)/ n ] D(x) .

Therefore, to obtain an unbiased estimate of the theoretical variance D(x) need empirical variance S 2 (x) multiply by n/(n – 1) , i.e.

S 2 (x) = (x i - ) 2 f i/n n/(n – 1) = (x i - ) 2 f i /(n – 1) .

In practice, this correction is made when calculating the variance estimate in cases where n< 30 .

There can be several valid unbiased estimates. For example, to estimate the center of dispersion of a normal distribution, along with the arithmetic mean, the median can be taken . The median is also an unbiased consistent estimate of the grouping center. Of two consistent unbiased estimates for the same parameter, it is natural to give preference to the one with less variance.

Such the estimate whose variance is the smallest relative to the parameter being estimated is called effective. For example, from two estimates of the center of dispersion of a normal distribution M(x) an effective assessment is , not , since the variance is less than the variance . The comparative effectiveness of these estimates with a large sample is approximately equal to: D() / D= 2/ = 0,6366.

In practice, this means that the center of the population distribution (let’s call it 0) is determined by with the same accuracy for n observations as for 0.6366 n observations using the arithmetic mean.

4.4. Properties of sample means and variances.

1. If the sample size is large enough, then based on the law of large numbers with a probability close to unity, it can be argued that the arithmetic mean and variance S 2 will differ as little as possible from M(x) And D(x ), i.e.

M(x) ,S 2 (x) D(x ), and variance D() , whatever the sample size n, as long as the number of samples is large enough.

4. When variance D(x ), the population is unknown, then for large values n With a greater probability of a small error, the dispersion of sample means can be calculated approximately by the equality:

D() = S 2 (x)/n,

Where S 2 (x) = (x i - ) 2 f i/n - variance of a large sample.

Return