EBC-Estimator of Multidimensional Bayesian Threshold in Case of Two Classes
- 10.2991/jsta.d.200824.001How to use a DOI?
- Estimator; Multidimensional Bayesian threshold; Mixture with varying concentrations
Some threshold-based classification rules in case of two classes are defined. In assumption, that a learning sample is obtained from a mixture with varying concentration, the empirical-Bayesian classification (EBC)-estimator of multidimensional Bayesian threshold is constructed. The conditions of convergence in probability of estimator are found.
- © 2020 The Authors. Published by Atlantis Press B.V.
- Open Access
- This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).
The model of a mixture of several probability distributions was mentioned for the first time by Newcomb  and Pearson . Such mixtures naturally arise in many areas. In particular, in the theory of reliability and time of life, mixtures of gamma distributions  are used. Examples of the use of mixtures of normal distributions in the processing of biological and physiological data are given in . In Slud , a mixture of two exponential distributions is used to describe the debugging process of the software. Some applications of the model of mixtures in medical diagnostics were given in [6,7].
The technique of a nonparametric analysis of mixtures where concentrations changes from observation to observation develops, actively. The problem of distributions estimating in case at known concentrations is considered in the works of Maiboroda [8,9]. Estimates of concentrations in two-component mixtures in work . Works by Sugakova  and Ivanko  are devoted to the evaluation of component distribution densities. The correction algorithms for weighted empirical distribution functions are proposed in .
For the theoretical study of problems of nonparametric regression the nonhomogeneous weighted empirical distribution functions used by Stoune . These were applied by Maiboroda in the tasks of analyzing the mixture. In particular, in Maiboroda  found conditions under which the weighted empirical distribution functions are unbiased and minimal estimators of unknown distribution functions of components of the mixture.
Object classification by its numerical characteristic is an important theoretical problem and has practical significance, for example, the definition of a person as “not healthy,” if the temperature of its body exceeds 37°C. To solve this problem we consider the threshold-based rule of type
According to this rule, an object is classified to the first class if its characteristic does not exceed a threshold 37°C; otherwise, an object is classified to belong to the second class. The empirical-Bayesian classification (EBC) [15,16] and minimization of the empirical risk (MER) [17,18] are widely used methods to estimate the best threshold. The case when the learning sample is obtained from a mixture with varying concentrations is considered in  and the asymptotic of both methods of estimating is investigated.
However, it is often necessary to classify an object in case of more than one threshold, for example, the definition of a person as “not healthy,” if the temperature of its body exceeds 37°C or lower than 36°C. Another example: the person is sick, if the level of its hemoglobin exceeds 84 units or lower than 72 units. Accordingly, one can apply the classifiers of typeor
The case of two thresholds and three prescribed classes deserves special attention. An example is the classification of the disease stages. Thus, during the diagnosis of breast cancer a tumor marker CA 15-3 is used. If the value is less than 22 IU/mL, then the person is healthy; if its level is in the range from 22 to 30 IU/mL—precancerous conditions can be diagnosed; if the index is above 30 IU/mL—patient has cancer. When solving some technical problems it is needed to consider the substance in its various aggregate forms: gaseous, liquid, solid. The transition from state to state occurs at a specific temperature. According to this, a boiling point and a melting point are used. Accordingly, 6 classifiers of formscan be applied. This partial case was studied in .
2. SETTING OF THE PROBLEM
The problem of classification of an object from the observation of its numerical characteristic is studied. We assume that the object may belong to one of two prescribed classes. An unknown class number containing is denoted by . A classification rule (briefly, classifier) is a function that assigns a value to by using characteristic . In general, classification rule is defined as a general measurable function, but we restrict the consideration in this paper to the so-called threshold-based classification rules of the formsif and if , where is the multidimensional threshold.
The a priori probabilities , are assumed to be known. The characteristic is assumed to be random, and its distribution depends on , . The distributions are unknown, but they have continuous densities with respect to the Lebesgue measure.
The family of classifiers is denoted by .
Let, then the probability of error of such a classification rules are given by
Analogically, for :
A classification rule is called a Bayesian classification rule in the class , if attains its minimum at . The threshold for a Bayesian classification rule is called the Bayesian threshold:
Denote Bayesian threshold for classifier :where
Denotewhere or .
When determining the best threshold, one faces the problem of estimating the threshold by using a learning sample, whose members are classified correctly. We consider the Bayesian empirical classification method, in assumption, that a learning sample is obtained from a mixture with varying concentration.
The distribution functions (and, of course, densities ) are assumed to be unknown. One can estimate these functions from the data being a sample from a mixture with varying concentration, where are independent if is fixed and
Here is a known concentration in the mixture of objects of the first class at the moment when an observation is made .
To estimate the distribution functions , we use weighted empirical distribution functions
One can apply kernel estimators to estimate the densities of distributions:where is a kernel (the density of some probability distribution) and is a smoothing parameter [11,24].
The empirical-Bayesian estimator is constructed as follows. At first, one determines the set of all solutions of the equation
Second, one choosesas an estimator for , where therefore where or , or where and where or .
3. MAIN RESULTS
3.1. Choice of Classifier
The choice of the classifier or depends on the smallest root of the equation
If root of (4) minimizes , then the classifier is selected, but if it minimizes , then it is selected .
The statement follows from the properties and .
The next theorem can be proved analogically to Theorem 3.1.1.
If root of (3) minimizes , then the is selected, but if it minimizes , then it is selected .
The statement follows from the properties and .
3.2. The Convergence in Probability of EBC-Estimator
In what follows we assume that
(A). The threshold exists, is a unique of the global minimum of or ( is a global minimum point of or ).
(Bk). The limits , , exists and .
Let conditions (A) and (Bk) hold. Assume that densities and exist and are continuous, , , is a continuous function, and
Then , for , where and , .
According to Theorem 1 of , the assumptions of the theorem imply that in probability at every point . Therefore,in probability. For , let
Now we shall that
Since, is a point of minimum of , , and or (similarly, ), depending on the parity or the oddness of , is a continuous function, changes its sign in a neighborhood of the point . This means that there are and such thatand .
Thus, . Since is a continuous function, . Therefore (5) is proved.
Let assumptions of Lemma 3.2.1 for hold. Thenwhere for and for .
Fix , . Since () are continuous functions on , ,and condition (A) holds. Moreover such that (or ) for all for which .
Choose so thatfor all . Denote for and for .
Fix an arbitrary . Using the uniform convergence of to (or to ), we obtain the necessary statement for sufficiently large for according , .
Assume that conditions of Lemma 3.2.1 hold. Then (or ) in probability as , namely , (or , ) in probability as .
Since (5) for sufficiently large . If the event occurs, then there existssuch that for all given the event occurs (or for all ). Therefore, and given the event occurs for sufficiently large , , taking into account that
This completes the proof of the theorem, since , are arbitrary.
In this paper, we found the conditions of convergence in probability of the estimator for the Bayesian threshold constructed by the method of empirical-Bayesian classification for a sample from a mixture with variable concentrations.
CONFLICT OF INTEREST
The author has no conflicts of interest to declare.
I thank the reviewers whose insightful comments helped me to improve this paper.
Cite this article
TY - JOUR AU - Oksana Kubaychuk PY - 2020 DA - 2020/09/08 TI - EBC-Estimator of Multidimensional Bayesian Threshold in Case of Two Classes JO - Journal of Statistical Theory and Applications SP - 342 EP - 351 VL - 19 IS - 3 SN - 2214-1766 UR - https://doi.org/10.2991/jsta.d.200824.001 DO - 10.2991/jsta.d.200824.001 ID - Kubaychuk2020 ER -