Shannon information; mean residual life; mean reversed life
Abstract
The weighted entropy measure is a germane dynamic measure of uncertainty in reliability and survival studies. In this paper, the new results of weighted entropies with some characterizations are provided. Furthermore, we have presented some results for weighted entropy residual and weighted past residual of order statistics with some application of some reliability systems such as a series structure and a parallel structure. In addition, we introduced the lower bound for the weighted residual (past) entropy. Moreover, the stochastic orders based on weighted entropy are presented. Finally, we illustrate the usefulness of the proposed non-parametric estimators of weighted entropy by application to real data.
The weighted distributions have been utilized in many applications such as distributions theory, reliability, probability, ecology, bio-statistics and applied.
Consider the distribution function G. for a random variable Y≥0 with density function g.. Suppose
lY=infy∈ℝ1:Gy>0,uY=supy∈ℝ1:Gy<1,SY=lY,uY and w.∈R+ be a weighted function. The weighted random variable YW, having probability density function as:
gw(y)=w(y)g(y)/E[w(y)],−∞≤y≤∞,
where Ewy∈ℝ+. Let Y represents the life length of a “unit” in reliability studies, and life distribution, with survival function G¯Y, hazard rate function φ¯G.=gY./G¯Y., reversed hazard rate φG.=gY./GY., the geometric vitality function ϑY=ElnY|Y>0 and mean revered residual lifetime as
θ(t)=E[κ−Y|Y≤κ]=∫0κGY(u)duGY(κ),κ∈R+.
As reported by Ebrahimi and Pellery [1] and Asha and Rejeesh [2], the differential entropy HY demonstrate the expected uncertainty of gy. In addition, it measures how the distribution spreads over its domain, where there is an inverse relationship between the value of HY and concentration of the probability mass of Y.HY sometimes called a dynamic measure of uncertainty or Shannon information measure.
The differential entropy of random variable Y can be defined in the continuous case as follows:
Di Crescenzo and Longobardi [4] developed the following convex entropy measure:
HwY=−∫0∞vgYvlngYvdv,
or equivalently:
Hw(Y)=−∫0∞dy∫y∞gY(v)lngY(v)dv.
The uncertainty of the residual lifetime is discussed in Di Crescenzo and Longobardi [5], with the following measure:
HY,κ=1−Elnφ¯GY|Y>κ=−∫κ∞gYvG¯vlngYvG¯vdv,κ∈ℝ+,
where κ∈A=y∈ℝ+|G¯Yy>0. In addition, the past entropy has been widely researched. We can measure it as follows:
H¯GY,κ=1−ElnφGY|Y<κ=−∫0κgYvGvlngYvGvdv,κ∈ℝ+.
Di Crescenzo and Longobardi [4] defined the convex residual entropy as
HwY,κ=−∫κ∞ygYyG¯κlnygYyG¯κdy,κ∈ℝ+.
Furthermore, let Y1 and Y2 be two random variables with distribution functions G1. and G2., densities functions gY1. and gY2. and survival functions G¯1. and G¯2. respectively. Kullback and Leibler [6] introduced an information distance between two distributions G1 and G2 as follows:
IY1,Y2=∫0∞gY1ulngY1ugY2udu.
In addition, Ebrahimi and Kirmani [7] have demonstrated that the Kullback-Leibler discrimination information of Y1 and Y2 at time κ can be presented as
We can use Eq. (5) to distinguish between two residual lifetimes those have both survived up to time κ, where IRY1,Y2κ identifies with the relative entropy of Y1−κ|Y1〉κ and Y2−κ|Y2〉κ.
The purpose of this study is to develop and add more properties, characterizations, order statistics, some inequalities and stochastic orders of weighted differential entropies measures. In Section 2, definitions, notation, basic properties and characterizations are illustrated. The weighted entropy (residual and past residual) of order statistics with some application of reliability systems such as a series structure and a parallel structure are given in Section 3. In addition, we provided the lower bound for the weighted residual (past) entropy. The stochastic orders based on weighted entropy are developed in Section 4. Lastly, in Section 5, the suggested estimators of weighted entropy are presented. Furthermore, we illustrate the usefulness of the proposed non-parametric estimators of weighted entropy by application to real data.
Throughout this article, the term entropy is used instead of differential entropy and using abbreviation PL for past lifetime, WPE for weighted past residual entropy, WRE for weighted residual entropy, SS for the series system, PS for the parallel system, SE for small than or equal.
2. THE WEIGHTED DIFFERENTIAL ENTROPY
The weighted differential entropy WDE defined by Das [8] for random variable Ywith weighted functionwx=x as:
whenever the integral∫ℒuθfYu/EYθ1∨|lnuθfYu/EYθ|du<∞.
As a general case, we can be defined the generalized the WDE as the following definition:
Definition 2.1.
Given a functiony∈ℒ↦wy≥0,and anRV Y:ℱ→ℒ, with a probability density functiongY., survival function G¯. and mean EY. Therefore, the weighted differential entropy with weighted functionwy=yθis defined as
If fXx is the actual density function of random variable X and gYx is the density function determined by the researcher. Therefore, the weighted inaccuracy measure can be defined as
Rw(X,Y)=−∫ℒxfX(x)lngY(x)dx.
Next, we define the relative WDE of two densities.
Definition 2.2.
Let XandYbe two random variables with density function,s∈ℒ↦fXws≥0ands∈ℒ↦gYs≥0, and mean valuesEX. and EY., respectively. Therefore, the relative weighted differential entropy ofgYxrelative tofwxcan be defined as
ℝXw∥Y=ElnsfXsEXsgYs,=∫ℒvfXvEXXlnvfXvEXvgYvdv.
By using Eq. (4), we can define an alternative formulas of ℝXw∥Y as follows
where B.,. is beta function and Ψ. is psi function.
The following theorem states that this relationship actually characterizes the beta distribution.
Characterization Theorem 2.1:
Any random variableYwith distribution functionK, density functionfYx, meanEY, modeΓKY, geometric meanGY,entropy functionHYand weighted differential entropyξwYsatisfying the following relationship:
ξwY=HY−α−1ΓKY+EYΓKY+lnEY−lnGY+EY−1/α,
is either degenerate orYhas a beta distribution. Indeed, the degenerate case should be subsumed in the beta distribution withα,β∈R+.
Proof.
By using the following integral formula which is taken from Gradshteyn and Ryzhik ([10], formula 4.253(1), pp. 538):
∫ℒyθ−11−ycλ−1lnydy=1c2Bθc,λΨθc−Ψθc+λ,
provided that Reθ>0, Reλ>0, c>0. We denote the beta function with the symbol B(·,·) and the digamma function with Ψ(·). Therefore, we have
By using Example 2.3 in Di Crescenzo and Longobard ([4], pp.682), Eq. (4) and the recurrence relation of the digamma function, we can rewrite HwY as follows:
In next results we study the closure transformation property of the weighted entropy. We can now proceed analogously to Di Crescenzo and Longobardi [4] and introduce the following theorem.
Theorem 2.2.
SupposeUis RV with density functionfU.andψUis strongly convex, strictly increasing, continuous and differentiable function with derivativedduψu. Then
where Φ. is Error function, ϖμ,σ=μexpσ2/2−1 and Dxy=2x/2exp−y2/4F−x2,12,y22 is Parabolic cylinder function. We denote a confluent hypergeometric function of the first kind with the symbol F.,.,.. Further,
ξwU=ϖμ,σαμ,σ−lnϖμ,σ−ϖμ,σexp2lnμ+σ21−Φ−21/2σ,
where αμ,σ=ln2πσμexpσ2/2+exp2lnμ+σ21−Φ−21/2σ+1σ2−3/2+32μ2πexpσ24D−3−σ.
Furthermore,
ℝXw∥X=lnexp−β1μ,σ+2β2μ,σϖμ,σ/ϖμ,σ,
where:
β1μ,σ=exp2lnμ+σ21−Φ−21/2σ;
β2μ,σ=ln2πσEX+ϑwX+1σ2−3/2+32μ2πexpσ24D−3−σ
Example 2.2:
Let X be random variable having Chi distribution with density function
fXx=2π/2π/2γπΓπ/2xπ−1exp−x2π/2γ2,γ,x>0,π is a positive integer, with parameter μ,γ>0. From Eq. (7) we have
ϑwX=2π/2π/2γπΓπ/2∫0∞xπexp−x2π/2γ2lnxdx,
take u=x2π/2γ2, then we get
ϑwX=γ1/2π1/2Γπ/2∫0∞uπ2−0.5exp−ulnu/π/2γ21/2du.
By using Gradshteyn and Ryzhik ([10], 4.352(1)) for evaluate the below formula, we have the following result:
Since EU=0, we get that ξ1wU and ℝ1wUwU can not be found.
3. CONNECTION TO RELIABILITY THEORY
Suppose U1,U2,…,Un be i.i.d. lifetimes with probability density function g., distribution K., survival function K¯. and reversed hazard rate φK.. Therefore, the probability that any two (or more) observation in random sample take the same magnitude (the same value is equal to zero). Therefore, there exists a unique ordered arrangement of the sample observation according to magnitude. Let 0≤U1≤U2≤…≤Un<∞ be the corresponding order statistics. Therefore, Ur defines the lifetime of an (n−r+1)out of n system. Write gr.,Kr., φr. and φ¯r. as the distribution function, the probability density function, the RHR function of Ur and the hazard rate of Ur respectively. Then we have
It is to be noted that U1 represents an age of the series system. By using Eq. (13), with simple calculation, we have the weighted the residual entropy of U1 as
It follows from Proposition 1 in Bairamov et al. [11], and definition of mean residual lifetime of n−k+1-out-of-n system in Asadi and Bayramoglu [12] that
EU1|U1>κ=∫κ∞K¯1vdvK¯1κ=M1κsay,
and
K¯κ=M10M1κexp−∫0κM1−1κ1/n,
Then, Eq. (15) can be written in the following form
It is to be noted that Un refers to the lifetime of PS with survival function K¯n.=1−Kn.. Based on Eq. (13), we can define the weighted the residual entropy of Un as
In addition, we know that lnBnκ+κ1−Knκ≤0. Hence, we have
ξ1wUn,κ≤lnBnκ+κ1−Knκ.
In next result, we derive the lower bound for WPE of Un.
Proposition 3.1:
SupposeU≥0be a random variable with distribution functionKv. Then
ξ2wUn,κ≥−n2Eu2K2n−2ugu|U≤κκ−θnκ2Kn−1κ.
Proof.
Using Eq. (2), inequality −lny≥1−y, for y≥0, and since
∫0κvKn−1vgvdv≤∫0κnv2K2n−2vg2vdv.
The result follows.
4. STOCHASTIC ORDERS BASED ON WEIGHTED ENTROPY
In this section, we explore the possibility of application of stochastic orders.
Definition 4.1.
AssumeU1≥0andU2≥0be two random variables with density functionsg1andg2, distribution functionsGU1andGU2, reliability functionsG¯U1=1−GU1andG¯U2=1−GU2, the weighted entropy functionsξg1w.andξg2w., the convex residual entropy functions Hg1wU1,t and Hg2wU2,t and the weighted inaccuracy measures ℝU1w∥U1 and ℝU2w∥U2, respectively. We say that U1 is SE to U2 in the:
weighted entropy ordering (U1≤ξwU2) if ξg1wx≤ξg2wx, for all x≥0.
weighted inaccuracy ordering (U1≤ℝwU2) if ℝU1w∥U1≤ℝU2w∥U2.
convex residual entropy ordering (U1≤cwU2) if Hg1wU1,t≤Hg2wU2,t.
less uncertainty ordering (U1≤UU2) if HU1≤HU2.
Definition 4.2.
Let U1 and U2 be two random variables, then U1 is said to be SE to U2 in the convex order (U1≤cxU2). If
EϕU1≤EϕU2,
This is true for any convex functions ϕ.
Definition 4.3.
The random variable U1 is said to be increasing hazard rate, IHR, if, and only if,
G¯U1u+v/G¯U1u is decreasing in u≥0, for all v≥0.
Next result discusses the closure under increasing linear transformation of ≤ξw:
Theorem 4.1.
Suppose U1 and U2 are to be two random variables, let we define new functions as
V1=α1U1β1 and V2=α2U2β2, for all α1,α2∈ℝ+ and β1,β2∈ℝ+.
Let (i) U1≤ξwU2, (ii) α1≤α2, (iii) β1≤β2. Then V1≤ξwV2 if U1≤cxU2.
Proof.
Due to fact that φx=xlnx is convex function, and when U1≤cxU2 we get that EU1=EU2, if we suppose that ξU1wu is decreasing in u, and let U1≤ξwU2, α1≤α2 and β1≤β2. By apply Eq. (7) and Proposition 2.3, we have V1≤ξwV2.
Corollary 4.1.
Suppose the relationship between two random variables U1 and U2 as follows:
U1≤ξwU2.
Define V1=αU1β and V2=αU2β, α,β∈ℝ+. Then U1≤ξwU2 if U1≤cxU2.
In next theorem, we explain preservation properties and application of ≤ξw,≤ℝw and ≤U between two exponential RV's if their scale parameters are ordered.
Theorem 4.2.
Let two absolutely continuous random variables U1 and U2 with density function
fix=αiexp−αix,αi,x>0 and i=1,2.
If α1≥α2, then U1≤ξwU2.
If α1≤α2, then U1≤ℝwU2.
If X≤ℝwY then HU2≤UHU1.
Proof.
The result is obtained immediately from Remark 2.1.
Many studies explain the properties of repairable systems such as minimal repair. If the system has the virtual age Un−1 immediately after the n−1th repair, the functioning system obtained has the nth failure-time Yn distributed as
PrYn≤y|Un−1=u=Gy+u−GuG¯u,
where Gy is the failure time distribution of a new system U0=0. Let nth repair cannot remove the damages incurred before the (n−1)th repair and αn be the degree of the nth repair, now the time between n−1th failure and nth failure reduce from Yn to αnYn. If αn=1 for all n≥1 then it agrees with a minimal repair model.
Suppose Vn=∑i=1nYin≧1 with V0=0 which represents the time elapsed since the system was put in operation, or the associated counting process ℕt=supn≧1:Vn−1≦t. Kijima [17] proved that ℕtor Vn0∞ is a non-homogeneous Poisson process when αn=1 for all n≥1. Ebrahimi and Pellerey [1] defined the following definition:
Definition 4.4.
A point process ℕt,t≥0 consisting of interarrival times Y1,Y2,… is increasing (decreasing) in the
convex residual entropy order if
HwBi,t≤≥HwBj,t,for all t∈R+.
weighted entropy order if
ξwBi,t≤≥ξwBj,t
and 1≤i≤j≤n, where fk is the conditional probability density function of Yk=Vk−Vk−1 for all k=1,2,…, given Vk−1=vk−1,…,V1=v1, and
Bk=st[Yk|Vk−1=vk−1,…,V1=v1].
From Definition 5.3, we can note that if a point process is increasing (decreasing) means the uncertainty of the distribution is increasing (decreasing), i.e., the process is deterioration (improving).
Lemma 4.1.
Let Bk be as defined in Definition 5.3. Then, for k=1,2,…,
The stochastic point process ℕt=supn≧1:Vn−1≦t consisting of the time of nth failure Vk−1, k=1,2,… generated by a minimal repair policy is increasing (decreasing) in
convex residual entropy order ifG. is IHR,
weighted entropy order ifξwU is increasing for all u≥0.
Proof.
Similarly to lemma 5.1, we have HwBk+1,t=HwU,t+vk. In addition, by Theorem 3.1 in Di Crescenzo and Longobardi [4] we conclude that
and by Theorem 2.1 in Ebrahimi and Pellery [1] we have
HwU,t+vk=t+vkHU,t+vk+1G¯t+vkIt+vk.
By Theorem 2.5 in Ebrahimi and Pellery [1] we conclude that HU,t+vn−1≤HU,t+vn. By using Eqs. (20–22), and when a continuous distribution G. is IHR. It is obvious that
G¯t+vk≤G¯t+vk−1,
and
It+vk≥It+vk−1.
Then, we get HwU,t+vk≥HwU,t+vk−1. This complete the proof.
5. ENTROPY ESTIMATION
In this section, we introduce four the non-parametric estimators of Eq. (6) by using the same idea in Vasicek [9], Van Es [18], Ebrahimi et al. [19] and Al-Omari [20].
Let Z1,Z2,…,Zn is a sequence of the random sample with the distribution G and let Z1,Z2,…,Zn be the corresponding order statistics. Besides the sample distribution function Gnz=n−1∑i=1n1Zk≤z,1≤k≤n. Then, the on-parametric estimators can be expressed as:
Weighted Vasicek Entropy (Vξδ,nwZ): We can estimate of Eq. (8) by replacing Gwt by the empirical distribution Gnwt, and using a difference operator in place of the differential operator. Thus, Vξm,nwZ estimator of Eq. (8) can be represented as follows
Vξδ,nwZ=n−1∑i=1nlnn∑k=1nwZk∑j=1iwZj2δZi+δ−Zi−δ,
where δ∈ℕ+ know as a window size, δ<n/2,Zs=Z1 if s<1 and Zs=Zn if s>n.
Weightedδ-spacings Entropy (SEξδ,nwZ): Estimates of weighted entropy based on sample δ-spacings which introduced by Van Es [18], we can provide VEξδ,nwZ estimator of Eq. (6) as
where ψ. is the digamma function and lnδ−ψδ corrected bias entropy estimator.
Weighted Small weights Entropy (WSξδ,nwZ): As assign smaller weights in Vasicek [9], we obtain
WSξδ,nwZ=n−1∑i=1nlnn∑k=1nZk∑j=1iZjαiδZi+δ−Zi−δ,
where
αk={δ+k−1δ,1≤k≤δ2,δ+1≤k≤n−δδ+n−kδ,n−δ+1≤k≤n.
Modified Small weights Entropy (MSξδ,nwZ): As assign smaller weights in Ibrahimi et al. [21] we get
MSξδ,nwZ=n−1∑i=1nlnn∑k=1nwZk∑j=1iwZjβiδZi+δ−Zi−δ,
where
βi={32,1≤i≤δ,2,δ+1≤i≤n−δ,12,n−δ+1≤i≤n..
Example 5.1:
Let Z be random variable having exponential distribution with density function fZx=αexp−αz,α,z>0 with parameter α>0. From Eq. (7) we obtain,
ϑwZ=1αψ2−lnα,
where ψ. is Euler's psi function. By Example (2.1, a) in Di Crescenzo and Longobardi [4] we get,
HwZ=2−lnαα and ξ1wZ=2−lnα−ψ2.
Therefore,
ℝZwZ=−41−lnα−ψ2=−4HZ−ψ2.
Suppose α∈0,100. HwZ,ξwZ and ℝZwZ with weighted function wz=z is evaluated in Table 1.
α
H^wZ
ξ^wZ
ℝ^wZwZ
0.25
13.5452
2.9635
-9.9680
0.5
5.3863
2.2704
−7.1954
1
2
1.5772
−4.4228
5
0.0781
−0.0322
2.0150
10
−0.0303
−0.7254
4.7876
Table 1
Measures of weighted entropies of exponential distribution.
Now, a real data is illustrated to investigate the performance of suggested estimators.
Data Set: The following data set which is taken from Smith and Naylor [22], it represents the strength of 1.5 cm glass fibers measured at the National Physical Laboratory, England.
Shanker et al. [23] show the exponential density function (Exp0.663647) provided a better fit for this data. We compute the exact value of the weight entropy measure by the real data and compare this measure with Vξδ,nwZ,SEξδ,nwZ,WSξδ,nwZ and MSξδ,nwZ which shows in Table 2.
δ
ξδ,nwZ
Vξδ,nwZ
SEξδ,nwZ
WSξδ,nwZ
MSξδ,nwZ
1
1.9872
1.210967749
1.244680937583
1.232972422
1.237538804
2
1.9872
1.200196229
1.231653171583
1.943017269
1.298445816
3
1.9872
1.245455104
1.462350850583
2.308545839
1.323498873
4
1.9872
1.279410563
1.650373401583
2.537544185
1.337842382
5
1.9872
1.282220574
1.820823663583
2.721711858
1.415075846
Table 2
Weighted entropy measure for exponential distribution (0.663647).
REFERENCES
1.N. Ebrahimi and F. Pellerey, J. App. Prob., Vol. 32, No. 1, 1995, pp. 202-211.
2.G. Asha and C.J. Rejeesh, Metron., Vol. 73, No. 1, 2015, pp. 119-134.
3.A.I. Khinchin, Mathematical Foundation of Information Theory, Dover Publications, New York, 1957.
4.Di Crescenzo and M. Longobardi, Sci. Math. Japn., Vol. 64, No. 2, 2006, pp. 679-690.
5.Di Crescenzo and M. Longobardi, J. Appl. Probab., Vol. 39, No. 2, 2002, pp. 434-440.
6.S. Kullback and R.A. Leibler, Ann. Math. Stat., Vol. 22, No. 1, 1951, pp. 79-86.
7.N. Ebrahimi and S.N.U.A. Kirmani, Ann. Inst. Stat. Math., Vol. 48, No. 2, 1996, pp. 257-265.
8.S. Das, Commun. Stat. Theory Methods., Vol. 46, No. 12, 2017, pp. 5707-5727.
9.O. Vasicek, J. R. Stat. Soc. Ser. B., Vol. 38, No. 1, 1976, pp. 54-59.
10.I.S. Gradshteyn and I.M. Ryzhik, Table of Integrals, Series and Products, fifth, Academic Press, New York, 1994.
11.I. Bairamov, M. Ahsanullah, and I. Akhundov, J. Stat. Theory Appl., Vol. 1, No. 2, 2002, pp. 119-132.
12.M. Asadi and I. Bayramoglu, IEEE. Trans. Reliab., Vol. 55, No. 2, 2006, pp. 314-318.
13.M. Tavangar and M. Asadi, Metrika., Vol. 72, No. 1, 2010, pp. 59-73.
14.M. Asadi and I. Bayramoglu, Commun. Stat. Theory Methods., Vol. 34, No. 2, 2005, pp. 475-485.
15.M. Asadi, J. Stat. Plan. Infer., Vol. 136, No. 4, 2006, pp. 1197-1206.
16.R.C. Gupta, H.C. Taneja, and R. Thapliyal, Stat. Theory Appl., Vol. 13, No. 1, 2014, pp. 27-37.
17.M. Kijima, J. Appl. Probab., Vol. 26, No. 1, 1989, pp. 89-102.
18.B. Van Es, Scand. J. Stat., Vol. 19, No. 1, 1992, pp. 61-72.
19.N. Ebrahimi, K. Pflughoeft, and E. Soofi, Stat. Probab. Lett., Vol. 20, No. 3, 1994, pp. 225-234.
20.A.I. Al-Omari, J. Comput. Appl. Math., Vol. 261, No. 1, 2014, pp. 95-102.
TY - JOUR
AU - M. Ramadan
PY - 2018
DA - 2018/12/31
TI - Weighted Entropy Measure: A New Measure of Information with ts Properties in Reliability Theory and Stochastic Orders
JO - Journal of Statistical Theory and Applications
SP - 703
EP - 718
VL - 17
IS - 4
SN - 2214-1766
UR - https://doi.org/10.2991/jsta.2018.17.4.11
DO - 10.2991/jsta.2018.17.4.11
ID - Ramadan2018
ER -
