2. Non-parametric estimation of generalized past entropy function

In this section, we propose a non-parametric kernel type estimator for the generalized past entropy function.

Let $\{X_i; 1\leq i\leq n\}$ be a sequence of identically distributed rvs. Note that $X{_i}'s$ need not be mutually independent, that is, the observations are supposed to be $\alpha$-mixing. Rosenblatt [Reference Rosenblatt16] and Parzen [Reference Parzen14] proposed a non-parametric kernel type estimator of $f (x)$ that is given by

(2.1)\begin{equation} f_n (x) = \frac{1}{n b_n}\sum\limits_{j = 1}^n {K\left( {\frac{{x - X_j }}{{b_n }}} \right)}, \end{equation}

where $K(x)$ is a kernel of order $s$ satisfying the conditions:

1. a. $K(x) \geq 0$ for all $x$,

2. b. $\int\limits_{- +\infty }^{+\infty} K(x)dx = 1$,

3. c. $K(\cdot)$ is symmetric about zero and satisfies the Lipschitz condition, that is, there exists a constant $M$ such that $|K(x)-K(y)| \leq M |x-y|$,

4. d. $K_n (x) = \frac{1}{b_n} {K\left( {\frac{{x }}{{b_n }}} \right)}$, where $\{b_n\}$ is a bandwidth sequence of positive numbers such that $b_n \rightarrow 0$ and $nb_n \rightarrow +\infty$ as $n \rightarrow +\infty$.

The expressions for the bias and variance of $f_n (x)$ under $\alpha$-mixing dependence sample are respectively given by

(2.2)\begin{equation} Bias\mathop {}\limits^{} (f_n (x)) \backsimeq \frac{{b_n ^s \Delta_s}}{{s!}}f^{(s)} (x) \end{equation}

and

(2.3)\begin{equation} Var\mathop {}\limits^{} (f_n (x)) \backsimeq \frac{{ f(x)}}{{nb{}_n}}\Delta_K, \end{equation}

where $\Delta_s = \int\limits_{- +\infty }^{+\infty} {u^s K(u)\ du} $, $\Delta_K = \int\limits_{- +\infty }^{+\infty} {K^2 (u)\ du}$ and $f^{(s)}(\cdot)$ denotes the $s$-th derivative of $f$.

Based on (2.1), we propose a non-parametric kernel type estimator for $\overline H^{\beta} (f;t)$ that is defined as

(2.4)\begin{equation} \overline H_n^\beta (f;t) = \frac{1}{(1-\beta)}\left\{\log \int\limits_0 ^ t f_n^\beta (x)\ dx - \beta \log F_n (t) \right\}, \end{equation}

where $f_n (x)$ is a non-parametric estimator of $f (x)$ and is given in (2.1) and $ F_n (t)= \int\limits_0^t f_n (x) dx $ is a non-parametric estimator of cdf $ F (t)$. The bias and variance of $ F_n (t)$ are respectively given by (see, Maya [Reference Maya12])

(2.5)\begin{equation} Bias\mathop {}\limits^{}(F_n (t)) \backsimeq \frac{b_n^{s}\Delta_{s} }{s!}\int\limits_0^t f^{(s)}(x)\ dx \end{equation}

and

(2.6)\begin{equation} Var\mathop {}\limits^{} (F_n (t)) \backsimeq \frac{1}{nb_n} \Delta_K F(t). \end{equation}

3. Asymptotic properties of generalized past entropy function

In this section, some asymptotic properties of generalized past entropy function are established.

For computational simplicity, define the following

(3.1)\begin{equation} a_n (t) = \log\int\limits_0^t {f_n^{\beta} (x)\ dx},a(t) = \log\int\limits_0^t {f^{\beta} (x)\ dx}, \end{equation}

and

(3.2)\begin{equation} m_n (t)=\log F_n (t)\ \text{and} \ m(t)=\log F(t). \end{equation}

Therefore,

(3.3)\begin{equation} \overline H^\beta (f;t) = \frac{1}{(1-\beta)}\left[ a(t)- \beta\ m (t) \right] \end{equation}

and

(3.4)\begin{equation} \overline H_n^\beta (f;t) = \frac{1}{(1-\beta)}\left[ a_n (t)- \beta\ m_n (t) \right]. \end{equation}

In the following theorems, the weak and strong consistency properties of $\overline H_n^\beta (f;t)$ are proved.

Proof. By using Taylor’s series expansion, we have

\begin{equation*} \log\int\limits_0^t {f_n^{\beta} (x)\ dx}\backsimeq \log\int\limits_0^t {f^{\beta} (x)\ dx}+(f_n(t)-f(t))\frac{\int\limits_0^t\beta f^{\beta-1}(x)dx }{\int\limits_0^t f^\beta(x)dx}. \end{equation*}

Using (3.1), we get

(3.5)\begin{equation} a_n(t) \backsimeq a(t)+(f_n(t)-f(t))u(t), \end{equation}

where

\begin{equation*}u(t)=\frac{\int\limits_0^t\beta f^{\beta-1}(x)dx }{\int\limits_0^t f^\beta(x)dx}.\end{equation*}

Also,

\begin{equation*} \log F_n (t) \backsimeq\log F(t)+\frac{(F_n(t)-F(t))}{F(t)}. \end{equation*}

From (3.2), we write

(3.6)\begin{equation} m_n(t) \backsimeq m(t)+\frac{(F_n(t)-F(t))}{F(t)}, \end{equation}

Using (3.5) and (3.6), the bias and variance of $a_n(t)$ and $m_n(t)$ are given by

(3.7)\begin{equation} Bias\mathop {}\limits^{}(a_n (t)) \backsimeq \frac{b_n^{s}}{s!}\Delta_{s} f^{(s)}(t)u(t), \end{equation}

(3.8)\begin{equation} Var\mathop {}\limits^{} (a_n (t)) \backsimeq \frac{1}{nb_n} \Delta_Kf(t)u^2(t), \end{equation}

(3.9)\begin{equation} Bias\mathop {}\limits^{}(m_n (t)) \backsimeq \frac{b_n^{s}\Delta_{s} }{s!F(t)}\int\limits_0^t f^{(s)}(x)\ dx, \end{equation}

and

(3.10)\begin{equation} Var\mathop {}\limits^{} (m_n (t)) \backsimeq \frac{1}{nb_n} \frac{\Delta_K} {F(t)}. \end{equation}

From (3.7) and (3.8), as $ n \to +\infty$

\begin{equation*} \text{MSE} \left( {a_n (t)} \right) \to 0. \end{equation*}

Therefore, the estimator $a_n(t)$ is consistent (in the probability sense), i.e.,

\begin{equation*} {a_n (t)}\mathop \to \limits ^ p a(t). \end{equation*}

From (3.9) and (3.10), as $ n \to +\infty$

\begin{equation*} \text{MSE} \left( {m_n (t)} \right) \to 0. \end{equation*}

Therefore, the estimator $m_n(t)$ is consistent (in the probability sense), i.e.,

\begin{equation*} {m_n (t)}\mathop \to \limits ^ p m(t). \end{equation*}

Therefore

\begin{equation*} \overline H_n^\beta (f;t) =\frac{1}{(1-\beta)}\left[ a_n (t)- \beta\ m_n (t) \right] \mathop \to \limits^p \frac{1}{(1-\beta)}\left[a(t) -\beta\ m(t)\right]= \overline H^\beta (f;t) . \end{equation*}

That is, $\overline H_n^\beta (f;t)$ is a consistent (in the probability sense) estimator of $\overline H^\beta (f;t)$.

Theorem 3.2. Let $\overline H_n^\beta (f;t)$ be a non-parametric estimator of $\overline H^\beta (f;t)$ defined in (2.4) satisfying the assumptions given in Section 2. Suppose the kernel $K(\cdot)$ satisfies the requirements:

1. a. $K(u)\to 0$ as $|u|\to +\infty$,

2. b. $\int\limits_{- +\infty }^{+\infty} |K^{'}(u)| du \lt +\infty$,

3. c. $\int\limits_{- +\infty }^{+\infty} |u| K(u)\ du \lt +\infty$.

Let $\overline t=\sup \{t \in \mathbb R; F(t) \lt 1\}$ and $J$ be any compact subset of $(0,\ \overline t)$. Then

\begin{equation*} \lim \limits _{n \to +\infty} \sup \limits_ {t \epsilon J} \left|\overline H_n^\beta (f;t)-\overline H^\beta (f;t)\right|= 0 \ a.s. \end{equation*}

Proof. By direct calculations, we get

\begin{equation*} \begin{aligned} \overline H_n^\beta (f;t)-\overline H^\beta (f;t)&= \frac{1}{(1-\beta)}\left(a_n (t)-a(t)\right)-\frac{\beta}{(1-\beta)}\left(m_n (t)-m(t)\right).\\ \end{aligned} \end{equation*}

\begin{equation*}\nonumber \begin{aligned} |\overline H_n^\beta (f;t)-\overline H^\beta (f;t)| & \backsimeq \frac{u(t)}{(1-\beta)}\left|f_n (t)-f(t)\right|+\frac{\beta}{(1-\beta)}\left(\frac{| F_n (t)- F(t)|}{1- F(t)}\right). \end{aligned} \end{equation*}

By using Theorem 3.3 and Theorem 4.1 of Cai and Roussas [Reference Cai and Roussas5], the result is immediate. Thus, we conclude that $\overline H_n^\beta (f;t)$ is a strong consistent estimator of $\overline H^\beta (f;t)$.

In order to prove that $\overline H_n^\beta (f;t)$ is an integratedly uniformly consistent in quadratic mean estimator of $\overline H^\beta (f;t)$, we consider the following definition from Wegman [Reference Wegman21].

Definition 3.1. The density estimator $f_n(x)$ is said to be integratedly uniformly consistent in quadratic mean if the mean integrated squared error (MISE) approaches zero, i.e.,

\begin{equation*} \lim_{n \to +\infty} \mathbb{E} \left[ \int (f_n(x) -f(x))^2 \,dx \right] = 0 \end{equation*}

In the following theorem, we prove that $\overline H_n^\beta (f;t)$ is integratedly uniformly consistent in quadratic mean estimator of $\overline H^\beta (f;t)$.

Proof. MISE of $\overline H_n^\beta (f;t)$ is given by

(3.11)\begin{equation} \begin{aligned} {\rm MISE}(\overline H_n^\beta (f;t))&= {\rm E} \int\limits_{0}^{+\infty} \left[\overline H_n^\beta (f;t)-\overline H^\beta (f;t)\right]^2 dt\\ & =\int\limits_{0}^{+\infty} \left[[{\rm Bias}(\overline H_n^\beta (f;t))]^2+{\rm Var}(\overline H_n^\beta (f;t)) \right]dt\\ &= \int\limits_0^{+\infty}\left( \frac{b^{s}_n \Delta_s }{(1-\beta)s!} \left(f^{(s)}(t) u(t)-\frac{\beta}{F(t)} \int\limits_{0}^ {t} f^{(s)}(x) dx \right) \right)^2 dt\\ & +\int\limits_0^{+\infty} \left(\frac{\Delta_K}{nb_n(1-\beta)^{2}} \left(f(t) u^{2}(t)+\frac{\beta^{2}}{F(t)} \right)\right) dt.\\ \end{aligned} \end{equation}

We have, as  $ n \to +\infty$,

\begin{equation*} {\rm MSE}\left( \overline H_n^\beta (f;t)\right) = Bias(\overline H_n^\beta (f;t))]^2+{\rm Var}(\overline H_n^\beta (f;t)) \to 0. \end{equation*}

Therefore, from (3.11), we have

(3.12)\begin{equation} {\rm MISE}\left(\overline H_n^\beta (f;t)\right) \to 0, \text{as}~ n \to +\infty. \end{equation}

From (3.12), we can say that $\overline H_n^\beta (f;t)$ is integratedly uniformly consistent in quadratic mean estimator of $\overline H^\beta (f;t)$.

Thus the theorem is proved.

In the following theorem, we obtained the optimal bandwidth.

Theorem 3.4. Suppose $\overline H_n^\beta (f;t)$ is a non-parametric estimator of $ \overline H^\beta (f;t)$ defined in (2.4) satisfying the assumptions given in Section 2. Then the optimal bandwidth is given by

(3.13)\begin{equation} b^{'}_n=\left[\frac{\frac{1}{n} \int\limits_{0}^ {+\infty} \Delta_K \left(f(t) u^{2}(t)+\frac{\beta^{2}}{F(t)} \right) dt }{2s\int\limits_{0}^ {+\infty} \left(\frac{\Delta_s}{s!}\left(f^{(s)}(t) u(t)-\frac{\beta}{F(t)} \int\limits_{0}^ {t} f^{(s)}(x) dx \right)\right)^2dt} \right] ^{\frac{1}{2s+1}}. \end{equation}

Proof. By using (3.11), the asymptotic-MISE (A-MISE) is given by

\begin{equation*} \begin{aligned} \text{A-MISE}(\overline H^{\beta}_n (f;t)) & = {b^{2s}_{n}}\int\limits_0^{+\infty}\left(\frac{\Delta_s}{(1-\beta)s!}\left(f^{(s)}(t) -\frac{\beta}{F(t)}\int\limits_0^t f^{(s)}_{(x)} dx \right)\right)^2dt\\ & + \frac{1}{n b_n} \int\limits_0^{+\infty} \frac{\Delta_K}{(1-\beta)^{2}}\left(f(t)u^{2}(t) +\frac{\beta^{2}}{F(t)} \right) dt.\\ \end{aligned} \end{equation*}

By minimizing $\text{MISE}(\overline H^{\beta}_n (f;t) )$ with respect to the parameter $b_n$, we get the optimal bandwidth $b^{'}_n$.

$\frac{\partial \text{A-MISE}(\overline H^{\beta}_n (f;t))}{\partial b_{n}}=0\Rightarrow$

\begin{equation*} \begin{aligned} 2s b^{2s-1}_{n}\int\limits_0^{+\infty}\left\{\frac{\Delta_s}{(1-\beta)s!}\left(f^{(s)}(t) -\frac{\beta}{F(t)}\int\limits_0^t f^{(s)}_{(x)} dx \right) \right\} ^2dt &= \\ \frac{1}{n b^{2}_n} \int\limits_0^{+\infty} \frac{\Delta_K}{(1-\beta)^{2}}\left(f(t)u^{2}(t) +\frac{\beta^{2}}{F(t)} \right) dt. \end{aligned} \end{equation*}

$\Rightarrow$

\begin{equation*} b^{2s+1}_{n}=\frac{\frac{1}{n } \int\limits_0^{+\infty} \frac{\Delta_K}{(1-\beta)^{2}}\left(f(t)u^{2}(t) +\frac{\beta^{2}}{F(t)} \right) dt}{2s \int\limits_0^{+\infty}\left\{\frac{\Delta_s}{(1-\beta)s!}\left(f^{(s)}(t) -\frac{\beta}{F(t)}\int\limits_0^t f^{(s)}_{(x)} dx \right) \right\} ^2dt}. \end{equation*}

Therefore,

\begin{equation*} \begin{aligned} b^{'}_{n} &= \left[\frac{\int\limits_0^{+\infty} \Delta_K\left(f(t)u^{2}(t) +\frac{\beta^{2}}{F(t)} \right) {\rm d}t}{2s \int\limits_0^{+\infty}\left\{\frac{\Delta_s}{s!}\left(f^{(s)}(t) -\frac{\beta}{F(t)}\int\limits_0^t f^{(s)}_{(x)} dx \right) \right\} ^2{\rm d}t}\right]^{\frac{1}{2s+1}}n^{-\frac{1}{2s+1}} \\ &= O\left(n^{-\frac{1}{2s+1}}\right), s=1,2,....\\ \end{aligned} \end{equation*}

The following lemma is used to derive the asymptotic normality of $\overline H^{\beta}(f;t)$ and is established in Theorem 3.4.

Lemma 3.1. Let $K(x)$ be a kernel of order $s$, let $\{\alpha (i)\}$ be the mixing coefficients and let $\{b_n\}$ be a sequence of numbers satisfying the assumptions given in Section 2. Then, for all fixed point $x$, with $f(x) \gt 0$,

(3.14)\begin{equation} \left(nb_n\right)^\frac{1}{2}\left\{\frac{(f_n (x)- f(x))}{\sigma_f}\right\} \end{equation}

has a standard normal distribution as $n \to +\infty $ with $\sigma_{f} ^2 \backsimeq \frac{1}{n}f(x)\Delta_k$.

Theorem 3.5. Let $\overline H^{\beta}_n (f;t)$ be a non-parametric estimator of $\overline H^{\beta}(f;t)$, $K(x)$ be a kernel of order $s$ satisfying the assumptions given in Section 2. Then for fixed $t$,

(3.15)\begin{equation} {\sqrt {nb_n} }\left\{\frac{{(\overline H^{\beta}_n (f;t) -\overline H^{\beta} (f;t))}}{{\sigma _{\overline H}}}\right\} \end{equation}

has a standard normal distribution as $n \to +\infty$ with

(3.16)\begin{equation} \sigma^2_{\overline H} \backsimeq \frac{\Delta_K }{nb_n(1-\beta)^2}\left\{f(t)u^2(t)+ \frac{\beta^2}{F(t)} \right\}. \end{equation}

Proof.

\begin{align*} \sqrt {n b_n}\left(\overline H^{\beta}_n (f;t)-\overline H^{\beta}(f;t) \right)&= \frac{1}{(1-\beta)}(nb_n)^{\frac{1}{2}} \left\{\log\! \int\limits_0^t f_{n}^{\beta}(x)\ dx- \log\! \int\limits_0^t f^{\beta}(x)\ dx\right\} \\ &\quad - \frac{\beta}{(1-\beta)}(nb_n)^{\frac{1}{2}}\left\{\log F_n (t)-\log F (t)\right\}\\ &\backsimeq \frac{1}{(1-\beta)}(nb_n)^{\frac{1}{2}} \left\{(f_n (t)-f(t))u(t) \right\} \\ &\quad- \frac{\beta }{(1-\beta)}(nb_n)^{\frac{1}{2}}\frac{\int\limits_0^t (f_n(x)-f(x))\ dx}{F(t)}. \end{align*}

By using the asymptotic normality of $f_n (x)$ given in Lemma 1, the proof is immediate.

4. Simulation

A simulation study is conducted to evaluate the performance of the proposed estimator $\overline H^{\beta}_n (f;t)$. Here the process $\left\{X_i\right\}$ is generated form exponential AR(1), with correlation coefficient $\phi=0.2$ and parameter $\lambda=1$. The Gaussian kernel is employed as the kernel function for estimation. The estimated value, bias, and MSE of the proposed estimator are calculated for various sample sizes. We vary the values of $\beta$ and the bandwidth parameter is determined using the plug-in method as proposed by Sheather [Reference Sheather and Jones20]. The result for the exponential AR(1) process are presented in Table 1.

Table 1. Estimated value, bias ad $MSE$ of $\overline H^{\beta}_n (f;t)$ for the Exponential AR(1) process along with the corresponding theoretical value $\overline H^{\beta} (f;t)$.

From Table 1, it is evident that both the $MSE$ and bias of the estimators decrease as the sample size increases. The reduction in $MSE$ signifies that the estimators’ predictions become closer to the true values with larger sample sizes, reflecting improved accuracy and efficiency in estimation. Similarly, the decreasing bias highlights the growing precision of the estimator.

5. Numerical examples

To demonstrate the practical applicability of the proposed estimator discussed in Section 2, we analyze real-world data consisting of the failure times (in months) of 20 electric carts used for internal transport and delivery in a large manufacturing facility (see Zimmer et al. [Reference Zimmer, Keats and Wang23]). The bootstrapping procedure is employed to determine the optimal value of $b_n$ (see Efron [Reference Efron8]).

For estimation, we utilize the Gaussian kernel function:

\begin{equation*} K(z) = \frac{1}{\sqrt{2\pi}} \exp \left( \frac{-z^2}{2} \right). \end{equation*}

At each value of $t$, we compute the biases and mean-squared errors of $\overline{H}_{n}^{\beta} (f;t)$ using 250 bootstrap samples of size 20. Table 2 presents the bootstrap biases and mean-squared errors for the estimator $\overline{H}_{n}^{\beta} (f;t)$. Figure 1 compares the theoretical value $\overline{H}^{\beta} (f;t)$ with the estimated value $\overline{H}_n^{\beta} (f;t)$. From Figure 1, it is evident that for the given dataset, the generalized past entropy function increases over time.

Figure 1. Plots of estimates of generalized past entropy function for the first failure of 20 electric carts.

Table 2. Bootstrap bias and mean-squared error of $\overline H_{n}^{\beta}(f;t)$ for the real data set.

Next, we aim to examine the asymptotic normality of the estimator presented in Theorem 3.5. To achieve this, we conduct a numerical evaluation of $\overline{H}^{\beta}_n (f;t)$ and analyze it through Monte Carlo simulations. Let $X$ follow an exponential distribution with parameter $\lambda$, where the mean is given by $\mu = 1/\lambda$. Then, the past entropy function of order $\beta$ is expressed as

(5.1)\begin{equation} \overline H^{\beta} (f;t)=\frac{1}{1-\beta}\left[(\beta-1)\log\lambda-\log\beta-\beta\log(1-e^{-\lambda t})+\log(1-e^{-\lambda\beta t})\right]. \end{equation}

Before obtaining our estimator, it is necessary to fix a function $K$ and a sequence $\{b_n\}_{n\in\mathbb N}$ which satisfy the assumptions given in Section 2. Here, we consider

(5.2)\begin{eqnarray} K(x)&=&\frac{1}{\sqrt{2\pi}}\exp\left(-\frac{x^2}{2}\right), \ \ x\in\mathbb R \end{eqnarray}

(5.3)\begin{eqnarray} b_n&=&\frac{1}{\sqrt{n}}, \ \ n\in\mathbb N. \end{eqnarray}

By using these assumptions, we obtain

\begin{equation*} \nonumber \Delta_K=\frac{1}{2\sqrt\pi}, \end{equation*}

and, for the exponential distribution, we have

\begin{equation*} \nonumber u(t)=\frac{\beta^2}{\lambda(\beta-1)}\frac{1-e^{-\lambda(\beta-1)t}}{1-e^{-\lambda\beta t}}. \end{equation*}

We aim to verify whether the quantity defined in (3.15) follows a standard normal distribution. Using the exprnd function in MATLAB, we generate 500 samples, each of size $N = 50$, from an exponential distribution with parameter $\lambda$. For each simulated sample, we extract only the values that do not exceed a fixed threshold $t$, and the count of these selected values is used as $n$ in our evaluations. Subsequently, using the fixed parameters, we compute the quantity given in (3.15) for each sample. To assess its asymptotic normality, we construct a histogram based on the 500 computed values. Specifically, we explore various choices for $\lambda$, $\beta$, and $t$, as illustrated in Figure 2.

Figure 2. Histograms of (3.15) with parameters given in Section 5 and different choices of $\alpha$, $\beta$ and $t$. a) $\lambda=1$, $\beta=2$, $t=2$ b) $\lambda=1$, $\beta=2$, $t=4$ c) $\lambda=1$, $\beta=0.5$, $t=2$ d) $\lambda=2$, $\beta=0.5$, $t=2$.

6. Conclusion

This paper examines non-parametric estimators for the generalized past entropy function using kernel-based estimation, with $\alpha$-mixing dependent observations. The asymptotic properties of the proposed estimator are analyzed and established. Additionally, a simulation study and real data analysis are carried out to evaluate the performance of the proposed estimator. We obtained that the estimator is performing good on both simulation study and data analysis owing to bias and $MSE$. As discussed earlier these estimate can be used in selecting a reliable system from the other available competing models.

Future research will extend beyond the $\alpha$-mixing dependence condition to explore the inferential properties of generalized past entropy function under $\phi$-mixing and $\rho$-mixing dependence conditions.