2.1. Hierarchical Bayesian modeling

Section 2.1 follows an explanation provided by the authors precious work (Bull et al., Reference Bull, Di Francesco, Dhada, Steinert, Lindgren, Parlikad, Duncan and Girolami2023). Consider data recorded from a population of K engineering structures. The population can be denoted,

(1) $$ {\left\{{\mathbf{x}}_k,{\mathbf{y}}_k\right\}}_{k=1}^K={\left\{{\left\{{x}_{ik},{y}_{ik}\right\}}_{i=1}^{N_k}\right\}}_{k=1}^K $$

where $ {y}_k $ is a target response vector for inputs $ {x}_k $ and $ \left\{{x}_{ik},{y}_{ik}\right\} $ are the $ {i}^{\mathrm{th}} $ pair of observations in group $ k $ . There are $ {N}_k $ observations in each group and thus $ {\Sigma}_{k=1}^K{N}_k $ observations in total. The aim is to learn a set of $ K $ predictors related to a regression or classification task. This paper focuses on regression, where the tasks satisfy,

(2) $$ {\left\{{y}_{ik}={f}_k\left({x}_{ik}\right)+{\unicode{x025B}}_k\right\}}_{k=1}^K $$

and the output $ {y}_{ik} $ is determined by evaluating one of $ K $ latent functions with separate additive noise $ {\varepsilon}_k $ . The mapping $ {f}_k $ is assumed to be correlated between members in the population. The models should be improved by learning the parameters in a joint inference over the whole population. In machine learning this is referred to as multitask learning; in statistics, such data are usually modeled with hierarchical models (Kreft and De Leeuw, Reference Kreft and De Leeuw1998; Gelman and Hill, Reference Gelman and Hill2006).

In practice, some members in a population may possess extensive historical data, while members that may have been more recently deployed will have very limited data for training. Learning separate independent models for each group might lead to unreliable results for data-poor members, while a single regression model of all the data (complete pooling) would result in poor generalization. Hierarchical Bayesian models can learn separate models for each member, while encouraging the parameters of these models to be correlated (Murphy, Reference Murphy2012). The established theory is summarized here.

Consider K linear regression models,

(3) $$ {\left\{{\mathbf{y}}_k={\varPhi}_k{\alpha}_k+{\unicode{x025B}}_k\right\}}_{k=1}^K $$

where $ {\Phi}_k=\left[\mathbf{1},{\mathbf{x}}_k\right] $ is the $ {N}_k\times 2 $ design matrix; $ {\alpha}_k $ is the $ 2\times 1 $ vector of weights; and the noise vector is $ {N}_k\times 1 $ and normally distributed $ {\unicode{x025B}}_k\sim N\left(0,{\sigma^2}_k\mathbf{1}\right) $ . $ \mathbf{1} $ is a vector of ones, $ \mathbf{I} $ is the identity matrix, and $ N\left(m,s\right) $ is the normal distribution with mean $ m $ and (co)variance $ s $ . The likelihood of the target response vector is then

(4) $$ {\mathbf{y}}_k\mid {\mathbf{x}}_k\sim \mathcal{N}\left({\Phi}_k{\alpha}_k,{\sigma_k}^2\mathbf{I}\right) $$

(5) $$ \therefore {y}_k\mid {x}_k\sim \mathcal{N}\left({\alpha_1}^{(k)}+{\alpha_2}^{(k)}{x}_{ik},{\sigma_k}^2\right) $$

following the Bayesian methodology, one can set a common hierarchy of prior distributions over the weights (slope and intercept) for each member in the population, typically normal distributions are used for the weights of each group and inverse-Gamma for the variance parameter.

(6) $$ {\left\{{\alpha}_K\right\}}_{k=1}^K\overset{i.i.d.}{\sim}\mathcal{N}\left({\mu}_{\alpha },\operatorname{diag}\left\{{\sigma}_{\alpha}^2\right\}\right) $$

(7) $$ {\mu}_{\alpha}\sim \mathcal{N}\left({\mathbf{m}}_{\alpha },\operatorname{diag}\left\{{\mathbf{s}}_{\alpha}\right\}\right) $$

(8) $$ {\sigma}_{\alpha}\overset{i.i.d.}{\sim}\mathcal{I}\mathcal{G}\left(a,b\right) $$

In words, Equation (6) assumes that the weights $ {\left\{{\alpha}_K\right\}}_{k=1}^K $ are normally distributed $ N\left(\cdot \right) $ with mean $ {\mu}_{\alpha } $ and covariance $ \mathit{\operatorname{diag}}\left\{{\sigma^2}_{\alpha}\right\} $ . Equation (7) states that prior expectation of the weights $ {\alpha}_k $ is normally distributed with mean $ {\mu}_{\alpha } $ and covariance $ \mathit{\operatorname{diag}}\left\{{\mathbf{s}}_{\alpha}\right\} $ . Equation (8) states that the prior deviation of the slope and intercept is inverse-Gamma distributed with shape $ a $ and scale $ b $ . A general representation of hierarchical regression can be seen in the direct graphical model in Figure 1.

Figure 1. A graphical model representing the linear mixed model with partial pooling.

The parent nodes $ \left\{{\mu}_{\alpha },{\sigma^2}_{\alpha}\right\} $ are inferred from the data, so Equations (6)–(8) encode prior belief of the dependence between latent variables. If these parent nodes were a fixed value, rather than inferred, each model would be independent, preventing the flow of information between domains, this structure allows data-sparse domains to borrow statistical strength from those that are data rich (Murphy, Reference Murphy2012).

Huang and Beck (Reference Huang and Beck2015) and Huang et al. (Reference Huang, Beck and Li2019) present an early example of using hierarchical Bayesian models to represent engineering structures for SHM. Bull et al. (Reference Bull, Di Francesco, Dhada, Steinert, Lindgren, Parlikad, Duncan and Girolami2023) used hierarchical models to improve the survival analysis of a truck fleet and power prediction in a wind farm. Di Francesco et al. (Reference Di Francesco, Chryssanthopoulos, Faber and Bharadwaj2021) used hierarchical models to account for incomplete and imperfect data in an inspection-planning setting.

2.2. Decision theory

Hierarchical models allow one to quantify beliefs about the states of interest and do reasoning under uncertainty. In engineering these predictions are used to take actions in the real world. Decision theory gives a framework to use the uncertainty quantification provided by a Bayesian approach to make optimal choices in many engineering problem settings.

The idea of a “rational” decision-maker is defined as a decision-maker that acts to maximize the expected utility of their actions; this is expressed via the von Neumann–Morgenstern theorem (Morgenstern et al., Reference Morgenstern, Von Neumann, Kuhn and Rubinstein1964), which states that, for two decidable actions $ a $ and $ b $ :

(9) $$ a\succcurlyeq b\iff EU(a)\ge EU(b) $$

where $ \succcurlyeq $ denotes a weak preference, indicating that a decision-maker favors $ a $ at least as much $ b $ ; and $ EU\left(\cdot \right) $ denotes the expected utility associated with doing an action. In Morgenstern et al. (Reference Morgenstern, Von Neumann, Kuhn and Rubinstein1964), the expected utility is derived. Consider a stochastic event $ X $ , which has mutually exclusive outcomes of which $ x\in \mathcal{X} $ are conditionally dependent on a decision $ D $ between actions $ a $ and $ b $ . The expected utility of action $ a $ is computed as follows:

(10) $$ EU(a)=\sum \limits_{x\in \mathcal{X}}P\left(X=x|D=a\right)\cdot U\left(X=x,D=a\right) $$

$ P\left(X|D=a\right) $ is the probability of the outcome of $ X $ given that action $ a $ is executed. $ U $ denotes a utility function that maps $ U:\mathcal{X}\times \mathcal{A}\to \mathrm{\mathbb{R}} $ . Conveniently, if the utility of $ D $ is independent of the variable, X $ U\left(X,D\right) $ can be expressed as the sum of two utility functions, $ U(X) $ and $ U(D) $ That separately describe the utilities associated with outcomes and actions. Equation (10) can then be written as,

(11) $$ EU(a)=\left[\sum \limits_{x\in \mathcal{X}}P\left(X=x|D=a\right)\cdot U\left(X=x\right)\right]+U\left(D=a\right) $$

For a single decision $ D $ over a finite set of actions $ \mathcal{A} $ , an optimal action $ {a}^{\ast } $ can be defined such that the maximum expected utility (MEU) is achieved, where,

(12) $$ \mathrm{MEU}(D)=\underset{a\in \mathcal{A}}{\max } EU(a) $$

and,

(13) $$ {a}^{\ast }=\underset{a\in \mathcal{A}}{\arg \max } EU(a) $$

From Equations (11) and (12), one can see an equivalence between expected utility and risk, which is defined as the product of a probability and a cost. One limitation of utility/risk-based decision theory is that it can be difficult to obtain the probability distributions in Equations (10) and (11). However, via the Bayesian framework outlined in Section 2.1, one can acquire posterior distributions, based on one’s beliefs about an action, that can be used as the probabilities required for Equations (11) and (12). Additionally, the costs or utilities required for these equations can be elicited from asset owners, allowing the expected utility of an action to be estimated.

2.3. Active Learning

To implement a Bayesian hierarchical model, such as the one proposed in Section 2.1, labeled data are required. As discussed in Section 1, for many engineering applications, and particularly for SHM, collecting labels for data is very expensive. It requires sending a domain expert to inspect the physical asset and often requires the operation of the asset to be temporarily halted. These costs motivate reducing the number of inspections and only inspecting measurements that would most improve the predictive model. AL is a set of tools that do this. Additionally, an offline active learner would not be suitable for SHM. SHM systems are required to analyze data as it arrives throughout the life of the monitored system and decisions about interventions can be required immediately upon arrival of this data. One of the major challenges of an online active learner in SHM is that once the decision is made not to query a label, access is lost to this information. It is not possible to retrieve the label once the opportunity has passed.

Consider data, $ X={{\left\{{\mathbf{x}}_i\right\}}^N}_{i=1} $ , which have hidden labels, $ Y={{\left\{{\mathbf{y}}_i\right\}}^N}_{i=1} $ , which can be acquired by paying the costs associated with inspection. The process of choosing and labeling these data points is referred to as querying. An active learner aims to learn a mapping of the observations, $ X $ , to the labels, $ Y $ , while keeping queries to a minimum. A general heuristic for AL is presented by Bull et al. (Reference Bull, Rogers, Wickramarachchi, Cross, Worden and Dervilis2019a) and adapted to regression in Figure 2.

Figure 2. An AL heuristic. Source: Bull et al. (Reference Bull, Rogers, Wickramarachchi, Cross, Worden and Dervilis2019a).

3.1 The dataset

A dataset described in the authors previous work (Wickramarachchi, Reference Wickramarachchi2019) measures deterioration over the life of machining tools during a turning process. The experimental set-up is shown in Figure 3. The workpiece is rotated around the dashed A-line and the tool makes four passes along the workpiece. Each pass starts at point S and ends at point E. After four passes, the tool is inspected, and measurements are taken of the workpiece and tool. This process continued until tool failure. The investigation was repeated for seven nominally identical tools, which form a population.

Figure 3. Schematic showing the experimental set-up used for data acquisition. Source: Wickramarachchi (Reference Wickramarachchi2019).

The deterioration is measured indirectly from the roughness of the workpiece. As the machining tool deteriorates, the surface quality of the workpiece will deteriorate and the roughness will increase. The measurements from this experiment can be seen in Figure 4. It can be seen that, in general, surface roughness increases with the distance a given tool has cut along the workpiece, this distance is termed sliding distance. Several tools show a sharp decrease in surface roughness at the second measurement point—this reading is believed to occur due to an initial sharpening of the tool early in the cutting process.

Figure 4. Experimental surface roughness measurements.

Because of the nature of the experiment, the measurements of surface roughness are very noisy, which can lead to robustness issues when modeling. Combined with the high noise, the shallow gradient of the deterioration makes it difficult to learn the parameters of the regression. This motivates the use of a Bayesian hierarchical model adapted from Section 2.1.

3.2. The hierarchical model

Because of the natural degradation of tools during the machining process, and its effect on surface finish, tools must be replaced regularly. While each tool may be produced to the same specification and made from the same materials, there will be variations in the manufacturing process that manifest as variations between the physical properties of the tools and differences in behavior between the tools; this can be an issue for standard modeling techniques. However, this variation lends itself well to a hierarchical Bayesian model because these types of model account for variations within a population while taking advantage of the statistical similarities between them. An additional benefit of hierarchical models is their suitability to the online setting and sparse datasets. This is particularly useful for tool condition monitoring where researchers may need to make predictions as soon as the machining process has begun and with only a few data points to learn a model. Additionally, the usual benefits of Bayesian modeling apply (uncertainty quantification, prior information, etc.). The hierarchical framework set out in Section 2.1 is adapted below to the case study. Here, there is a population of $ K=7 $ similar tools, with a target response vector $ {y}_k $ , the roughness measurements for each tool. The input vectors $ {x}_k $ are the sliding distance measurements for each tool (how far the tool has cut across the work piece). $ \left\{{x}_{ik},{y}_{ik}\right\} $ are the $ {i}^{\mathrm{th}} $ pair of observations in tool $ k $ . There are $ {N}_k $ observations in each member and thus $ {\Sigma}_{k=1}^K{N}_k $ observations in total. The aim is to learn a set of $ K $ predictors related to the regression task. The type of hierarchical model used for this analysis is a linear mixed model Kreft and De Leeuw, Reference Kreft and De Leeuw1998, so for each member in the population, a gradient $ {m}_k $ and intercept $ {c}_k $ are learnt. A graphical representation of this model can be seen in Figure 5.

Figure 5. A graphical model representing the linear mixed model with partial pooling.

The likelihood of this model is

(14) $$ {\left\{{y}_{ik}\right\}}_{k=1}^K\sim \mathrm{Cauchy}\left({y}_{mean},{\gamma}_k\right) $$

where the location parameter is the equation of a straight line,

(15) $$ {y}_{mean}={m}_k\cdot {x}_{ik}+{c}_k $$

The Cauchy distribution was chosen as the likelihood because of the noisy nature of the data. Cauchy distributions are particularly suited to these types of measurements because of the larger probability density at the extremes, compared to normal distributions another more typical choice. This makes the model less susceptible to outliers and extreme values. Following the Bayesian methodology, one can set prior distributions over the slope for the groups, which encode our prior knowledge of the parameter values.

(16) $$ {\left\{{m}_k\right\}}_{k=1}^K\sim \mathrm{Normal}\left({\mu}_m,{\sigma}_m\right) $$

(17) $$ {\mu}_m\sim \mathrm{Gamma}\left(k,\theta \right) $$

(18) $$ {\sigma}_m\sim \mathrm{HalfCauchy}\left({s}_{\sigma_m}\right) $$

where the slopes are $ \mathrm{normally} $ distributed, with mean $ {\mu}_m $ and standard deviation $ {\sigma}_m $ . Equation (17) shows the prior expectation of the slopes is $ \mathrm{Gamma} $ distributed with shape $ k=1 $ and scale $ \theta =1 $ . The Gamma distribution was a natural choice because we want to encode that the gradients are always positive over the life of a tool. Equation (18) shows that the prior deviation of the slope is $ \mathrm{HalfCauchy} $ distributed with location parameter equal to zero and scale parameter $ {s}_{\sigma_m}=25 $ . As recommended by Gelman (Reference Gelman2006), the variance priors for this hierarchical model are set to be weakly informative. The priors for the intercepts are

(19) $$ {\left\{{c}_k\right\}}_{k=1}^K\sim \mathrm{Normal}\left({\mu}_c,{\sigma}_c\right) $$

(20) $$ {\mu}_c\sim \mathrm{Normal}\left({\overline{\mu}}_c,{s}_{\mu_c}\right) $$

(21) $$ {\sigma}_c\sim \mathrm{HalfCauchy}\left({s}_{\sigma_c}\right) $$

where the intercepts are $ \mathrm{normally} $ distributed, with mean $ {\mu}_c $ and standard deviation $ {\sigma}_c $ . Equation (20) shows the prior expectation of the intercepts is also $ \mathrm{normally} $ distributed with mean $ {\overline{\mu}}_c=0 $ and standard deviation $ {s}_{\mu_c}=1 $ . Equation (21) shows that the prior deviation of the intercept is $ \mathrm{HalfCauchy} $ distributed with location parameter equal to zero and scale parameter $ {s}_{\sigma_c}=25 $ .

The systematic application of graph-theoretic algorithms has led to a number of probabilistic programming languages. Here, models are implemented in NumPyro (Phan et al., Reference Phan, Pradhan and Jankowiak2019. The parameters are inferred using MCMC, via the no U-turn implementation of Hamiltonian Monte Carlo (Hoffman and Gelman, Reference Hoffman and Gelman2014). Throughout, the burn-in period is 1000 iterations and 2000 iterations are used for inference.

To motivate a population-based approach, the hierarchical model will be compared to a model with complete pooling and an independent model with no pooling at all. In the complete pooling approach, a single regression for all the data is learnt. For the independent model, a regression is learnt for each member in the population, but it is assumed there is no correlation between them. Hierarchical modeling is somewhere in between, where a separate regression can be learnt for each member, while encouraging the parameters of these models to be correlated Murphy (Reference Murphy2012).

To simulate tool replacement, some measurements from Figure 4 will be hidden from the models, this emulates new tools with scarce data. In the following figures, the green line shows the samples from the posterior distribution over the (parameterized) latent linear functions, and the gray area shows the area in which 90% of the posterior samples fall.

Figure 6 shows that with complete-pooling, the model struggles with poor generalization and the physical variations between tools mean that a single regression performs poorly on many tools. Even with access to a tools full history, such as Tool 2 and Tool 3, a complete pooling method may not perform well.

Figure 6. Predictions using a complete pooling method.

The model with no pooling can be seen in Figure 7. For tools that have enough historic data, the mean and variance predictions are reasonable. However, this model performs poorly with scarce data. For these tools, the model struggles to learn the parameters of the regression. The variance of these tools are over-estimated and makes poor predictions about the hidden data, when there is not enough data the model relies on vague priors. Over predicting the variance of the surface roughness could be problematic for asset owners if they use this model to inform decisions. For example, unnecessary inspections may be triggered or tools may be replaced prematurely, increasing the costs of production.

Figure 7. Predictions without any pooling method.

Finally, the hierarchical model can be seen in Figure 8. For tools with plentiful data, this model behaves comparably to the no-pooling model. However, for data-scarce tools, there is a large reduction in the estimated variation and improvements in the predicted mean. This model is able to draw on the statistical strength of other tools to help predictions. The model remembers the data from other tools, captured by the prior shared between models of machining operations, and has learnt how similar tools behave.

Figure 8. Predictions using a partial pooling method.

It can be seen in Table 1 that the information sharing provided by a partial-pooling model can improve regression accuracy, shown through a reduction in the total mean squared error as compared to the complete-pooling and no-pooling models. It should be noted that while partial-pooling approaches can achieve improved performance across a population, for individual members in the population, the partial-pooling will occasionally score worse in terms of MSE. For example, in “Tool 2,” the surface roughness measurements are very elevated compared to any other member in the population and returns to the population mean towards the end of tool life. The no-pooling approach over fits to these data, while the partial pooling approach does a better job at modeling the underlying process, masked by noisy measurement. Importantly, the partial pooling method makes good predictions near the end of tool life, the portion of tool life that is most critical for making decisions about replacing the tool.

Table 1. Mean squared error of partial pooling, complete pooling, and no pooling methods

These results are in accordance with the results of the authors previous work (Bull et al., Reference Bull, Di Francesco, Dhada, Steinert, Lindgren, Parlikad, Duncan and Girolami2023; Dardeno et al., Reference Dardeno, Worden, Dervilis, Mills and Bull2024) and motivate sharing information across a population to improve predictions.

3.3. Decision process and Active Learning

Tool health deterioration can lead to tool failure. When monitoring tools, such as the ones that produced the data in this case study, a manufacturer may have an interest in avoiding tool failure because of the safety implications and costs associated with damage to the workpiece. Typically, inspections are used to observe the health state of the tool. However, performing an inspection has its own costs. Tool inspections halt production and labor and expertise are required to conduct an inspection. Optimizing this decision process, whether or not to inspect an asset, can improve economic efficiency. Often there is a limited budget for inspections and ideally inspections should only be conducted when necessary. Employing decision-theoretic approach as shown in Section 2.2 to guide decision-making for inspections provides a risk-based AL approach to more efficiently allocate inspection budgets.

A common quality control criterion for the machining of engineering components is maintaining a surface roughness below some threshold level $ {S}_{\mathrm{crit}} $ , beyond which the component is no longer fit for purpose. A high-quality surface finish can significantly improve the fatigue strength, corrosion resistance, and creep life of machined parts (Sharkawy et al., Reference Sharkawy, El-Sharief and Soliman2014). If the surface finish is damaged via inadequate modeling or control of the machining process, the part may need to be discarded or re-machined; this has an associated utility, $ {C}_{\mathrm{workpiece}} $ . Additionally, during the machining process, an inspection can be conducted to gain access to a noisy observation of the surface roughness, with a utility $ {C}_{\mathrm{inspection}} $ . Throughout the machining process, the tool can be replaced, with a cost $ {C}_{\mathrm{tool}} $ . Here, there is a decision, inaction, to allow the tool to continue machining with a risk to damage the workpiece, and action, to inspect/replace the tool.

The EU, as seen in Equation (10), needs to be determined for each of the possible actions, and the action with maximum expected utility should be chosen in accordance with Equation (13). One limitation of the dataset used in this case study is that inspections and tool replacements can only be triggered at specific discrete time steps because the data were collected at periodic intervals; a dataset with less restrictive inspections points is under production and will be part of future work.

The expected utility of three actions need to be considered to evaluate this decision process. Inaction, to do nothing and allow the tool to continue machining, to inspect the tool and/or to replace the tool. If an inspection occurs, there is another opportunity to decide to replace the tool based on the new information acquired from the inspection.

Again, Equation (10) can be used to calculate the expected utility of a decision. To compute the utility associated with inaction, not inspecting the tool at time step $ t $ , one can use Equation (22). The two outcomes of inaction are the surface roughness reaching or exceeding $ {S}_{\mathrm{crit}} $ before the next opportunity to intervene, time step $ t+1 $ , or the surface roughness not reaching or exceeding $ {S}_{\mathrm{crit}} $ . The utility of not exceeding $ {S}_{\mathrm{crit}} $ is assigned a value of 0, so the second term in Equation (22) equals 0. The probability $ {P}_{t+1}\left(S>{S}_{\mathrm{crit}}\right) $ can be estimated from the model; here, it is the proportion of samples from the Hamiltonian Monte Carlo (HMC) simulation that exceed $ {S}_{\mathrm{crit}} $ before the next opportunity to inspect.

(22) $$ EU\left(D=\mathrm{do}\ \mathrm{nothing}\right)={P}_{t+1}\left(S>{S}_{crit}\right)\times U\left(S>{S}_{crit}\right)+P\left(S<{S}_{crit}\right)\times U\left(S<{S}_{crit}\right) $$

When computing the utility of inspection at a given time step, again, there are two outcomes based on whether the surface roughness will exceed $ {S}_{crit} $ . Again, probabilities of the outcomes can be estimated from the HMC samples. The equation to compute the EU of inspection can be seen in Equation (23). It includes the probability that the tool needs to be replaced, which is the probability that $ S $ is already greater than $ {S}_{\mathrm{crit}} $ at that time step, as well as the probability that $ S $ is not greater than $ {S}_{\mathrm{crit}} $ but will be by the next time step. For the current linear model, this is equivalent to $ {P}_{t+1}\left(S>{S}_{\mathrm{crit}}\right) $ and could be evaluated a such from the HMC samples. Again, these probabilities are estimated from HMC samples.

(23) $$ EU\left(D=\mathrm{inspection}\right)={C}_{tool}\times \left({P}_{t+1}\left(S>{S}_{\mathrm{crit}}\right)\right)+{C}_{inspection} $$

The criteria required to trigger the replacement of a tool can be seen in Equation (24). It is the probability at which the risk associated with damaging the work piece (the probability of damage occurring multiplied by the utility associated with it) becomes greater than the cost of replacing the tool.

(24) $$ {\displaystyle \begin{array}{l}P\left({T}_f<T\right)\times {C}_{\mathrm{workpiece}}\ge {C}_{\mathrm{tool}}\\ {}\hskip4em P\left({T}_f<T\right)\ge \frac{C_{\mathrm{tool}}}{C_{\mathrm{workpiece}}}\\ {}\hskip4em P\left({T}_f<T\right)\ge \alpha \end{array}} $$

Equation (24) shows that tools should be replaced at the earliest time, T, that $ P\left({T}_f<T\right) $ is equal to or greater than the ratio, $ \alpha $ . $ P\left({T}_f<T\right) $ is the assessment of time-to-failure after one has updated beliefs with the results of any inspections and where failure is $ S $ exceeding $ {S}_{\mathrm{crit}} $ . Intuitively, as $ {C}_{tool} $ increases, the probability requirements are increased and so replacements are more difficult to trigger, the system prioritizes extending tool life. As $ U\left(S>{S}_{crit}\right) $ increases, the probability requirements are reduced and so replacements are more easily triggered and the system prioritizes the surface quality of the workpiece.

A complete decision-theoretic approach would consider the expected value of information associated with improving the model based on the new data from an inspection. For the current work, it is assumed that inspecting the tool would provide zero improvement to the predictions of the model as the full value of information calculation is very computationally expensive.

At every potential inspection point, the hierarchical model makes predictions about the surface roughness. These predictions are based on the currently available data and are used to inform the decision analysis detailed above. A diagram showing this process is shown in Figure 9.

Figure 9. The decision theoretic active-learning heuristic.

The online decision-theoretic AL approach for inspection planning presented in this work is compared to a conventional inspection plan featuring periodic inspections. With both approaches, the model has access to the full measurement history of four of the tools and very limited data for the rest of the tools (the authors found that, in this case, without access to at least four full tool datasets, the hierarchical model does not have enough data to draw on for accurate predictions).

In general, the parameters of the decision process, $ {S}_{crit},{C}_{inspection},{C}_{tool},U\left(S>{S}_{crit}\right) $ can be elicited from expert familiar with the inspection process. Here, the values are set by hand to reflect a situation common in industry, where the cost of damaging the workpiece is much greater than the cost of replacing the tool (It is worth noting that optimal actions are invariant under affine transformations of the utility function.):

(25) $$ {\displaystyle \begin{array}{l}U\left(S>{S}_{crit}\right)=1\\ {}\hskip3em {C}_{tool}=0.25\\ {}\hskip1.48em {C}_{inspection}=0.05\\ {}\hskip3.24em {S}_{crit}=0.9\mu m\end{array}} $$

To assess the performance of these monitoring systems, each will be compared to a monitoring system with access to the full measurement history of every tool. This monitoring system will be seen as the gold standard as it represents the limiting case of using all possible information within the dataset and thus the point at which it decides to replace the tool can be considered to be the optimal point of replacement. If the other monitoring systems choose to replace the tool later than the optimal point, it is assumed the roughness has exceeded $ {S}_{crit} $ (despite what the noisy measurements might suggest) and the workpiece will be considered damaged with a cost $ {C}_{workpiece} $ . If the monitoring systems choose to replace the tool earlier than the optimal replacement time, then some portion of the tool life is wasted; the cost of which can be calculated using Equation (26). The optimal monitoring system can choose to replace the tool at any point throughout the life of the tools, that is, it can observe several measurements that are greater than $ {S}_{\mathrm{crit}} $ and then realize it should have replaced the tool before these measurements. The other monitoring systems are performing in an online manner, so do not have this luxury. The combined costs of both inspections and suboptimal replacements, will be compared.

(26) $$ {C}_{\mathrm{early}\ \mathrm{replacement}}=\frac{t_{\mathrm{replacement}}}{t_{\mathrm{optimal}\ \mathrm{replacement}}}\times {C}_{tool} $$

As mentioned previously, the first four tools are to be seen as historic data that the manufacturer has collected from previous tools. The hierarchical model can leverage this data to inform the population-level distributions. The active-learning procedure tool replacements are not calculated for these tools, only Tools 5–7. During the original experiment the measurements were taken at a time step that relates to 6.02 km sliding distance Wickramarachchi, Reference Wickramarachchi2019. Figure 10 shows the optimal replacement for Tools 5–7 based on the “gold standard” model.

Figure 10. Benchmark replacements determined using all available information.