3.1. MOEAs proposed for Multiobjective Optimization Problems (MOPs)

In the early 1960s, Rosenberg proposed an evolutionary algorithm (EA) to solve MOPs. However, just from 1980s that evolutionary multiobjective optimization (EMO) and MOEAs have become a hot topic in the evolutionary computing research. (Mao-Guo et al., Reference Mao-Guo, Li-Cheng, Dong-Dong and Wen-Ping2009).

Multiobjective optimization methods based on EAs can be grouped into three research stages. In the first MOEAs (1993-1998), such as MOGA (Fonseca and Fleming, Reference Fonseca and Fleming1993), NPGA (Horn et al., Reference Horn, Nafpliotis and Goldberg1994), and NSGA (Srinivas and Deb, Reference Srinivas and Deb1994), the selection method was formally based on non-dominance classification and the population diversity strategies adopted a fitness sharing mechanism. Works developed around 1995–2002 focused in the refinament of the MOEAs, proposing selection methods based on external archiving strategies and different approaches for the individuals ranking of the population in a non-dominant environment. NSGA-II (Deb et al., Reference Deb, Pratap, Agarwal and Meyarivan2002), SPEA (Zitzler and Thiele, Reference Zitzler and Thiele1999), SPEA-II (Zitzler et al., Reference Zitzler, Laumanns and Thiele2001), and PAES (Knowles and Corne, Reference Knowles and Corne1999) are some examples of this category of algorithms.

Since then, several multiobjective algorithms have emerged, most investigating problems with 2–3 objectives. Among them, we can mention ParEGO, one of the first and simplest extensions of the EGO algorithm for MOPs (Knowles, Reference Knowles2006). ParEGO sequentially scales the multiobjective problem with weights that are iteratively updated to explore the Pareto front. In 2007, Zhang and Li (Reference Zhang and Li2007) proposed a Decomposition-based MOEA (MOEA/D) combining traditional mathematical programming methods with EAs. Ho and Tay (Reference Ho and Tay2007) presented an efficient approach to solve the flexible multiobjective job-shop combining evolutionary algorithm and guided local search. In 2010, an improved GA based on the principle of immunity and entropy is used to solve the multiobjective flexible job-shop scheduling problem proposed by (Wang et al., Reference Wang, Gao, Zhang and Shao2010). Pizzuti proposed in 2011 the Multiobjective Genetic Algorithms for Networks (MOGA-Net) to discover communities in networks using genetic algorithms (Pizzuti, Reference Pizzuti2011). The method optimizes two objective functions introduced in 2009 for Lancichinetti et al. (Reference Lancichinetti, Fortunato and Kertész2009), that proved to be effective in detecting modules in complex networks. The first objective function employs the concept of community scoring to measure the quality of a network’s community division. The second function defines the concept of capability of nodes belonging to a module and iteratively locates the modules with the highest sum of capability of the node, hereinafter referred to as community capability. In 2012, a method that creates projections of current solutions and propagates equidistant interpolations in a non-dominant environment was presented by (Gu et al., Reference Gu, Liu and Tan2012) and demonstrated to obtain uniformly distributed optimal Pareto solutions. A new opposition self-adaptive hybridized differential evolution algorithm was proposed by Chong and Qiu (2016) (Chong and Qiu, Reference Chong and Qiu2016) to deal with continuous MOPs. Bradford et al. (2018) (Bradford et al., Reference Bradford, Schweidtmann and Lapkin2018) proposed an algorithm to approximate Pareto sets in a small number of function evaluations. The algorithm extends the well-known Thompson sampling (TS) method from the multi-armed bandit community (Thompson, Reference Thompson1933) to continuous multiobjective optimization. The algorithm was named ‘Thompson Sampling Efficient multiobjective Optimization’ (TSEMO).

The increase in the number of objectives brought an additional challenge, since the Pareto Fronts also increased in size. Therefore, it was common for EAs to find clusters of solutions in the search space that belonged to the same Front. Thus, problems with 4 or more objectives started to be classified as many-objective problems. Algorithms such as NSGA-II and SPEA2 presented difficulties in finding the best solutions for this type of problem and several algorithms and approaches have been proposed to solve many-objective problems. One of the most successfully used indicator-based MOEAs is the S-Metric-Selection EMOA (SMS-EMOA) (Emmerich et al., Reference Emmerich, Beume and Naujoks2005). SMS-EMOA invokes the non-dominated classification, which is used as a ranking criterion. Then it uses the hypervolume indicator as a selection mechanism to rule out the individual contributing the smallest hypervolume (the worst ranked). Deb and Saxena (2006) (Deb and Saxena, Reference Deb and Saxena2006) proposed the Principal Component Analysis-NSGA-II algorithm called PCA-NSGA-II. This algorithm combines a reduction technique with NSGA-II to deal with the redundance of objectives in MaOPs. In fact, many real-world problems have M objectives, while the true Pareto front is smaller than M-dimensions, that is, some of the objectives are redundant. herefore, the authors use the PCA method to determine the true Pareto optimal. In 2007, Sato et al. (Reference Sato, Aguirre and Tanaka2007) proposed a method to control the area of dominance of solutions in order to induce the proper classification of solutions for the problem in question, improving the selection and the performance of MOEAs in combinatorial optimization problems. The proposed method can control the degree of expansion or contraction of the dominance area of the solutions using a user-defined S parameter. In their experiments, the authors used the NSGA-II to solve MKP instances with five objectives. In order to solve large-scale problems, Garza-Fabre et al. proposed three different ways to rank solutions of the same Pareto front, which were called Global Detriment (GD), Profit (Pf) and Distance to the Best known solution (GB). Aiming to evaluate their proposal, the authors used a generic multiobjective GA (Garza-Fabre et al., Reference Garza-Fabre, Pulido and Coello2009). Said et al. (2010) (Said et al., Reference Said, Bechikh and Ghédira2010) proposed a new dominance relationship called r-dominance (dominance based on reference solution) that creates a strict partial order between equivalent Pareto solutions and is able to differentiate between partially non-dominated solutions based on a solution provided by user aspiration level vector. In ther experiments, the authors implemented r-dominance in the NSGA-II and evaluated it on problems from 2 to 10 objectives. Singh et al. (2011) (Singh et al., Reference Singh, Isaacs and Ray2011) presented the Pareto Evolutionary Singing Algorithm (PCSEA). The authors proposed a new approach that identifies a reduced set of objectives rather than dealing with the true dimensionality of true MaOP. Furthermore, the PCSEA does not approach the entire Pareto front, but seeks a specific set of non-dominated solutions. More specifically, the authors suggested using Pareto front limits called corner solutions to predict the dimensionality of the true Pareto front. Many-Objective Meta-heuristic Based on the R2 Indicator (MOMBI) (Gómez and Coello, Reference Gómez and Coello2013), is an algorithm proposed in 2013, that produces an non-dominated classification scheme based on utility functions. The main idea is to cluster solutions that optimize the set of utility functions and give them the first ranking. These solutions will then be removed and a second rank will be identified in the same way. The process will continue until all members of the population are classified. Asafuddoula et al. (2013) proposed the Decomposition Based Evolutionary Algorithm for Many-Objective Optimization with Systematic Sampling and Adaptive Epsilon Control (DBEA-Eps) (Asafuddoula et al., Reference Asafuddoula, Ray and Sarker2013). This algorithm is based on decomposition generating a structured set of reference points, which uses an adaptive epsilon comparison to managing the balance between convergence and diversity, and adopting an adaptive epsilon formulation to deal with constraints. Deb and Jain (2014) (Deb and Jain, Reference Deb and Jain2014) proposed a variation of the NSGA for many-objective optimization (NSGA-III), which was later extended to solve generic constrained many-objective optimization problems (Jain and Deb, Reference Jain and Deb2014). Elarbi et al. (2016) (Elarbi et al., Reference Elarbi, Bechikh, Said and Hung2016) proposed a new dominance relation called TSD-dominance to deal with many-objective problems and introduced it in NSGA-II. TSD-NSGA-II represents a new version of many-objective NSGA-II, where Pareto dominance is replaced by TSD dominance. It was considered highly competitive in the treatment of both restricted and unrestricted problems. In Cheng et al. (2016) (Cheng et al., Reference Cheng, Jin, Olhofer and Sendhoff2016) a reference vector-driven evolutionary algorithm for many-objective optimization was presented that balances convergence and diversity of solutions in a high-dimensional objective space. Lafetá et al. (2016) (Lafeta et al., Reference Lafeta, Bueno, Brasil and Oliveira2016) proposed MEANDS, an evolutionary algorithm for many-MOPs based on sub-populations, which represent sets of non-dominated solutions for different instances of subproblems, combining from two to all objectives. Lafetá and Oliveira (2019) (Lafeta and Oliveira, Reference Lafeta and Oliveira2019) present a new version of MEANDS, the MEANDS-II, which adapts to many-MOPs with a very large search space. In 2017, Elarbi et al. (Reference Elarbi, Bechikh, Gupta, Said and Ong2017) proposed the RPD-NSGA-II, an algorithm that integrates a new form of dominance, called RP-dominance, and NSGA-II. RP-dominance assigns penalties to some solutions, so it can classify solutions from the same front. Li et al. (2019) (Li et al., Reference Li, Wei, Song and Liu2019) proposed an efficient objective reduction method called ORSAP. The method is divided into two steps: initially, it uses a sampling algorithm to collect points that can represent objectives, calculating the improvements to the objectives. Then affinity propagation is adopted to cluster the objectives so that redundant ones can be grouped. Therefore, only the centroid objectives are kept in the non-redundant set. Li et al. (2020) (Li et al., Reference Li, He, Liang and Quan2020) proposed the Dividing-based Many-objective Evolutionary Algorithm for large-scale Feature Selection (DMEA-FS). The algorithm models the variables to adjust the solutions to a new search space that is more organized and easier to select the most promising solutions.

3.2. MOEAs proposed for Dynamic Multiobjective Optimization Problems (DMOPs)

Different strategies have been proposed to adapt traditional EAs to deal with DMOPs. Each strategy behaves differently when it comes to changing the environment (modifying the objectives and constraints of the problem). The most intuitive strategy would be to restart the population of MOEAs at each change of environment, but it proved to be a weak approach in (Branke, Reference Branke2001). The majority of the methods proposed for DMOPs in the literature use the knowledge of the population from the previous environment to streamline the search in the new environment (Jin and Branke, Reference Jin and Branke2005). The strategies employed during the change of environment usually concern with boosting convergence and diversity for the new environment. Three main types of strategies can be observed in the related work (Azzouz et al., Reference Azzouz, Bechikh and Said2017): (i) introduction of diversity (Deb and Karthik, Reference Deb and Karthik2007); (ii) forecast (Koo et al., Reference Koo, Goh and Tan2010); and (iii) memory (Cámara et al., Reference Cámara, Ortega and Toro2007). Some of the most relevant MOEAs proposed for DMOPs are presented following, where they are grouped for each kind of strategy.

(i) Introduction of diversity: Deb and Karthik (Reference Deb and Karthik2007) extended the well-known NSGA-II algorithm to deal with DMOPs, introducing diversity with each detection of change. This extension was named DNSGA-II and it is described in the next section since it is one of the MOEAs investigated in the experiments of Section 5. The DOMOEA algorithm (Orthogonal multiobjective EA for DMOP) was introduced in (Zeng et al., Reference Zeng, Chen, Zheng, Shi, de Garis, Ding and Kang2006), where two types of crossovers were used to improve diversity and intensify convergence: orthogonal crossover (Zeng et al., Reference Zeng, Chen, Zheng, Shi, de Garis, Ding and Kang2006) and linear crossover (Wright, Reference Wright1991). In 2007, Zheng et al. (Zheng, Reference Zheng2007) proposed a MOEA adapted for DMOPs called DMOEA. Basically, when a change of environment happens, DMOEA selects a set of the best individuals to mutate using hypermutation (Cobb, Reference Cobb1990) and the rest of the population is replaced by new individuals generated at random. In 2009, Chen et al. (Reference Chen, Li and Chen2009) proposed to explicitly maintain genetic diversity, considering it as an additional objective in the optimization process. They presented the Individual Diversity MultiObjective Optimization Evolutionary Algorithm (IDMOEA), which uses a new method for assessing the preservation of diversity, called Individual Diversity Evolutionary Method (IDEM). The purpose of IDEM is to add a useful selective pressure aimed at Pareto Front and to maintain diversity. An adaptation of the DNSGA-II was proposed by Azzouz et al. (Reference Azzouz, Bechikh and Ben Said2015) in 2015 to deal with dynamic constraints, replacing the constraint handling mechanism used with a more elaborate and self-adaptive penalty function. The resulting algorithm is called Dynamic Constrained NSGA-II (DC-NSGA-II). A new strategy for introducing diversity was proposed by Liu et al. (Reference Liu, Peng, Liu and Liu2020) in 2020, thus generating a MOEA called DDIS-MOEA/D-DE. In this proposal, the information generated during the evolution is recorded in preparation to assess the intensity of the change. Two strategies for introducing diversity are used to maintain the balance between convergence and diversity when an environmental change is detected. An improved inverse modeling is used for the drastic changes, while random initialization of partial solutions is used for the light changes. In 2020 Lafetá and Oliveira (Reference Lafetá and Oliveira2020a) proposed DNSGA-II*; another adaptation of DNSGA-II, where an external archive (Bosman and Thierens, Reference Bosman and Thierens2003) was introduced. A variation of the NSGA-III (Deb and Jain, Reference Deb and Jain2014) was also proposed in (Lafetá and Oliveira, Reference Lafetá and Oliveira2020a) adapting it to dynamic MOPs and it was called DNSGA-III.

(ii) Forecast: Hatzakis and Wallace (Reference Hatzakis and Wallace2006) presented in 2006 a forecasting technique called feed-forward prediction strategy in order to estimate the location of the Pareto Optimal. Then, an anticipated population (named forecast set) is used to accelerate the discovery of the next set of solutions. In 2007, Zhou et al. (Reference Zhou, Jin, Zhang, Sendhoff and Tsang2007) proposed the prediction of the new locations of various Pareto solutions in the decision space once a change is detected. Then, individuals in the reinitialized population are generated around these predicted points. The method called Dynamic Multiobjective Evolutionary Algorithm with Predicted Re-Initialization (DMEA/PRI) works with two strategies to reinitialize the population. The first strategy predicts the locations of new individuals, based on changes from previous points. The population is then partially or completely replaced by the new individuals generated based on this forecast. The second strategy includes a predicted Gaussian noise to the population, where the variation is estimated according to previous changes. In 2008, Roy and Mehnen (Reference Roy and Mehnen2008) proposed an adaptation of DNSGA-II (Deb and Karthik, Reference Deb and Karthik2007) that uses prediction and convenience functions. While in DNSGA-II, the diversity is introduced by adding random solutions, the parent population in this proposition is discarded and only the children are reevaluated before the algorithm is restarted. The objective functions are transformed using preference functions (Mehnen et al., Reference Mehnen, Trautmann and Tiwari2007) in order to guide the search toward the most interesting parts of the Pareto Front, according to the preferences of an expert or decision maker. In 2010, Koo et al. (Reference Koo, Goh and Tan2010) proposed a new evolutionary approach based on prediction to be applied during the change of environment. Based on the historic of previous solutions, the approach attempts to predict the direction and magnitude of the next environment change using the weighted average of these solutions. Updated individuals will remain in the vicinity of the new Pareto Front Pool and will help the rest of the population to converge. The algorithm known as population prediction strategy (PPS) (Zhou et al., Reference Zhou, Jin and Zhang2014) was presented in 2014 for continuous DMOPs. It tries to predict the entire population instead of forecasting some isolated points. It consists of dividing the set of solutions into two parts: central point and collectors. When the environment changes, the next central point is predicted using the sequence of central points found during the search, and the previous collectors are used to predict the next collectors. Thus, the PPS initializes the population by combining the central points and the collectors. Muruganantham et al. (Reference Muruganantham, Tan and Vadakkepat2016) proposed in 2015 a dynamic multiobjective evolutionary algorithm that uses the Kalman filter (Zai et al., Reference Zai, DeMarco and Lipo1992) as a prediction model. When the environment changes, the Kalman filter is applied to the population directing the search to the new Pareto Front in the objective space. This algorithm is based on the MOEA with Decomposition based on Differential Evolution (MOEA/D-DE) (Li and Zhang, Reference Li and Zhang2008) and is called the Kalman Filter prediction based DMOEA (MOEA/D-KF). In 2019, Cao et al. (Reference Cao, Xu, Goodman and Li2019) presents a new prediction model combined with a MultiObjective Evolutionary Algorithm based on Decomposition (MOEA/D) (Zhang and Li, Reference Zhang and Li2007) to solve DMOPs. In this model, the movement of the Pareto set over time is represented with respect to the translation of the centroid, and the other solutions are considered to have the same movement as the centroid. A prediction model is built based on the historic locations of recent centroids, which is used to estimate the centroid movement in the next environment change. Then, the new locations of the other solutions are also predicted based on their current locations. According to the authors, the model proposed in (Cao et al., Reference Cao, Xu, Goodman and Li2019) is based on two other works previously cited here (Koo et al., Reference Koo, Goh and Tan2010; Zhou et al., Reference Zhou, Jin and Zhang2014).

(iii) Memory: An algorithm was proposed in 2009 by Wang and Li (Reference Wang and Li2009) to solve dynamic multiobjective problems. It was called the multi-strategy ensemble MOEA (MS-MOEA) and it is one of the MOEAs investigated in the present work. In this algorithm, convergence is accelerated using a new generation dependency mechanism based on adaptive genetics and differential operators. It uses a Gaussian mutation and a memory-like strategy to reinitialize the population. In 2017, Azzouz et al. (Reference Azzouz, Bechikh and Said2017) presented an algorithm to deal with DMOPs with very severe changes. They introduced an adaptive hybrid population management strategy using memory and a random method to deal efficiently with dynamic environments. The Dynamic MultiObjective Evolutionary Algorithm Based on a Dynamic Evolutionary Environment model (DEE-DMOEA) was proposed by Zou in 2019 (Zou et al., Reference Zou, Li, Yang, Zheng, Peng and Pei2019). The algorithm records the evolutionary information generated to guide the search process. When the environment changes, the algorithm assists in the adaptation of the population to the new environment, building a dynamic evolutionary model that increases the diversity of the population. In 2020, Hu et al. (Reference Hu, Zheng, Zou, Yang, Ou and Wang2020) presented a new evolutionary algorithm based on the Intensity of Environmental Change (IEC). The algorithm divides each individual into two parts based on the feedback of the evolutionary information of the Pareto sets in the current and previous environments. The micro and macro-change decision components are implemented considering different scenarios, in order to build an efficient information exchange between dynamic environments.

The majority of DMOP algorithms are based on single-population frameworks. However, some multi-populational MOEAs have also been investigated. Goh and Tan (Reference Goh and Tan2009) presented in 2009 a co-evolutionary multiobjective algorithm based on competitiveness and cooperation to solve DMOPs. Aiming to overcome the difficulty of the decomposition problem and the interdependencies of the subcomponents resulting from co-EAs, the problem is decomposed into several subcomponents along the decision variables. These subcomponents are optimized by different sub-populations in the iterative process that involves competitiveness and cooperation. The algorithm is known as dynamic Competitive Cooperative EA (dCOEA). In 2013, Shang et al. (Reference Shang, Jiao, Ren, Li and Wang2013) proposed the quantum immune clonal co-evolutionary algorithm (QICCA) to solve DMOPs. QICCA is a multi-population algorithm using the strategies Immune Clonal Function and Clonal Selection generally applied to artificial immune systems (Shang et al., Reference Shang, Jiao, Gong and Lu2005). The relationship of competitiveness and cooperation are used to improve the exchange of information between populations, encouraging a better diversity in the search process. D-MEANDS (Lafetá and Oliveira, Reference Lafetá and Oliveira2020b) is one of the MOEAs investigated in this paper and it is described in next section. It is a multi-population model based on MEANDS (Lafeta et al., Reference Lafeta, Bueno, Brasil and Oliveira2016), previously proposed for static MOPs, which employs both introduction to diversity and memory strategies to deal with environmental changes.

D-MEANDS-MD is a variation of D-MEANDS proposed in the present paper and it can also be classified as a multipopulational MOEA using introduction to diversity and memory strategies whenever a environmental change happens. As we have pointed in the introduction, the present work and the previous ones (Lafetá and Oliveira, Reference Lafetá and Oliveira2020a; Lafetá and Oliveira, Reference Lafetá and Oliveira2020b) are, to the best of our knowledge, the first that investigate EAs for dynamic many-objective problems (DMaOPs). However, it is worth to say that the authors in (Koo et al., Reference Koo, Goh and Tan2010) investigated test functions with 2 and 5 objectives. But, on the contrary of the present work, their investigation were totally based on a traditional multiobjective perspective and they did not stick to particular aspects of problems with many-objective formulations.

4.1. DNSGA-II*

The Dynamic Nondominated Sorting Genetic Algorithm II (DNSGA-II) (Deb and Karthik, Reference Deb and Karthik2007) is an adaptation of the well-known NSGA-II algorithm proposed by Deb and colleagues (Deb et al., Reference Deb, Pratap, Agarwal and Meyarivan2002). This variation was adapted in (Deb and Karthik, Reference Deb and Karthik2007) for dynamic problems using a strategy to introduce diversity whenever the environment used to evaluate the solutions faces some perturbation. The typical evolutionary behavior of NSGA-II was kept for DNSGA-II, except when an environment change happens. It consists of dividing the population into hierarchical fronts, where a solution from an upper front has a better fitness than another solution from a lower front. The fronts are classified based on the concept of Pareto dominance, where a solution that is not dominated by any other would be classified on the uppermost front. They are called the nondominated solutions of the current population and the goal is to approximate this uppermost front to the Pareto Optimum at the end of any arbitrary run. The other solutions are classified in a hierarchy of fronts where the solutions of an upper front dominates the solutions of lower fronts. The crowding distance metric is used to differentiate the solutions of the same front. This distance gives a better fitness for the solutions that are more isolated in the front.

In the DNSGA-II framework, a percentage of the current population is modified to promote diversity whenever the environment changes. Two strategies for this modification were evaluated in DNSGA-II (Deb and Karthik, Reference Deb and Karthik2007) defining different versions of this algorithm. The first version was called DNSGA-II-A, where a percentage of the population is kept intact for the next generation after the environment change (a kind of elitism) and other new individuals are generated at random replacing the worst solutions. The second version was called DNSGA-II-B, where the elitism was also used to keep a percentage of the best solutions while the new individuals that replace the worst ones are generated by applying mutation over the elite. The strategy used in the works (Lafetá and Oliveira, Reference Lafetá and Oliveira2020a; Lafetá and Oliveira, Reference Lafetá and Oliveira2020b) was DNSGA-II-A. Moreover, Lafetá and Oliveira proposed in (Lafetá and Oliveira, Reference Lafetá and Oliveira2020a) a change in DNSGA-II, where an external archive (Bosman and Thierens, Reference Bosman and Thierens2003) was introduced. The external archive stores all the nondominated solutions found in the population until the current generation, but it does not participate in the genetic operators selection. On that way, any nondominated solution is kept to end of the run, even if it disappears from the upper front due to size limit of the population. This strategy is very usual in different multiobjective models, like the MOEA/D (Zhang and Li, Reference Zhang and Li2007). This variation of the DNSGA-II algorithm was called DNSGA-II*. It was clear in (Lafetá and Oliveira, Reference Lafetá and Oliveira2020a) that DNSGA-II* outperforms DNSGA-II, with a little extra processing time.

4.2. MS-MOEA

The evolutionary process of the Multi-strategy Ensemble MOEA (MS-MOEA) (Wang and Li, Reference Wang and Li2010) is based on the concept of nondominance, as its population tends to maintain nondominated solutions and discards those that are dominated. Because the population has a fixed size, at the beginning of the search it is normal that not all individuals are nondominated. New nondominated descendants may be generated along the search and they replace the dominated individuals of the population. On the other way, if the current population does not have dominated individuals, the substitution is made at random. Since some nondominated solutions can be removed from the population, the algorithm stores all nondominated solutions found during the search in an internal archive. As this archive participates in the evolutionary process providing solutions to some matings, it is not considered an external archive like the one applied to DNSGA-II*. At ever change of environment some solutions stored in the archive suffer mutation and the new individuals are inserted in the population for the next environment. However, only a part of the population is made using mutation of the nondominated solutions found in the previous environment. The rest of the population of the new environment is generated randomly.

4.3. DDIS-MOEA/D-DE

The MOEA/D based on differential evolution and dynamic diversity introduction strategy (DDIS-MOEA/D-DE) (Liu et al., Reference Liu, Peng, Liu and Liu2020) is an algorithm for dynamic multiobjective problems that uses the same evolutionary process as MOEA/D-DE (Li and Zhang, Reference Li and Zhang2008), proposing a dynamic adaptive strategy according to the severity of change. MOEA/D-DE is an evolutionary multiobjective algorithm that decomposes the problem into several subproblems, which are evolved simultaneously. The decomposition is made by a scalarization function, which has the role of transforming each individual in a single scalar value (fitness). The current population is made up of the best solutions for each subproblem. The neighborhood of each subproblem is defined based on the distances between their weighting coeficient vectors. It is very important to highlight that the MOEA/D-DE uses differential evolution operators. The operators for genetic algorithms used in this paper were proposed (Tasgetiren et al., Reference Tasgetiren, Pan, Kizilay and Suer2015). Tasgetiren et al. (2015) proposed a model of differential evolution operators for the DMKP using a probability model based on the sigmoid function inspired by (Wang and Zheng, Reference Wang and Zheng2013). The main idea is to employ variable mutation strategies combined with a uniform crossover to generate a trial individual. The DDIS-MOEA/D-DE uses two strategies to create population during the change of environment. The first strategy occurs when the severity of change is high, so the algorithm takes advantage of the population from the previous environment and replaces part of the population with random individuals. However, when the severity of change is small, the algorithm keeps the population of the previous environment and applies a mutation in part of this population. In the experiments, the environment changes are controlled to keep the severity of change as low as possible, so the second strategy is always used. The number of individuals that mutate during the change of environment is equivalent to 20% of the population, as used in (Liu et al., Reference Liu, Peng, Liu and Liu2020).

4.4. D-MEANDS

DMEANDS is strongly based on its predecessor MEANDS. The Many-objective Evolutionary Algorithm Based on Nondominated Decomposed Sets (MEANDS) (Lafeta et al., Reference Lafeta, Bueno, Brasil and Oliveira2016) was proposed aimed at discrete and static MaOPs. The MEANDS framework manipulates several sub-populations (or subsets) of nondominated solutions. Each sub-population assesses the nondominance of individuals based on a subset of objectives. MEANDS initializes its population by generating a sample of random individuals of size A and verifying the dominance relation to insert the nondominated solutions into each subset considering the group of objectives related to each subpopulation. The insertion and removal process of individuals in each subset are independent because each subpopulation uses a different group of objectives. Each subset stores only nondominated solutions extracted from the random sample. For that, each individual x of the sample is sequentially evaluated and it is inserted in the subset when it is not dominated by any other individuals. Besides, if x dominates some other solution y previously inserted in the subset, then y is removed from this subpopulation. Therefore, during the population initialization, despite starting this process by generating a sample with A individuals, not all of them will survive. Each subset stores the current Pareto Front related to its group of objectives and the algorithm works with only first front solutions found for the subproblem related to these objectives. Each new individual generated by crossover is tested to be inserted into all the subsets. For each subset, the dominance is evaluated to decide the inclusion (or not) according to the respective group of objectives. This is the main reason that MEANDS generates only one individual per generation. Besides, each subset is scored according to its contribution to the search process (convergence score). The highest-scoring subsets participate more often in the crossover process. The convergence score is reset at each 100 generations to avoid stagnation. The parent selection has two steps, choosing two subsets and then choosing an individual from each one. The subset is selected by a tournament based on the convergence score and the individual is selected at random from each selected subset. The size of each subset is unlimited and it does not allow to insert duplicate individuals. Figure 1 shows the flowchart of the MEANDS algorithm.

The Dynamic Many-objective Evolutionary Algorithm Based on Nondominated Decomposed Sets (D-MEANDS) (Lafetá and Oliveira, Reference Lafetá and Oliveira2020b) was later proposed to deal with dynamic many-objective problems (DMaOPs). The adaptation made to manipulate dynamic MOPs is that, in the occurrence of changes in the environment, the individuals in the subset related to the set of all objectives - known as archive - are used to repopulate the new subsets. Note that not all individuals extracted from the archive of the previous environment will be reinserted in the new subsets. It depends on the nondominance of each old solution related to the subset of objectives considered in each sub-population. Besides, random individuals are also generated to repopulate the subsets considering the new environment objectives.

Figure 1. Flowchart: evolutionary strategy of MEANDS

4.5. D-MEANDS-MD

The Dynamic Many-objective Evolutionary Algorithm Based on Nondominated Decomposed Sets with Memory and Diversity (D-MEANDS-MD) uses the basic framework of MEANDS (Lafeta et al., Reference Lafeta, Bueno, Brasil and Oliveira2016), including the repopulation strategy used in D-MEANDS (Lafetá and Oliveira, Reference Lafetá and Oliveira2020b) when the environment faces some change. Therefore, D-MEANDS-MD employs a subjacent structure of subsets (or sub-populations) of nondominated solutions. Each sub-population stores the nondominated solutions found by evolutionary process when just a related subset of objectives is considered. For example, in a problem with 4 objectives, each nondominated set is related to one of the possible combinations of a subset of objectives (two by two and three by three) and the last one considers all the 4 objectives. The dominance in each subset is based on the set of objectives for what it refers. The current population corresponds to the union of the nondominated solutions stored in all these sub-populations. The population is initialized with a sample of random individuals of size A, where each individual could be inserted in each subset depending to the nondominance in respect to the current solutions. The individuals with the worst genetic information probably will not be included in any subset. The set that uses all the objectives to define the dominance relation is called archive, because it has a behavior similar to the external archive used in several MOEAs. However, the archive participates of the evolutionary process.

D-MEANDS-MD employs a crossover based on the degree of convergence (Lafeta et al., Reference Lafeta, Bueno, Brasil and Oliveira2016), which generates a single child. In each generation, only one crossover occurs and each descendant generated is tested to be stored in each subset, depending on the respective dominance relation. During the attempt to insert the new descendant, if it is a nondominated solution for the respective subset, it removes the solutions that are dominated by it. In this way some current individuals can be removed during the insertion of a new one. Each sub-population has no size limitation and is classified by a score called the degree of convergence, where the best classified subsets have evolved more than other sets. That is, the sub-populations that have received more inclusions of new individuals in the last generations will receive a greater grade named degree of convergence. This degree is used in the crossover process divided in two stages. First, it selects two distinct sub-populations, where the selection of each one is based on the tournament-4 using the degree of convergence. Second, a parent individual is randomly selected in each subset. In order to prevent the algorithm from becoming biased in choosing the same group of subsets, the degree of convergence is restarted every 100 generations. Figure 2 shows the pseudocode of the proposed D-MEANDS-MD.

Figure 2. Pseudocode of D-MEANDS-MD

Similar to the adaptation incorporated in D-MEANDS (Lafetá and Oliveira, Reference Lafetá and Oliveira2020b) to deal with dynamic optimization problems, when the environment faces any change, the population will be reinitialized but not exclusively at random. When the environment change happens, the individuals in the current archive, which considers all the objectives in the dominance relation, are used to repopulate all subsets of the new population. Therefore, this restocking strategy applied to the event of changes can be seen as a memory strategy, as it conserves part of the old population to start a new one. However, to inject some diversity into such changes, random individuals are also generated to repopulate the subsets considering the new environmental objectives. The number of random individuals is equal to the initial sample size A minus the number of nondominated solutions extracted from the previous environment.

In the previous D-MEANDS (Lafetá and Oliveira, Reference Lafetá and Oliveira2020b) the diversity is applied only when the population resumes right after a disturbance of the environment. However, after our initial experiments using D-MEANDS reported in Section 5.1, it was possible to observe that when a high number of environment changes is employed, the convergence of the population based on the memory of the previous environment could not be as fast as the more dynamic problems need. In such cases, it proved important to inject some randomness over the generations between two changes. The new variation called D-MEANDS-MD employs a crossover with some degree of randomness, driven by a new balance parameter called as $B_{MD}$ . Each crossover has a $B_{MD}$ percentual chance of being employed exactly like D-MEANDS. Otherwise, a new individual is built at random to be the second parent of crossover. All the experiments in this paper uses $B_{MD}$ =90%. Therefore, the proposed MOEA employs both memory and diversity strategies to balance the power of pre-existing nondominated individuals with the diversity of totally new individuals aiming to extract the best performance of the evolutionary search. Section 5.5 presents some experimental results showing the difference obtained when employing this degree of randomness (DMEANDS vs DMEANDS-MD). The percentual used in this balance (10% randomness and 90% memory) was empirically adjusted for the instances of the problem in tackle (DMKP). However, $B_{MD}$ can be tunned for new scenarios or even for new dynamic problems.

5.1. Experiments using $DNSGA-II^*$ , MS-MOEA and D-MEANDS

In this section, the performance of the three algorithms investigated in (Lafetá and Oliveira, Reference Lafetá and Oliveira2020b) are evaluated using a number of environment changes and a number of objectives higher than that previously used. Lafetá and Oliveira showed that the increase in the number of environment changes made DNSGA-II considerably improves its results compared with the other two algorithms. However, the instances were limited to only 4 objectives when 10 environment changes were employed. This limitation was made mainly due to the employment of parametric metrics that greatly increased the computational cost. On the other hand, since non-parametric metrics are used in the present work, it enabled us to analyze the behavior of these algorithms in more complex scenarios up to instances with 8 objectives and 20 environment changes.

Figure 3 shows the results of the AD metric for all the 27 scenarios. As can be noted, D-MEANDS has the best values in almost all instances. The exceptions are three scenarios with 30 items: one involving 6 objectives (EC=15) and two with 8 objectives (EC=15 and EC=20). Even in these 3 scenarios D-MEANDS found results similar to the best found by MS-MOEA. By comparing MS-MOEA and $DNSGA-II^*$ , the second outperforms the first in some instances (eight of them), but MS-MOEA is better than $DNSGA-II^*$ in the most of the 27 scenarios. Based on AD results, it is possible to conclude that D-MEANDS finds more distributed sets of solutions.

Figure 4 shows the HV metric results. In 9 out of the 27 instances, the $DNSGA-II^*$ outperforms the other algorithms. These scenarios use 4 or 6 objectives. However, in the majority of scenarios (including all the 9 scenarios with 8 objectives), D-MEANDS achieved the best performance. It was possible to notice that the increased number of objectives highlights D-MEANDS performance. On the other hand, it can also be observed that the increased number of environment changes decays D-MEANDS metrics in respect to the other MOEAs. Moreover, the increment of items to be packed also contributes to the decay on D-MEANDS performance. Comparing MS-MOEA and $DNSGA-II^*$ , the first outperforms the later in most HV results.

The T-test (Mankiewicz, Reference Mankiewicz2000) was used to assess the significance of the D-MEANDS performance compared with the others and the results are shown in Tables 1, 2, and 3. This hypothesis test was done with 99% confidence. Each row of the tables represents the comparison of D-MEANDS with other algorithm and that the column indicates the metric. Green cells show that D-MEANDS was significantly higher and red cells show it was significantly lower. In white, there was no significant difference. We can see the results in Table 1 for instances with EC=10. Tables 2 and 3 show similar results for EC=15 and EC=20, respectively. The hypothesis test confirms that there are significant evidences of the superiority of D-MEANDS in AD metric in almost all scenarios and that, in general, D-MEANDS returns a better performance in HV metric.

The execution time of the algorithms is shown in Figure 5. Clearly, the $DNSGA-II^*$ processing time is much lower than the other algorithms. This observation becomes more evident with the increasing complexity of the instances. D-MEANDS has an execution time close to that of $DNSGA-II^*$ in the instances of 4 objectives, but as the number of objectives increases, its execution time increases much more than $DNSGA-II^*$ . D-MEANDS processing time is shorter than that of MS-MOEA in all instances of 4 and 6 objectives but they are close when formulations with 8 objectives were used (D-MEANDS is still faster in about half of scenarios).

D-MEANDS found better AD and HV results in most instances. It also scales better when the number of objectives is increased up to 8. However, as we increase the number of environment changes (EC) and the number of items to be packed, which represents more difficult instances, D-MEANDS performance decays when compared with the other algorithms. On the other hand, $DNSGA-II^*$ proved to be very competitive in some instances demanding a lower execution time, even when the number of objectives is increased.

Figure 4. Hypervolume results (HV). The figures from (1) to (9) are the results of 4 objectives. The figures from (10) to (18) are the results of 6 objectives. The figures from (19) to (27) are the results of 8 objectives

Table 1. Hypothesis test T of the instances with EC=10. Comparing D-MEANDS vs All algorithms

Table 2. Hypothesis test T of the instances with EC=15. Comparing D-MEANDS vs All algorithms

Figure 5. Time Execution. The figures from (1) to (9) are the results of 4 objectives. The figures from (10) to (18) are the results of 6 objectives. The figures from (19) to (27) are the results of 8 objectives

5.2. Experiments including D-MEANDS-MD

We investigated the possible causes for D-MEANDS performance decay when the number of environment changes are increased. It was possible to observe that in such scenarios, the convergence of this MOEA is not so fast as needed mainly because there is a severe decay in the variability of the nondominated solutions, since the basic MEANDS and D-MEANDS frameworks discard all the information of dominated solutions. Therefore, it becomes clear the need of a new mechanism to inject some randomness what led us to the proposition of D-MEANDS-MD explained in Section 4.5.

Table 3. Hypothesis test T of the instances with EC=20. Comparing D-MEANDS vs All algorithms

In this session, the proposal D-MEANDS-MD is confronted with the other MOEAs. From the previous section, we repeat the values of $DNSGA-II^*$ and D-MEANDS metrics to investigate the impact of the implemented modification in respect to the two best MOEAs, being that DNSGA-II has been evidenced by its lower computational time while D-MEANDS returned the best performance in AD and HV metrics. Moreover, we also implemented a variation of $DNSGA-II^*$ called here $DNSGA-II^+$ where the same diversity mechanism proposed for D-MEANDS-MD was incorporated. That is, 10% of crossover matings in $DNSGA-II^+$ use a random parent. In Table 4 we can see the parameters used by D-MEANDS and D-MEANDS-MD.

Table 4. D-MEANDS and D-MEANDS-MD parameters

Figure 6. Diversity results (AD). The figures from (1) to (9) are the results of 4 objectives. The figures from (10) to (18) are the results of 6 objectives. The figures from (19) to (27) are the results of 8 objectives

Figure 7. Hypervolume results (HV). The figures from (1) to (9) are the results of 4 objectives. The figures from (10) to (18) are the results of 6 objectives. The figures from (19) to (27) are the results of 8 objectives

Figure 6 shows the results related to AD metric: D-MEANDS-MD performed better than all the other algorithms, followed by the original D-MEANDS performance. $DNSGA-II^+$ showed no improvement compared with $DNSGA-II^*$ . Analyzing the results of the HV metric in Figure 7, D-MEANDS-MD overcame all the other algorithms, followed by D-MEANDS performance. As observed with the other metric, $DNSGA-II^+$ showed no improvement compared with $DNSGA-II^*$ .

Table 5. Hypothesis test T of the instances with EC=10. Comparing D-MEANDS-MD vs All algorithms

Table 6. Hypothesis test T of the instances with EC=15. Comparing D-MEANDS-MD vs All algorithms

Table 7. Hypothesis test T of the instances with EC=20. Comparing D-MEANDS-MD vs All algorithms

Therefore, the introduction of a randomness mechanism is $DNSGA-II^*$ did not return any significant improvement. On the other hand, this mechanism had a positive robust impact on D-MEANDS. The D-MEANDS-MD proposal improved the results of all instances in both metrics. As a consequence, the new algorithm overcame all the other algorithms.

The T-test was used to assess the significance of D-MEANDS-MD results compared with the others. This hypothesis test was done with 99% confidence and one can see the results in Tables 5, 6, and 7 for instances with EC=10, EC=15, and EC=20, respectively. Each row of the table represents the comparison of D-MEANDS-MD with other algorithm and the column indicates the metric. It is possible to observe that the proposal D-MEANDS-MD significantly outperforms all the other algorithms in all 27 investigated instances.

Figure 8 shows the processing time related to each investigated MOEA. D-MEANDS and D-MEANDS-MD were faster than the $DNSGA-II^*$ and $DNSGA-II^+$ in some instances using 4 objectives. But when using formulations with 6 or 8 objectives, $DNSGA-II^*$ and $DNSGA-II^+$ are clearly faster. The increase in the number of objectives shows that D-MEANDS-MD takes a longer processing time than D-MEANDS. However, D-MEANDS-MD execution time proved to be similar to D-MEANDS: it has a much greater increase in processing time than the two algorithms $DNSGA-II^*$ and $DNSGA-II^+$ as we increase the number of objectives. Therefore, although D-MEANDS-MD showed the best performance in terms of multiobjective metrics, it is as time expensive as its precursor D-MEANDS.

Additionally, we speculated if $DNSGA-II^*$ would achieve better results than D-MEANDS and D-MEANDS-MD if have extra time to perform its evolutionary search. Aiming to answer this question we ran extra experiments, in which we increased the number of generations of $DNSGA-II^*$ . For instances of 30 items, we multiply by 5 the number of generations of $DNSGA-II^*$ , for 50 items we multiply by 10 and for 100 items we multiply by 40. With such increased number of generations, in all instances DNSGA-II* surpassed the execution time of the other two algorithms. We redid the experiments of this section comparing $DNSGA-II^*$ with D-MEANDS and D-MEANDS-MD, while the parameters of D-MEANDS and D-MEANDS-MD are kept fixed. Due to lack of space, we did not include these new results here. However, we could confirm that $DNSGA-II^*$ did not surpass D-MEANDS and D-MEANDS-MD in AD and HV metrics even using extra processing time.

5.3. Experiments including DDIS-MOEA/D-DE

Aiming at comparing D-MEANDS-MD and a more recent algorithm in the literature, a new experiment was carried out. The DDIS-MOEA/D-DE presented in Section 4.3, uses the same evolutionary strategy as the MOEA/D-DE with a dynamic strategy based on diversity. The chosen algorithm was proposed recently and according to the authors in (Liu et al., Reference Liu, Peng, Liu and Liu2020) has returned good results in a controlled and continuous dynamic environment. The following parameters were used in DDIS-MOEA/D-DE: 500 generations, 100% crossover, 10% mutation, population size = 100, and 20% of its population mutates during the change of environment. $DNSGA-II^+$ and D-MEANDS were also maintained in order to compare the performance of DDIS-MOEA/D-DE with another algorithm that uses dynamic strategy based on diversity ( $DNSGA-II^+$ ) and the precursor of D-MEANDS-MD (D-MEANDS). In this experiment, only instances with EC=20 were considered.

Figure 8. Time Execution. The figures from (1) to (9) are the results of 4 objectives. The figures from (10) to (18) are the results of 6 objectives. The figures from (19) to (27) are the results of 8 objectives

Figure 9 shows the AD metric achieved by each DMOEA in the investigated scenarios. DDIS-MOEA/D-DE presents worse results than $DNSGA-II^*$ , DMEANDS and DMEANDS-MD in all instances. In Figure 10 we can see the results of the HV metric, where the DDIS-MOEA/D-DE also presents the worst results in all instances compared with the other algorithms. The execution time of the algorithms is shown in Figure 11. As observed in the last section, D-MEANDS and D-MEANDS-MD significantly worsen their execution time as the number of objectives increases. For 4 and 6 objectives, both algorithms get a shorter time than DDIS-MOEA/D-DE, but they demand more processing time with 8 objectives. Thus, we can see that DDIS-MOEA/D-DE presents a better behavior than DMEANDS and DMEANDS-MD in respect to the execution time as the number of objectives increases. However, when compared with $DNSGA-II^*$ it demands more processing time in all instances. Therefore, we can see that between the two algorithms that use dynamic strategy based on diversity, $DNSGA-II^*$ performed better in DMKP.

Figure 9. Diversity performance for instances with EC=20. Graphs from (1) to (3) show the results for 4 objectives. Graphs from (4) to (6) show the results for 6 objectives. Graphs from (7) to (9) show the results for 8 objectives

Figure 10. Hypervolume performance for instances with EC=20. Graphs from (1) to (3) show the results for 4 objectives. Graphs from (4) to (6) show the results for 6 objectives. Graphs from (7) to (9) show the results for 8 objectives

Figure 11. Time Execution performance for instances with EC=20. Graphs from (1) to (3) show the results for 4 objectives. Graphs from (4) to (6) show the results for 6 objectives. Graphs from (7) to (9) show the results for 8 objectives

Although not presented in the present paper, similar experiments were performed using the execution time as stop criteria (instead of number of generations). In such experiments, each execution stopped for a time limit of 30 seconds. The main conclusions were: (i) D-MEANDS-MD performed better in HV and AD metrics than the other MOEAs and (ii) D-MEANDS-MD and $DNSGA-II^*$ outperforms DDIS-MOEA/D-DE. However, in these supplementary experiments, we could not guarantee that all the algorithms were submitted to exactly the same number of environmental changes. In future investigations, we intend to implement a new framework for experiments where it will be able to compare the MOEAs with the same execution time and the same number of environmental changes.

The T-test was used to assess the significance of D-MEANDS-MD performance compared with the other investigated algorithms. The test was done with 99% confidence and the results are presented in Table 8 for instances with EC=20. Each column of the table represents the comparison of D-MEANDS-MD with other algorithm with respect to number of the items, and the row indicates the metric and number of objectives considered. It is possible to note that the D-MEANDS-MD is significantly superior to the other algorithms in all instances.

Table 8. Hypothesis test T of the instances with EC=20. Comparing D-MEANDS-MD vs All algorithms

5.4. Analyzing the dominance of the final set of solutions

In the experiments reported in the last sections, the MOEAs were evaluated regarding AD and HV metrics, which give absolute values to make a comparison among several algorithms. These metrics translate some characteristics of the Pareto approximation found by a specific algorithm, such as diversity and convergence to Pareto Optimum, to numerical values that make these multiple comparison possible. However, its is common in multiobjective investigation, to perform a direct comparison of two algorithms regarding the quality of the set of solutions found by each one. In this section, the Coverage of two Sets (CS) metric is used to perform a direct comparison between the algorithms with the best performances on the last experiments: $DNSGA-II^*$ , D-MEANDs and D-MEANDS-MD.

CS definition: (Zitzler, Reference Zitzler1999): Consider two Pareto sets X and Y. Function F evaluates the dominance relationship of Pareto X to Y, returning a value between 0 and 1. If $F=1$ means that all solutions of Y are dominated by some point of X, while $F=0$ means that no solutions of Y are dominated by X. Note that it does not mean that there is some point of Y that dominates a solution of X. Formally, the CS metric is calculated as follow:

(8) \begin{align}F(X,Y)=\frac{|{y\epsilon Y | \exists x \epsilon X \,:\, x \succeq y}|}{|Y|}\end{align}

This experiment was performed using the scenarios with the higher number of changes, that is, using only EC=20 instances. Each result represents the mean of 100 comparisons between the Pareto set of two algorithms (X and Y). Tables 9, 10, and 11 present the values of the CS metric achieved using the DMOEAs in scenarios with 30, 50, and 100 items, respectively. The row represents the Pareto set X and the column represents the set Y. As closer the value is to 1, as more solutions of the column algorithm (Y) are dominated by solutions of the row algorithm (X). Each algorithm was represented by a lowercase letter: a= $DNSGA-II^*$ , b=D-MEANDS and c=D-MEANDS-MD. We can see in Table 9 (instances of 30 items), $DNSGA-II^*$ finds very few solutions that dominate the solutions of D-MEANDS and D-MEANDS-MD. On the other hand, about 70% of the $DNSGA-II^*$ solutions are dominated by the DMEANS and D-MEANDS-MD solutions. Regarding D-MEANDS-MD and D-MEANDS the CS difference is not so evident (between 2% and 4%). However, in all scenarios D-MEANDS-MD have more solutions dominated by D-MEANDS than the opposite. In the instances of 50 (Table 10) and 100 items (Table 11), as the number of items increases, as more solutions of $DNSGA-II^*$ are dominated by the other two algorithms (around 80% and 90% in all instances of 50 and 100 items, respectively). The performances of the D-MEANDS and D-MEANDS-MD are similar to that observed with 30 items. The difference in CS is not so evident between both D-MEANDS versions (up to 8% of difference). However, in all scenarios this slight difference favours D-MEANDS.

Table 9. CS metric in instances of 30 items and EC=20, using a= $DNSGA-II^*$ ; b=D-MEANDS; c=D-MEANDS-MD algorithms

Table 10. CS metric in instances of 50 items and EC=20, using a= $DNSGA-II^*$ ; b=D-MEANDS; c=D-MEANDS-MD algorithms

Table 11. CS metric in instances of 100 items and EC=20, using a= $DNSGA-II^*$ ; b=D-MEANDS; c=D-MEANDS-MD algorithms

The CS metric compares two multiobjective algorithms and gives us an idea of which one presents a better convergence to the Pareto Optimum. This is a relative result because it does not quantify the magnitude of this difference on convergence. Nevertheless, one can see that D-MEANDS presents a better convergence to Pareto than D-MEANDS-MD. Therefore, a question arose after obtaining these results: why is D-MEANDS-MD better in the HV metric than D-MEANDS, as showed in Section 5.2, if CS results show that D-MEANDS has a better convergence? We believe that the answer is related to the D-MEANDS-MD mechanism to increase diversity in the population (and indeed diversity is of great importance to dynamic problems). On the other hand, the hypervolume is influenced by both convergence and diversity, being that the greater the convergence and diversity of an algorithm, the better the value of HV. Therefore, we conclude that D-MEANDS-MD presents a better diversity and a worse convergence than D-MEANDS. However, we speculate that the difference in diversity is more significant than the difference on convergence, at least for their contribution to HV calculus, resulting in significant higher values of HV when using the new version of D-MEANDS proposed in the present work. Although the hypervolume is the most widely metric used to evaluate multiobjective algorithms (because it is non-parametric and gives this mixed perspective of diversity and convergence), in the future, it could be interesting to use other convergence metrics to have an absolute measure of how much D-MEANDS convergence is higher than D-MEANDS-MD. For example, by calculating the convergence metric discussed in (Deb and Jain, Reference Deb and Jain2002).

Figure 12. Parallel coordinates of the solutions found by D-MEANDS-MD (black dots and lines) and DDIS-MOEA/D-DE (blue) in their best runs (out of 100). DMKP instance with 4 objectives, 100 items and EC=20. The reference W0 used for HV calculus is also represented in red

Figure 13. Parallel coordinates of the solutions found by (a)(c)(e) D-MEANDS-MD and (b)(d)(f) D-MEANDS in their best runs (out of 100). DMKP instance with 100 items and EC=20. The graphs (a) (b) refer to the instance of 4 objectives, (c) (d) of 6 objectives and (e) (f) of 8 objectives

Figure 14. Average of the HVs of the start and end of each environment of 100 executions of D-MEANDs and D-MEANDS-MD in the instance of 6 objectives 30, 50 and 100 items with EC=20

5.5. Deeping the hypervolume analysis

Sections 5.2 and 5.3 show the superiority of D-MEANDS-MD over the other algorithms regarding the hypervolume metric when submitted to the same number of fitness evaluations. In this section we will continue to analyze the hypervolume but using other information to deeper evaluate these numerical results. Figures 12 and 13 show parallel coordinates graphs, which are commonly used to visualize set of solutions in multiobjective problems with many objectives (more than 3). Figure 12 shows two sets of final solutions: the black lines and points represent the final Pareto approximation found by D-MEANDS-MD in the best run in respect to HV, and the light blue lines and points represent the final set found by the best run of DDIS-MOEA/D-DE. They were obtained in the DMKP instance with 4 objectives, 100 items and EC=20. Besides, the graph also represents the reference solution (in red) used for HV calculus: w0=(0,0,0,0). Each line represents one evolved solution and it links the four values found in each objective. The axis X, Y, W, and Z represent the dimensions of the four objectives involved in the problem. The hypervolume calculated for the set found by D-MEANDS-MD is HV=4.4738E+7, while for the DDIS-MOEA/D-DE is HV=2.0549E+7. Corroborating that their hypervolumes are very different (D-MEANDS-MD value is much higher), it is clear to observe in the graph of Figure 12 the differences between the two Pareto approximations. It also makes clear that, despite the numerical values of the hypervolume, the set found by D-MEANDS-MD is clear better than the other.

However, when we try to visualize closer results, the analisys is not so clear, because the solutions from the two sets may mix in the graph. It was what happened when we try to compare the sets of solutions found by D-MEANDS-MD and D-MEANDS in the same instance of DMKP: 4 objectives, 100 items and EC=20. The hypervolume calculated for the set found by D-MEANDS is HV=3.9697E+7 (for D-MEANDS-MD, as pointed early, is HV=4.4738E+7). Therefore, the results are plotted in separate graphs, Figure 13(a) for D-MEANDS-MD and Figure 13(b) for D-MEANDS. Moreover, we noticed that when including w0=(0,0,0,0) in such graphs, the comparison of the final sets are more difficult to do. The inclusion of w0 causes the graphs scales to mix the solutions embarrassing the comparison. Therefore, w0 was not included in Figure 13.

Comparing the sets of solutions in Figure 13(a) and (b) it is possible to observe that in x and w dimensions, both sets appear to occupy a similar interval of objective values. However, considering the z dimension, the set found by D-MEANDS-MD (Figure 13(a)) stands out with a much higher values than the other (Figure 13(b)). On the other hand, considering the y dimension, the D-MEANDS-MD tends to occupy a little slower interval than the other, but the difference is not significant as the observed in dimension z. Therefore, the difference in dimension z was more preponderant in the HV calculus resulting in a higher value for the set of solutions found by D-MEANDS-MD. It is also important to highlight that the set found by D-MEANDS-MD presents points more distributed in the axis. This spreading of points causes the hypervolume to be higher, once HV calculus also represents the diversity of the final solutions, because this decreases the overlapping area of the hypercubes. This observation combined with the verification in the last subsection (by using CS metric) that in general D-MEANDS-MD does not dominate more solutions of D-MEANDS make us to conclude that both version of DMEANDS algorithm have similar convergence to the Pareto Front but the D-MEANDS-MD presents a better diversity which makes higher values of HV and AD metrics. This is a direct consequence of the new strategy of diversity included in D-MEANDS-MD.

Increasing the number of objectives returns a set of non-dominated solutions of high cardinality, making it difficult to visually analyze the graphs. However, the graphs with 6 and 8 (Figure 13) objectives were presented only for a better understanding of the shape of the curves in these other spaces. Overall, the two approaches are similar, making it difficult to visually determine which was superior. However, the HV metric shows that DMEANDS-MD performed better than its predecessor.

A final analysis was performed to investigate the change promoted by the insertion of the diversity strategy in D-MEANDS-MD considering the intervals between consecutive environmental changes. In Section 5.2, it is said that the modification implemented in D-MEANDS-MD causes the convergence to sets with better solutions in the evolutionary process observed from one EC to the other. A final experiment was designed to show that this MOEA is able to find populations with higher hypervolume in the interval between two ECs. Figure 14 presents three graphs comparing D-MEANDS-MD and D-MEANDS hypervolume performance relative to three DMKP instances with 6 objectives, where the number of items are 30, 50, and 100, respectively. Each algorithm was performed 100 times. Each run has a total of 100 generations and one environmental change was applied at each 25 generations, resulting in three ECs referred here as $C_{25}$ , $C_{50}$ , and $C_{75}$ . The hypervolume for the current population was calculated in some points of the evolutionary search: (a) at the start of the run (initial population); (b) before the occurrence of each EC; (c) after the reinitialization of the population; and (d) at the end of the run (final solution). It results in eight control points where HV was calculated in each run. The graphs in Figure 14 present the average of HV calculated in each of these eight control points for each MOEA.

Considering the first graph (DMKP instance with 30 items), both algorithms start from similar values of HV (P1) but at the end of the first environment (P2), after 25 generations, its is possible to observe that the HV related to the D-MEANDS-MD solutions are higher (in average). After the first change ( $C_{25}$ ) and reinicialization (P3) in both MOEAs the HV value decays, as expected. However, in general, D-MEANDS-MD tends to generate a population with a slight higher HV at the beginning of the next environment, since it departs from a better set before the EC occurrence. For this reason D-MEANDS-MD reinitializes in point P3 generating populations with a slight higher HV value (compared to D-MEANDS value). In the other ECs the behavior is similar: in the points P4 and P6 (after 25 generations from the previous EC) the HV value achieved in D-MEANDS-MD is clearly higher than D-MEANDS. Moreover, after $C_{50}$ and $C_{75}$ changes in points P5 and P7, D-MEANDS-MD departs from sets of solutions with higher HV than D-MEANDS population. As a consequence, at the end of the run (P8), D-MEANDS-MD population in average has HV values higher than D-MEANDS. Similar observations can be done for the other two graphs in Figure 14 (50 and 100 items). Since in all intervals of evolution between consecutive ECs, D-MEANDS-MD found solution sets with higher HV than those found by D-MEANDS, we can point that D-MEANDS-MD strategy exhibits a better convergence to fronts with higher HD at each environment.