2.1. Modeling approach

Building energy models (BEMs) are most commonly classified in relation to their construction based on physical or data-driven approaches. Approaches based on explicit physical relations are commonly defined as “white-box” (WB) models, while purely data-driven models are named “black-box” (BB) models. In between these, semi-physical models or “gray-box” (GB) models are placed. Each one has its own advantages and caveats depending on the particular application.

WB models are built based on the explicit formulation of energy flows and balances in buildings. Complex models are built based on the aggregation of a large number of equations, each of which models a small subset of the energy system. With initial developments more than 50 years ago, WB models have grown in complexity and scale, and several simulation tools, such as Energy Plus (National Renewable Energy Laboratory (NREL), 2017), TRNSYS (University of Wisconsin—Madison Solar Energy Laboratory, 1975), ESPr (University of Strathclyde. Energy Systems Research Unit, 2002), and so forth, are accepted by the Building Energy Simulation community, mostly for their use in the design of new buildings. In recent years, specific libraries (Lawrence Berkley National Laboratory, 1998) for the general-purpose Modelica language (Modelica Association, 1997) have also gained popularity, particularly for HVAC and control applications.

One of the advantages of WB models is that they are built based on design data and do not require historical data to develop the model. This is in line with their main applications as support tools for building designers but poses problems as there is a frequently documented energy “performance gap” (Johnston et al., Reference Johnston, Miles-Shenton and Farmer2015; Zou et al., Reference Zou, Xu, Sanjayan and Wang2018) from the design to the post-construction phase. This gap is typically in the range of a factor of 2.

Although not originally designed for this purpose, WB models have also been used as simulation environments for building control applications. Energy Plus is commonly used as a unique simulator (Kwak et al., Reference Kwak, Huh and Jang2015; Jia et al., Reference Jia, Jin, Sun, Hong and Spanos2019; Brandi et al., Reference Brandi, Savino Piscitelli, Martellacci and Capozzoli2020; Coraci et al., Reference Coraci, Brandi, Hong and Capozzoli2023, Reference Coraci, Brandi, Hong and Capozzoli2024) or as part of larger cosimulation environments (Cucca and Ianakiev, Reference Cucca and Ianakiev2020; Brandi et al., Reference Brandi, Coraci, Borello and Capozzoli2022). This software is also used when scaling up simulation environments to deal with city-scale applications (Pinto et al., Reference Pinto, Deltetto and Capozzoli2021a; Pinto et al., Reference Pinto, Piscitelli, Vázquez-Canteli, Nagy and Capozzoli2021b), which, in some cases, require developing surrogate models (Pinto et al., Reference Pinto, Deltetto and Capozzoli2021a). In many cases, several WB models are integrated into a cosimulation environment, where EnergyPlus is the core tool for building physics, and other systems are incorporated for building occupancy (Naspi et al., Reference Naspi, Arnesano, Stazi, D’Orazio and Revel2018; Jia and Srinivasan, Reference Jia and Srinivasan2020) and HVAC modeling (Cucca and Ianakiev, Reference Cucca and Ianakiev2020). Here, cosimulation environments and tools to embed WB models as functional mockup units (Lawrence Berkley National Laboratory, 2019) have substantially facilitated the integration of WB models among them and with other general-purpose programming languages.

Modelica is typically the most used tool for the modeling of HVAC and controls, sometimes also performing the (simplified) modeling of buildings (Guo et al., Reference Guo, Fu, Liu and O’Neill2024) or by building cosimulation environments with EnergyPlus (Cucca and Ianakiev, Reference Cucca and Ianakiev2020). Although the main trend is to use the above-mentioned tools, similar approaches have been used to integrate other models, such as IES Virtual Environment (IESVE) (Naspi et al., Reference Naspi, Arnesano, Stazi, D’Orazio and Revel2018), or to couple models with real-life pieces of equipment under hardware-in-the-loop (Calfa et al., Reference Calfa, Yang, Li, Chen, O’Neill and Wen2023) approaches.

The power of WB models resides in the capacity to create virtual models of any building and energy system based only on information already available during the preliminary design stage of buildings. Although this has made it popular for the development of simulation environments for research purposes, in most research works, the quality of the simulation environment has not been checked against real building performance data (Mason and Grijalva, Reference Mason and Grijalva2019) or accuracy has been validated only at aggregated levels, which provides potential limitations to the performance and applicability of control applications investigated on top of them.

Fully opposed to WB, BB models do not require knowledge of the configuration of the physical system and are built only based on historical data of input and output variables of the environment (building) to be modeled.

BB models are used by authors when WB or GB models cannot be used due to the absence of engineering information or large scale of integrated phenomena, for example, for the estimation of energy loads of a District Heating (DH). Lumbreras et al. (Reference Lumbreras, Garay-Martinez, Arregi, Martin-Escudero, Diarce, Raud and Hagu2022) present a procedure in which a heat load prediction model in buildings is performed based on data from smart meters with hourly resolution. By applying a set of different statistical procedures (interquartile range filtering criteria, remove nonvalid values, etc.) for preprocessing data sources, doing an extensive data analysis (detection of different heating patterns in different buildings and time periods) and training different ML algorithms (decision trees and multivariance regressions) authors created a valid general dataset that contains historical energy usage patterns that are used to feed a relatively simple model with no prior knowledge of the building.

In Lopez-Villamor et al. (Reference Lopez-Villamor, Eguiarte, Arregi, Garay-Martinez and Garrido-Marijuan2024), the authors present a methodology to create a BB model that can be used to train autoregressive models to predict thermal loads in commercial buildings and optimize their thermal consumption. A set of historical weather data, occupancy information, and indoor set point temperatures are used to develop individual predictions for 168 data clusters, each representing one individual time of the week. Each cluster is used to train an independent autoregressive exogenous sub-model to calibrate the BB model. The model is validated using a typical k-fold cross-validation procedure, and it is compared against other models with a lesser complexity.

Liang et al. (Reference Liang, Zhu, Chen, Chen, Jin and Du2023) present a unified framework for endowing BB models with rejection ability. Authors explain that the obtained data for creating a BB model may not be useful if the collected data for creating it are limited to certain operational conditions or outside the scope of training data. For that reason, the rejection methodology rejects the possible outcomes that are not reliable for being used in a prediction methodology, ensuring that the created BB model is going to be more precise for developing control-oriented applications.

The accuracy of these models is highly reliant on the amount and quality of the data. Previous works have showcased the relevance of social behavior (Zhan et al., Reference Zhan, Liu, Chong and Yan2020; Lumbreras et al., Reference Lumbreras, Diarce, Martin, Garay-Martinez and Arregi2023) and climate patterns (Lumbreras et al., Reference Lumbreras, Diarce, Martin, Garay-Martinez and Arregi2023; Borgato et al., Reference Borgato, Prataviera, Bordignon, Garay-Martinez and Zarrella2024; Hajri et al., Reference Hajri, Garay-Marinez, Macarulla and Ben Sassi2024) as key information leading to different energy performances of buildings. Thus, data availability should be properly balanced to develop BB models suitable for modeling energy performance for all this variability.

A key issue when developing a BB model for control application is that observations should properly represent the variation not only of model disturbances (climate, occupancy, etc.) but also of control inputs to the system (i.e., temperature set points, activation of fans, etc.). This last item is commonly difficult to achieve, as historical observations are commonly biased due to the existing operational criteria (i.e., fixed set-points and operational schedules).

GB models are between the previous approaches (Li et al., Reference Li, O’Neill, Zhang, Chen, Im and DeGraw2021). A reduced set of formulas with physical correspondence is defined, and its parameters are calibrated with operational data. With this approach, GB models integrate the best of WB and BB models as they are physically significant and fit the data at the same time. GB models are reduced-order models that capture the main dynamics of buildings, where the resulting performance of many energy transfer paths is formulated. Reference works in building physics model building performance in equations (Bacher and Madsen, Reference Bacher and Madsen2011; Wang et al., Reference Wang, Chen and Li2019), and procedures have been developed to identify model structures and calibrate parameters automatically (Leprince et al., Reference Leprince, Madsen, Miller, Real, van der Vlist, Basu and Zeiler2022a; Leprince et al., Reference Leprince, Miller, Madsen, Basu, van der Vlist and Zeiler2022b), as well as integration into building control systems (De Coninck et al., Reference De Coninck, Magnusson, Åkesson and Helsen2016; Blum et al., Reference Blum, Wang, Weyandt, Kim, Wetter, Hong and Piette2022).

In most cases, GB models are represented by using a Resistance-Capacitance (RC) structure, where resistance represents energy transfer and capacitance represents energy storage. Recent identification processes driven by statistical significance show that models with 2–3 capacitances are suitable to represent the heat dynamics in buildings (Leprince et al., Reference Leprince, Madsen, Miller, Real, van der Vlist, Basu and Zeiler2022a).

Although GB models are suitable for energy performance modeling, they are most focused on the performance of the structural components in buildings and do not typically consider the interaction with HVAC systems and social behavior associated with building occupants and their associated loads (i.e., lighting). With some complex exceptions (De Coninck and Helsen, Reference De Coninck and Helsen2016; Blum et al., Reference Blum, Wang, Weyandt, Kim, Wetter, Hong and Piette2022), these are either left out of the models or integrated in a very simplistic manner.

In this work, we take advantage of the simplicity of GB models to create the basis for building energy performance simulators. We complement the GB model with specific additions to integrate HVAC and occupancy modeling.

There is an underlying discussion on the time and expertise required to build a suitable energy model, where WB and GB models are commonly attributed to require greater efforts in this process, but this has been substantially reduced in recent times by means of automated WB model generation procedures and automated GB model identification and calibration techniques. In our case, we trust that already stable models such as (Bacher and Madsen, Reference Bacher and Madsen2011) can be directly used in our environment, and model calibration approaches such as those developed in (Leprince et al., Reference Leprince, Madsen, Miller, Real, van der Vlist, Basu and Zeiler2022a; Leprince et al., Reference Leprince, Miller, Madsen, Basu, van der Vlist and Zeiler2022b) will be able to generate models for other buildings in an efficient way.

2.2. Implementation

Intelligent energy operators are commonly implemented by using informatics tools. These tools try to mimic human behavior by doing intelligent decision-making processes in which the data from the environment is understood and suitable actions are executed. However, to be able to implement these processes (commonly named intelligent agents), valid data from the target environment and the installed building active system should be provided. Thus, researchers could use the information from the developed physical models to use them as the backbone of their technical implementations. The models will play a critical role in the creation of these agents because they will be able to provide solid and valid data to fit the decisions of the agent according to the current conditions of the target environment.

Depending on the physical model approach (WB, GB, and BB) and the informatics programming language (Java, Python, C, etc.) used, different technical tools and architectures should be considered.

WB-based solutions are developed by using third-party applications (simulation tools such as EnergyPlus or Modelica). This implies that a logical connection between the model and the technical implementation is required. The most common approach is usually performing a cosimulation, where the logical code is written and connected with the simulation environment. There is a need to develop/parametrize a custom model within the simulation environment (a quite expert task) or rely on predefined configurations.

Simulation tools are commonly written in C or C++. Considering the preeminence of Python as a programming language in the scientific community, authors have tried to create “bridges” to train their control-oriented implementations.

For example, Sinergym (Campoy-Nieves et al., Reference Campoy-Nieves, Manjavacas, Jiménez-Raboso, Molina-Solana and Gómez-Romero2025) is a Python library allowing a direct connection with Energy Plus to develop RL algorithms by using the Gymnasium library (Towers et al., Reference Towers, Kwiatkowski, Terry, Balis, De Cola, Deleu, Goulão, Kallinteris, Krimmel, Perez-Vicente, Pierré, Schulhoff, Jet Tai, Tan and Younis2024) and interact at each time step. On each RL episode, a new EnergyPlus instance is created, and interaction is performed until the episode finishes. Sinergym has drawbacks linked to the complexity of installation and configuration of the system. Although a Docker image is supplied, there is still a need to install several additional components. Additionally, the training procedure is computationally intensive due to the need to execute several instances of EnergyPlus.

ALFALFAFootnote ¹ is a web-based application for running WB models from different simulation engines (EnergyPlus, Modelica, etc.) that allows uploading custom models using a web-based interface or Python programming code and executing these with various configurations (e.g., simulation start time, simulation end time, etc.). The configuration of “external clock” allows defining how the model advances in time through a “Stepping manner,” which is an important feature for ML developments. It also allows for modifying model input points by using a Python dictionary (e.g., set a value of an outside air damper to 0.7 from the original Energy Plus model). ALFALFA requires that a full server–client architecture is deployed, a specific format is used (Open Studio Workflow, OSW) with its own format and file configuration, and that manual efforts are performed to connect simulations with developments.

BOPTEST (Blum et al., Reference Blum, Arroyo, Huang, Drgoňa, Jorissen, Walnum, Chen, Benne, Vrabie, Wetter and Helsen2021) is a framework with many similarities to ALFALFA, but a more generalized purpose system to focus on benchmarking building control performance over pre-defined scenarios. The main difference between BOPTEST and ALFALFA is that the former is focused only on testing scenarios, and the latter is focused on training and testing scenarios. Despite being a good tool for rapid-testing purposes, it has the same problems as ALFALFA and does not provide options to customize training or processes or customize BEMs outside the provided repository of test cases.

Advanced Controls Test Bed (ACTB)Footnote ² is presented as a middle solution between BOPTEST and ALFALFA. ACTB develops a “high-fidelity” model by coupling building envelope and load modeling (EnergyPlus) with HVAC models and controls (Modelica). Interfaces are provided for developing MPC or RL controls. Although the software is complete in terms of modeling and control execution, the usage becomes more complicated than the previously presented systems, as two independent simulation tools need to be configured. Also, this environment is in early development stages.

The previously analyzed frameworks were used in several contributions. For example, Manjavacas et al. (Reference Manjavacas, Campoy-Nieves, Jiménez-Raboso, Molina-Solana and Gómez-Romero2024) used the Sinergym library to compare the performance of different RL implementations with various buildings and climate conditions.

Kang et al. (Reference Kang, Velazquez and Kane2024) utilized ALFALFA to create a Building Operations Emulator system to allow Building BEMs to control the building elements as if they were controlling them in a real scenario. A bridge from ALFALFA is used to obtain the data points of the BACnet standard protocol and allow the direct interaction between a BEM and a virtual building. They demonstrate the potential to obtain realistic responses from virtual buildings to train control-oriented applications.

Yang et al. (Reference Yang, Filonenko, Arendt and Veje2020) used BOPTEST tool to evaluate an MPC controller where a Modelica model was used to simulate electric loads and thermal dynamics of a building and its HVAC system, and which factors affect the performance of the proposed MPC.

Dey et al. (Reference Dey, Marzullo, Zhang and Henze2023) used ACTB to test two RL algorithms against a rule-based controller in a multi-zone building energy simulation. The authors reported the necessity of using a specific connector to effectively connect their RL implementation with the simulation environment.

GB models are directly integrated with the existing code. After having developed the model, the representative model equations are directly written in the programming language used as a set of functions or methods. These functions will be called by the logical programming pipeline, considering the detected input variables such as time, weather, electricity prices, and so on. The results of the methods will be the evolution of the thermal behavior of the building and its associated costs.

The authors in Talib et al. (Reference Talib, Park, Im and Joe2023) perform an experimentation in which a GB model based on a RC-network is compared against a BB model based on Artificial Neural Networks (ANNs). To create each model, the authors used experimental data from the research building in Oak Ridge National Laboratory. In the case of RC-Network, the authors built it using first-order ordinary differential equations, considering the heat transfer equations of a 3R2C-based thermal network of the zone using a state-space formulation. The ANN network was built directly by taking measured historical raw data from building sensors from September 20 to September 28 of 2019, and applying different statistical procedures to feed ANN networks (Pearson correlation, data reshaping, etc.). According to the authors, both models used the same number of input parameters to perform a fair comparison (although ANN models require fewer input parameters). Results reveal that the GB model outperforms the ANN model in terms of accuracy and prediction stability. However, the authors explain that the performance of ANN models could be improved by executing a hyperparameter optimization procedure, a feature engineering process, an ensemble method, and weight initialization.

Other authors (Kumar et al., Reference Kumar, Rawlings, Wenzel and Risbeck2023) create an ANN and an RC model to enhance the usage of an MPC-based controller. The RC network is created as a data source for estimating how the thermal environment changes over time, and the ANN is created to add thermal disturbances to the model to create a more realistic environment. To create the RC model, the authors used data from the target environment (a total of 1 week of data measurements of 1 min). To create the ANN, the authors used 20 weeks of data. To compare the performance of their solution, the authors compare different MPC solutions with and without the ANN-based thermal disturbance estimator and the RC model estimations. Results reveal that the usage of their ANN thermal disturbance and the RC network model estimations result in an accurate and optimized MPC controller against others.

Abtahi et al. (Reference Abtahi, Athienitis and Delcroix2024) present a manuscript in which they used a Control-Oriented Archetype Model (Candanedo et al., Reference Candanedo, Vallianos, Delcroix, Date, Saberi Derakhtenjani, Morovat, John and Athienitis2022) as a GB model to represent the zone/building environment with the aim of building an online economic MPC (E-MPC) to simulate optimal heating load management. The case study was applied in a research house (Hydro-Québec) using an 11R6C network. To compare the results of the developed solutions, the authors compared the actual indoor temperature versus a 24 h-ahead predictions done by their CAM implementation. Results revealed that near-term predictions are more accurate than the long-term ones, and that the energy savings on those near-term predictions were around 30% of reduction in daily heating cost compared to the reference operation.

Gokhale et al. (Reference Gokhale, Claessens and Develder2022) present an implementation of a Physics Informed Neuronal Networks (PINNs). The primary objective is to develop a prediction network that forecasts room temperature and power consumption for subsequent time steps in a target building. This network will consider a set of input variables, despite the absence of information about certain environmental elements, such as solar radiation and other factors. Thus, the aim is to build a PINN that can estimate the hidden states of the environment and provide accurate values. To build the PINN, the authors used an RC model to define the loss function and the optimization strategy of the ANN. To validate it, the authors used two approaches: (1) a synthetic validation dataset and (2) real-world data obtained from a cold storage unit. Results showed that the two developed PINNs provide good prediction rates with an error rate <0.25 °C.

BB models are commonly implemented by loading raw data from the target model in the implementation, which alleviates the requirement of having prior knowledge of the building to build a thermal model. However, the data obtained from these sensors should be used carefully to select only the most meaningful values, ensuring the trustworthiness of the final implementation. As explained in GB models, these models could also be directly integrated with the existing programming language. However, certain operations to load and adjust the data should be done, for example, load data from an Excel file or plain text, delete nonvalid values, and so on.

Taking into account the aforementioned lines, Knudsen et al. (Reference Knudsen, Georges, Skeie and Petersen2021) used a test facility located at the Norwegian University of Science and Technology to adjust the water flow of a hydronic radiator to optimize its usage. To do it, a novel low-cost E-MPC controller was created by using a BB-based model. The model was created by changing the set point of the building from 21 to 24 °C. The E-MPC model was trained with 14 days of model data and tested with 4 days of the same model. Results showed that their E-MPC controllers consumed 1.3% more energy than a constant setpoint baseline of 21 °C but reduced the energy costs by 22.5%.

Lin et al. (Reference Lin, Lin, Lin, Wang and Jiang2022) built a computational fluid dynamics model using the software 6SigmaDC to create an experimental platform that provides data from the tool to emulate the cooling state of a data center room with different installed racks (20 racks arranged in 4 rows). The emulated environment creates data from two different scenarios: (1) when the computer room reaches a thermal equilibrium (specific workload configuration and air conditioning parameter settings) and (2) when the computer room is in a transient scenario because there are dynamic changes in its parameters (dynamic change of parameter settings to predict the future system state). The aim is to apply and compare the performance of ML models to estimate the temperature predictions based on the possible workloads of their servers to adjust the installed cooling system and save energy. Results demonstrated that XGBoost (an extreme gradient boosting tree algorithm) and LightGBM (a gradient boosting framework for using tree-based learning algorithms) are superior to others (ANNs, Support Vector Machines, etc.) in prediction accuracy and computational overhead.

Kamel et al. (Reference Kamel, Sheikh and Huang2020) studied what are the most meaningful inputs that are required for forecasting energy consumption in residential buildings (cooling load, heating load, hot water, and ventilation fan). To perform the study, the authors used a tool to generate synthetic data based on a program called the Building Energy Optimization Tool, which is developed by the National Renewable Energy Laboratory. The overall idea is to generate a dataset to emulate a BB modeling approach, which will be cleaned and curated in different ways using classical statistics techniques (e.g., Pearson correlation) to apply a regression algorithm to compute its root mean square error to determine which process is the best for determining the most interesting inputs that could be used for forecasting the energy consumption.

In Mugnini et al. (Reference Mugnini, Coccia, Polonara and Arteconi2020), the authors compared GB (RC-network) versus BB (data-driven) models with the aim of comparing what is the real performance of a simple MPC controller designed to minimize the total energy cost while ensuring that the demand satisfaction is guaranteed. To do this, an MPC controller was designed using MATLAB, and it is fed with the data coming from both models. The BB model was created from a set of datasets generated by a TRNSYS model, and the GB model is modeled according to the target building. Results showed that, unless the performance of both approaches is similar, the GB approach is better because it follows the system dynamics more accurately than the BB. The BB model implementation exceeds the upper comfort limit by 1°; thus, it seems that a small error in prediction leads to excessive temperature inside the building.

3.1. Building physics model

Building physics (heat transfer, thermal inertia, etc.) is simulated through a lumped capacitance model grouping the effective thermal inertia of the system into several discrete lumps (Vivian et al., Reference Vivian, Zarrella, Emmi and De Carli2017). This model is a representation of the heat transfer dynamics of an Full Physical System (FPS). A validated state-space model of a building is taken from (Bacher and Madsen, Reference Bacher and Madsen2011). The selected model (referenced as TiTmTeThTsAeRia) consists of a reduced set of equations modeling the heat dynamics of the building. These deal with heat transfer through the building envelope, the impact of solar radiation, and thermal mass in the building, as well as short-term dynamics associated with the sensors. A basic heat input model is also presented. According to the authors’ statement, the proposed model and parameter identification techniques are suitable for control applications. The model has been properly validated by the authors and is considered suitable to characterize the interaction of the building with local climate.

The dynamics of the building are linked to four states as defined in Equations (3.1)–(3.4) (the nomenclature is described in Appendix A on Table A1). These equations are adapted from Equations (13)–(28) from Bacher and Madsen (Reference Bacher and Madsen2011), considering only relevant parameters in the TiTmTeTℎTsAeRia model. Stochastic terms are disregarded as they are only required in the identification process.

(3.1) $$ \frac{d{T}_s}{dt}=\frac{1}{C_s}\cdot \frac{\left({T}_i-{T}_s\right)}{R_s} $$

(3.2) $$ \frac{d{T}_h}{dt}=\frac{1}{C_h}\cdot \left[\frac{\left({T}_i-{T}_h\right)}{R_{ih}}+{Q}_h\right] $$

(3.3) $$ \frac{d{T}_i}{dt}=\frac{1}{C_i}\cdot \left[\frac{\left({T}_s-{T}_i\right)}{R_s}+\frac{\left({T}_h-{T}_i\right)}{R_{ih}}+\frac{\left({T}_e-{T}_i\right)}{R_{ie}}+{A}_w\cdot {I}_s\right] $$

(3.4) $$ \frac{d{T}_e}{dt}=\frac{1}{C_e}\cdot \left[\frac{\left({T}_i-{T}_e\right)}{R_{ie}}+\frac{\left({T}_a-{T}_e\right)}{R_{ea}}+{A}_e\cdot {I}_s\right] $$

The parameters in the model are associated with a 120 m² single-story building located in Denmark, but the methodology defined in this work allows getting equivalent parameters for other buildings of interest. The application of this model for control-oriented applications requires its extension to incorporate key systems and phenomena not included in the original model, such as building ventilation, occupancy, and heat production and distribution systems.

3.2. Extensions of the model

3.2.1. Ventilation

Despite ventilation being mandatory according to most modern building regulations, it is not explicitly considered in the original model. An additional energy flow is added to Equation (3.4) based on ventilation rates defined by the American Society of Heating (2015). A fixed ventilation rate is taken, considering the most limiting factor among the maximum occupation rates or floor area-based occupation rates. The effective energy flow is determined according to Equation (3.5), which is then converted into an effective thermal resistance in Equation (3.6). Accordingly, Equation (3.3) is replaced with Equation (3.7).

(3.5) $$ {Q}_{vent}=1.21\cdot {V}_{vent}\cdot \left({T}_a-{T}_i\right) $$

(3.6) $$ {R}_{vent}=\frac{1}{1.21\cdot {V}_{vent}} $$

(3.7) $$ \frac{d{T}_i}{dt}=\frac{1}{C_i}\cdot \left[\frac{\left({T}_s-{T}_i\right)}{R_s}+\frac{\left({T}_h-{T}_i\right)}{R_{ih}}++\frac{\left({T}_e-{T}_i\right)}{R_{ie}}+{A}_w\cdot {I}_s+\frac{\left({T}_a-{T}_i\right)}{R_{vent}}\right] $$

3.2.2. Heat distribution and diffusion system

The model presented in Equations (3.1)–(3.4) is based on electric heaters, where no physical limitation is introduced to the diffusion capacity of the heating system. This approach is in line with the focus of Bacher and Madsen (Reference Bacher and Madsen2011) on the modeling of the heat dynamics of the building. However, modification is necessary to allow the description and manipulation of real heating systems. An exponential equation for radiator systems in Truschel (Reference Truschel2002) is adopted as per Equation (3.8).

(3.8) $$ {Q}_h={Q}_{h, nom}\cdot {\left[\frac{\left({T}_{\mathit{\sup}, effect}-{T}_i\right)}{\left({T}_{\mathit{\sup}, design}-{T}_{i, design}\right)}\right]}^n $$

3.2.3. Heat production technology

A typical configuration of heat pumps is assumed (Equations [3.9] to [3.12]), in which the thermodynamic heating cycle is operated whenever the required heating power ( $ {Q}_h $ ) is within the heating capacity of the heat pump ( $ {Q}_{hp,\mathit{\max}} $ ). During peak periods, in-line heaters are used to boost heat production for the remaining required energy ( $ {Q}_{h, boost} $ ).

(3.9) $$ {Q}_{h, hp}=\mathit{\min}\left({Q}_h,{Q}_{hp,\mathit{\max}}\right) $$

(3.10) $$ {Q}_{h, boost}=\mathit{\max}\left({Q}_h-{Q}_{h, hp},{Q}_{boost,\mathit{\max}}\right) $$

(3.11) $$ COP=2.60+0.0465\cdot {T}_a-0.0187\cdot {T}_{\mathit{\sup}, effect} $$

(3.12) $$ {P}_{h, el}=\frac{Q_{h, hp}}{COP}+{Q}_{h, boost} $$

Equation (3.11) is formulated in accordance with De Coninck and Helsen (Reference De Coninck and Helsen2016), where empirical correlations of two air–water heat pumps are presented. Coefficients of both heat pumps have been averaged, and only significant terms have been considered. Considering typical operational controls in heat pump systems, these are not activated until relevant heating loads are required. This is due to low performance and high wear in those conditions. For this reason, a minimum heat load of $ {Q}_{hp,\mathit{\min}} $ is defined. For heat loads below this threshold, no heating is provided.

3.2.4. Power balance in the heating system

There is a need to perform an energy balance of heat production and diffusion systems, achieved by adapting the effective supply temperature to the physical constraints of the system. The heat delivery capacity of the system is calculated for the supply temperature set by the controller ( $ {T}_{\mathit{\sup}, effect}={T}_{\mathit{\sup}, set} $ ) and compared with the actual capacity of the heat production system. If the delivery capacity exceeds the heat production capacity, then an effective supply temperature is calculated ( $ {T}_{\mathit{\sup}, effect}<{T}_{\mathit{\sup}, set} $ ) with Equation (3.10) so that the energy balance in Equation (3.13) is ensured.

(3.13) $$ {Q}_h={Q}_{hp,\mathit{\max}}+{Q}_{boost,\mathit{\max}} $$

3.2.5. Internal heat gains and occupancy model

An occupancy model is defined based on a mathematical algorithm by Page et al. (Reference Page, Robinson, Morel and Scartezzini2008) using Markov chains (Feller, Reference Feller1968) to generate occupant patterns based on the probability distribution of the target place. The model generates time series of zeros (absence) and ones (presence), reproducing arrivals and departures of the target place and alternating short periods of presence and absence. In this regard, the algorithm takes a monthly and an hourly presence probability distribution to generate occupant patterns. For this implementation, a common office occupation probability pattern with a maximum occupancy of 12 persons has been adopted to create stochastic vectors of 2-year occupation data in hourly intervals. The heat load associated with occupancy (Equation [3.14]) is calculated based on well-grounded engineering calculations (ASHRAE American Society of Heating Refrigerating and Air-Conditioning Engineers, 2017) assuming a metabolic rate of 77 W/m ² for male occupants and 54 W/m ² for female occupants and a surface area of 1.8 m ² per occupant, resulting in a specific heat load factor (Fiℎg) of 117.81 W/pers.

(3.14) $$ {Q}_{ig}={F}_{ihg}\cdotp Ocup $$

3.3. Input/output variables

Measured climate data and reliable energy costs are obtained to build a realistic scenario for using the model. Climate data to feed the air temperature ( $ {T}_a $ ) and global radiance ( $ {I}_s $ ) in the provided equations were extracted from local climate stations placed next to the used model. The obtained data has a time resolution of 1 min and corresponds to 2017 and 2019.

For energy data, historical electricity prices have been obtained through a local power exchange company. Hourly electricity prices in EUR/kWℎ from 2017 to 2019 have been used to calculate the total cost of activating the heat pump by multiplying these costs with the electrical power ( $ {P}_{h, el} $ ) obtained through Equation (3.12). The HVAC system is controlled with three signals activating the heating system ( $ AC{T}_H $ ), activating the ventilation system ( $ AC{T}_{Vent} $ ), and setting the supply temperature to the radiator system ( $ {T}_{\mathit{\sup}, set} $ ).

4.1. Architecture description

Figure 1 depicts the following architecture for creating the simulation environment. It is mainly composed of two modules: (1) the data extraction and model definition module (right section), and (2) the simulation engine and the data endpoints (left section).

Figure 1. General overview of the architecture. At the right, the data extraction and model formulas; at the left, and the simulation engine and data endpoints.

The end user uses the Representational State Transfer (REST) application as an access point to the simulation tool to send the required information to start a simulation procedure. The simulation engine acts as a “middle” connector between Application Programming Interface (API) endpoints and the simulation core and executes the required code to perform the simulation process. The simulation engine executes the model formulas and adds the required exogenous and occupant estimator values to the model. Results are sent to the end user once the simulation procedure is completed.

4.1.2. Simulation engine and data endpoints

The simulation engine and data extraction endpoints module comprise the software components required to create a simulation environment and allow third parties to use the simulation data to train and evaluate their algorithms. This module uses and manages the different components presented in Section 4.1.1 and acts as a middleware communication channel between these components and requests from different implementations. As depicted previously in Figure 1, the module is divided into two different components: (1) the simulator engine and (2) the REST application service.

The simulator engine allows for interactive simulation of the behavior of a building when an active element is used (e.g., an HVAC system). The code is composed of one class that allows defining how to load the different data sources (exogenous data, occupants, model formulas, etc.) and other additional configurations such as the Δ time gap between iterations or the total simulation time. The simulator engine aims at supporting a decision-making process; thus, each iteration represents a simulated time iteration (t) in the equations in Section 3 (each iteration satisfies the time difference parameter dt). Hence, on each interaction, values are calculated using the previously calculated results and a Δ time.

By default, the Δ time is set to 10 s and the starting thermal values (Te, Ts, and Te) are set to 20.0 °C, but it is possible to adjust these values via class instance parameters. In addition, it is possible to configure the total simulation time. By default, the system provides a yearly simulation from January 1 of the initial year to December 31 of the final year, but it is possible to configure the final date by giving it in a year–month–day format. The maximum time steps to be simulated are calculated by taking the time difference (δ) between the initial and final dates, and the defined Δ time gap (dt).

Algorithm 1 shows a pseudo-code to illustrate the logic followed by the simulator engine. As illustrated, the code loads the required exogenous data and initializes the required values to calculate the operational costs (C_op), the building internal gains (Q_ig), and the variation of the thermal values of the building (dT_s, dT _ℎ, dT_i, and dT_e) when an installed HVAC device is powered on or off (ACT_HP). The decision to activate or not to activate the HVAC device (ACT_HP) is decided by the researcher’s algorithm.

Algorithm 1. Simulation engine logical execution.

1: Δ ← 10                                                                                                                          ⊲ In seconds

2: t_max ← max_timesteps                                                                                 ⊲ Total yearly simulation

3: exogenous ← exogenous_data.

4: O ← generated_occupant_estimation.

5: T_s, T_h, T_i, T_e ← 20                                                                                                   ⊲ Degree Celsius

6: for t = 1, 2,…, t_max do.

7:       electricity_price, T_a ← exogenous _t

8:       Q_h, P_el ← calculate_cop(ACT_HP, T_i, T_a)

9:       C_op ← calculate_costs(P_el, electricity_price)

10:     Q_ig ← calculate_internal_gain(O_t)

11:     Q_Vent ← calculate_vent_resistance(T_a, T_i)

12:     dT_s ← calculate_t_s(Δ, T_i, T_s)

13:     dT _ℎ ← calculate_t_h(Δ, Q_h, T_i, T_h)

14:     dT_i ← calculate_t_i(Δ, Q_ig, T_s, T_i, T_h, T_e, Q_Vent)

15:     dT_e ← calculate_t_e(Δ, T_i, T_a, T_e)

16:     T_s ← dT_s                                                                                                  ⊲ Update with new values

17:     T_h ← dT _ℎ

18:     T_i ← dT_i

19:     T_e ← dT_e

20:     save_results()

21: end for

The REST application service is the final and optional component to externally interact with the simulator engine. The goal is to create a confidential training workbench to allow third parties to obtain reliable simulated data while the knowledge of the model equations is preserved. Thus, the REST application service isolates the model equations and, at the same time, acts as direct access to the data generated by the simulation engine. The reason for creating a confidential training workbench is that some thermal formulas need to be protected due to closed-source licensing reasons. For example, there are HVAC manufacturers that provide closed-source model formulas to emulate the operational states of their systems to evaluate them in certain scenarios. In the present article, the developed REST API receives a one-hot vector array according to the available actions described in Section 3.3.

An endpoint called train is coded to show how the simulation engine can be used. Table 1 depicts an example of a JSONFootnote ³ code data to send to the API. The action vector contains the actions taken in each time step to control the active element of the building (ACTℎp). The datetime_start contains the initial simulation date. The datetime_end contains the final simulation date. Finally, the simulation_steps value defines the Δ time gap value to calculate the total simulated time steps.

Table 1. JSON needed to send the data to the simulation environment

Table 2 shows an example of data received from the API. If a simulation can be run successfully (the send values are correct and valid), the API will return a large JSON array containing the results of each sent action. In this example, the data depicted in Figure 2 was sent and it returned to the global horizontal irradiation, date, Ta, and Ts values. As illustrated, every 10 s, a JSON entry is added to the resulting JSON array showing how the simulator environment values are changing according to the control formulas applied and the actions sent to the active element.

Table 2. JSON data received by the API

4.2. Used technical tools

The architecture depicted in Section 4.1 is implemented using different technical tools and libraries. The Python programming languageFootnote ⁴ is used as the backbone of the implementation. It allows for easy readability of the programmed code to developers who do not have a broad experience in hard-type languages (e.g., Java or C) and contains several powerful Data Science libraries for doing AI research. The Nginx reverse web/proxy serverFootnote ⁵ is used to manage the API requests, perform the required redirections to call the API instances, and manage the data transactions. The UbuntuFootnote ⁶ operating system is used because it is free, easy to use, and allows for a custom compilation of different data science libraries.

To be able to run the API, several technical libraries are used. Flask is a micro-framework that allows the creation of web services using a few code lines. In this implementation, it is used to create the REST application service. Flask-Security and Itsdangerous libraries are used to extend Flask’s capabilities by adding an extra security layer to web services, such as signed tokens, reCAPTCHA,Footnote ⁷ and email sending when there is an error in the web service. Gunicorn is a Web Server Gateway InterfaceFootnote ⁸ server designed to forward requests from external sources to web application services, or frameworks written in Python. In this implementation, Gunicorn creates an internal Linux socket to pass the information to the Nginx reverse proxy server: the idea is that Nginx handles the connections and uses the internal socket as a middle communication between the end users and the Gunicorn web server. Jsonschema is a Python implementation for JSON schema vocabulary.Footnote ⁹ It is used to validate the data sent by the end users to ensure that the requests contain reliable and valid data. Finally, the Arrow library enhances the usage of dates and times in Python, allowing easy management of different times, especially when there is an interest of handling time zones.

5.1. Technical validation

Technical validation is aimed at validating whether the developed workbench can be used on a regular machine and escalated onto different systems. In this article, the authors used a laptop computer based on an INTEL CORE CPU 9850H, with 6 cores and 12 logical processors, 64 GB of RAM memory DDR4, and 1 T Solid State Drive GEN 3 inside an M.2 factor connection. The central processing unit (CPU), memory, and execution time values were analyzed using the psutilFootnote ¹⁰ Python library, which allows obtaining information about the running processes and system utilization. The technical validation was conducted using two different approaches: (1) a cumulative simulation and (2) an individual simulation. In both cases, the training workbench was fed with weather and electrical prices datasets, from 2016 to 2019.

5.1.1. Cumulative simulation

The cumulative simulation performs a continuous annual simulation to stress the machine capacity of storing the given values into memory and study the impact of a continuous, long training procedure. The aim is to study the suitability of the developed workbench to be escalated when several years of external datasets are given.

The simulation starts by only simulating the impact of 1-year data and is escalated to simulate a total of 4 years. On each execution, the CPU, memory, and elapsed time parameters are tracked. Table 3 depicts the results of the test; the column N.Years contains the total continuous simulated years, the column CPU contains the used mean CPU percentage, the column Memory contains the mean memory consumption in MB, and the column E.Time contains the elapsed time in minutes.

Table 3. Performance escalation results of the simulation environment

As observed in the given results, CPU usage remains similar in each test and is within logical values considering that the target machine is not an enterprise-grade machine (e.g., a server with an Intel Xeon family processor).

Memory consumption grows as more yearly data are processed and stored in memory. The values obtained are reasonable, considering that a consumer-grade computer has 16 GB of RAM memory. Surprisingly, the memory consumption is not linear among the different tests; for example, the difference between the 2- and 1-year simulations is 173.49 MB, whereas the difference between the 3- and 2-year simulations is 118.71 MB. This is because the provided external weather and electricity price datasets contain different values and have a direct impact on the internal equations exposed in Section 3.

The execution time for each year is around 5 min. In terms of real usage, this is again reasonable considering the complexity of the designed equations.

5.1.2. Individual simulation

The individual simulation simulates the required resources to simulate one specific year. The test analyses the variation of the required resources among different years to study the variations of the computer resources. Table 4 reveals the results obtained by the validation procedure.

Table 4. Individual results of the simulation environment

Results reveal a reasonable similarity between different years. The leap year (2016) is not the most demanding one, which confirms that the use of machine resources is more linked to the requirements of the target equations to compute the data given by the external datasets.

5.2. Model validation

The simulation engine has been validated regarding its physical performance. This has been done by means of a definition of test sequences, where backward identification of its fundamental engineering parameters has been performed. Tests regarding steady state (thermal resistance and solar heat gain) and transient (time constants) performance of the building and heat pump (coefficient of performance) have been conducted as follows:

• • Experiment 1. Steady state 20 to 0 °C indoor–outdoor temperature difference without solar radiation. The overall thermal resistance of the building is compared with Bacher and Madsen (Reference Bacher and Madsen2011) with an accuracy of 1%.

• • Experiment 2. Steady state 20 to 0 °C indoor–outdoor temperature difference with 300 W/m ² of incident solar radiation. The overall gA value of the wall of the building is compared with Bacher and Madsen (Reference Bacher and Madsen2011) with an accuracy of 0.3%

• • Experiment 3. Stepwise 20/0 °C variation of indoor temperature with 0 °C outdoor temperature and no solar radiation. The longest time constant of the building is compared with Bacher and Madsen (Reference Bacher and Madsen2011) with accuracy better than 0.1 h.

• • Experiment 4. Transient outdoor temperature. The coefficient of performance of the heat pump is compared with De Coninck and Helsen (Reference De Coninck and Helsen2016) with an accuracy better than 2%. This value is better than the resolution provided for this parameter in standardized rating processes (only one decimal figure).

7.1. Technical extensions

The current implementation would benefit from the following technical extensions regarding security and usability:

• - The addition of more security measures and endpoints to enhance the experience of the REST application service.

• - The development of alternative interaction procedures for the training of control-oriented solutions. The current implementation only defines a single procedure for training ML algorithms through a direct interaction between the researcher’s inputs and the simulation engine. In this regard, the ability to make modifications to the simulation engine itself by providing, for example, a custom weather file or custom electricity prices will definitely enhance the training experience. This modification allows the researcher to use the model as a base archetype for training more generalized algorithms and adapting to a defined location. In addition, adding more endpoints allows us to define the granularity of the data to be extracted from the simulator.

• - The addition of a graphical interface to allow direct modification of the simulation equations and parameters. This is largely linked to the potentialities for model extensions stated in Section 7.2.

7.2. Model extension and escalation

The current work proves the feasibility of extending a thermal model of an unoccupied building into a comprehensive model comprising occupancy, ventilation, and heating appliances. However, further extensions are required to match the complexities of real-world buildings and thermal interactions:

• - The thermal model of the building is limited by its mono-zonal and thermal-only approach. Extensions of the model for multi-zonal contexts could be considered.

• - Advanced controls for building energy systems would potentially require better modeling of comfort in line with the requirements of the (Fanger, Reference Fanger1970) model, comprising at least a humidity parameter.

• - The modeling of building occupants is currently limited to their presence and a quite simple induced heat load model. Their impact on the building configuration (i.e., opening of windows, activation of shading systems, etc.) and variable heat loads (i.e., separate load patterns for meetings, computer use, rest, etc.) could substantially improve the model.

• - With regard to technical systems, the inclusion of more complex ventilation systems (e.g., mechanical ventilation with heat recovery and bypass), various heat diffusion systems (e.g., fan-coils and radiant surfaces) and heat production systems (i.e., solar thermal, geothermal, and boilers) could be considered.

• - The present model does not allow for cooling applications. Such an extension would probably require modifications to the technical systems (e.g., mechanical chillers) and to the building model (e.g., incorporation of a humidity model and dew point calculations)

• - The current model requires explicit model definition. In the current implementation, models such as Bacher and Madsen (Reference Bacher and Madsen2011) are proposed. Recent works, such as Leprince et al. (Reference Leprince, Madsen, Miller, Real, van der Vlist, Basu and Zeiler2022a), developed a generalized approach to the calibration of RC models, ultimately resulting in datasets of calibrated RC models in Leprince et al. (Reference Leprince, Miller, Madsen, Basu, van der Vlist and Zeiler2022b). Thus, an interesting extension of the presented work would be to perform a specific instance of an RC model and adjust its equation parameters using the given values of their datasets.

The simulator represents the transient behavior of a building in a reduced set of equations, which makes it suitable to be escalated to larger multi-zone buildings (more complex models) or aggregated (many models running in parallel) into district-scale simulators. All this considers the need for the integration of the corresponding HVAC equations. Considering the relatively small number of equations (substantially lower than for WB and BB models) should allow for efficient computation time, enabling research in intelligent energy operators delivering specific actions to each zone.