2.1. HVAC control for building system

Controlling building systems is a complex challenge due to the influence of fluctuating factors like weather and occupant behavior, making accurate predictions difficult. This is particularly true for HVAC systems, where Model Predictive Control (MPC) has long been explored for its optimization potential. Recent years have seen a surge in interest toward intelligent buildings that leverage weather and temperature forecasts to further optimize HVAC control, as demonstrated in several studies (Afram and Janabi-Sharifi, Reference Afram and Janabi-Sharifi2014; Carli et al., Reference Carli, Cavone, Ben Othman and Dotoli2020; Yao and Shekhar, Reference Yao and Shekhar2021; Blum et al., Reference Blum, Wang, Weyandt, Kim, Wetter, Hong and Piette2022; Hou et al., Reference Hou, Li, Nord and Huang2022; Taheri et al., Reference Taheri, Hosseini and Razban2022). For instance, Taheri et al. (Reference Taheri, Hosseini and Razban2022) provide a comprehensive review of recent advancements in MPC techniques, outlining guidelines for optimal design. Carli et al. (Reference Carli, Cavone, Ben Othman and Dotoli2020) bridge the gap between theory and practice by proposing an IoT-based architecture for implementing MPC in real-world HVAC systems. Furthermore, Blum et al. (Reference Blum, Wang, Weyandt, Kim, Wetter, Hong and Piette2022) demonstrated the practical implementation of MPC in a real office building using an open-source Modelica-based toolchain, achieving a 40% reduction in HVAC energy consumption over a 2-month trial while analyzing practical challenges and implementation effort. Addressing the inherent uncertainties in weather forecasting, Hou et al. (Reference Hou, Li, Nord and Huang2022) introduce an error model based on easily measurable data to enhance MPC performance. They proposed an error model incorporating readily available data to improve MPC performance under weather forecast uncertainty, achieving a 3.4% energy cost reduction and a 73% decrease in room temperature deviations compared to rule-based control. Previous studies have highlighted the potential of MPC for energy-saving in building systems, particularly in HVAC. Yao and Shekhar (Reference Yao and Shekhar2021) reviewed MPC applications in buildings, emphasizing their ability to handle uncertain parameters like weather forecasting, ambient temperature, and solar radiation. Afram and Janabi-Sharifi (Reference Afram and Janabi-Sharifi2014) compared MPC with other control methods, highlighting its advantages in improving energy efficiency and maintaining indoor comfort. They demonstrated the potential for pre-heating or pre-cooling buildings during off-peak periods to reduce electricity costs.

While MPC has shown promise, predicting building system behavior remains challenging due to inherent uncertainties. Researchers have explored occupant comfort prediction as a proxy for building performance (Rajith et al., Reference Rajith, Soki and Hiroshi2018; Choi et al., Reference Choi, Park, Kim and Moon2022; Skaloumpakas et al., Reference Skaloumpakas, Sarmas, Mylona, Cavadenti, Santori and Marinakis2023). Choi et al. (Reference Choi, Park, Kim and Moon2022) developed a deep learning model for real-time clothing insulation (R-CLO) estimation, demonstrating its potential for comfort-aware HVAC control by predicting energy consumption based on clothing and impacting thermal comfort assessed using the Predicted Mean Vote (PMV) index. Rajith et al. (Reference Rajith, Soki and Hiroshi2018) achieved a 20–40% energy consumption reduction during summer by automating HVAC control based on user feedback, sensor data, and a neural network/multi-layered perceptron (MLP) for time series prediction, prioritizing user comfort by minimizing unnecessary feedback requests. Skaloumpakas et al. (Reference Skaloumpakas, Sarmas, Mylona, Cavadenti, Santori and Marinakis2023) proposed a neural network model to overcome PMV limitations in accurately reflecting perceived comfort by integrating user feedback and temperature sensor data, predicting next-hour internal and external temperatures to improve comfort prediction.

However, implementing room-specific MPC models across entire buildings presents significant challenges, including high cost and data acquisition complexities. The rise of generative AI, particularly unsupervised learning models, offers a potential solution by eliminating the need for extensive training data. This breakthrough could significantly reduce data collection and annotation costs, paving the way for wider adoption in industry.

2.2. Generative AI potentials as a simulator

Building system prediction necessitates expertise in computer science and mathematics, including data science and statistics. For effective HVAC control, generative AI’s ability to predict and regress based on various sensor inputs becomes crucial. Recent studies have explored the potential of LLMs for automating data science and statistical analysis. Tu et al. (Reference Tu, Zou, Su and Zhang2024) demonstrated the effectiveness of LLMs in data analysis tasks, including data cleaning, analysis, and report generation, while also highlighting their potential as expert tutors for enhancing data science and programming learning outcomes. Specifically, their evaluation showed LLMs could perform these tasks with high quality. Cheng et al. (Reference Cheng, Li and Bing2023) investigated GPT-4’s capabilities as a data analyst, employing various databases to perform end-to-end automated data analysis and comparing its performance with human data scientists. Their findings revealed that GPT-4 possesses moderate to advanced data science skills but lags behind humans in tasks involving graphical analysis. Imani et al. (Reference Imani, Du and Shrivastava2023) explored the limitations of LLMs in solving arithmetic reasoning tasks, where a single correct answer exists. They demonstrated that using zero-shot Chain-of-Thought (CoT) prompting enhances LLM performance in finding mathematical predictions and solutions. Zhu et al. (Reference Zhu, Dai and Sui2024) investigated LLMs’ understanding of the fundamental mathematical concept of numbers. They concluded that LLMs maintain a compressed internal representation of numbers, making accurate reconstruction difficult but suggesting partial understanding. Raj et al. (Reference Raj, Rosati and Majumdar2022) highlighted the sensitivity of pre-trained LLMs to prompt variations, even when the prompts are semantically similar, leading to significant differences in output. Their research proposed a metric for assessing the consistency of LLM outputs, showing a strong correlation with human evaluation.

LLMs are actively being researched as simulators for decision-making across various fields, including autonomous driving, robotics, agent-based control, and social simulation. In autonomous driving, LLMs simulate vehicle trajectories and predict actions like lane changes based on sensor data (Mao et al., Reference Mao, Qian, Zhao and Wang2023; Sha et al., Reference Sha, Mu, Jiang, Chen, Xu, Luo, Li Eben, Tomizuka, Zhan and Ding2023; Fu et al., Reference Fu, Li, Wen, Dou, Cai, Shi and Yu2024). Robotics research utilizes LLMs for environment-aware decision-making, translating ambiguous natural language instructions into precise reward functions, and voice control (Cui et al., Reference Cui, Siddharth, Palleti, Shivakumar, Liang and Sadigh2023; Gramopadhye and Szafir, Reference Gramopadhye and Szafir2023; Salzmann et al., Reference Salzmann, Chiang, Ryll, Sadigh, Parada and Bewley2023). Other studies are also investigating the automatic optimization of simulator parameters using LLMs to automatically derive reward functions (Yu et al., Reference Yu, Gileadi, Fu, Kirmani, Lee, Arenas, Chiang, Erez, Hasenclever, Humplik, Ichter, Xiao, Xu, Zeng, Zhang, Heess, Sadigh, Tan, Tassa and Xia2023; Ma et al., Reference Ma, Liang, Wang, Huang, Bastani, Jayaraman, Zhu, Fan and Anandkumar2024). For predicting real-world behavior, LLMs are being employed to simulate social interactions observed in media and to model human actions and decision-making processes (Park et al., Reference Park, Popowski, Cai, Morris, Liang and Bernstein2022; Song et al., Reference Song, Wu, Washington, Sadler, Chao and Su2023a; Gui and Toubia, Reference Gui and Toubia2024).

2.3. LLM-driven agents and world-modeling

Agentic AI, a current focus of research, is characterized by its ability to autonomously manage all processes required to achieve a human-specified goal. This encompasses breaking down complex tasks, planning execution sequences, executing individual tasks, evaluating outcomes against the goal, and, crucially, interacting with the external environment. This interaction may involve generating and executing code, collecting and analyzing information from Supplementary Materials, and utilizing external APIs. The significant versatility of Agentic AI is driving discussion about its associated benefits and risks. Acharya et al. (Reference Acharya, Kuppan and Divya2025) provide a comprehensive overview of Agentic AI, covering its characteristics, methodologies, applications (e.g., health care, finance, and manufacturing), and challenges (e.g., scalability, ethics, and regulation). It highlights the transformative potential of Agentic AI’s autonomous decision-making while emphasizing the need for ongoing research in areas like goal alignment, multi-agent coordination, and ethical considerations. Li et al. (Reference Li, Chen, Yan, Shen, Xu, Wu, Zhang, Zhou, Chen, Cheng, Shi, Zhang, Huang and Zhou2023) developed ModelScope-Agent, a customizable, general-purpose agent system leveraging open-source LLMs. This system demonstrates improved API call accuracy through a novel training strategy and provides a flexible framework for building real-world applications using over 1000 public AI models. Shavit et al. (Reference Shavit, Agarwal, Brundage, Adler, O’Keefe, Campbell, Lee, Mishkin, Eloundou, Hickey, Slama, Ahmad, McMillan, Vallone, Passos and Robinson2023) discuss the potential benefits and risks of Agentic AI, including increased efficiency and tailored recommendations, alongside potential harms. It proposes actionable practices for users, developers, and implementers, such as task appropriateness evaluation, action space constraints, and activity monitoring, to mitigate risks and address societal impacts. Sivakumar (Reference Sivakumar2024) explores the application of Agentic AI to enhance Artificial Intelligence for IT Operations (AIOps) platforms, focusing on predictive analytics, machine learning, and autonomous decision-making to improve operational efficiency and resource allocation. It highlights the potential for real-time resource optimization and discusses challenges related to ethics, data security, and the transition to self-governing systems.

Research has increasingly shown that agents can be successfully trained to learn world models, resulting in significant generalization capabilities across diverse tasks. The potential for transfer to real-world applications makes this approach particularly attractive for industry. A world model is a learned representation that approximates the structure of the world based on limited observational data from a specific environment. This learned representation supports the inference of causal factors from observations and allows for the prediction of future or unseen events based on those factors. West et al. (Reference West, Lu, Dziri, Brahman, Li, Hwang, Jiang, Fisher, Ravichander, Chandu, Newman, Koh, Ettinger and Choi2024) investigate the “Generative AI Paradox,” where generative models exhibit strong output generation but weaker understanding. Through analysis of language and vision modalities, they demonstrate that generative capability does not necessarily depend on understanding, suggesting a structural difference between generative AI and human intelligence. Dinu et al. (Reference Dinu, Leoveanu-Condrei, Holzleitner, Zellinger and Hochreiter2024) introduce SymbolicAI, a logic-based framework for concept learning and flow management in LLMs, addressing the issue of hallucination. By treating LLMs as semantic parsers, SymbolicAI integrates generative models with symbolic reasoning, providing a foundation for research in areas like program synthesis and autonomous agent design. Wang et al. (Reference Wang, Xian, Chen, Wang, Wang, Fragkiadaki, Erickson, Held and Gan2024) propose Generative Simulation, a new paradigm for robot skill learning, and introduces RoboGen, a generative robot agent. RoboGen uses a self-guided cycle of proposing skills, generating simulation environments, and autonomously acquiring those skills, demonstrating a broader range of task and skill generation compared to traditional human-created datasets. Ha and Schmidhuber (Reference Ha, Schmidhuber, Bengio, Wallach, Larochelle, Grauman, Cesa-Bianchi and Garnett2018) construct a generative neural network, termed a World Model, for reinforcement learning, enabling agents to learn and plan actions within a learned representation of the environment. This model comprises a Vision Model (V), a Memory RNN (M), and a Controller (C), allowing for efficient learning and transfer of policies to real-world environments, surpassing traditional deep Reinforcement Learning (RL) methods. Hafner et al. (Reference Hafner, Lillicrap, Ba and Norouzi2020) propose Dreamer, a reinforcement learning agent that learns long-term behaviors by backpropagating value gradients through imagined trajectories in the latent space of a learned Recurrent State Space Model (RSSM). Dreamer achieves superior data efficiency and performance compared to both model-free and model-based approaches on visual control tasks. Hafner et al. (Reference Hafner, Lillicrap, Norouzi and Ba2021) introduce DreamerV2, the first world-model-based reinforcement learning agent to achieve human-level performance on the Atari benchmark. Key improvements over the original Dreamer include the use of categorical latent states and Kullback–Leibler (KL) balancing in the world model, enabling more effective representation learning. Hafner et al. (Reference Hafner, Pasukonis, Ba and Lillicrap2023) present DreamerV3, a scaled-up and generalized reinforcement learning agent that achieves significant advancements in autonomous learning, including solving the Minecraft diamond collection task without human demonstrations. DreamerV3’s improvements in decision-making, stability, and scalability open new possibilities for real-world reinforcement learning applications, including learning from diverse data sources.

Drawing inspiration from recent examples of Agentic AI, we propose that the inherent simulation capabilities of these agents, specifically their internal representations of the environment, can function as a world model. This article details an experimental investigation to test this hypothesis. We collected observations of an office environment using IoT sensors and feedback from occupants. An Agentic AI was then employed to infer the optimal HVAC setpoints that would maintain a comfortable environment while minimizing energy usage. We rigorously evaluated the accuracy of the Agentic AI’s predictions in approximating the real office environment, employing both mathematical expressions and supporting rationales.

2.4. Multimodal foundation models

Beyond language, research is rapidly advancing in multimodal generative AI, which processes inputs and outputs across multiple modalities, including images, audio, video, 3D models, and robotic sensor data (Fei et al., Reference Fei, Lu, Gao, Yang, Huo, Wen, Lu, Song, Gao, Xiang and Sun2022; Yang et al., Reference Yang, Li, Lin, Wang, Lin, Liu and Wang2023b; GeminiTeam, 2024b; Liang et al., Reference Liang, Zadeh and Morency2024; Yang et al., Reference Yang, Sun, Weihs, VanderBilt, Herrasti, Han, Wu, Haber, Krishna, Liu, Callison-Burch, Yatskar, Kembhavi and Clark2024b; Qin et al., Reference Qin, Shi, Yu, Wang, Zhou, Li, Yin, Liu, Sheng, Shao, BAI, Ouyang and Zhang2025). Fei et al. (Reference Fei, Lu, Gao, Yang, Huo, Wen, Lu, Song, Gao, Xiang and Sun2022) developed a foundational model pre-trained on large-scale multimodal data, demonstrating promising results in adapting to various tasks with a single model. MFMs are believed to improve model explainability and enhance imagination capabilities. Liang et al. (Reference Liang, Zadeh and Morency2024) presented a comprehensive survey of MFMs, classifying various approaches for handling multimodal inputs and outputs across diverse tasks and providing an overview of current trends in the field. Such MFMs have become commercially available and are being adopted at a remarkable pace (GeminiTeam, 2024b). Yang et al. (Reference Yang, Li, Lin, Wang, Lin, Liu and Wang2023b) proposed enhancing LLMs’ capabilities through visual marker input, demonstrating that GPT-4V can understand visual markers on images, enabling visual reference-based prompting and inspiring this study’s approach to incorporating image information for captioning tasks. Furthermore, Yang et al. (Reference Yang, Zhang, Li, Zou, Li and Gao2023a) introduced Set-of-Marks (SoM), a novel visual prompting method for large multimodal models (LMMs) like GPT-4V, which divides input images into regions with varying granularity and uses alphabet, numbers, masks, or boxes as marks to enhance GPT-4V’s recognition abilities. OpenAI’s Sora is a general-purpose visual model trained on video data, capable of generating high-resolution videos of varying durations, aspect ratios, and resolutions, up to 1 min in length (Qin et al., Reference Qin, Shi, Yu, Wang, Zhou, Li, Yin, Liu, Sheng, Shao, BAI, Ouyang and Zhang2025). While Sora demonstrates capabilities in video generation from text and image prompts, video extension, and style transfer, it exhibits limitations in accurately simulating complex physical interactions. Yang et al. (Reference Yang, Sun, Weihs, VanderBilt, Herrasti, Han, Wu, Haber, Krishna, Liu, Callison-Burch, Yatskar, Kembhavi and Clark2024b) introduce Holodeck, a system that automatically generates 3D simulation environments from user prompts, addressing the challenge of manual 3D environment creation in Embodied AI. Holodeck uses GPT-4 to acquire commonsense knowledge, reconstruct scenes with 3D assets, and optimize layouts, demonstrating superior performance compared to procedural baselines and supporting zero-shot object navigation for trained agents.

This study investigates the effectiveness of generative AI’s environmental understanding when provided with an overhead view of an office environment, including spatial continuous distributions of room temperatures as heatmaps and occupant locations, as input to MFMs.

2.5. Agentic AI for logic control

Generative AI is expanding its input and output modalities, leading to increased multimodal utilization, including the ability to output program code for controlling devices. Recent research has demonstrated success in using Agentic AI for the automatic generation of control code and utilizes the code for Programmable Logic Controllers (PLCs), commonly used in industry, holding promise for accelerating industrial automation. In the case of building HVAC systems, particularly large-scale HVAC systems in buildings, centralized control systems are commonly used. This allows for the direct integration of control code generated by generative AI, enabling dynamic adjustments to the control system. Yang et al. (Reference Yang, Wu, Zhang, Zhang, Liu, Lian, Ren and Tian2024a) propose AutoPLC, an LLM-based approach for automatically generating vendor-specific Structured Text (ST) code for PLCs. AutoPLC utilizes a knowledge base, a retrieval module, and a flexible code checker with self-correction capabilities, outperforming baseline models on several ST code generation benchmarks. Koziolek et al. (Reference Koziolek, Gruener and Ashiwal2023) investigate the potential of LLMs to generate PLC and Distributed Control System (DCS) control logic from natural language prompts. Using ChatGPT and a set of 100 prompts, they demonstrate the generation of syntactically correct IEC 61131-3 Structured Text code, highlighting the potential for LLMs to enhance control engineer productivity. Hu et al. (Reference Hu, Lu and Clune2025) introduce Automated Design of Agentic Systems (ADAS) as a new research domain and propose Meta Agent Search, an algorithm where a meta-agent programs and evaluates new agents iteratively. Agents discovered by this method significantly outperform manually designed agents on various benchmarks, demonstrating strong cross-domain transferability. Dou et al. (Reference Dou, Liu, Jia, Xiong, Zhou, Shan, Huang, Shen, Fan, Xi, Zhou, Ji, Zheng, Zhang, Huan and Gui2024) present StepCoder, a code generation framework with Curriculum of Code Completion Subtasks (CCCS) and Fine-Grained Optimization (FGO), along with a new dataset, APPS+. StepCoder demonstrates superior performance compared to baseline models on multiple code generation benchmarks, emphasizing improvements in exploration for reinforcement learning. Xue et al. (Reference Xue, Lu, Huang, Wang, Ouyang and Bai2024) introduce ComfyBench, a benchmark for evaluating LLM-based agents’ ability to design Collaborative AI Systems using the ComfyUI platform and propose ComfyAgent, a framework for autonomous design. While ComfyAgent outperforms baseline agents, the results highlight the ongoing challenges in achieving fully autonomous design of complex AI systems. Si et al. (Reference Si, Zhang, Yang, Liu and Yang2024) present Design2Code, a real-world benchmark for evaluating Multimodal Large Language Models (MLLMs) on code generation from visual designs. Experiments using this benchmark demonstrate that GPT-4V outperforms other baselines and can even generate web designs considered superior to the originals, highlighting the model’s understanding of modern web design principles.

2.6. HVAC control with generative AI

Studies by Song et al. (Reference Song, Zhang, Zhao and Bian2023b) and Ahn et al. (Reference Ahn, Kim, Cho and Chae2023) have explored the use of LLMs for HVAC control on simulation. Song et al. (Reference Song, Zhang, Zhao and Bian2023b) employed a virtual building simulation model (BEAR) to evaluate the performance of GPT-4 by OpenAI (2023) in controlling HVAC. Their research provided inputs like indoor/outdoor temperature, solar irradiance, setpoint temperature, and power status to GPT-4, and it outputted a value between −1 and 1, indicating how much warmer/cooler the virtual environment should be. The study found that GPT-4’s zero-shot inference achieved performance comparable to reinforcement learning methods, although it lagged behind Model Predictive Control (MPC). Ahn et al. (Reference Ahn, Kim, Cho and Chae2023) applied ChatGPT to autonomous building system operation, validating its effectiveness on a virtual office building simulation model. Their research demonstrated success in minimizing energy consumption across various building systems, including HVAC, chillers, cooling towers, and pumps. Compared to deep reinforcement learning, ChatGPT achieved a 16.8% reduction in energy consumption, while deep reinforcement learning resulted in a 24.1% reduction. The study highlighted the promise of ChatGPT-based control due to its lower learning cost compared to reinforcement learning, making it potentially viable for real-world deployment.

While these studies demonstrate the potential of using GPT-like general-purpose LLMs for zero-shot HVAC control with minimal training costs, they were all based on simulations. This research takes a step forward by conducting a real-world case study in an actual office environment. We aim to leverage multimodal generative AI to control HVAC and minimize energy consumption while considering office workers’ comfort. Our approach integrates real-time data from various IoT sensors and occupants’ feedback, providing a novel framework for exploring the practical application of generative AI in building system control.

3.1. Case study modeling

A 322 $ {\mathrm{m}}^2 $ office space in the Yokohama Dia Building in Yokohama, Japan, was rented for a field experiment. The office layout is shown in Figures 1 and 2. An average of 18.6 people worked in the office during the experimental period. The room was equipped with seven HVAC units, each with a thermometer to measure the spatial temperature within the office. The objective of this experiment was to minimize energy consumption while considering the comfort of occupants.

Figure 1. CG rendering of the office used in this experiment, and the opposing view.

Figure 2. The overhead map that shows the location of office workers and the spatial distribution of office temperature via MELRemo-IPS IoT sensor device. Triangles ( $ \varDelta $ ) indicate the installation locations of the MELRemo-IPS sensors in Figure 3a, while inverted triangles (∇) represent the positions of the Soracom IoT sensors in Figure 3b. Human figures represent the location of office workers. The color distribution visualizes the estimated temperature in degrees Celsius.

Song et al. (Reference Song, Zhang, Zhao and Bian2023b) conducted research utilizing generative AI for HVAC control. They provided the following reward function as a prompt to the AI:

(1) $$ \left(1.0-\frac{\sum_{0\le i<n}\mid {a}_i\mid }{n}\right)+\alpha \cdot \left(1-\frac{\sum_{0\le i<n}{\left({t}_i-T\right)}^2}{T\cdot n}\right) $$

where $ {a}_i $ represents the action value (−100 to 0) for how much to open the air conditioner valve in the $ i $ th room in the simulation, $ n $ represents the number of rooms, $ T $ represents the target room temperature setting, and $ {t}_i $ represents the indoor temperature of a specific room. This equation aims to minimize the difference between the target temperature and the current temperature while minimizing valve operation to ensure energy efficiency.

As a preliminary experiment, we used this reward function as a prompt and provided hypothetical room temperatures to GPT-4 (OpenAI, 2023) to evaluate its feasibility as a simulator for HVAC temperature setting. Specifically, we assumed one HVAC unit per room and set the target room temperature to 22°C for cooling. We then instructed GPT-4 to calculate the action values for hypothetical current room temperatures of [19°C, 23°C, 25°C, 20°C]. The output results were [−30, −20, −60, −20]. This indicated significant fluctuations in the action values, with only about 30% of the trials in the preliminary experiment producing outputs with consistent magnitude relationships.

3.2. Human feedback

We hypothesized that increasing the constraints within the reward function might stabilize GPT-4’s output. Therefore, in addition to the objective of “controlling air conditioners to create an optimally tempered environment,” we explicitly added “providing instructions to office workers based on their feedback to enhance their comfort” to the prompt as a role for the LLMs. Specifically, we provided feedback information such as “Currently, I am in Room 1, and my comfort level is 5 on a scale of 1 to 10,” along with the aforementioned office environment simulation information. The output resulted in action values of [−10, −30, −50, −15] and a human-directed instruction stating, “Since you are currently in Room 1 with a comfort level of 5, we recommend moving to a room closer to the target temperature. Since Room 3 is warmer than the target temperature and you currently feel cold in Room 1, we recommend moving to Room 3.” This result demonstrates that while generative AI may produce fluctuating results when seeking precise solutions, adding linguistic explanations alongside action values enables logical and concrete explanations. Furthermore, incorporating human feedback into the generative AI model increases the simulation’s specificity and leads to more relevant responses. Based on this preliminary experiment, we decided to incorporate office worker feedback into the prompts for our main study, enabling the generative AI to perform simulations that consider subjective sensory information.

3.3. Energy consumption estimation

The reward function used in Song et al. (Reference Song, Zhang, Zhao and Bian2023b)’s input to the generative AI was an equation related to valve control for air conditioners installed individually in each room. However, since the target office for this field experiment has seven HVAC units in one room, it became necessary to modify the reward function equation. Therefore, based on the research by Semitsu et al. (Reference Semitsu, Kaizu, Komatsu, Nakamura, Akagi and Hamada2023), we decided to use the following equation as the reward function, representing the estimated energy consumption $ {P}_E $ of each HVAC unit:

(2a) $$ \Delta T(t)={T}_{\mathrm{set}}(t)-{T}_{\mathrm{current}}(t) $$

(2b) $$ D(t)=\left\{\begin{array}{ll}1,& \mathrm{if}\;\Delta T(t)\ge 1.5\\ {}0,& \mathrm{otherwise}\end{array}\right. $$

(2c) $$ {P}_E=3\hskip0.1em \mathrm{kW}\cdot \frac{1}{H}{\sum}_{t=0}^{H-1}D(t) $$

The variable $ t $ represents time (in hours), $ {T}_{\mathrm{set}}(t) $ represents the set temperature at time $ t $ (°C), and $ {T}_{\mathrm{current}}(t) $ represents the current temperature at time $ t $ (°C). $ \Delta T(t) $ represents the difference between the set temperature and the current temperature at time $ t $ (°C). Additionally, $ D(t) $ is a variable that indicates whether the difference between the set temperature and the current temperature at time $ t $ is greater than or equal to 1.5°C (1: greater than or equal to 1.5°C, 0: less than 1.5°C), and $ H $ represents the measurement duration in hours. In this experiment, $ H $ is set to 2. Semitsu et al. (Reference Semitsu, Kaizu, Komatsu, Nakamura, Akagi and Hamada2023) developed a machine learning model to predict air conditioning energy consumption in an office, utilizing the difference between set temperature and current room temperature as an explanatory variable and finding that a difference exceeding 1.5°C resulted in an approximate +3 kW impact on power consumption. In this experiment, we predicted the hourly power consumption for each HVAC unit and included it in the prompt.

3.4. Office overhead map analysis

To enhance the specificity of generative AI simulations, overhead imagery incorporating room layout, temperature distribution, and occupant locations were introduced (Figure 2). This leveraged [MELRemo-IPS], a commercial system that maps office worker locations and indoor temperature onto floor plans. Worker locations are determined via Bluetooth signal strength between registered devices and central units (see Figure 3a). Spatial temperature prediction utilizes data from HVAC-integrated thermal sensors. Furthermore, a separate commercial product from [SORACOM] provides localized illuminance and temperature measurements.

Figure 3. In this experiment, (a) was used to track occupant locations and estimate spatial temperature distribution, while (b) was employed for measuring illuminance and obtaining pinpoint temperature.

Leveraging GPT-4V’s image-to-text capabilities and employing the prompt engineering technique of Yang et al. (Reference Yang, Li, Lin, Wang, Lin, Liu and Wang2023b), we provided the model with overhead imagery and prompted it to describe occupant positions, room temperature, and spatial information (see Figure 6). While minor inaccuracies in localized temperature and precise positioning were observed, the model effectively identified general “hot” and “cold” zones. Consequently, these descriptions were integrated, anticipating a positive impact on HVAC control strategy. This dynamic approach incorporates real-time office information into the prompt, enabling the multimodal foundation model to reflect the dynamically changing office environment.

3.5. Proposed system concept and framework

This article proposes a system that utilizes generative AI to optimize the environment within an office. As illustrated in Figure 4, the system operates as follows: environmental data from the office, including IoT sensor information and feedback from office workers, are sensed in the physical domain and transmitted to the cloud. In the cyber domain, within the cloud, prompts are generated based on the acquired office environment data, and these prompts are used as input to a generative AI model. The generative AI model is tasked with predicting optimal HVAC setpoint temperatures for different zones or units within the office. These predicted setpoints are then communicated to office occupants, who can implement the changes using thermostats to control the HVAC system. The resulting changes in the HVAC system, in turn, affect the office environment. This iterative cycle allows the system to adapt to the dynamically changing office environment. The optimization of temperature setpoints using generative AI in the cloud represents the cyber-domain component of the system, while the HVAC control and data acquisition from the office constitute the physical-domain component. Therefore, this loop exemplifies a Cyber-Physical System (CPS).

Figure 4. Conceptual diagram of the proposed system, illustrating the cyber-physical loop: real-world office environment data is collected, used by generative AI in the cloud to predict optimal HVAC temperatures, and then applied to control the physical HVAC system.

To realize AI systems driven by machine learning and deep learning, incorporating human intervention in specific decision-making and control processes is referred to as “Human-in-the-Loop” (HITL). However, this study proposes an “Office-in-the-Loop” system that goes beyond relying solely on human feedback. Instead, it integrates a comprehensive range of office information, including IoT sensor data and real-time power consumption predictions, as constraints and inputs for the generative AI model. This approach utilizes diverse data sources to guide the generative AI’s decision-making and control processes, going beyond human feedback alone.

We hypothesized that generative AI could be used as a simulator for real-world optimization and implemented the following flow (see Figure 5): (1) We deployed IoT sensors in a real-world office to acquire environmental data and obtained subjective feedback on comfort from office workers, which were sent to a Prompt Generator. (2) The IoT sensor data was analyzed to create a map of the office layout drawing, showing the estimated spatial distribution of office worker locations and indoor temperatures. (3) This map was input to a multimodal generative AI to generate a textual description, which was also sent to the Prompt Generator. (4) The Prompt Generator automatically summarized the received information and created tasks with clearly stated Objectives and Specific Instructions, allowing LLMs to reason in a more realistic scenario. Every 2 h, the IoT sensor data was input to LLMs as Few-shot Examples via Demonstrations as Historical Data. (5) The text automatically generated by the Prompt Generator was input to LLMs to predict HVAC temperature settings that could achieve the objectives. (6) The actual HVAC temperature in the real-world office was changed. Since the office environment changes dynamically, this was further sensed by IoT sensors, returning to the process in step 1).

Figure 5. Overview of our system for HVAC control using a multimodal foundation model. Leveraging generative AI as a simulator to achieve optimal HVAC forecasting that adapts to dynamically changing real-world office environments.

4.1. Prompt design

Preliminary experiments (Section 3) demonstrated that integrating human feedback alongside energy efficiency and comfort constraints within a generative AI simulation framework yields more realistic and practical suggestions. This study explores effective prompting strategies for applying generative AI to a practical scenario. Beginning with a review of existing research (Section 3.1), we modify our approach to the specific case study by incorporating feedback mechanisms (Section 3.2), estimated power consumption data (Section 3.3), and office map information (Section 3.4). The final prompt was validated in a dedicated preliminary experiment conducted in a real office environment to confirm the effectiveness. Moreover, incorporating office layout imagery enables the model to dynamically assess room state, including spatial temperature distribution and occupant locations. Based on these findings, we carefully designed our prompts.

First, as illustrated in Figure 6, we provide the Objectives and Specific Instructions within the prompt. Here, we assign the role of an HVAC manager tasked with maintaining the room temperature at the target level while optimizing energy consumption. We sequentially feed external environmental information, such as outside temperature and sunlight intensity, along with details regarding HVAC placement and control parameters within the room, prompting the model to predict the optimal set temperature for each unit.

Figure 6. Our prompt engineering for controlling HVAC.

Next, we provide the following information to the prompt as the current situation: (1) room temperature of each area with an HVAC unit, (2) set temperature of each HVAC unit, (3) estimated power consumption of each HVAC unit (calculated using Equation (2)), (4) feedback from office workers sent to each HVAC unit, and (5) external environmental information including outside temperature and sunlight intensity.

Subsequently, as depicted in Figure 2, we input the office layout image into the prompt and utilize a multimodal foundation model to generate descriptions of the spatial temperature distribution and office worker locations within the office.

Finally, we provide Demonstrations containing Historical Data, including past LLM-generated set temperatures, office environment information, and external environment information, to the prompt.

To ensure the LLM’s outputs are comprehensible and reliable, we have incorporated the concept of Self-Consistency (Wang et al., Reference Wang, Wei, Schuurmans, Le, Chi, Narang, Chowdhery and Zhou2023). Specifically, we provide a range for the control values (e.g., 18°C to 26°C) within the prompts as a guardrail and perform multiple inferences to make a comprehensive judgment. The final HVAC setting value occurring most frequently over 10 trials was selected for realizing a more stable prediction. During the experiment, human operators double-checked and ensured the safety of the temperature settings. The experiment spans from 8 AM to 6 PM, with the generative AI predicting optimal set temperatures for each HVAC unit every 2 h. Therefore, the Demonstrations accumulate as examples every 2 h. Office workers provide subjective feedback on their thermal sensation to each HVAC unit using a seven-point scale ranging from −3 to +3 (−3 being cold, 0 being comfortable, and + 3 being hot), similar to the PMV index. Notably, we pair the LLM-generated set temperatures for each HVAC unit with the actual room temperature of the corresponding area 2 h after setting, feedback from office workers, and the estimated power consumption of each HVAC unit. As the Demonstrations increase, the LLM’s simulated estimates are updated with real-world data, promoting adaptation and more accurate predictions aligned with the real-world environment as the trials progress.

In this article, we extend the previously discussed prompt engineering by investigating the impact of allowing the generative AI to validate its own predictions using the acquired environmental data. This is compared against the performance of Self-Consistency (Wang et al., Reference Wang, Wei, Schuurmans, Le, Chi, Narang, Chowdhery and Zhou2023). We introduce Data-Driven Reasoning, a technique that prompts the model to provide rational explanations for its predictions, grounded in the environmental data. Our findings indicate that, combined with zero-shot CoT, Data-Driven Reasoning not only facilitates the generation of human-interpretable explanations based on the data but also improves prediction accuracy, achieving effective results with a reduced number of trials. A detailed analysis of this is presented in the Ablation Study, Section 6.

4.2. Experimental design

This study was conducted as a real-world experiment in an actual office environment for a total of one and a half months, divided into two separate experimental periods. The first period ran from 15 January to 9 February, while the second period lasted from 26 February to 8 March. Three different approaches for controlling the HVAC system were compared:

• • Baseline: The baseline HVAC system operated in its default automatic mode, maintaining a constant set temperature of 23°C, which is the standard winter setting for this particular office. While the baseline system (e.g., [PEFY-P140M-E1]) lacks modern AI-powered features, its automatic mode heuristically adjusts heating/cooling based on temperature fluctuations. Specifically, if the temperature deviates from the setpoint by 1.5°C for 3 min, the system activates the appropriate mode. This represents a common, though simplistic, commercial HVAC setup.

• • LLMs + MFMs: This approach integrates Large Language Models (LLMs) with Multimodal Foundation Models (MFMs). The MFM analyzes overhead office imagery, extracting environmental information such as occupant count and temperature distribution. This information, combined with estimated energy consumption data and two-hourly occupant comfort feedback, is input to the LLMs, which subsequently generate optimal HVAC set temperatures.

• • LLMs: This approach was similar to LLMs + MFMs, but without the use of the MFM to analyze overhead images.

While the experiment plan was finalized 3 weeks prior to execution, the specific execution dates for the proposed method and baseline were not pre-determined with respect to weather conditions. To ensure the robustness and reliability of the baseline measurements, we performed each step of the process twice over multiple weeks. Due to the large room (322 $ {\mathrm{m}}^2 $ ), room temperature has not changed rapidly in response to HVAC setpoint adjustments, indicating a significant thermal inertia. This is due to the large heat capacity such as air, furniture, and walls. Short control intervals would be insufficient to induce a perceivable temperature change for occupants. Therefore, to account for this thermal inertia and ensure a significant impact on room temperature, a control frequency of 2 h was selected for this experiment.

6.1. Rational explanation in data-driven reasoning

As an ablation study, the same prompts used in the 7 March experiment were input to modern MFMs (Gemini-1.5-pro-exp-0827, GeminiTeam, 2024a) and GPT-4o-20240513 (OpenAI, 2024). Optimal settings for the seven HVAC units were predicted based on the mode of 10 trials. The modern MFMs tended to produce a more stable setpoint prediction when the mode of 10 trials was taken same as Self-Consistency (Wang et al., Reference Wang, Wei, Schuurmans, Le, Chi, Narang, Chowdhery and Zhou2023). Table 5 indicates a tendency toward convergence to similar results. The Turn number represents the prediction results at 2-h intervals, starting at 10:00 AM. Although the generative AI models produce similar predictions for the afternoon, their predictions diverge in the morning. This divergence can likely be attributed to the more substantial fluctuations in both outdoor and indoor temperatures during the morning hours, coupled with the area-specific variations in feedback. Consequently, the models may infer multiple plausible optimal temperatures from similar input information.

Table 5. Variations in daily predictions across different generative AI models on 7 March

Note: “–” indicates inactivity; all values are in degrees Celsius. Italics denote area numbers.

Table 6 presents the output and reasoning generated by Gemini1.5 (GeminiTeam, 2024a), a multimodal generative AI, when tasked with predicting HVAC settings. It seems that target temperatures for each area are determined based on the current room temperature and feedback. Furthermore, the HVAC setpoint is determined by considering factors such as proximity to windows and illuminance intensity to achieve these target temperatures. Given the improved reasoning capabilities of these state-of-the-art models, it is reasonable to expect that incorporating human-understandable explanations into their outputs will enhance the acceptability and adoption of generative AI for industrial equipment control in industrial practical scenarios.

Table 6. Results of querying Gemini1.5 (GeminiTeam, 2024a) on March 7 (Turn 4 prompt) for the rationale behind HVAC settings

Furthermore, acknowledging the remarkable recent advancements in Agentic AI, we investigated whether the simulation capabilities of these models, particularly large-scale generative AI models designed for multi-stage deep reasoning, could yield more rational and mathematically sound explanations. This investigation is based on the hypothesis that the internal representations of these modern generative AI models, trained with vast numbers of parameters, sufficiently learn real-world phenomena and can therefore function as world models.

For example, OpenAI’s o1 (OpenAI, 2025) is trained with large-scale reinforcement learning to reason using CoT prompting, enabling it to respond with deliberate contextual consistency. Similarly, Google’s Gemini 2.0 (Pichai et al., Reference Pichai, Hassabis and Kavukcuoglu2024) is reportedly developed for Agentic AI, capable of understanding the world more deeply and reasoning autonomously over multiple steps.

Initially, we examined how predictions varied when Agentic AI models were provided with prompts summarizing historical information obtained from our experiments, including past IoT sensor data, office worker feedback, and HVAC settings. Previous experiments revealed that predicting precise optimal temperatures with generative AI models was challenging, with variations of around 2°C observed across trials. Therefore, we employed Self-Consistency, a prompt engineering technique that involves multiple inference runs and selecting the mode of the outputs, to stabilize predictions. However, Self-Consistency incurs a cost proportional to the number of inference runs required to determine the mode.

In this study, we discovered that incorporating Data-Driven Reasoning, where the generative AI explains its rationale based on provided data, had the effect of stabilizing the predicted optimal HVAC temperatures, causing them to converge to a consistent value. The following prompt, adapted from the main experiment, was used for the Data-Driven Reasoning runs:

Therefore, for the days during the experimental period where only LLMs were used, we conducted 10 inference runs using Self-Consistency with modern models such as o1 and Gemini 2.0 to determine the mode. We then evaluated how closely a single inference run using Data-Driven Reasoning with the same model matched the Self-Consistency result, using the following methods:

• • HVAC ON/OFF prediction: The HVAC ON/OFF predictions in our experiments were treated as a binary classification task. We assessed whether the Data-Driven Reasoning prediction matched the mode of the 10 Self-Consistency trials, calculating Precision and Recall.

• • HVAC temperature prediction: For HVAC temperature predictions, we calculated the success rate of the single Data-Driven Reasoning prediction falling within $ \pm $ 1°C and $ \pm $ 0.5°C of the mean of the 10 Self-Consistency trials. We also calculated the Mean Absolute Error (MAE) between the Data-Driven Reasoning prediction and the mean of the Self-Consistency trials.

• • Temperature parameter: The temperature parameter for the generative AI models was set to 0, as used in the main experiments. However, for o1, the default value was used, as this model does not support temperature modification.

For the experiments, we utilized the following Generative AI models: GPT-4 (gpt-4-1106-preview, OpenAI, 2023), GPT-4o (2024-05-01-preview, OpenAI, 2024), OpenAI o1 (2024-12-01-preview, OpenAI, 2025), Claude 3.5 Sonnet V2 (claude-3-5-sonnet-v2@20241022, Anthropic, 2024), Gemini 1.5 Pro (gemini-1.5-pro-002, GeminiTeam, 2024a), and Gemini 2.0 Flash (gemini-2.0-flash-exp, Pichai et al., Reference Pichai, Hassabis and Kavukcuoglu2024).

Figure 13 illustrates the prediction spread for the optimal HVAC temperature when historical data from the experimental period was provided as input to each generative AI model without employing Data-Driven Reasoning, instead using Self-Consistency with 10 inference trials to determine the mode. Utilizing the prompt structure shown in Figure 6 without Data-Driven Reasoning and performing 10 inferences via Self-Consistency tends to narrow the prediction spread of the optimal temperature. However, the results indicate that, depending on the model, the distribution containing 50% of the data (i.e., interquartile range) still varies by approximately 2 °C across the dataset.

Figure 13. Box plot of HVAC temperature predictions from 10 inferences using Self-Consistency without Data-Driven Reasoning.

Table 7 presents the results comparing Data-Driven Reasoning against Self-Consistency for two tasks: binary classification (i.e., predicting HVAC ON/OFF state) and temperature prediction (i.e., predicting the optimal setpoint). Each numerical value in the table represents the average performance over the 13-day experimental period, where the historical information was fed into each generative AI model to predict the HVAC operational strategy.

Table 7. Performance comparison of different models for HVAC control tasks

First, regarding the Binary Classification task (see Table 7), the accuracy exhibited significant variation depending on the generative AI model used. NotablyMFMs such as GPT-4o and Gemini 1.5 Pro, as well as the agentic AI model Gemini 2.0, consistently predicted the HVAC ON/OFF state with stability. For these models, the accuracy achieved with a single prediction using Data-Driven Reasoning was remarkably close to the accuracy obtained using the mode of 10 trials with Self-Consistency alone, demonstrating high performance. Conversely, models like OpenAI o1 and Claude 3.5 Sonnet showed a tendency for their ON/OFF instructions to fluctuate across trials, resulting in somewhat lower accuracy. These findings suggest that while model-specific accuracy differences exist, employing Data-Driven Reasoning enables high-accuracy prediction of HVAC operational states for the binary classification task.

Next, focusing on the Temperature Prediction task (see Table 7), we examined the accuracy of predictions falling within $ \pm $ 1°C of the mean temperature predicted by Self-Consistency. In this task too, multimodal foundation models like Gemini 1.5 Pro and the agentic AI Gemini 2.0 demonstrated stable performance, predicting the optimal HVAC setpoint temperature with high accuracy, exceeding 90%. Furthermore, Gemini 2.0 achieved an accuracy of over 90% even within the stricter range of $ \pm $ 0.5°C from the Self-Consistency mean prediction. Multimodal models like GPT-4o and Gemini 1.5 also achieved accuracies exceeding 80% within the $ \pm $ 1°C range. Referring back to the distribution of predicted temperatures using Self-Consistency shown in Figure 13, it is evident that while the prompt engineering strategy from the proposed method (see Figure 6) helps reduce prediction variance, consistently achieving temperature predictions with over 90% accuracy within a $ \pm $ 0.5°C range remains a challenging task. However, the effectiveness of our proposed approach, particularly the Data-Driven Reasoning component, becomes evident when considering the initial prediction spread. As shown by the box plot in Figure 13, the interquartile range (IQR) of the Self-Consistency predictions spans approximately 2°C, representing the central 50% of the data and indicating significant initial variability. Despite this wide initial distribution, our method enables all tested generative AI models to achieve over 50% accuracy within the narrow $ \pm $ 0.5°C range, with Gemini 2.0 remarkably surpassing 90%. This demonstrates that Data-Driven Reasoning effectively mitigates the inherent prediction variance, yielding highly precise and reliable temperature setpoints. Furthermore, examining the Mean Absolute Error (MAE) in Table 7, the average prediction errors for Gemini 2.0, Gemini 1.5, and GPT-4 were below 0.5°C, indicating that their responses were stable relative to the given information. In contrast, OpenAI o1 sets a fixed default temperature parameter, causing its predictions to deviate significantly depending on the target, thus resulting in the lowest accuracy for this task in our experiments. These results collectively indicate that although performance fluctuations exist depending on the specific generative AI model used, employing Data-Driven Reasoning offers the potential to reduce the computational and financial costs associated with prompt engineering techniques like Self-Consistency, which require multiple inferences to determine the most frequent output. This reduction in overhead is a significant consideration for practical deployment in industrial scenarios.

6.2. Impact of advanced agentic AI on energy consumption

We simulated the potential energy savings achievable by operating the HVAC system with optimal temperature settings predicted by state-of-the-art Agentic AI, using the experimental history data as input. The regression coefficient $ {b}_{LLMs}=-49.512 $ obtained from the regression analysis (see Table 4) represents the change in total daily energy consumption when switching from the baseline control strategy without LLM to the LLM-based control strategy. This coefficient cannot be directly applied to temperature differences. Therefore, we used the regression coefficient $ {b}_{LLMs} $ to estimate the impact on the energy by the change in the HVAC ON/OFF by comparing control strategies, between the one based on GPT-4 used in the original experiment and that of the new Agentic AI models.

First, using the 13-day experimental history data employing only the LLM, we prepared prompts containing one day’s worth of demonstration information. These prompts were used as input for each Agentic AI. We employed the Self-Consistency method, performing 10 trials and selecting the mode as the predicted optimal HVAC settings (temperature and ON/OFF state) for each Agentic AI. For each HVAC unit, the estimated energy change was calculated based on the agreement between the Agentic AI’s prediction and the GPT-4 prediction used in the experiment. If the predictions matched, the estimated energy change was set to 0. If the Agentic AI predicted OFF while GPT-4 predicted ON, the estimated energy change was the negative of the average energy impact per HVAC unit. Conversely, if the Agentic AI predicted ON while GPT-4 predicted OFF, the estimated energy change was the positive of the average energy impact per HVAC unit.

The average energy impact per unit is calculated by dividing the regression coefficient $ {b}_i $ by the number of HVAC units ( $ N $ ), assuming equal contribution from each unit. The total estimated daily energy change was calculated by summing the $ \Delta {E}_i $ values for all seven HVAC units. This change was then added to the average daily energy consumption of GPT-4 during the experiment (42.92 kWh) to obtain the estimated total daily consumption of the new model. Furthermore, we calculated the percentage change in energy consumption of each Agentic AI relative to the GPT-4-based operation and baseline. The results are shown in Table 8.

Table 8. Model comparison of estimated energy consumption and reduction from baseline

Note: The power consumption values for Baseline and GPT-4 represent the actual usage measured during the experiments, as presented in Table 1.

Based on the results in Table 8, projections suggest a high likelihood that utilizing newer generative AI models—specifically GPT-4o, OpenAI o1, and Gemini 1.5—for HVAC operation could achieve further reductions in energy consumption compared to the GPT-4 model used in the actual experiments. For Gemini 2.0, the projected energy consumption was equivalent to that of the actual GPT-4 operation, attributed to the lack of relative changes in its predicted HVAC ON/OFF switching behavior. Conversely, predictions from Claude 3.5 frequently indicated activating the HVAC (i.e., ON state) during periods when the actual control (i.e., implemented by GPT-4) maintained an OFF state; consequently, its use is projected to increase energy consumption relative to GPT-4. However, it is anticipated that all evaluated generative AI models can deliver substantial energy savings compared to the baseline HVAC automatic operation mode (i.e., operation without generative AI).

6.3. Mathematical modeling of office environments using agentic AI

As an additional experiment, we investigated whether the detailed historical information obtained in our experiments, including IoT sensor data and worker feedback, could be leveraged by a state-of-the-art Agentic AI to generate a mathematical formulation for optimizing the trade-off between office environment comfort and cost reduction on a daily basis. Typically, optimizing the control of office equipment involves accumulating a large amount of data and then using reinforcement learning to train the weights of a human-defined model. In this experiment, we provided the following additional prompt to the Agentic AI, instructing it to perform the office environment modeling itself and to derive optimal coefficients and weights using the historical information, and examined whether the AI could explain it.

Tables 9–11 show the predicted optimal HVAC temperatures when all prompts up to Turn 5 on 7 March are given, along with the explanations of the office environment modeled from the daily office environment data described by each Agentic AI. While all methods yielded the same optimal HVAC temperature setting prediction, the resulting mathematical formulations of the office environment differed across the models.

Table 9. Results of querying GPT-4o (OpenAI, 2024) on 7 March (Turn 5 prompt) for the rationale behind HVAC settings with mathematical reasoning

Note: The results of the optimal temperature prediction for HVACs 1 through 7 in the “Optimal settings” are shown in order from the beginning of the array. 0 indicates shutdown, and each numerical value represents the temperature in degrees Celsius.

Looking at the results of GPT-4o in Table 9, the model calculated weights that prioritize reducing power consumption over improving comfort, and provided a justification for this. However, it failed to adequately explain the equations for energy consumption and comfort.

The results of OpanAI o1 in Table 10 show that the model consists of three component equations: one relating to the temperature perceived as comfortable by office workers and the current room temperature, another relating to the outside temperature and the current set temperature, and a third relating to sudden changes in the set temperature. This model emphasizes comfort in its equation explanation. Notably, it provides a comprehensive explanation of the phenomena, including how a large difference between the outside temperature and the set temperature leads to significant power consumption, and how abruptly changing the set temperature oneself places a large load on the HVAC system, increasing power consumption. All of these equations are constructed based on the acquired data, resulting in higher explanatory power compared to the results of GPT-4o.

Table 10. Results of querying OpanAI o1 (OpenAI, 2025) on 7 March (Turn 5 prompt) for the rationale behind HVAC settings with mathematical reasoning

Note: The results of the optimal temperature prediction for HVACs 1 through 7 in the “Optimal settings” are shown in order from the beginning of the array. 0 indicates shutdown, and each numerical value represents the temperature in degrees Celsius.

The results of Gemini 2.0 in Table 11 show that the weight settings prioritize the comfort of office workers. Regarding the comfort equation, the sum of the difference between the ideal room temperature and the current room temperature, plus the feedback on the current room temperature, is inverted. This indicates that a smaller difference between the ideal temperature and the current temperature, and feedback closer to 0 (indicating comfort), is better. On the other hand, the equation for energy consumption is designed such that the smaller the discrepancy between the set temperature and the room temperature, the higher the overall evaluation.

Table 11. Results of querying Gemini2.0 (Pichai et al., Reference Pichai, Hassabis and Kavukcuoglu2024) on 7 March (Turn 5 prompt) for the rationale behind HVAC settings with mathematical reasoning

Note: The results of the optimal temperature prediction for HVACs 1 through 7 in the “Optimal settings” are shown in order from the beginning of the array. 0 indicates shutdown, and each numerical value represents the temperature in degrees Celsius.

As shown in Tables 9–11, the variations in the explanations provided by each Agentic AI demonstrate that even when using the same prompts, including sensing data of the office environment and worker feedback information, to predict the same optimal temperature setting, the mathematical explanations of how the office environment was modeled and how the optimal setting temperature was derived differ.

An interesting point of this experiment is that when Agentic AI is used to provide mathematical explanations for complex real-world events, it can provide rational explanations based on various explanatory variables of the office environment obtained on that day. Specifically, the problem of designing the weights of the cost function included in the equation typically requires collecting a large amount of data and performing reinforcement learning or physical modeling to construct a highly accurate simulator. However, the results of this experiment suggest that Agentic AI has the potential to be used as a simplified simulation even though in real-world complex scenarios.