The appeal of quadruped robots lies not only in their ability to mimic the diverse and agile locomotion of animals but also in their potential to integrate with human artistic expression to achieve complex multi-skill movements. Continuous multi-skill motion in quadruped robots requires the realization of diverse, continuous, and long-horizon behaviors, involving a broader state space and more complex motion generation and transitions. This presents significant challenges, including sparse rewards, incomplete data, long-horizon motion planning, and the design of fine-grained motion transitions. In this work, we categorize quadruped robot skills into three types: rhythmic motions, expressive motions, and high-dynamic motions and generate reference trajectories for each category using central pattern generators, animation design, and motion capture, respectively. We then design an asymmetric neural network architecture and employ an imitation–reinforcement learning algorithm to train policies for generating these three types of motions. By composing multiple motion skill trajectories, we avoid long-horizon motion planning; by leveraging reinforcement learning, we enable smooth and continuous skill transitions; and by introducing a two-stage reference state initialization curriculum, the robot is able to switch from arbitrary states to the target motion skill. Moreover, during training, the policy imitates only key characteristics of the reference motions rather than strictly tracking fixed trajectories, making it more robust. Finally, we achieve robust motion skill generation and seamless transitions on a quadruped robot equipped with a 6-DoF manipulator, validating the effectiveness and feasibility of the proposed multi-skill generation and transition method.

This work explores Reinforcement Learning (RL) for the circular design of planar truss linkages using available bars and pins. A bipartite graph representation and elementary action formulation enable agents to assemble mechanisms in a physics-based environment. Results for a force-inverter design problem show 98.5% success for fixed-stock training and 66.0% for shuffled stocks. The method demonstrates RL’s potential for inventory-constrained mechanism synthesis, with future work targeting scalable, indexing-invariant architectures and more flexible connection actions.

We present FORGE (Framework for Optimization and Reinforcement-driven Generative Engineering), a probabilistic programming framework for generative design that unifies declarative, symbolic modeling and reinforcement learning (RL). FORGE can learn and refine a design generator through RL based on simulator-derived rewards. We demonstrate FORGE across several vehicle domains. FORGE creates an extensible, interpretable foundation for generative engineering. It can act as both a data generator for machine learning and a design optimizer, offering a practical alternative to purely neural methods.

The applicability and scalability of design adaptations utilizing reinforcement learning can be broadened by using graph-based approaches instead of rigid vector- or grid-based ones. However, graph-based approaches often require a high number of simulations to converge. To reduce the simulation effort in the mechanical optimisation, the reinforcement learning setup is enriched with task-specific causal and physically based information. This work systematically investigates the influence of this additional information on the efficiency of design adaptations using a factorial test design.

Motivation and gap: PID-family controllers remain a pragmatic choice for many robotic systems due to their simplicity and interpretability, but tuning stable, high-performing gains is time consuming and typically non-transferable across robot morphologies, payloads, and deployment conditions. Fuzzy gain scheduling can provide interpretable online adjustment, yet its per-joint scaling and consequent parameters are platform dependent and difficult to tune systematically.

Proposed approach: We propose a hierarchical framework for cross-platform tuning of a learnable fuzzy gain-scheduled PID (LF-PID). The controller uses shared fuzzy membership partitions to preserve common error semantics, while learning per-joint scaling and Takagi–Sugeno consequent parameters that schedule PID gains online. Combined with physics-constrained virtual robot synthesis, meta-learning provides cross-platform initialization from robot physical features, and a lightweight reinforcement learning (RL) stage performs deployment-specific refinement under dynamics mismatch. Starting from three base simulated platforms, we generate 232 physically valid training variants via bounded perturbations of mass ($\pm$10%), inertia ($\pm$15%), friction ($\pm$20%), and damping ($\pm$30%).

Results and insight: We evaluate cross-platform generalization on two distinct systems (a 9-DOF serial manipulator and a 12-DOF quadruped) under multiple disturbance scenarios. The RL adaptation stage improves tracking performance on top of the meta-initialized controller, with up to 80.4% error reduction in challenging high-load joints (12.36$^\circ$$\rightarrow$2.42$^\circ$) and 19.2% improvement under parameter uncertainty. We further identify an optimization ceiling effect: online refinement yields substantial gains when the meta-initialized baseline exhibits localized deficiencies, but provides limited improvement when baseline quality is already uniformly strong.

Semantic textual similarity (STS) is to measure semantic equivalence between sentences; it plays an important role in natural language processing (NLP) tasks. The major core of STS is text representation. This paper studies how to obtain a sentence skeleton for text representation in STS. Unlike most existing syntax models, we propose a skeleton-based reinforcement learning method to identify the skeleton and construct sentence representations. Parallel networks are adopted to extract features of different dimensions in the sentence. In the framework of parallel networks, two sentence representation models are designed: context constrained LSTM (CC-LSTM) and Adorned skeleton LSTM (AS-LSTM). CC-LSTM builds the sentence representation by constraining the word context. AS-LSTM constructs the sentence representation through using the identified skeleton and its qualifiers. Our approach achieves good results without using external resources. Especially AS-LSTM, which outperforms the state-of-the-art without using external resources in the SICK dataset.

Preference-based reinforcement learning (PbRL) significantly simplifies the design of reward functions in reinforcement learning (RL) tasks. However, because of the tasks’ complexity, intransitive preferences, and sensitivity to preference errors, PbRL requires substantial feedback to achieve the desired performance. This extensive reliance on expert input notably increases the burden on participants. We have developed a novel framework: Self-teacher-learning preference-based reinforcement learning (STL-PbRL). In the teacher-led (TL) module, agents learn more reliable reward prediction models (RM) through TL PbRL. In the self-learning (SL) module, agents utilizing the preference comparison approach for trajectory segments integrate sparse but critical, easily designed task-oriented information into the feedback process. The STL-PbRL framework incorporates the SL module to refine the RM initially generated by the TL module. We have demonstrated that this integration significantly enhances RM by enabling RM to converge toward an optimal reward model that effectively supports achieving a training policy that meets task objectives. This streamlined and efficient STL-PbRL framework enables a more accurate and efficient training process. Our experimental results confirm that the SL module seamlessly integrates with existing PbRL algorithms, significantly reducing the need for feedback and alleviating the impact of errors in preference indications. This efficiency and effectiveness highlight that STL-PbRL innovates, simplifies, and enhances the RL training process across various applications.

We propose a deep reinforcement learning (RL) framework designed to optimize the hedging of specific, user-defined risk factors—referred to as targeted risks—in financial instruments affected by multiple sources of uncertainty. Our methodology uses Shapley value decompositions to establish source of risk grouping’s contribution to the projected contract cash flows, providing a clear attribution of the profit and loss to distinct risk categories. Leveraging this decomposition, we apply deep RL to hedge only the targeted risks, while leaving non-targeted risks mostly unaffected. In addition, we introduce a joint neural network architecture in which the agent network utilizes risk estimates from a risk measurement neural network to stabilize the hedging strategy, taking into account local risk dynamics. Numerical experiments show that our approach outperforms traditional methods, such as delta hedging and traditional deep hedging, significantly reducing targeted risks in variable annuities while maintaining flexibility for broader applications.

This study presents a framework that combines Bayesian inference with reinforcement learning to guide drone-based sampling for methane source estimation. Synthetic gas concentration and wind observations are generated using a calibrated model derived from real-world drone measurements, providing a more representative testbed that captures atmospheric boundary layer variability. We compare three path planning strategies—preplanned, myopic (short-sighted), and non-myopic (long-term)—and find that non-myopic policies trained via deep reinforcement learning consistently yield more precise and accurate estimates of both source location and emission rate. We further investigate centralized multi-agent collaboration and observe comparable performance to independent agents in the tested single-source scenario. Our results suggest that effective source term estimation depends on correctly identifying the plume and obtaining low-noise concentration measurements within it. Precise localization further requires sampling in close proximity to the source, including slightly upwind. In more complex environments with multiple emission sources, multi-agent systems may offer advantages by enabling individual drones to specialize in tracking distinct plumes. These findings support the development of intelligent, data-driven sampling strategies for drone-based environmental monitoring, with potential applications in climate monitoring, emission inventories, and regulatory compliance.

Legged robots have demonstrated remarkable potential for dynamic locomotion and terrain adaptability, making them a prominent focus of research. However, achieving robust and agile bipedal running remains challenging due to the complex dynamics of legged locomotion. In this paper, we propose a reinforcement learning framework for robust bipedal running, incorporating a simple reference trajectory generator and an asymmetric actor-critic architecture. The reference generator, based on kinematics, provides diverse trajectory references while preserving key gait characteristics, facilitating efficient policy exploration. To mitigate the simulation-to-reality gap, we extract latent variables encoding environmental and motion information from dual historical observations. Our method simplifies the trajectory generation process while maintaining effective guidance for learning. Extensive simulation and physical experiments demonstrate that, compared to model-based and learning-based baselines, our approach achieves higher agility, more accurate velocity tracking, and stronger disturbance rejection while preserving gait stability. The resulting controller exhibits spring–mass running dynamics that remain robust on both flat and uneven terrains.

This study analyses the dynamics of the global rare earth element market, with a focus on China’s dominant role as the primary supplier, which is crucial for the energy transition and digitalization. Using a game-theoretic approach, the research examines a potential duopoly market structure that may emerge over time, as well as potential shifts in supply from China to other countries in this scenario. It considers China’s low marginal costs and factors such as resource extraction and discoveries. Additionally, the study examines the strategic market interactions, the role of technological advancements, and policy support in shaping market outcomes. The methodology assumes that agents have limited foresight and use a learned value function to strategically assess outcomes based on their own and others’ actions, while accounting for environmental constraints.

The penetration strategy of hypersonic vehicles in hostile environments is a critical factor in determining their effectiveness in completing reconnaissance or strike missions. Reinforcement learning (RL), as an end-to-end method, exhibits inherent advantages in addressing complex problems. However, existing research indicates that to enhance the efficiency of RL-based strategies, further advancements are necessary to reduce training costs and improve generalisation capabilities. This paper introduces a RL-based cooperative guidance law for multi-hypersonic vehicles, incorporating the estimated remaining time-of-flight and the absolute value of the bank angle obtained through a predictor-corrector method. The observation space and reward function are specifically designed to simplify the complex decision-making problem into a single-value decision problem, thereby reducing computational complexity and training costs. The proposed guidance law integrates the observation space, reward function and action space within the reinforcement learning framework to control flight trajectories, flight time and penetration of no-fly zones, ensuring compliance with multiple constraints. Model training and simulation tests conducted under multiple constraints demonstrate that the proposed approach reduces the training iterations required for the reinforcement learning agent and improves decision-making efficiency. Furthermore, simulations under different no-fly zone distributions confirm the proposed guidance approach’s high generalisation ability.

In this paper, we investigate a competitive market involving two agents who consider both their own wealth and the wealth gap with their opponent. Both agents can invest in a financial market consisting of a risk-free asset and a risky asset, under conditions where model parameters are partially or completely unknown. This setup gives rise to a nonzero-sum differential game within the framework of reinforcement learning (RL). Each agent aims to maximize his own Choquet-regularized, time-inconsistent mean-variance objective. Adopting the dynamic programming approach, we derive a time-consistent Nash equilibrium strategy in a general incomplete market setting. Under the additional assumption of a Gaussian mean return model, we obtain an explicit analytical solution, which facilitates the development of a practical RL algorithm. Notably, the proposed algorithm achieves uniform convergence, even though the conventional policy improvement theorem does not apply to the equilibrium policy. Numerical experiments demonstrate the robustness and effectiveness of the algorithm, underscoring its potential for practical implementation.

Goldman (2001) asks how novices can trust putative experts when background knowledge is scarce. We develop a reinforcement-learning model, adapted from Barrett, Skyrms, and Mohseni (2019), in which trust arises from experience rather than prior expertise labels. Agents incrementally weight peers who outperform them. Using a large dataset of human probability judgments as inputs, we simulate communities that learn whom to defer to. Both a strictly individual-learning variant and a reputation-sharing variant yield performance-sensitive deference, the latter accelerating convergence. Our results offer an empirically grounded account of how communities identify and trust experts without blind deference.

This paper analyzes individual behavior in multi-armed bandit problems. We use a between-subjects experiment to implement four bandit problems that vary based on the horizon (indefinite or finite) and the number of bandit arms (two or three). We analyze commonly suggested strategies and find that an overwhelming majority of subjects are best fit by either a probabilistic “win-stay lose-shift” strategy or reinforcement learning. However, we show that subjects violate the assumptions of the probabilistic win-stay lose-shift strategy as switching depends on more than the previous outcome. We design two new “biased” strategies that adapt either reinforcement learning or myopic quantal response by incorporating a bias toward choosing the previous arm. We find that a majority of subjects are best fit by one of these two strategies but also find heterogeneity in subjects’ best-fitting strategies. We show that the performance of our biased strategies is robust to adapting popular strategies from other literatures (e.g., EWA and I-SAW) and using different selection criteria. Additionally, we find that our biased strategies best fit a majority of subjects when analyzing a new treatment with a new set of subjects.

Reinforcement learning (RL) has demonstrated computational efficiency and autonomy in solving unmanned aerial vehicle (UAV) obstacle avoidance problems. However, practical applications still remain challenges, such as three-dimensional manoeuvres, dynamic obstacles and kinematic constraints. This paper proposes a real-time obstacle avoidance method based on RL and a kinematic model, where the RL framework outputs 3D-axis velocity to enable continuous UAV manoeuver control. To perceive large-scale, highly dynamic obstacles, we establish a 3D safety separation model and construct a modular observation matrix to enhance perception capability. The Soft Actor-Critic (SAC) algorithm is adopted to enhance stochastic exploration in high-dimensional state space while incorporating flight uncertainty. Under simulation, the proposed method effectively avoids dynamic obstacles. The optimised state space boosts learning speed and performance. This provides an effective solution for UAV autonomous obstacle avoidance in complex environments.