Preference-based reinforcement learning (PbRL) significantly simplifies the design of reward functions in reinforcement learning (RL) tasks. However, because of the tasks’ complexity, intransitive preferences, and sensitivity to preference errors, PbRL requires substantial feedback to achieve the desired performance. This extensive reliance on expert input notably increases the burden on participants. We have developed a novel framework: Self-teacher-learning preference-based reinforcement learning (STL-PbRL). In the teacher-led (TL) module, agents learn more reliable reward prediction models (RM) through TL PbRL. In the self-learning (SL) module, agents utilizing the preference comparison approach for trajectory segments integrate sparse but critical, easily designed task-oriented information into the feedback process. The STL-PbRL framework incorporates the SL module to refine the RM initially generated by the TL module. We have demonstrated that this integration significantly enhances RM by enabling RM to converge toward an optimal reward model that effectively supports achieving a training policy that meets task objectives. This streamlined and efficient STL-PbRL framework enables a more accurate and efficient training process. Our experimental results confirm that the SL module seamlessly integrates with existing PbRL algorithms, significantly reducing the need for feedback and alleviating the impact of errors in preference indications. This efficiency and effectiveness highlight that STL-PbRL innovates, simplifies, and enhances the RL training process across various applications.

There have been significant interests and efforts in the field of impedance control on robotic manipulation over last decades. Impedance control aims to achieve the desired mechanical interaction between the robotic equipment and its environment. This paper gives the overview and comparison of basic concepts and principles, implementation strategies, crucial techniques, and practical applications concerning the impedance control of robotic manipulation. This work attempts to serve as a tutorial to people outside the field and to promote discussion of a unified vision of impedance control within the field of robotic manipulation. The goal is to help readers quickly get into the problems of their interests related to impedance control of robotic manipulation and to provide guidance and insights in finding appropriate strategies and solutions.

We present a general framework for modeling a class of mechanical systems for robotic manipulation, consisting of articulated limbs with redundant tendinous actuation and unilateral constraints. Such systems, that include biomorphically designed devices, are regarded as a collection of rigid bodies, inter-acting through connections that model both joints and contacts with virtual springs. Methods previously developed for the analysis of force distribution in multiple whole-limb manipulation are generalized to this broader class of mechanisms, and are shown to provide a basis for the control of co-contraction and internal forces that guarantee proper operation of the system. In particular, in the presence of constraints such as those due to limited friction between surfaces or object fragility, the choice of tendon tensions is crucial to the success of manipulation. An algorithm is described that allows to evaluate efficiently set-points for the control of tendon actuators that “optimally” (in a sense to be described) comply with the given constraints.