Hostname: page-component-77c78cf97d-bzm8f Total loading time: 0 Render date: 2026-04-23T08:34:08.136Z Has data issue: false hasContentIssue false
  • English
  • Français

Integrating admittance control and residual reinforcement learning for robotic assembly with visual and force feedback

Published online by Cambridge University Press:  29 December 2025

Shi Li ,
Bin Li Open the ORCID record for Bin Li [Opens in a new window]  and
Jianbo Yu
Shi Li
Affiliation:
Academy for Engineering and Technology, Fudan University, Shanghai, China
Bin Li
Affiliation:
College of Automation Science and Engineering, South China University of Technology, Guangzhou, China
Jianbo Yu*
Affiliation:
School of Microelectronics, Fudan University, Shanghai, China
*
Corresponding author: Jianbo Yu; Email: jb_yu@fudan.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper presents an innovative hybrid approach that integrates traditional control strategies with deep reinforcement learning for robotic assembly. By fusing multimodal information from visual and force feedback, the method leverages admittance control to ensure safe force feedback while using deep reinforcement learning to process visual input, enabling precise control and real-time correction of assembly actions. This multi-sensor feedback mechanism not only enhances the stability and accuracy of the assembly process but also improves the robot’s robustness and adaptability in uncertain environments. Additionally, a twin-delay deep deterministic policy gradient algorithm based on residual reinforcement learning is proposed. The design of a task-specific reward function, which simultaneously considers visual goals, force compliance, and contact stability, effectively addresses challenges such as difficult state information acquisition and sparse rewards in assembly tasks. This improves the robot’s interaction capabilities and task execution efficiency in real-world environments. Experimental results demonstrate that the method designed in this paper effectively reduces the training time for reinforcement learning from 400 epochs to 100 epochs, significantly decreases the magnitude of contact forces during the assembly process, and shortens the contact time.

Keywords

residual reinforcement learningmultimodal informationadmittance controlrobot assembly

Information

Type
Research Article
Information
Robotica , Volume 44 , Issue 1 , January 2026 , pp. 37 - 51
Copyright
© The Author(s), 2025. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Zhu, Z. and Hu, H., “Robot learning from demonstration in robotic assembly: A survey,” Robotics 7(2), 17 (2018).CrossRefGoogle Scholar
Marvel, J. A., Bostelman, R. and Falco, J., “Multi-robot assembly strategies and metrics,” ACM Comput. Surv. (CSUR) 51(1), 1–32 (2018).CrossRefGoogle ScholarPubMed
Sun, K., Zhang, J., Liu, J., Yu, R. and Song, Z., “DRCNN: Dynamic routing convolutional neural network for multi-view 3D object recognition,” IEEE Trans. Image Process. 30, 868–877 (2020).CrossRefGoogle ScholarPubMed
Li, C., Zheng, P., Yin, Y., Wang, B. and Wang, L., “Deep reinforcement learning in smart manufacturing: A review and prospects,” CIRP J. Manuf. Sci. Technol. 40, 75–101 (2023).CrossRefGoogle Scholar
Shi, Q., Lam, H.-K., Xuan, C. and Chen, M., “Adaptive neuro-fuzzy PID controller based on twin delayed deep deterministic policy gradient algorithm,” Neurocomputing 402, 183–194 (2020).CrossRefGoogle Scholar
Dankwa, S. and Zheng, W., “Twin-Delayed DDPG: A Deep Reinforcement Learning Technique to Model a Continuous Movement of an Intelligent Robot Agent,” In: Proceedings of the 3rd International Conference on Vision, Image and Signal Processing 2019 (ACM, 2019) pp. 1–5.CrossRefGoogle Scholar
Tang, S.-W., Zhou, Z.-J., Hu, C.-H., Zhao, F.-J. and Cao, Y., “A new evidential reasoning rule-based safety assessment method with sensor reliability for complex systems,” IEEE Trans. Cybern. 52(5), 4027–4038 (2020).CrossRefGoogle Scholar
Faccio, M., Bottin, M. and Rosati, G., “Collaborative and traditional robotic assembly: A comparison model,” Int. J. Adv. Manuf. Technol. 102(5−8), 1355–1372 (2019).CrossRefGoogle Scholar
Fernando, H. and Surgenor, B., “An unsupervised artificial neural network versus a rule-based approach for fault detection and identification in an automated assembly machine,” Robot. Comput. Integr. Manuf. 43, 79–88 (2017).CrossRefGoogle Scholar
Jiang, Y., Huang, Z., Yang, B. and Yang, W., “A review of robotic assembly strategies for the full operation procedure: Planning, execution and evaluation,” Robot. Comput. Integr. Manuf. 78, 102366 (2022b).CrossRefGoogle Scholar
Stolt, A., Linderoth, M., Robertsson, A. and Johansson, R., “Force Controlled Robotic Assembly Without a Force Sensor,” In: 2012 IEEE International Conference on Robotics and Automation (IEEE, 2012) pp. 1538–1543.CrossRefGoogle Scholar
Peña-Cabrera, M., Lopez-Juarez, I., Rios-Cabrera, R. and Corona-Castuera, J., “Machine vision approach for robotic assembly,” Assembly Autom. 25(3), 204–216 (2005).CrossRefGoogle Scholar
Singh, R., Seneviratne, L. and Hussain, I., “Robust pollination for tomato farming using deep learning and visual servoing,” Robotica 43(1), 86–109 (2025).CrossRefGoogle Scholar
Wang, G., Liu, Q., Ma, Q. and Qin, J., “Camera network-based visual servoing for aerial interceptor quadrotors,” Robotica 43(3), 1157–1170 (2025).CrossRefGoogle Scholar
Hong, L., Wang, X. and Zhang, D., “Robust hybrid visual servoing for hovering control of autonomous underwater vehicles in unstructured environments,” Ocean Eng. 339, 122103 (2025).CrossRefGoogle Scholar
Jorg, S., Langwald, J., Stelter, J., Hirzinger, G. and Natale, C., “Flexible Robot-Assembly Using a Multi-Sensory Approach,” In: Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference On Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), vol. 4 (IEEE, 2000) pp. 3687–3694.CrossRefGoogle Scholar
Jiang, J., Yao, L., Huang, Z., Yu, G., Wang, L. and Bi, Z., “The state of the art of search strategies in robotic assembly,” J. Ind. Inf. Integr. 26, 100259 (2022a).Google Scholar
Johannink, T., Bahl, S., Nair, A., Luo, J., Kumar, A., Loskyll, M., Ojea, J. A., Solowjow, E. and Levine, S., “Residual Reinforcement Learning for Robot Control,” In: 2019 International Conference on Robotics and Automation (ICRA) (IEEE, 2019) pp. 6023–6029.CrossRefGoogle Scholar
Staessens, T., Lefebvre, T. and Crevecoeur, G., “Adaptive control of a mechatronic system using constrained residual reinforcement learning,” IEEE Trans. Ind. Electron. 69(10), 10447–10456 (2022).CrossRefGoogle Scholar
Wen, S., Shu, Y., Rad, A., Wen, Z., Guo, Z. and Gong, S., “A deep residual reinforcement learning algorithm based on soft actor-critic for autonomous navigation,” Expert Syst. Appl. 259, 125238 (2025).CrossRefGoogle Scholar
Garcia-Hernando, G., Johns, E. and Kim, T.-K., “Physics-based Dexterous Manipulations with Estimated Hand Poses and Residual Reinforcement Learning,” In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE, 2020) pp. 9561–9568.CrossRefGoogle Scholar
Luo, J., Solowjow, E., Wen, C., Ojea, J. A., Agogino, A. M., Tamar, A. and Abbeel, P., “Reinforcement Learning on Variable Impedance Controller for High-Precision Robotic Assembly,” In: 2019 International Conference on Robotics and Automation (ICRA) 2019 (IEEE, 2019) pp. 3080–3087.CrossRefGoogle Scholar
Al-Amin, M., Tao, W., Doell, D., Lingard, R., Yin, Z., Leu, M. C. and Qin, R., “Action recognition in manufacturing assembly using multimodal sensor fusion,” Procedia Manuf. 39, 158–167 (2019).CrossRefGoogle Scholar
Nottensteiner, K., Sachtler, A. and Albu-Schäffer, A., “Towards autonomous robotic assembly: Using combined visual and tactile sensing for adaptive task execution,” J. Intell. Robot. Syst. 101(3), 49 (2021).CrossRefGoogle Scholar
Tan, J., “A method to plan the path of a robot utilizing deep reinforcement learning and multi-sensory information fusion,” Appl. Artif. Intell. 37(1), 2224996 (2023).CrossRefGoogle Scholar
Goh, H. H., Huang, Y., Lim, C. S., Zhang, D., Liu, H., Dai, W., Kurniawan, T. A. and Rahman, S., “An assessment of multistage reward function design for deep reinforcement learning-based microgrid energy management,” IEEE Trans. Smart Grid 13(6), 4300–4311 (2022).CrossRefGoogle Scholar
Icarte, R. T., Klassen, T. Q., Valenzano, R. and McIlraith, S. A., “Reward machines: Exploiting reward function structure in reinforcement learning,” J. Artif. Intell. Res. 73, 173–208 (2022).CrossRefGoogle Scholar
Bing, Z., Zhou, H., Li, R., Su, X., Morin, F. O., Huang, K. and Knoll, A., “Solving robotic manipulation with sparse reward reinforcement learning via graph-based diversity and proximity,” IEEE Trans. Ind. Electron. 70(3), 2759–2769 (2022).CrossRefGoogle Scholar
Devidze, R., Radanovic, G., Kamalaruban, P. and Singla, A., “Explicable reward design for reinforcement learning agents,” Adv. Neur. Inf. Process. Syst. 34, 20118–20131 (2021).Google Scholar
Schoettler, G., Nair, A., Luo, J., Bahl, S., Ojea, J. A., Solowjow, E. and Levine, S., “Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards,” In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2020 (IEEE, 2020) pp. 5548–5555.CrossRefGoogle Scholar
Eschmann, J., Reward Function Design in Reinforcement Learning,” In: Reinforcement Learning Algorithms: Analysis and Applications (Springer, 2021) pp. 25–33.Google Scholar