Hostname: page-component-6766d58669-bp2c4 Total loading time: 0 Render date: 2026-05-23T15:34:06.212Z Has data issue: false hasContentIssue false
  • English
  • Français

A pixel-level grasp detection method based on Efficient Grasp Aware Network

Published online by Cambridge University Press:  18 September 2024

Haonan Xi Open the ORCID record for Haonan Xi [Opens in a new window] ,
Shaodong Li Open the ORCID record for Shaodong Li [Opens in a new window]  and
Xi Liu
Haonan Xi
Affiliation:
School of Electrical Engineering, Guangxi University, Nanning, China
Shaodong Li*
Affiliation:
School of Electrical Engineering, Guangxi University, Nanning, China
Xi Liu
Affiliation:
School of Electrical Engineering, Guangxi University, Nanning, China
*
Corresponding author: Shaodong Li; Email: lishaodongyx@126.com
Get access Rights & Permissions [Opens in a new window]

Abstract

This work proposes a novel grasp detection method, the Efficient Grasp Aware Network (EGA-Net), for robotic visual grasp detection. Our method obtains semantic information for grasping through feature extraction. It efficiently obtains feature channel weights related to grasping tasks through the constructed ECA-ResNet module, which can smooth the network’s learning. Meanwhile, we use concatenation to obtain low-level features with rich spatial information. Our method inputs an RGB-D image and outputs the grasp poses and their quality score. The EGA-Net is trained and tested on the Cornell and Jacquard datasets, and we achieve 98.9% and 95.8% accuracy, respectively. The proposed method only takes 24 ms for real-time performance to process an RGB-D image. Moreover, our method achieved better results in the comparison experiment. In the real-world grasp experiments, we use a 6-degree of freedom (DOF) UR-5 robotic arm to demonstrate its robust grasping of unseen objects in various scenes. We also demonstrate that our model can successfully grasp different types of objects without any processing in advance. The experiment results validate our model’s exceptional robustness and generalization.

Keywords

graspingattention mechanismgrasp detectionrobotic controlcontrol of robotic systems

Information

Type
Research Article
Information
Robotica , Volume 42 , Issue 9 , September 2024 , pp. 3190 - 3210
Copyright
© The Author(s), 2024. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Tian, H., Wang, T., Liu, Y., Qiao, X. and Li, Y., “Computer vision technology in agricultural automation-a review,” Inf Process Agric 7(1), 1–19 (2020).Google Scholar
Sanchez, J., Corrales, J.-A., Bouzgarrou, B.-C. and Mezouar, Y., “Robotic manipulation and sensing of deformable objects in domestic and industrial applications: A survey,” Int J Robot Res 37(7), 688–716 (2018).CrossRefGoogle Scholar
Hu, Y., Wu, X., Geng, P. and Li, Z., “Evolution strategies learning with variable impedance control for grasping under uncertainty,” IEEE Trans Ind Electron 66(10), 7788–7799 (2018).CrossRefGoogle Scholar
Hota, R. K. and Kumar, C. S., “Effect of design parameters on strong and immobilizing grasps with an underactuated robotic hand,” Robotica 40(11), 3769–3785 (2022).CrossRefGoogle Scholar
Yan, W., Deng, Z., Chen, J., Nie, H. and Zhang, J., “Precision grasp planning for multi-finger hand to grasp unknown objects,” Robotica 37(8), 1415–1437 (2019).CrossRefGoogle Scholar
Weng, T., Pallankize, A., Tang, Y., Kroemer, O. and Held, D., “Multi-modal transfer learning for grasping transparent and specular objects,” IEEE Robot Autom Lett 5(3), 3791–3798 (2020).CrossRefGoogle Scholar
Fang, H., Fang, H.-S., Xu, S. and Lu, C., “Transcg: A large-scale real-world dataset for transparent object depth completion and a grasping baseline,” IEEE Robot Autom Lett 7(3), 7383–7390 (2022).CrossRefGoogle Scholar
Sajjan, S., Moore, M., Pan, M., Nagaraja, G., Lee, J., Zeng, A. and Song, S., “Clear Grasp: 3d Shape Estimation of Transparent Objects for Manipulation,” In: 2020 IEEE International Conference on Robotics and Automation (ICRA), (IEEE, 2020) pp. 3634–3642.CrossRefGoogle Scholar
Ju, Z., Yang, C., Li, Z., Cheng, L. and Ma, H., “Teleoperation of Humanoid Baxter Robot Using Haptic Feedback,” In: 2014 International Conference on Multisensor Fusion and Information Integration for Intelligent Systems (MFI), (IEEE, 2014) pp. 1–6.CrossRefGoogle Scholar
Dong, M. and Zhang, J., “A review of robotic grasp detection technology,” Robotica 41(12), 3846–3885 (2023).CrossRefGoogle Scholar
Marwan, Q. M., Chua, S. C. and Kwek, L. C., “Comprehensive review on reaching and grasping of objects in robotics,” Robotica 39(10), 1849–1882 (2021).CrossRefGoogle Scholar
Yu, S., Zhai, D.-H., Xia, Y., Wu, H. and Liao, J., “Se-resunet: A novel robotic grasp detection method,” IEEE Robot Autom Lett 7(2), 5238–5245 (2022).CrossRefGoogle Scholar
Morrison, D., Corke, P. and Leitner, J., “Learning robust, real-time, reactive robotic grasping,” Int J Robot Res 39(2-3), 183–201 (2020).CrossRefGoogle Scholar
Kumra, S., Joshi, S. and Sahin, F., “Antipodal Robotic Grasping using Generative Residual Convolutional Neural Network,” In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), (IEEE, 2020) pp. 9626–9633.CrossRefGoogle Scholar
Wang, S., Zhou, Z. and Kan, Z., “When transformer meets robotic grasping: Exploits context for efficient grasp detection,” IEEE Robot Autom Lett 7(3), 8170–8177 (2022).CrossRefGoogle Scholar
Yu, S., Zhai, D.-H. and Xia, Y., “Egnet: Efficient robotic grasp detection network,” IEEE Trans Ind Electron 70(4), 4058–4067 (2022).CrossRefGoogle Scholar
Wang, D., Liu, C., Chang, F., Li, N. and Li, G., “High-performance pixel-level grasp detection based on adaptive grasping and grasp-aware network,” IEEE Trans Ind Electron 69(11), 11611–11621 (2021).CrossRefGoogle Scholar
Lu, N., Cai, Y., Lu, T., Cao, X., Guo, W. and Wang, S., “Picking out the impurities: Attention-based push-grasping in dense clutter,” Robotica 41(2), 470–485 (2023).CrossRefGoogle Scholar
Tian, H., Song, K., Li, S., Ma, S., Xu, J. and Yan, Y., “Data-driven robotic visual grasping detection for unknown objects: A problem-oriented review,” Expert Syst Appl 211, 118624 (2023).CrossRefGoogle Scholar
Kundu, O., Dutta, S. and Kumar, S., “A novel method for finding grasping handles in a clutter using RGBD Gaussian mixture models,” Robotica 40(3), 447–463 (2022).CrossRefGoogle Scholar
Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Ojea, J. A. and Goldberg, K., “Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics,” (2017).arXiv preprint arXiv: 1703.09312.Google Scholar
Zhou, X., Lan, X., Zhang, H., Tian, Z., Zhang, Y. and Zheng, N., “Fully Convolutional Grasp Detection Network with Oriented Anchor Box,” In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), (IEEE, 2018) pp. 7223–7230.CrossRefGoogle Scholar
Kim, D., Li, A. and Lee, J., “Stable robotic grasping of multiple objects using deep neural networks,” Robotica 39(4), 735–748 (2021).CrossRefGoogle Scholar
Lenz, I., Lee, H. and Saxena, A., “Deep learning for detecting robotic grasps,” Int J Robot Res 34(4-5), 705–724 (2015).CrossRefGoogle Scholar
Redmon, J. and Angelova, A., “Real-Time Grasp Detection Using Convolutional Neural Networks,” In: 2015 IEEE international conference on robotics and automation (ICRA), (IEEE, 2015) pp. 1316–1322.CrossRefGoogle Scholar
Ronneberger, O., Fischer, P. and Brox, T., “U-net: Convolutional Networks for Biomedical Image Segmentation,” In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, (Springer, 2015) pp. 234–241. Proceedings, Part III 18.CrossRefGoogle Scholar
Gu, Q., Su, J. and Bi, X., “Attention Grasping Network: A Real-Time Approach to Generating Grasp Synthesis,” In: 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO), (IEEE, 2019) pp. 3036–3041.CrossRefGoogle Scholar
Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W. and Hu, Q., “ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks,” In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, (IEEE, 2020) pp. 11534–11542.CrossRefGoogle Scholar
Caldera, S., Rassau, A. and Chai, D., “Review of deep learning methods in robotic grasp detection,” Multimodal Technol Interact 2(3), 57 (2018).CrossRefGoogle Scholar
Rodriguez, A., Mason, M. T. and Ferry, S., “From caging to grasping,” Int J Robot Res 31(7), 886–900 (2012).CrossRefGoogle Scholar
Bicchi, A. and Kumar, V., “Robotic Grasping and Contact: A Review,” In: Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation Symposia Proceedings, (IEEE, 2000) pp. 348–353.Google Scholar
Tan, T., Alqasemi, R., Dubey, R. and Sarkar, S., “Formulation and validation of an intuitive quality measure for antipodal grasp pose evaluation,” IEEE Robot Autom Lett 6(4), 6907–6914 (2021).CrossRefGoogle Scholar
LeCun, Y., Bengio, Y. and Hinton, G., “Deep learning,” Nature 521(7553), 436–444 (2015).CrossRefGoogle ScholarPubMed
Van Hasselt, H., Guez, A. and Silver, D., “Deep Reinforcement Learning with Double Q-learning,” Proc AAAI Conf Artif Intell 30(1) (2016).CrossRefGoogle Scholar
Niu, M., Lu, Z., Chen, L., Yang, J. and Yang, C., “Vergnet: Visual enhancement guided robotic grasp detection under low-light condition,” IEEE Robot Autom Lett 8(12), 8541–8548 (2023).CrossRefGoogle Scholar
Kumra, S., Joshi, S. and Sahin, F., “Gr-convnet v2: A real-time multi-grasp detection network for robotic grasping,” Sensors 22(16), 6208 (2022).CrossRefGoogle Scholar
Yu, S., Zhai, D.-H. and Xia, Y., “Skgnet: Robotic grasp detection with selective kernel convolution,” IEEE Trans Autom Sci Eng 20(4), 2241–2252 (2022).Google Scholar
Chu, F.-J., Xu, R. and Vela, P. A., “Real-world multiobject, multigrasp detection,” IEEE Robot Autom Lett 3(4), 3355–3362 (2018).CrossRefGoogle Scholar
Zhang, F., Leitner, J., Milford, M., Upcroft, B. and Corke, P., “Towards vision-based deep reinforcement learning for robotic motion control,” (2015).arXiv preprint arXiv: 1511.03791.Google Scholar
Hu, J., Shen, L. and Sun, G., “Squeeze-and-Excitation Networks,” In: Proceedings of the IEEE conference on computer vision and pattern recognition, (IEEE, 2018) pp. 7132–7141.CrossRefGoogle Scholar
He, K., Zhang, X., Ren, S. and Sun, J., “Deep Residual Learning for Image Recognition,” In: Proceedings of the IEEE conference on computer vision and pattern recognition, (IEEE, 2016) pp. 770–778.CrossRefGoogle Scholar
Jiang, Y., Moseson, S. and Saxena, A., “Efficient Grasping from rgbd Images: Learning using a New Rectangle Representation,” In: 2011 IEEE International conference on robotics and automation, (IEEE, 2011) pp. 3304–3311.Google Scholar
Shukla, P., Pramanik, N., Mehta, D. and Nandi, G. C., “Generative model based robotic grasp pose prediction with limited dataset,” Appl Intell 52(9), 9952–9966 (2022).CrossRefGoogle Scholar
Kushwaha, V., Shukla, P. and Nandi, G. C., “Generating quality grasp rectangle using Pix2Pix GAN for intelligent robot grasping,” Mach Vision Appl 34(1), 15 (2023).CrossRefGoogle Scholar
Asif, U., Tang, J. and Harrer, S., “ Graspnet: An Efficient Convolutional Neural Network for Real-Time Grasp Detection for Low-Powered Devices,” In: IJCAI, (2018) pp. 4875–4882.Google Scholar
Saxena, A., Driemeyer, J. and Ng, A. Y., “Robotic grasping of novel objects using vision,” Int J Robot Res 27(2), 157–173 (2008).CrossRefGoogle Scholar