Hostname: page-component-76fb5796d-5g6vh Total loading time: 0 Render date: 2024-04-28T23:27:30.969Z Has data issue: false hasContentIssue false
  • English
  • Français

Selective 6D grasping with a collision avoidance system based on point clouds and RGB+D images

Published online by Cambridge University Press:  18 October 2023

Caio Cristiano Barros Viturino Open the ORCID record for Caio Cristiano Barros Viturino [Opens in a new window]  and
Andre Gustavo Scolari Conceicao Open the ORCID record for Andre Gustavo Scolari Conceicao [Opens in a new window]
Caio Cristiano Barros Viturino
Affiliation:
Postgraduate Program in Electrical Engineering, Federal University of Bahia, Salvador, Bahia, Brazil
Andre Gustavo Scolari Conceicao*
Affiliation:
LaR - Robotics Laboratory, Department of Electrical and Computer Engineering, Federal University of Bahia, Salvador, Bahia, Brazil
*
Corresponding author: Andre Gustavo Scolari Conceicao; E-mail: andre.gustavo@ufba.br
Get access Rights & Permissions [Opens in a new window]

Abstract

In recent years, deep learning-based robotic grasping methods have surpassed analytical methods in grasping performance. Despite the results obtained, most of these methods use only planar grasps due to the high computational cost found in 6D grasps. However, planar grasps have spatial limitations that prevent their applicability in complex environments, such as grasping manufactured objects inside 3D printers. Furthermore, some robotic grasping techniques only generate one feasible grasp per object. However, it is necessary to obtain multiple possible grasps per object because not every grasp generated is kinematically feasible for the robot manipulator or does not collide with other close obstacles. Therefore, a new grasping pipeline is proposed to yield 6D grasps and select a specific object in the environment, preventing collisions with obstacles nearby. The grasping trials are performed in an additive manufacturing unit that has a considerable level of complexity due to the high chance of collision. The experimental results prove that it is possible to achieve a considerable success rate in grasping additive manufactured objects. The UR5 robot arm, Intel Realsense D435 camera, and Robotiq 2F-140 gripper are used to validate the proposed method in real experiments.

Keywords

graspingdeep learning methodsrobotic manipulatorscomputer visionmanufacturing
Type
Research Article
Information
Robotica , Volume 41 , Issue 12 , December 2023 , pp. 3772 - 3787
Copyright
© The Author(s), 2023. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Morrison, D., Corke, P. and Leitner, J., “Closing the loop for robotic grasping: A real-time, generative grasp synthesis approach,” ArXiv Preprint ArXiv:1804.05172 (2018).Google Scholar
Mahler, J., Matl, M., Satish, V., Danielczuk, M., DeRose, B., McKinley, S. and Goldberg, K., “Learning ambidextrous robot grasping policies,” Sci. Robot. 4(26), (2019).CrossRefGoogle ScholarPubMed
Mousavian, A., Eppner, C. and Fox, D., “6-DOF graspnet: Variational grasp generation for object manipulation,” In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2019) pp. 2901–2910.Google Scholar
Chen, W., Liang, H., Chen, Z., Sun, F. and Zhang, J., “Improving object grasp performance via transformer-based sparse shape completion,” J. Intell. Robot. Syst. 104(3), 1–14 (2022).CrossRefGoogle Scholar
Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Ojea, J. A. and Goldberg, K., “Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics,” ArXiv Preprint ArXiv:1703.09312 (2017).Google Scholar
de Souza, J. P. C., Costa, C. M., Rocha, L. F., Arrais, R., Moreira, A. P., Pires, E. S. and Boaventura-Cunha, J., “Reconfigurable grasp planning pipeline with grasp synthesis and selection applied to picking operations in aerospace factories,” Robot. Comput. Integr. Manuf. 67, 102032 (2021).CrossRefGoogle Scholar
Breyer, M., Chung, J. J., Ott, L., Siegwart, R. and Nieto, J., “Volumetric grasping network: Real-time 6 dof grasp detection in clutter,” ArXiv Preprint ArXiv:2101.01132 (2021).Google Scholar
Lenz, I., Lee, H. and Saxena, A., “Deep learning for detecting robotic grasps,” Int. J. Robot. Res. 34(4-5), 705–724 (2015).CrossRefGoogle Scholar
Redmon, J. and Angelova, A., “Real-time grasp detection using convolutional neural networks,” In: 2015 IEEE International Conference on Robotics and Automation (2015) pp. 1316–1322.Google Scholar
Steffens, C. R., Messias, L. R. V., Drews-Jr, P. J. L. and Botelho, S. S. C., “On robustness of robotic and autonomous systems perception,” J. Intell. Robot. Syst. 101(3), 1–17 (2021).CrossRefGoogle Scholar
Viturino, C. C. B., Junior, U. M. P., Conceição, A. G. S. and Schnitman, L., “Adaptive artificial potential fields with orientation control applied to robotic manipulators,” IFAC-PapersOnLine 53(2), 9924–9929 (2020).CrossRefGoogle Scholar
Muñoz, J., López, B., Quevedo, F., Barber, R., Garrido, S. and Moreno, L., “Geometrically constrained path planning for robotic grasping with Differential Evolution and Fast Marching Square,” Robotica 41(2), 414–432 (2022).CrossRefGoogle Scholar
Kumra, S. and Kanan, C., “Robotic grasp detection using deep convolutional neural networks,” In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (2017) pp. 769–776.Google Scholar
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W. and Abbeel, P., “Domain randomization for transferring deep neural networks from simulation to the real world,” In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (2017) pp. 23–30.Google Scholar
Morrison, D., Corke, P. and Leitner, J., “Multi-view picking: Next-best-view reaching for improved grasping in clutter,” In: International Conference on Robotics and Automation (2019) pp. 8762–8768.Google Scholar
Kim, D., Li, A. and Lee, J., “Stable robotic grasping of multiple objects using deep neural networks,” Robotica 39(4), 735–748 (2021).10.1017/S0263574720000703CrossRefGoogle Scholar
Bohg, J., Morales, A., Asfour, T. and Kragic, D., “Data-driven grasp synthesis–a survey,” IEEE Trans. Robot. 30(2), 289–309 (2013).CrossRefGoogle Scholar
Kober, J. and Peters, J., “Imitation and reinforcement learning,” IEEE Robot. Autom. Mag. 17(2), 55–62 (2010).CrossRefGoogle Scholar
Maitin-Shepard, J., Cusumano-Towner, M., Lei, J. and Abbeel, P., “Cloth grasp point detection based on multiple-view geometric cues with application to robotic towel folding,” In: IEEE International Conference on Robotics and Automation (2010) pp. 2308–2315.Google Scholar
Ciocarlie, M., Hsiao, K., Jones, E. G., Chitta, S., Rusu, R. B. and Şucan, I. A., “Towards reliable grasping and manipulation in household environments,” In: Experimental Robotics (Springer, Cham, 2014), 241–252.CrossRefGoogle Scholar
Hernandez, C., Bharatheesha, M., Ko, W., Gaiser, H., Tan, J., van Deurzen, K., de Vries, M., Van Mil, B., van Egmond, J., Burger, R. and others, “Team delft’s robot winner of the amazon picking challenge 2016,” In: Robot World Cup (Springer, Cham, 2016) pp. 613–624.Google Scholar
Konidaris, G., Kuindersma, S., Grupen, R. and Barto, A., “Robot learning from demonstration by constructing skill trees,” Int. J. Robot. Res. 31(3), 360–375 (2012).CrossRefGoogle Scholar
Levine, S., Pastor, P., Krizhevsky, A. and Quillen, D., “Learning hand-eye coordination for robotic grasping with large-scale data collection,” In: International Symposium on Experimental Robotics (Springer, Cham, 2016) pp. 173–184.Google Scholar
Kappler, D., Bohg, J. and Schaal, S., “Leveraging big data for grasp planning,” In: IEEE International Conference on Robotics and Automation (ICRA) (2015) pp. 4304–4311.CrossRefGoogle Scholar
Danielczuk, M., Kurenkov, A., Balakrishna, A., Matl, M., Wang, D., Martin-Martin, R., Garg, A., Savarese, S. and Goldberg, K., “Mechanical search: Multi-step retrieval of a target object occluded by clutter,” In: IEEE International Conference on Robotics and Automation (ICRA) (2019) pp. 1614–1621.Google Scholar
Mnyusiwalla, H., Triantafyllou, P., Sotiropoulos, P., Roa, M. A., Friedl, W., Sundaram, A. M., Russell, D. and Deacon, G., “A bin-picking benchmark for systematic evaluation of robotic pick-and-place systems,” IEEE Robot. Autom. Lett. 5(2), 1389–1396 (2020).CrossRefGoogle Scholar
Bekiroglu, Y., Marturi, N., Roa, M. A., Adjigble, K. J. M., Pardi, T., Grimm, C., Balasubramanian, R., Hang, K. and Stolkin, R., “Benchmarking protocol for grasp planning algorithms,” IEEE Robot. Autom. Lett. 5(2), 315–322 (2019).CrossRefGoogle Scholar
van Vuuren, J. J., Tang, L., Al-Bahadly, I. and Arif, K. M., “A benchmarking platform for learning-based grasp synthesis methodologies,” J. Intell. Robot. Syst. 102(3), 1–16 (2021).CrossRefGoogle Scholar
Costa, F. S., Nassar, S. M., Gusmeroli, S., Schultz, R., Conceição, A. G. S., Xavier, M., Hessel, F. and Dantas, M. A. R., “FASTEN IIoT: An open real-time platform for vertical, horizontal and end-to-end integration,” Sensors 20(19), 5499 (2020). doi: 10.3390/s20195499.CrossRefGoogle ScholarPubMed
Arrais, R., Veiga, G., Ribeiro, T. T., Oliveira, D., Fernandes, R., Conceição, A. G. S. and Farias, P. C. M. A., “Application of the open scalable production system to machine tending of additive manufacturing operations by a mobile manipulator,” In: Progress in Artificial Intelligence. EPIA 2019. Lecture Notes in Computer Science, vol. 11805 (Springer, Cham, 2019) pp. 345–356. doi: 10.1007/978-3-030-30244-3_29.Google Scholar
Roveda, L., Maroni, M., Mazzuchelli, L., Praolini, L., Shahid, A. A., Bucca, G. and Piga, D., “Robot end-effector mounted camera pose optimization in object detection-based tasks,” J. Intell. Robot. Syst. 104(1), 1–21 (2022).CrossRefGoogle Scholar
Viturino, C. C. B., de Oliveira, D. M., Conceição, A. G. S. and Junior, U., “6D robotic grasping system using convolutional neural networks and adaptive artificial potential fields with orientation control,” In: Latin American Robotics Symposium (LARS), 2021 Brazilian Symposium on Robotics (SBR), and Workshop on Robotics in Education (WRE) (2021) pp. 144–149.CrossRefGoogle Scholar
Michel, O., “Webots: Professional mobile robot simulation,” J. Adv. Robot. Syst. 1(1), 39–42 (2004).Google Scholar
Zhou, Q.-Y., Park, J. and Koltun, V., “Open3D: A modern library for 3D data processing,” ArXiv Preprint ArXiv:1801.09847 (2018).Google Scholar
He, K., Gkioxari, G., Dollár, P. and Girshick, R., “Mask R-CNN,” In: Proceedings of the IEEE International Conference on Computer Vision (2017) pp. 2961–2969.Google Scholar
Ren, S., He, K., Girshick, R. B. and Sun, J., “Faster R-CNN: Towards real-time object detection with region proposal networks,” CoRR. abs/1506.01497 (2015).Google Scholar
Xie, S., Girshick, R., Dollár, P., Tu, Z. and He, K., “Aggregated residual transformations for deep neural networks,” In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017) pp. 1492–1500.Google Scholar
Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B. and Belongie, S., “Feature pyramid networks for object detection,” In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017) pp. 2117–2125.Google Scholar
Kingma, D. P. and Welling, M., “Auto-encoding variational bayes,” ArXiv Preprint ArXiv:1312.6114 (2013).Google Scholar
Qi, C. R., Yi, L., Su, H. and Guibas, L. J., “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” ArXiv Preprint ArXiv:1706.02413 (2017).Google Scholar

Viturino and Conceicao supplementary material

Viturino and Conceicao supplementary material

Download Viturino and Conceicao supplementary material(Video)
Video 6.1 MB