Hostname: page-component-89b8bd64d-x2lbr Total loading time: 0 Render date: 2026-05-08T01:11:57.302Z Has data issue: false hasContentIssue false

End-to-end deep learning-based framework for path planning and collision checking: bin-picking application

Published online by Cambridge University Press:  13 February 2024

Mehran Ghafarian Tamizi
Affiliation:
Department of Electrical and Computer Engineering, University of Victoria, Victoria, BC, Canada
Homayoun Honari
Affiliation:
Department of Mechanical Engineering, University of Victoria, Victoria, BC, Canada
Aleksey Nozdryn-Plotnicki
Affiliation:
Apera AI, Vancouver, BC, Canada
Homayoun Najjaran*
Affiliation:
Department of Electrical and Computer Engineering, University of Victoria, Victoria, BC, Canada Department of Mechanical Engineering, University of Victoria, Victoria, BC, Canada
*
Corresponding author: Homayoun Najjaran; Email: najjaran@uvic.ca
Rights & Permissions [Opens in a new window]

Abstract

Real-time and efficient path planning is critical for all robotic systems. In particular, it is of greater importance for industrial robots since the overall planning and execution time directly impact the cycle time and automation economics in production lines. While the problem may not be complex in static environments, classical approaches are inefficient in high-dimensional environments in terms of planning time and optimality. Collision checking poses another challenge in obtaining a real-time solution for path planning in complex environments. To address these issues, we propose an end-to-end learning-based framework viz., Path Planning and Collision checking Network (PPCNet). The PPCNet generates the path by computing waypoints sequentially using two networks: the first network generates a waypoint, and the second one determines whether the waypoint is on a collision-free segment of the path. The end-to-end training process is based on imitation learning that uses data aggregation from the experience of an expert planner to train the two networks, simultaneously. We utilize two approaches for training a network that efficiently approximates the exact geometrical collision checking function. Finally, the PPCNet is evaluated in two different simulation environments and a practical implementation on a robotic arm for a bin-picking application. Compared to the state-of-the-art path-planning methods, our results show significant improvement in performance by greatly reducing the planning time with comparable success rates and path lengths.

Information

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2024. Published by Cambridge University Press
Figure 0

Figure 1. Path planning for pick-and-place operation.

Figure 1

Figure 2. Path planning and collision checking network (PPCNet).

Figure 2

Figure 3. End-to-end training process of PPCNet: top row: the imitation learning and data aggregation processes for training the planner network. In this process, based on random queries and the data aggregation process, the planning network dataset is constructed. The dataset outputs the tuple $(q_t,q_T,q_{t+1})$ indicating the current, goal, and next configurations. Finally, the loss function $L_{planner}$ is used to backpropagate through the network’s weights. bottom row: population-based probability estimation and collision checker network training processes. In this process, for calculating the population-based probability, KD-tree is employed. Furthermore, using the collision checking network dataset and the respective loss function ($L_{binary}$ or $L_{Population})$, the network is updated.

Figure 3

Figure 4. Post-processing procedure: (a) The generated path by the planner, (b) Path after binary state contraction, (c) Path after resampling.

Figure 4

Algorithm 1. Training process of PPCNet

Figure 5

Algorithm 2. Path-generation process using PPCNet

Figure 6

Algorithm 3. Steer Function $(\boldsymbol{q}_{\boldsymbol{current}},\boldsymbol{q}_{\boldsymbol{next}})$

Figure 7

Figure 5. Experimental environments: (a) UR5 scene, (b) UR5 scene with a wall as an obstacle, (c) Real-world implementation on Kinova Gen3.

Figure 8

Table I. Hyperparameters selection for planners.

Figure 9

Table II. Planning time and path length comparison of the proposed method and BI-RRT for 500 random pick-and-place queries.

Figure 10

Figure 6. Planning time comparison between different methods.

Figure 11

Figure 7. Cartesian space path for a randomly selected bin-picking scenario with the Kinova Gen3 robotic arm. In this scenario, the robot should pick three different objects and place them on the table.

Figure 12

Figure 8. Motion profile for a randomly selected bin-picking scenario with the Kinova Gen3 robotic arm. top row: joints angles plot middle row: joints velocities plot bottom row: joints accelerations plot.

Supplementary material: File

Ghafarian Tamizi et al. supplementary material movie

Ghafarian Tamizi et al. supplementary material movie
Download Ghafarian Tamizi et al. supplementary material movie(File)
File 35.1 MB