Hostname: page-component-89b8bd64d-x2lbr Total loading time: 0 Render date: 2026-05-06T23:10:57.248Z Has data issue: false hasContentIssue false
  • English
  • Français

Vision-based navigation using multi-rate feedback from optic flow and scene reconstruction

Published online by Cambridge University Press:  27 January 2016

A. R. Arvai ,
J. J. Kehoe  and
R. Lind
R. Lind*
Affiliation:
Aerospace Engineering, University of Florida, Florida, USA
*
ricklind@ufl.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Vision-based control is being aggressively pursued for autonomous systems. Such control is particularly valuable for path planning to achieve mission objectives like target tracking and obstacle avoidance. This paper presents a multi-rate strategy that utilises a fast-rate optic flow approach and a slow-rate scene reconstruction approach. The vehicle uses scene reconstruction to generate an accurate map for path planning; however, optic flow is used to avoid obstacles while the scene reconstruction is computed. A switch element is used in the feedback path to determine whether information relating to the reconstructed map or the optical flow should be used for navigation. The resulting controller is able to generate flight trajectories and perform obstacle avoidance within a computational cost which is reasonable given performance demands and computational resources on a wide range of aircraft. A simulation demonstrates the performance of an aircraft that uses the multi-rate controller to avoid an obstacle which is only observed after a turn. Essentially, the fast-rate optic flow indicates the presence of the obstacle during the time that slow-rate scene reconstruction is being performed. The resulting flight path is able to follow mission objectives and avoid a collision.

Information

Type
Research Article
Information
The Aeronautical Journal , Volume 115 , Issue 1169 , July 2011 , pp. 411 - 420
Copyright
Copyright © Royal Aeronautical Society 2011 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

1. Lucas, B. and Kanade, T. An iterative image registration technique with an application to stereo vision, Proceedings of the DARPA Image Understanding Workshop, 1981, pp 121–130.Google Scholar
2. Horn, B.K. and Schunk, B.G. Determining optical flow, Artificial Intelligence, 1981, 17, pp 185–203.Google Scholar
3. Song, K.-T. and Huang, J.-H. Fast optical flow estimation and its application to real-time obstacle avoidance, IEEE International Conference on Robotics and Automation, 2001, 3, pp 2891–2896.Google Scholar
4. Ruffier, F., Viollet, S., Amic, S. and Franceschini, N. Bio-inspired optical flow circuits for the visual guidance of micro-air vehicles, International Symposium on Circuits and Systems, May 2003, 3, pp 846–849.Google Scholar
5. Barrows, G.L. Future visual microsensors for mini/micro-UAV Applications, 7th IEEE International Workshop on Cellular Neural Networks and their Applications, July 2002, pp 498–506.Google Scholar
6. Barrows, G.L., Chahl, J.S. and Srinivasan, M.V. Biomimetic visual sensing and flight control, 2002 Bristol UAV Conference, April 2002.Google Scholar
7. Barrows, G.L. and Neely, C. Mixed-mode VLSI optic flow sensors for in-flight control of a micro air vehicle, SPIE, July 2000.Google Scholar
8. Young, G.-S., Hong, T.-H., Herman, M. and Yang, J.C.S. Obstacle detection for a vehicle using optical flow, Intelligent Vehicles Symposium, June 1992, pp 185–190.Google Scholar
9. Baratoff, G., Toepfer, C., Wende, M. and Neumann, H. Real-time navigation and obstacle avoidance from optical flow on a Space-variant map, IEEE International Symposium on Intelligent Control, September 1998, pp 289–294.Google Scholar
10. Castro, A.P.A., Silva, J.D. and Simoni, P.O. Image based autonomous navigation with fuzzy logic control, International Joint Conference on Neural Networks, July 2001, 3, pp 2200–2205.Google Scholar
11. Merrell, P.C., Lee, D.J. and Beard, R.W. Obstacle Avoidance for Unmanned Air Vehicles Using Optical Flow Probability Distributions, SPIE Optics East, Robotics Technologies and Architectures, Mobile Robot XVII, October 2004, Volume 5609-04.Google Scholar
12. Ma, Y., Soatto, S., Kosecka, J. and Sastry, S.S. An Invitation to 3D Vision, Springer-Verlag Publishers, New York, NY, USA, 2004.Google Scholar
13. Ullman, S. The Interpretation of Visual Motion, MIT Press, Cambridge, MA, USA, 1979.Google Scholar
14. Kurdila, A., Nechyba, M., Prazenica, R., Dahmen, W., Binev, P., Devore, R. and Sharpley, R. Vision-based control of micro air vehicle: progress and problems in estimation, IEEE Conference on Decision and Control, December 2004, pp 1635–1642.Google Scholar
15. Wang, H. and Brady, M. A structure-from-motion algorithm for robot vehicle guidance, Intelligent Vehicles Symposium, July 1992, pp 30–35.Google Scholar
16. Stephens, M.J., Blissett, R.J., Charnley, D., Sparks, E.P. and Pike, J.M. Outdoor vehicle navigation using passive 3D vision, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 1989, pp 556–562.Google Scholar
17. Kanade, T., Amidi, O. and Ke, Q. Real-time and 3D vision for autonomous small and micro air vehicles, IEEE Conference on Decision and Control, December 2004, pp 1655–1662.Google Scholar
18. Sinopoli, B., Micheli, M., Donato, G. and Koo, T.J. Vision based navigation for an unmanned aerial vehicle, IEEE International Conference on Robotics and Automation, 2001, 2, pp 1757–1764.Google Scholar
19. Forsyth, D.A. and Ponce, J. Computer Vision: A Modern Approach, Prentice-Hall Publishers, Upper Saddle River, NJ, USA, 2003.Google Scholar
20. Lorigo, L.M., Brooks, R.A. and Grimsou, W.E.L. Visually-guided obstacle avoidance in unstructured environments, IEEE International Conference on Intelligent Robots and Systems, September 1997, 1, pp 373–379.Google Scholar
21. Yao, Y.-S. and Chellappa, R. Dynamic feature point tracking in an image sequence, 12th IAPR International Conference on Pattern Recognition, October 1994, 1, pp 654–657.Google Scholar
22. Longuet-Higgins, H. A computer algorithm for reconstructing a scene from two projections, Nature, 1981, 293, pp 133–135.Google Scholar
23. Huang, T. and Faugeras, O. Some properties of the e-matrix in two-view motion estimation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 1989, 11, (12), pp 1310–1312.Google Scholar
24. Zhang, Z., Deriche, R., Faugeras, O. and Luong, Q.-T. A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry, Artificial Intelligence J, 1995, 78, pp 87–119.Google Scholar
25. Grimson, W. From Images to Surfaces: A Computational Study of the Human Early Visual System, MIT Press, Cambridge, MA, USA, 1981.Google Scholar
26. Faugeras, O. Three-Dimensional Computer Vision, MIT Press, Cambridge, MA, USA, 1993.Google Scholar
27. Okutami, M. and Kanade, T. A multiple-baseline stereo system, IEEE Transactions on Pattern Analysis and Machine Intelligence, 1993, 15, (4), pp 353–363.Google Scholar
28. Koenderink, J. and Van Doorn, A. Affine structure from motion, J Optical Society of America: Part A, 1990, 8, pp 377–385.Google Scholar
29. Tomasi, C. and Kanade, T. Shape and motion from image strems under orthography: a factorization method, Int J Computer Vision, 1992, 9, (2), pp 137–154.Google Scholar
30. Hartley, R. and Zisserman, A. Multiple View Geometry in Computer Vision, Cambridge University Press, Cambridge, UK, 2000.Google Scholar
31. Barron, A.R., Choen, A., Dahmen, W. and DeVore, R. Approximation and learning by greedy algorithms, Annals of Statistics, 2008, 36, (1), pp 64–94.Google Scholar
32. Zabarankin, M., Kurdila, A., Prokopyev, O., Goel, A., Boginski, V., Causey, R., Uryasev, S. and Pardalos, P. Vision-based trajectory planning for autonomous micro air vehicles in urban environments, IEEE Robotics and Automation Magazine, in review.Google Scholar
33. Webb, T.P., Prazenica, R.J., Kurdila, A.J. and Lind, R. Vision-based state estimation for uninhabited aerial vehicle, AIAA-2005-5869, August 2005.Google Scholar
34. Causey, R.S, Lind, R. and Prazenica, R.J. Effects of camera uncertainty on vision-based feedback, AIAA-2005-6097.Google Scholar
35. Roberts, P.J., Walker, R.A. and O’Shea, P. Fixed wing UAV navigation and control through Integrated GNSS and vision, AIAA-2005-5867, August 2005.Google Scholar
36. Kehoe, J.J., Causey, R.S., Arvai, A. and Lind, R. Optical flow to estimate angular rates and aerodynamic angles of nonlinear aircraft, submitted to American Control Conference, June 2006.Google Scholar
37. Stevens, B.L. and Lewis, F.L. Aircraft control and simulation, Wiley, 2003.Google Scholar
38. Kehoe, J.J., Causey, R.S., Abdulrahim, M. and Lind, R. Waypoint navigation for a micro air vehicle using vision-based attitude estimation, Aeronaut J, December 2006, 110, (1114), pp 821–829.Google Scholar