Pierre Sermanet
PhD student in vision, machine learning and robotics
News | Videos | Software | Publications | LAGR | Eurobot | Resume
LAGR
  LAGR: Learning Applied to Ground Robots
    Description | Publications | Press | More details


Project Overview: An autonomous robot with long-range vision that can navigate and learn.


Long-range mapping & planning: The long-range vision-space mapping and planning system exploiting the long-range vision system.
  • This video shows the long-range hyperbolic-polar mapping and planning system allowing long-range local navigation in our LAGR project.

  • This mapping and planning scheme is decribed in the JFR08 and IROS08 papers below.
Short-range dynamics planning: A very simple and efficient maneuver learning technique for smooth obstacle avoidance.
  • We compare a trivial linear-steering technique (in red) for steering the robot to a simple and efficient (learned) dynamics planner (in green).
  • The trivial linear-steering was quite successful during phase 1 of the project as shown at the beginning, but still would hit many obstacles in difficult situations.
  • The learned-dynamics planner however smoothly avoids obstacles and never hits any obstacles even in complicated and cluttered environments.
  • This video was put together by Marco Scoffier.

Description

  • Team: New York University / Net-Scale Technologies
  • Time Period: December 2004 - January 2008
  • Mission: DARPA's objectives
  • Principal Investigators: Yann LeCun (NYU), Urs Muller (Net-Scale Technologies)
  • Members: Pierre Sermanet, Raia Hadsell, Ayse Erkan, Jan Ben, Jeff Han, Matt Grimes, Marco Scoffier
  • Robot manufacturer: NREC (Carnegie-Mellon University)

  • Description: The goal is to demonstrate benefits of learning algorithms in unstructured outdoor robotics, using passive vision only and without any prior knowledge of testing locations. Our team has been selected as one of 8 participants in the LAGR project funded by the US Government (DARPA). Each team receives an indentical copy of the LAGR robot for fair comparison and improvement quantization.

  • Technologies: Neural networks, Image/Sensor processing, Navigation, lush/C
  • Sensors: Cameras only (2 Stereo pairs), GPS
  • Results: CMU state-of-the-art Baseline has been beaten by a factor of 2.3 in mid 2006 by standard government tests.

Journal Publications

JFR 09 "A Multi-Range Architecture for Collision-Free Off-Road Robot Navigation",
P. Sermanet, R. Hadsell, M. Scoffier, M. Grimes, J. Ben, A. Erkan, C. Crudele, U. Muller, Y. LeCun,
in Journal of Field Robotics, Special Issue on LAGR [43 pages]
JFR 09 "Learning Long-Range Vision for Autonomous Off-Road Driving",
R. Hadsell, P. Sermanet, A. Erkan, M. Scoffier, K. Kavukcuoglu, U. Muller, Y. LeCun,
in Journal of Field Robotics, Special Issue on LAGR

Video Publications

AAAI
LAB-RS		 08
08     		"DARPA LAGR Program: Learning Applied to Long-Range Vision using a Collision-Free Navigation Platform"
P. Sermanet, R. Hadsell, M. Scoffier, M. Grimes, J. Ben, A. Erkan, C. Crudele, U. Muller, Y. LeCun,
in video competitions of Association for the Advancement of Artificial Intelligence (AAAI)
and Learning and Adaptive Behavior in Robotic Systems (LAB-RS)

Refeered Conference Publications

IROS 08 "Mapping and Planning under Uncertainty in Mobile Robots with Long-Range Perception" ,
P. Sermanet, R. Hadsell, M. Scoffier, U. Muller, Y. LeCun,
in Proc. of Intelligent Robots and Systems [6 pages]
IROS 08 "Deep Belief Net Learning in a Long-Range Vision System for Autonomous Off-Road Driving" ,
R. Hadsell, A. Erkan, P. Sermanet, M. Scoffier, U. Muller, Y. LeCun,
in Proc. of Intelligent Robots and Systems [6 pages]
NESCAI 08 Also presented at North East Student Colloquium on Artificial Intelligence
ISR 08 "Learning Maneuver Dictionaries for Ground Robot Planning" ,
P. Sermanet, M. Scoffier, C. Crudele, U. Muller, Y. LeCun,
in 39th International Symposium on Robotics [6 pages]
IAV 07 "Speed-Range Dilemmas for Vision-Based Navigation in Unstructured Terrain",
P. Sermanet, R. Hadsell, J. Ben, A. Naz Erkan, B. Flepp, U. Muller, Y. LeCun,
Proc. 6th IFAC Symposium on Intelligent Autonomous Vehicles [6 pages]
RA 07 "A Multi-Range Vision Strategy for Autonomous Offroad Navigation",
R. Hadsell, A. N. Erkan, P. Sermanet, J. Ben, U. Muller, Y. LeCun,
in Proc. of Robotics and Applications [8 pages]
IROS 07 "Adaptive Long Range Vision in Unstructured Terrain" ,
A. N. Erkan, R. Hadsell, P. Sermanet, J. Ben, U. Muller, Y. LeCun,
in Proc. of Intelligent Robots and Systems [8 pages]
RSS 07 "Online Learning for Offroad Robots: Using Spatial Label Propagation to Learn Long-Range Traversability"
R. Hadsell, P Sermanet, J. Ben, A. Erkan, J. Han, B. Flepp, U. Muller, Y. LeCun,
in Proc. of the Robotics Science and Systems Conference [8 pages]
NESCAI 07 Also presented at North East Student Colloquium on Artificial Intelligence

Other Publications & Presentations

NIPS 07 "Self-Supervised Learning From High Dimensional Data for Autonomous Off-Road Driving" ,
A. N. Erkan, R. Hadsell, P. Sermanet, K. Kavukcuoglu, M. A. Ranzato, U. Muller, Y. LeCun,
presented at NIPS Workshop: Robotic Challenges for Machine Learning
CVPR 06 "Driving and Learning Strategies for Offroad Robots",
R. Hadsell, P. Sermanet, J. Ben, J. Han, A. N. Erkan, S. Chopra, Y. Sulsky, B. Flepp, U. Muller, Y. Lecun,
Demo poster at Conference on Computer Vision and Pattern Recognition
 Snowbird 06 "On-Line Learning of Long-Range Obstacle Detection for Offroad Robots",
R. Hadsell, P. Sermanet, J. Ben, J. Han, S. Chopra, M.A. Ranzato, Y. Sulsky, B. Flepp, U. Muller, Y. LeCun,
Snowbird Workshop on Learning [3 pages]

Press

  • NYU LAGR featured on Discovery Science Channel
    • Our work on the LAGR program was featured on the Discovery Science channel in January 2010 in the Sci-Fi Science series by Michio Kaku:
      Physics of the Impossible: How to build an Intelligent Robot
    • In this episode, Michio Kaku's goal is to design an intelligent robot. His design includes a learning brain that constantly rewires itself as it performs new tasks.
    • The learning brain is illustrated by our convolutional neural network which has learned to recognize obstacles in natural scenes and also adapts constantly to new obstacles it has never seen before.


    Last update: February 27th, 2009