Pete Florence's Home Page

News

✨ Socratic Models pre-print out!
Giving a tutorial on "Vision-Based Robot Learning" at CVPR 2022 with Michael Ryoo, Shuran Song, and Andy Zeng on June 20, 2022.
4 papers at ICRA 2022! Thanks to all co-authors on these.
🎙️ Did a podcast! It's always great chatting with Eric Jang and Kevin Zakka -- thanks for setting this up, Kevin!
✍️ We wrote a blog post for the Google AI blog on Implicit BC!
XIRL was a best paper finalist at CoRL 2021.
IEEE Robotics and Automation Letters Best Paper Award (2020) for “Self-Supervised Correspondence in Visuomotor Policy Learning”.

Research

Reinforcement Learning with Neural Radiance Fields.
Danny Driess, Ingmar Schubert, Pete Florence, Yunzhu Li, Marc Toussaint. On arXiv.
[ pdf ] [ website with code and videos ]

SMs

Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language.
Andy Zeng, Maria Attarian, Brian Ichter, Krzysztof Choromanski, Adrian Wong, Stefan Welker, Federico Tombari, Aveek Purohit, Michael Ryoo, Vikas Sindhwani, Johnny Lee, Vincent Vanhoucke, Pete Florence. On arXiv.
[ pdf ] [ website with code and videos ]

Heravi

Visuomotor Control in Multi-Object Scenes Using Object-Aware Representations.
Negin Heravi, Ayzaan Wahid, Corey Lynch, Pete Florence, Travis Armstrong, Jonathan Tompson, Pierre Sermanet, Jeannette Bohg, Debidatta Dwibedi
[ pdf ]

NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields.
Lin Yen-Chen, Pete Florence, Jonathan T. Barron, Tsung-Yi Lin, Alberto Rodriguez, Phillip Isola. In ICRA 2022.
[ pdf ] [ website with code and videos ]

ikp

VIRDO: Visio-tactile Implicit Representations of Deformable Objects.
Youngsun Wi, Pete Florence, Andy Zeng, Nima Fazeli. In ICRA 2022.
[ pdf ]

Implicit Kinematic Policies: Unifying Joint and Cartesian Action Spaces in End-to-End Robot Learning.
Aditya Ganapathi, Pete Florence, Jake Varley, Kaylee Burns, Ken Goldberg, Andy Zeng. In ICRA 2022.
[ pdf ]

Trajectory Optimization with Optimization-Based Dynamics.
Taylor A. Howell, Simon Le Cleac'h, Sumeet Singh, Pete Florence, Zachary Manchester, Vikas Sindhwani. In RA-L and ICRA 2022.
[ pdf ] [ code ]

Implicit Behavioral Cloning.
Pete Florence, Corey Lynch, Andy Zeng, Oscar Ramirez, Ayzaan Wahid, Laura Downs, Adrian Wong, Johnny Lee, Igor Mordatch, Jonathan Tompson. In CoRL 2021.
[ pdf ] [ code ] [ site with video ] [ blog post ]

XIRL: Cross-embodiment Inverse Reinforcement Learning.
Kevin Zakka, Andy Zeng, Pete Florence, Jonathan Tompson, Jeannette Bohg, Debdidatta Dwibedi. In CoRL 2021.
[ pdf ] [ code ] [ site with video ]
★ Best Paper Finalist, CoRL 2021
★ Oral Presentation, CoRL 2021

iNeRF

iNeRF: Inverting Neural Radiance Fields for Pose Estimation.
Yen-Chen Lin, Pete Florence, Jonathan T. Barron, Alberto Rodriguez, Phillip Isola, Tsung-Yi Lin. In IROS 2021.
[ pdf ] [ site with video ]

bags

Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks.
Daniel Seita, Pete Florence, Jonathan Tompson, Erwin Coumans, Vikas Sindhwani, Ken Goldberg, Andy Zeng. In ICRA 2021.
[ pdf ] [ site with video ]

Transporter Network architecture

Transporter Networks: Rearranging the Visual World for Robotic Manipulation.
Andy Zeng, Pete Florence, Jonathan Tompson, Stefan Welker, Jonathan Chien, Maria Attarian, Travis Armstrong, Ivan Krasin, Dan Duong, Vikas Sindhwani, Johnny Lee. In CoRL 2020.
[ pdf ] [ site with videos ]
★ Best Presentation Finalist, CoRL 2020
★ Oral Presentation, CoRL 2020

Keypoints into the future

Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learning.
Lucas Manuelli, Yunzhu Li, Pete Florence, Russ Tedrake. In CoRL 2020.
[ pdf ] [ site with video ]

Pete Florence

Dense Visual Learning for Robot Manipulation.
Peter Florence. PhD thesis, Massachusetts Institute of Technology, September 2019.
[ pdf ] [ presentation video ]

Pete Florence

Self-Supervised Correspondence in Visuomotor Policy Learning.
Peter Florence, Lucas Manuelli, Russ Tedrake. In RA-L and ICRA 2020.
[ pdf ] [ site ] [ video ]
★ Best Paper Award, RA-L 2020
Also at: NeurIPS 2019 Workshop on Robot Learning: Control and Interaction in the Real World

kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation.
Lucas Manuelli*, Wei Gao*, Peter Florence, Russ Tedrake. In ISRR 2019.
[ pdf ] [ site with videos ]
Also at: CVPR 2019 Workshop on 3D Scene Understanding for Vision, Graphics, and Robotics

Pete Florence

DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, Steven Lovegrove. In CVPR 2019.
[ pdf ] [ video ] [ code ] [ talk ]
★ Best Paper Finalist, CVPR 2019
★ Oral Presentation, CVPR 2019

Pete Florence

Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation.
Peter R. Florence*, Lucas Manuelli*, and Russ Tedrake. In CoRL 2018.
[ pdf ] [ video ] [ code ]
★ Best Paper Award, CoRL 2018
★ Best Technical Paper, 2018 Amazon Robotics Best Paper Awards in Manipulation
Press: [ WIRED ] [ CNN ] [ Newsweek ] [ Engadget ] [ VentureBeat ] [ more ]

Pete Florence

LabelFusion: A Pipeline for Generating Ground Truth Labels for Real RGBD Data of Cluttered Scenes.
Pat Marion*, Peter R. Florence*, Lucas Manuelli*, and Russ Tedrake. In ICRA 2018. [ website with code, video, paper ]
★ Finalist, Best Vision Paper Award, ICRA 2018

Pete Florence

NanoMap: Fast, uncertainty-aware proximity queries with lazy search over local 3d data.
Peter R. Florence, John Carter, Jake Ware, and Russ Tedrake. In ICRA 2018.
[ pdf ] [ video ] [ code ]
Press: [ IEEE Spectrum ] [ BBC ] [ The Verge ] [ Engadget ] [ Air & Space Magazine ] [ TechCrunch ] [ more ]

Pete Florence

High-speed autonomous obstacle avoidance with pushbroom stereo.
Andrew J. Barry, Peter R. Florence, and Russ Tedrake. In Journal of Field Robotics (JFR) 2018.
[ pdf ] [ video ]

Pete Florence

Integrated perception and control at high speed: Evaluating collision avoidance maneuvers without maps.
Peter R. Florence, John Carter, and Russ Tedrake. In WAFR 2016.
[ pdf ] [ presentation ]

Pete Florence

Aggressive quadrotor flight through cluttered environments using mixed integer programming.
Benoit Landry, Robin Deits, Peter R. Florence, and Russ Tedrake. In ICRA 2016.
[ pdf ][ video ]

Patents

(Pending) Peter Raymond Florence, Christopher David Sachs, Kent W. Ryorchuk. Systems and methods for probabilistic semantic sensing in a sensory network. WO2015134879 A1 (2014)
Christopher David Sachs, Peter Raymond Florence. Dynamic spatially-resolved lighting using composited lighting models. U.S. Patent No. 9763306 (2014)

Technical Reports

Lucas Manuelli, Peter R. Florence. Reinforcement Learning for Autonomous Driving Obstacle Avoidance using LIDAR.
[ pdf ] [ video ] (2015)

Engineering Education for Kids

My friend Andy Barry and I started Stage One Education together. We develop and teach hands-on engineering workshops for kids across the country. Over 6,000 students to date have taken our workshops.