no code implementations • 6 May 2024 • Jang Hyun Cho, Boris Ivanovic, Yulong Cao, Edward Schmerling, Yue Wang, Xinshuo Weng, Boyi Li, Yurong You, Philipp Krähenbühl, Yan Wang, Marco Pavone
Our experiments on outdoor benchmarks demonstrate that Cube-LLM significantly outperforms existing baselines by 21. 3 points of AP-BEV on the Talk2Car dataset for 3D grounded reasoning and 17. 7 points on the DriveLM dataset for complex reasoning about driving scenarios, respectively.
no code implementations • 29 Mar 2024 • Zhiwen Fan, Wenyan Cong, Kairun Wen, Kevin Wang, Jian Zhang, Xinghao Ding, Danfei Xu, Boris Ivanovic, Marco Pavone, Georgios Pavlakos, Zhangyang Wang, Yue Wang
This pre-processing is usually conducted via a Structure-from-Motion (SfM) pipeline, a procedure that can be slow and unreliable, particularly in sparse-view scenarios with insufficient matched features for accurate reconstruction.
1 code implementation • 25 Mar 2024 • Xunjiang Gu, Guanyu Song, Igor Gilitschenski, Marco Pavone, Boris Ivanovic
High-definition (HD) maps have played an integral role in the development of modern autonomous vehicle (AV) stacks, albeit with high associated labeling and maintenance costs.
no code implementations • 26 Feb 2024 • Gautam Singh, Yue Wang, Jiawei Yang, Boris Ivanovic, Sungjin Ahn, Marco Pavone, Tong Che
While modern best practices advocate for scalable architectures that support long-range interactions, object-centric models are yet to fully embrace these architectures.
no code implementations • 8 Feb 2024 • Boyi Li, Yue Wang, Jiageng Mao, Boris Ivanovic, Sushant Veer, Karen Leung, Marco Pavone
Adapting driving behavior to new environments, customs, and laws is a long-standing problem in autonomous driving, precluding the widespread deployment of autonomous vehicles (AVs).
1 code implementation • 3 Nov 2023 • Jiawei Yang, Boris Ivanovic, Or Litany, Xinshuo Weng, Seung Wook Kim, Boyi Li, Tong Che, Danfei Xu, Sanja Fidler, Marco Pavone, Yue Wang
We present EmerNeRF, a simple yet powerful approach for learning spatial-temporal representations of dynamic driving scenes.
no code implementations • 1 Sep 2023 • Yulong Cao, Boris Ivanovic, Chaowei Xiao, Marco Pavone
This works aims to address this by developing a framework that employs reinforcement learning with human preference (RLHF) to enhance the realism of existing traffic models.
2 code implementations • NeurIPS 2023 • Boris Ivanovic, Guanyu Song, Igor Gilitschenski, Marco Pavone
The field of trajectory forecasting has grown significantly in recent years, partially owing to the release of numerous large-scale, real-world human trajectory datasets for autonomous vehicles (AVs) and pedestrian motion tracking.
1 code implementation • 16 Jul 2023 • Shuhan Tan, Boris Ivanovic, Xinshuo Weng, Marco Pavone, Philipp Kraehenbuehl
In this work, we turn to language as a source of supervision for dynamic traffic scene generation.
no code implementations • 10 Jun 2023 • Ziyuan Zhong, Davis Rempe, Yuxiao Chen, Boris Ivanovic, Yulong Cao, Danfei Xu, Marco Pavone, Baishakhi Ray
Realistic and controllable traffic simulation is a core capability that is necessary to accelerate autonomous vehicle (AV) development.
no code implementations • 3 Apr 2023 • Fan-Yun Sun, Jonathan Tremblay, Valts Blukis, Kevin Lin, Danfei Xu, Boris Ivanovic, Peter Karkus, Stan Birchfield, Dieter Fox, Ruohan Zhang, Yunzhu Li, Jiajun Wu, Marco Pavone, Nick Haber
At inference, given one or more views of a novel real-world object, FINV first finds a set of latent codes for the object by inverting the generative model from multiple initial seeds.
no code implementations • 13 Dec 2022 • Peter Karkus, Boris Ivanovic, Shie Mannor, Marco Pavone
To enable the joint optimization of AV stacks while retaining modularity, we present DiffStack, a differentiable and modular stack for prediction, planning, and control.
no code implementations • 26 Oct 2022 • Filippos Christianos, Peter Karkus, Boris Ivanovic, Stefano V. Albrecht, Marco Pavone
Reasoning with occluded traffic agents is a significant open challenge for planning for autonomous vehicles.
no code implementations • 11 Oct 2022 • Ruixiang Zhang, Tong Che, Boris Ivanovic, Renhao Wang, Marco Pavone, Yoshua Bengio, Liam Paull
Humans are remarkably good at understanding and reasoning about complex visual scenes.
2 code implementations • 23 Sep 2022 • Boris Ivanovic, James Harrison, Marco Pavone
Learning-based behavior prediction methods are increasingly being deployed in real-world autonomous systems, e. g., in fleets of self-driving vehicles, which are beginning to commercially operate in major cities across the world.
1 code implementation • 26 Aug 2022 • Danfei Xu, Yuxiao Chen, Boris Ivanovic, Marco Pavone
We empirically validate our method, named Bi-level Imitation for Traffic Simulation (BITS), with scenarios from two large-scale driving datasets and show that BITS achieves balanced traffic simulation performance in realism, diversity, and long-horizon stability.
1 code implementation • CVPR 2022 • Yuxiao Chen, Boris Ivanovic, Marco Pavone
In this work, we present ScePT, a policy planning-based trajectory prediction model that generates accurate, scene-consistent trajectory predictions suitable for autonomous system motion planning.
no code implementations • CVPR 2022 • Xinshuo Weng, Boris Ivanovic, Kris Kitani, Marco Pavone
This is typically caused by the propagation of errors from tracking to prediction, such as noisy tracks, fragments, and identity switches.
1 code implementation • 18 Oct 2021 • Xinshuo Weng, Boris Ivanovic, Marco Pavone
Recently, there has been tremendous progress in developing each individual module of the standard perception-planning robot autonomy pipeline, including detection, tracking, prediction of other agents' trajectories, and ego-agent trajectory planning.
1 code implementation • 7 Oct 2021 • Boris Ivanovic, Yifeng Lin, Shubham Shrivastava, Punarjay Chakravarty, Marco Pavone
As a result, perceptual uncertainties are not propagated through forecasting and predictions are frequently overconfident.
1 code implementation • 7 Oct 2021 • Boris Ivanovic, Marco Pavone
Detecting other agents and forecasting their behavior is an integral part of the modern robotic autonomy stack, especially in safety-critical scenarios entailing human-robot interaction such as autonomous driving.
no code implementations • 28 Sep 2021 • Rachel Luo, Shengjia Zhao, Jonathan Kuck, Boris Ivanovic, Silvio Savarese, Edward Schmerling, Marco Pavone
When deploying machine learning models in high-stakes robotics applications, the ability to detect unsafe situations is crucial.
no code implementations • 21 Jul 2021 • Boris Ivanovic, Marco Pavone
Forecasting the behavior of other agents is an integral part of the modern robotic autonomy stack, especially in safety-critical scenarios with human-robot interaction, such as autonomous driving.
1 code implementation • 26 Apr 2021 • Boris Ivanovic, Kuan-Hui Lee, Pavel Tokmakov, Blake Wulfe, Rowan Mcallister, Adrien Gaidon, Marco Pavone
Reasoning about the future behavior of other agents is critical to safe robot navigation.
1 code implementation • 2 Dec 2020 • Simon Schaefer, Karen Leung, Boris Ivanovic, Marco Pavone
To achieve seamless human-robot interactions, robots need to intimately reason about complex interaction dynamics and future human behaviors within their motion planning process.
1 code implementation • NeurIPS 2020 • Masha Itkina, Boris Ivanovic, Ransalu Senanayake, Mykel J. Kochenderfer, Marco Pavone
Discrete latent spaces in variational autoencoders have been shown to effectively capture the data distribution for many real-world problems such as natural language understanding, human intent prediction, and visual scene representation.
1 code implementation • 16 Sep 2020 • Boris Ivanovic, Amine Elhafsi, Guy Rosman, Adrien Gaidon, Marco Pavone
Reasoning about human motion is a core component of modern human-robot interactive systems.
no code implementations • 12 Sep 2020 • Haruki Nishimura, Boris Ivanovic, Adrien Gaidon, Marco Pavone, Mac Schwager
This paper presents a novel online framework for safe crowd-robot interaction based on risk-sensitive stochastic optimal control, wherein the risk is modeled by the entropic risk measure.
no code implementations • 10 Aug 2020 • Boris Ivanovic, Karen Leung, Edward Schmerling, Marco Pavone
Human behavior prediction models enable robots to anticipate how humans may react to their actions, and hence are instrumental to devising safe and proactive robot planning algorithms.
3 code implementations • ECCV 2020 • Tim Salzmann, Boris Ivanovic, Punarjay Chakravarty, Marco Pavone
Reasoning about human motion is an important prerequisite to safe and socially-aware robotic navigation.
Ranked #2 on Trajectory Prediction on ETH
1 code implementation • ICCV 2019 • Boris Ivanovic, Marco Pavone
Developing safe human-robot interaction systems is a necessary step towards the widespread integration of autonomous agents in society.
1 code implementation • 16 Jun 2018 • Boris Ivanovic, James Harrison, Apoorva Sharma, Mo Chen, Marco Pavone
Our Backward Reachability Curriculum (BaRC) begins policy training from states that require a small number of actions to accomplish the task, and expands the initial state distribution backwards in a dynamically-consistent manner once the policy optimization algorithm demonstrates sufficient performance.
1 code implementation • 6 Mar 2018 • Boris Ivanovic, Edward Schmerling, Karen Leung, Marco Pavone
This work presents a methodology for modeling and predicting human behavior in settings with N humans interacting in highly multimodal scenarios (i. e. where there are many possible highly-distinct futures).
Robotics Human-Computer Interaction