no code implementations • 17 Dec 2023 • Xinghao Zhu, Devesh K. Jha, Diego Romeres, Lingfeng Sun, Masayoshi Tomizuka, Anoop Cherian
Automating the assembly of objects from their parts is a complex problem with innumerable applications in manufacturing, maintenance, and recycling.
no code implementations • 11 Dec 2023 • Lingfeng Sun, Devesh K. Jha, Chiori Hori, Siddarth Jain, Radu Corcodel, Xinghao Zhu, Masayoshi Tomizuka, Diego Romeres
Designing robotic agents to perform open vocabulary tasks has been the long-standing goal in robotics and AI.
no code implementations • 10 Oct 2023 • Giulio Giacomuzzo, Alberto Dalla Libera, Diego Romeres, Ruggero Carli
First, instead of directly modeling the inverse dynamics components, we model as GPs the kinetic and potential energy of the system.
no code implementations • 27 Jun 2023 • Chiori Hori, Puyuan Peng, David Harwath, Xinyu Liu, Kei Ota, Siddarth Jain, Radu Corcodel, Devesh Jha, Diego Romeres, Jonathan Le Roux
This paper introduces a method for robot action sequence generation from instruction videos using (1) an audio-visual Transformer that converts audio-visual features and instruction speech to a sequence of robot actions called dynamic movement primitives (DMPs) and (2) style-transfer-based training that employs multi-task learning with video captioning and weakly-supervised learning with a semantic classifier to exploit unpaired video-action data.
no code implementations • 30 Jan 2023 • Fabio Amadio, Alberto Dalla Libera, Daniel Nikovski, Ruggero Carli, Diego Romeres
We propose a Model-Based Reinforcement Learning (MBRL) algorithm named VF-MC-PILCO, specifically designed for application to mechanical systems where velocities cannot be directly measured.
no code implementations • 2 Dec 2022 • Devesh K. Jha, Siddarth Jain, Diego Romeres, William Yerazunis, Daniel Nikovski
In this paper, we present a system for human-robot collaborative assembly using learning from demonstration and pose estimation, so that the robot can adapt to the uncertainty caused by the operation of humans.
no code implementations • 23 Oct 2022 • Tim Schneider, Boris Belousov, Georgia Chalvatzaki, Diego Romeres, Devesh K. Jha, Jan Peters
Robotic manipulation stands as a largely unsolved problem despite significant advances in robotics and machine learning in recent years.
no code implementations • 28 Sep 2022 • Seiji Shaw, Devesh K. Jha, Arvind Raghunathan, Radu Corcodel, Diego Romeres, George Konidaris, Daniel Nikovski
In this paper, we present constrained dynamic movement primitives (CDMP) which can allow for constraint satisfaction in the robot workspace.
no code implementations • 22 Mar 2022 • Yuki Shirai, Devesh K. Jha, Arvind Raghunathan, Diego Romeres
Generalizable manipulation requires that robots be able to interact with novel objects and environment.
no code implementations • 5 Mar 2022 • Yuki Shirai, Devesh K. Jha, Arvind Raghunathan, Diego Romeres
In our formulation, we explicitly consider joint chance constraints for complementarity as well as states to capture the stochastic evolution of dynamics.
no code implementations • 20 Nov 2021 • Devesh K. Jha, Diego Romeres, William Yerazunis, Daniel Nikovski
This can be used to learn a suitable representation of the skill that can be generalized to novel positions of one of the parts involved in the assembly, for example the hole in a peg-in-hole (PiH) insertion task.
no code implementations • 6 Jun 2021 • Arvind U. Raghunathan, Devesh K. Jha, Diego Romeres
PYROBOCOP is a lightweight Python-based package for control and optimization of robotic systems described by nonlinear Differential Algebraic Equations (DAEs).
no code implementations • 26 Apr 2021 • Alberto Dalla Libera, Fabio Amadio, Daniel Nikovski, Ruggero Carli, Diego Romeres
We tested the two strategies on a simulated manipulator with seven degrees of freedom, also varying the GP kernel choice.
no code implementations • 28 Jan 2021 • Fabio Amadio, Alberto Dalla Libera, Riccardo Antonello, Daniel Nikovski, Ruggero Carli, Diego Romeres
The algorithm relies on Gaussian Processes (GPs) to model the system dynamics and on a Monte Carlo approach to estimate the policy gradient.
no code implementations • 21 Jan 2021 • Fabio Amadio, Alberto Dalla Libera, Ruggero Carli, Daniel Nikovski, Diego Romeres
In this paper, we propose a Model-Based Reinforcement Learning (MBRL) algorithm for Partially Measurable Systems (PMS), i. e., systems where the state can not be directly measured, but must be estimated through proper state observers.
no code implementations • 14 Nov 2020 • Kei Ota, Devesh K. Jha, Diego Romeres, Jeroen van Baar, Kevin A. Smith, Takayuki Semitsu, Tomoaki Oiki, Alan Sullivan, Daniel Nikovski, Joshua B. Tenenbaum
The physics engine augmented with the residual model is then used to control the marble in the maze environment using a model-predictive feedback over a receding horizon.
no code implementations • 22 Jul 2020 • Yifang Liu, Diego Romeres, Devesh K. Jha, Daniel Nikovski
One of the main challenges in peg-in-a-hole (PiH) insertion tasks is in handling the uncertainty in the location of the target hole.
no code implementations • 25 Feb 2020 • Alberto Dalla Libera, Diego Romeres, Devesh K. Jha, Bill Yerazunis, Daniel Nikovski
In this paper, we propose a derivative-free model learning framework for Reinforcement Learning (RL) algorithms based on Gaussian Process Regression (GPR).
no code implementations • 22 Jan 2020 • Patrik Kolaric, Devesh K. Jha, Arvind U. Raghunathan, Frank L. Lewis, Mouhacine Benosman, Diego Romeres, Daniel Nikovski
Motivated by these problems, we try to formulate the problem of trajectory optimization and local policy synthesis as a single optimization problem.
Model-based Reinforcement Learning reinforcement-learning +1
no code implementations • 26 Dec 2019 • Devesh Jha, Arvind Raghunathan, Diego Romeres
The algorithm has achieved state-of-the-art performance when used in reinforcement learning across a wide range of tasks.
2 code implementations • 23 Oct 2019 • Jonathan Chang, Nishanth Kumar, Sean Hastings, Aaron Gokaslan, Diego Romeres, Devesh Jha, Daniel Nikovski, George Konidaris, Stefanie Tellex
We demonstrate that our model trained on 33% of the possible goals is able to generalize to more than 90% of the targets in the scene for both simulation and robot experiments.
no code implementations • 13 Sep 2018 • Jeroen van Baar, Alan Sullivan, Radu Cordorel, Devesh Jha, Diego Romeres, Daniel Nikovski
Another advantage when robots are involved, is that the amount of time a robot is occupied learning a task---rather than being productive---can be reduced by transferring the learned task to the real robot.
no code implementations • 13 Sep 2018 • Diego Romeres, Devesh Jha, Alberto Dalla Libera, William Yerazunis, Daniel Nikovski
We propose the system presented in the paper as a benchmark problem for reinforcement and robot learning, for its interesting and challenging dynamics and its relative ease of reproducibility.
no code implementations • 13 Sep 2018 • Diego Romeres, Mattia Zorzi, Raffaello Camoriano, Silvio Traversaro, Alessandro Chiuso
This paper discusses online algorithms for inverse dynamics modelling in robotics.
no code implementations • 17 Mar 2016 • Diego Romeres, Mattia Zorzi, Raffaello Camoriano, Alessandro Chiuso
This paper presents a semi-parametric algorithm for online learning of a robot inverse dynamics model.
no code implementations • 17 Jan 2016 • Diego Romeres, Giulia Prando, Gianluigi Pillonetto, Alessandro Chiuso
We consider an on-line system identification setting, in which new data become available at given time steps.
no code implementations • 2 Jul 2015 • Diego Romeres, Gianluigi Pillonetto, Alessandro Chiuso
Unluckily, the stability of the predictors does not guarantee the stability of the impulse response of the system.