publications | Hanbit Oh

2024

ISPIL: Interactive Sub-Goal-Planning Imitation Learning for Long-Horizon Tasks With Diverse Goals

Cynthia Ochoa, Hanbit Oh, Yuhwan Kwon, and 2 more authors

IEEE Access, 2024

@article{oh2024ispil,
  title = {{ISPIL}: Interactive Sub-Goal-Planning Imitation Learning for Long-Horizon Tasks With Diverse Goals},
  author = {Ochoa, Cynthia and Oh, Hanbit and Kwon, Yuhwan and Domae, Yukiyasu and Matsubara, Takamitsu},
  journal = {IEEE Access},
  year = {2024},
  volume = {12},
  number = {},
  pages = {197616-197631},
  keywords = {Robots;Switches;Imitation learning;Training;Symbols;Planning;Cause effect analysis;Atoms;Artificial intelligence;Accuracy;Interactive imitation learning;learning-to-plan;hierarchical policy},
  doi = {10.1109/ACCESS.2024.3521302},
  url = {https://ieeexplore.ieee.org/abstract/document/10811934},
  equal = {test},
}

Leveraging Demonstrator-Perceived Precision for Safe Interactive Imitation Learning of Clearance-Limited Tasks

Hanbit Oh, and Takamitsu Matsubara

IEEE Robotics and Automation Letters, 2024

Awarded DOI Bib Website

IEEE Robotics and Automations Society Japan Chapter Young Award (IROS2024)

@article{oh2024dipil,
  title = {Leveraging Demonstrator-Perceived Precision for Safe Interactive Imitation Learning of Clearance-Limited Tasks},
  author = {Oh, Hanbit and Matsubara, Takamitsu},
  journal = {IEEE Robotics and Automation Letters},
  year = {2024},
  volume = {9},
  number = {4},
  pages = {3387-3394},
  keywords = {Task analysis;Collision avoidance;Uncertainty;Safety;Measurement;Standards;Robot learning;Learning systems;Imitation learning;Learning from demonstration;imitation learning},
  doi = {10.1109/LRA.2024.3366755},
  url = {https://ieeexplore.ieee.org/abstract/document/10438830},
}

2023

Bayesian Disturbance Injection: Robust Imitation Learning of Flexible Policies for Robot Manipulation

Hanbit Oh, Hikaru Sasaki, Brendan Michael, and 1 more author

Neural Networks, 2023

Abs DOI Bib PDF

Humans demonstrate a variety of interesting behavioral characteristics when performing tasks, such as selecting between seemingly equivalent optimal actions, performing recovery actions when deviating from the optimal trajectory, or moderating actions in response to sensed risks. However, imitation learning, which attempts to teach robots to perform these same tasks from observations of human demonstrations, often fails to capture such behavior. Specifically, commonly used learning algorithms embody inherent contradictions between the learning assumptions (e.g., single optimal action) and actual human behavior (e.g., multiple optimal actions), thereby limiting robot generalizability, applicability, and demonstration feasibility. To address this, this paper proposes designing imitation learning algorithms with a focus on utilizing human behavioral characteristics, thereby embodying principles for capturing and exploiting actual demonstrator behavioral characteristics. This paper presents the first imitation learning framework, Bayesian Disturbance Injection (BDI), that typifies human behavioral characteristics by incorporating model flexibility, robustification, and risk sensitivity. Bayesian inference is used to learn flexible non-parametric multi-action policies, while simultaneously robustifying policies by injecting risk-sensitive disturbances to induce human recovery action and ensuring demonstration feasibility. Our method is evaluated through risk-sensitive simulations and real-robot experiments (e.g., table-sweep task, shaft-reach task and shaft-insertion task) using the UR5e 6-DOF robotic arm, to demonstrate the improved characterisation of behavior. Results show significant improvement in task performance, through improved flexibility, robustness as well as demonstration feasibility.
@article{oh2022bdi, title = {{Bayesian Disturbance Injection}: Robust Imitation Learning of Flexible Policies for Robot Manipulation}, author = {Oh, Hanbit and Sasaki, Hikaru and Michael, Brendan and Matsubara, Takamitsu}, journal = {Neural Networks}, volume = {158}, pages = {42-58}, year = {2023}, issn = {0893-6080}, doi = {https://doi.org/10.1016/j.neunet.2022.11.008}, url = {https://www.sciencedirect.com/science/article/pii/S089360802200449X}, keywords = {Imitation learning, Disturbance injection, Human behavior characteristics, Robotic manipulation}, }

2022

Disturbance-injected Robust Imitation Learning with Task Achievement

Hirotaka Tahara, Hikaru Sasaki, Hanbit Oh, and 2 more authors

In IEEE International Conference on Robotics and Automation (ICRA), 2022

Abs DOI Bib

Robust imitation learning using disturbance injections overcomes issues of limited variation in demonstrations. However, these methods assume demonstrations are optimal, and that policy stabilization can be learned via simple augmentations. In real-world scenarios, demonstrations are often of diverse-quality, and disturbance injection instead learns sub-optimal policies that fail to replicate desired behavior. To address this issue, this paper proposes a novel imitation learning framework that combines both policy robustification and optimal demonstration learning. Specifically, this combinatorial approach forces policy learning and disturbance injection optimization to focus on mainly learning from high task achievement demonstrations, while utilizing low achievement ones to decrease the number of samples needed. The effectiveness of the proposed method is verified through experiments using an excavation task in both simulations and a real robot, resulting in high-achieving policies that are more stable and robust to diverse-quality demonstrations. In addition, this method utilizes all of the weighted sub-optimal demonstrations without eliminating them, resulting in practical data efficiency benefits.
@inproceedings{tahara2022disturbance, author = {Tahara, Hirotaka and Sasaki, Hikaru and Oh, Hanbit and Michael, Brendan and Matsubara, Takamitsu}, booktitle = {IEEE International Conference on Robotics and Automation (ICRA)}, title = {Disturbance-injected Robust Imitation Learning with Task Achievement}, year = {2022}, volume = {}, number = {}, pages = {2466-2472}, doi = {10.1109/ICRA46639.2022.9812376}, url = {https://arxiv.org/pdf/2205.04195.pdf}, }

2021

Bayesian Disturbance Injection: Robust Imitation Learning of Flexible Policies

Hanbit Oh, Hikaru Sasaki, Brendan Michael, and 1 more author

In IEEE International Conference on Robotics and Automation (ICRA), 2021

Abs DOI Bib PDF

Scenarios requiring humans to choose from multiple seemingly optimal actions are commonplace, however standard imitation learning often fails to capture this behavior. Instead, an over-reliance on replicating expert actions induces inflexible and unstable policies, leading to poor generalizability in an application. To address the problem, this paper presents the first imitation learning framework that incorporates Bayesian variational inference for learning flexible nonparametric multi-action policies, while simultaneously robustifying the policies against sources of error, by introducing and optimizing disturbances to create a richer demonstration dataset. This combinatorial approach forces the policy to adapt to challenging situations, enabling stable multi-action policies to be learned efficiently. The effectiveness of our proposed method is evaluated through simulations and real-robot experiments for a table-sweep task using the UR3 6-DOF robotic arm. Results show that, through improved flexibility and robustness, the learning performance and control safety are better than comparison methods.
@inproceedings{oh2021bdi, author = {Oh, Hanbit and Sasaki, Hikaru and Michael, Brendan and Matsubara, Takamitsu}, booktitle = {IEEE International Conference on Robotics and Automation (ICRA)}, title = {Bayesian Disturbance Injection: Robust Imitation Learning of Flexible Policies}, year = {2021}, volume = {}, number = {}, pages = {8629-8635}, doi = {10.1109/ICRA48506.2021.9561573}, url = {https://arxiv.org/pdf/2103.13623.pdf}, }