GSoC’22 RoboComp project: Reinforcement Learning for pick and place operations

30th June 2022

About me

Hey all!! This is Vamsi Anumula. I am currently working as a software engineer at Rakuten Mobile, Inc. I have completed my bachelors from India Institue of Technology, Hyderabad in Mechanical Engineering with a Minor in CS. I have been exploring the field of Deep Learning and Reinforcement Learning specifically, since the past two years from classic RL to current open research areas like meta learning, offline learning, model based RL.I have pursued a research internship in Robotics and Intelligent Systems Lab at IITH in Applied RL for Robotics. The Reinforcement Learning problem is the Artificial General Intelligence(AGI) problem and it has been a highly research oriented field which has been able to achieve superhuman performance in game playing, but still has to uncover its potential involving real world scenarios like robotics. With current research advances in Deep Learning and computational capability in an extraordinary pace, I believe we will be able to tap into its potential soon. I want to be part of the journey to AGI and this has been my motivation towards pursuing research in RL and robotics. …

About the Project

The aim of this project is to tackle the Robotic Manipulation task using Deep Reinforcement Learning. We will train a Kinova Gen3 arm simulated in CoppeliaSIm, to grasp and place objects at a desired position. Robotic Manipulation involves high dimensional state space and pretty complex dynamics, making it a challenging task to solve. A custom OpenAI Gym environment is created in Python, for simulating the Kinova Gen3 arm. The MDP for this problem is formulated with a continuous state space, discrete action space and a sparse reward setting. The agent will be trained using Model-Free Deep RL algorithms such as Deep Q-Network(DQN) and advanced techniques like Hind Sight Expereince Replay (HER) to allow sample-efficient learning for this complex environment. …

Vamsi Anumula