Robotics Researcher - Reinforcement Learning

JOB SUMMARY 

At Apptronik AI Core Robotics team (AIRcore) team, you will build a new generation of world-class large behavior and reasoning models for multi-resolution decision making in Robotics and build production grade robotics solutions and deliver this to customers.

You will build a stack of fundamental perception and decision-making algorithms to create a new generation of Robot controllers and build production grade robotics solutions and deliver this to production. Implement core algorithmic infrastructure in a task-agnostic manner and conduct research to build perception and controllers for high-dimensional physical systems such as full-body humanoids and bimanual dexterous hands.

ESSENTIAL DUTIES AND RESPONSIBILITIES or KEY ACCOUNTABILITIES 

This role touches all aspects of reinforcement learning for control, from research to development. Work may include RL training, simulation pipeline development, sim2real evaluation, as well as tooling and support. You will work on systems for training and finetuning multimodal foundations models for robotics at scale:

• Build a distributed reinforcement learning training framework.
• Build simulation infrastructure to enable locomanipulation
• Build RL policies in physical simulation and on robot hardware

Scale RL for locomotion, dexterous manipulation as well as bimanual, multiagent control.

TECH STACK 

• Python
• Pytorch, JAX and XLA
• CUDA (C++ and Triton)

EDUCATION and/or EXPERIENCE 

• Expertise in Reinforcement Learning, Imitation Learning and Inverse RL, specifically for continuous control.
• Familiarity with common algorithms (PPO, Actor-critic methods, Model-based methods) and training strategies for RL (reward shaping, curriculum design)
• Experience with simulation engines such as Mujoco, Isaac Lab, Isaac Sim, Drake or similar.
  
• Experience building training codebases for large-scale distributed ML models, in particular generative models such as transformers and diffusion. 
• Experience  ML Workflows and optimizing efficiency of distributed training systems and/or inference systems.
  
• Background in robotics is desired but not necessary. 
• Robotics Background in a subset of Dexterous Hands, Legged locomotion , Understanding of modeling and analysis used in robotics including kinematics, dynamics, motion planning, perception, task planning, and control theory
• Master's/ PhD degree in Computer Science, Machine Learning, Engineering, Robotics, or a related field. 
• Demonstrated track record. The portfolio can consist of: code, technical writing (white papers or blog posts), peer reviewed papers in Tier-1 Machine Learning, Robotics and Computer Vision
• 2 Years of relevant full-time experience post-PhD or 4 years of experience otherwise

PHYSICAL REQUIREMENTS  

• Prolonged periods of sitting at a desk and working on a computer
• Must be able to lift 15 pounds at times
• Vision to read printed materials and a computer screen
• Hearing and speech to communicate