JHU Hands-on Robot Learning Fall 2026
使用 SO-101 桌面机械臂学习数据采集、模仿学习与策略部署；含官方讲义、交互演示和实物作业总览。
- 单元
- 13 单元
- 发布
- 0 单元
- 内容
- 0 份
- 资料
- 38 条
Lab 与作业
入口按本课程官网公布的作业登记；代码、题目版本及访问要求以原站为准。
- A0–A3 · 官方机械臂作业总览 — A0 搭建校准；A1 手工控制；A2 单任务模仿；A3 多任务策略。实机作业需 SO-101 与相机；不把论文仓库视为作业起始代码。 · 课程来源
课程学习路径
00 · 课程参考资料
官方来源、作业与版本说明。
01 · 课程与学习资料
以 10 月 15 日更新的课程主页为准；作业为 A0–A3，部分旧指南侧栏编号仍为 A1–A4。
- 01
Introduction to modern robot learning
The data-driven robotics lifecycle; the state of the art in robot learning.
2026-09-03计划中 - 02
Essentials (whirlwind recap)
Robot design and payload computations; smart servos and how they work; electromechanical components of the SO-101; anatomy of a robot manipulator; interpreting motor datasheets; joint-space and Cartesian-space representations; forward and inverse kinematics; Jacobians and singularities; position-based feedback control; PID control; and a quick recap of modern machine learning.
2026-09-10官方资料已接入 - 03
Data for robot learning: teleoperation and other sources
Data sources for robot learning; What does/should a demonstration contain; commanded-vs-realized poses; leader–follower teleoperation; GELLO and gravity compensation; Task-space teleoperation; Wearable interfaces and UMI; Dexterous and whole-body human retargeting; and Open problems in scalable robot data collection.
2026-09-17官方资料已接入 - 04
Imitation learning and action chunking
Policies, states, observations, and Markov decision processes; Behavioral cloning; Covariate shift and compounding errors; Recovery data, DART, and DAgger; Policy evaluation and causal confusion; Action chunking and observation, prediction, and execution horizons; ACT architecture, training, and inference; Practical policy training and controller consistency; Temporal ensembling; and Tradeoffs in chunk length and policy context.
2026-09-24官方资料已接入 - 05
SO-101 assembly and calibration (in-class hands-on session).
In-class build session: arm assembly, motor configuration, calibration, camera checks, and basic safety.
2026-10-01计划中 - 06
Generative policies
Multimodality in robot learning; diffusion and flow matching; diffusion-model training and sampling; Diffusion Policy and conditioning; preprocessing, normalization, and closed-loop evaluation; relevant research on diffusion policies; action-chunk inference, prediction and execution horizons, real-time chunking, continuity, and reactivity.
2026-10-08官方资料已接入 - 07
Multitask policy learning and vision-language-action (VLA) models
Generalization and multitask policy learning; task-conditioned diffusion policies and Large Behavior Models; language-conditioned policies and early VLAs; representing heterogeneous robot data; action tokenization (FAST, VQ-BeT, OAT) and continuous action representations; Octo and π 0 ; learning from multiple data sources with GR00T; latent actions, simulation and video-based data augmentation; catastrophic forgetting and knowledge insulation; π 0.5 and open-world generalization.
2026-10-15官方资料已接入 - 08
VLA models, continued; Assignment 1 in-class demos
First half: continued discussion of VLAs and multitask policies. Second half: Assignment 1 demos of handcrafted manipulation skills.
2026-10-29计划中 - 09
Hands-on policy learning with LeRobot
Course-staff tutorial on policy-learning best practices: dataset preparation, training, checkpointing, deployment, debugging, and closed-loop evaluation.
2026-11-05计划中 - 10
Assignment 2 in-class demos — single-task policies
Demonstrate learned single-task policies; discuss task success, failure cases, and lessons from data collection, training, and deployment.
2026-11-12计划中 - 11
Coding agents as policies
Code as Policies and variants such as CaP-X; ENPIRE and related systems.
2026-11-19计划中 - 12
World models and policy learning
Learned dynamics and world models in robotics; action-centric versus goal-centric frameworks; policy learning, planning, prediction, and action extraction; reward and value models; and evaluation.
2026-12-03计划中 - 13
Assignment 3 / final demos
Multi-task policy evaluations, demos, and short presentations.
2026-12-10计划中