Undergraduate student — Hanoi, Vietnam
Building a real foundation in robotics and machine learning by constructing a complete smartphone-teleoperation + imitation-learning stack on a low-cost 6-DoF arm — everything from first principles, everything leaving a runnable artifact. Target: ICRA 2028.
A smartphone is the cheapest ubiquitous 6-DoF sensor in the world. This project turns one into a teleoperation device for a low-cost robot arm: WebXR streams the phone's pose at 60 Hz over WebSocket to a 50 Hz Python server (clutch-based delta-pose, One-Euro filtering, damped-least-squares IK) driving a 6-DoF SO-101 arm. Demonstrations are recorded as LeRobot datasets and used to train imitation-learning policies (ACT, SmolVLA fine-tuning).
The system is deliberately built from first principles — kinematics, deep learning, controls — rather than assembled from black boxes. Every topic studied must leave a runnable artifact in the repo. The full teleoperation stack will be released open-source (repo + reproducible README + video + blog) by the end of 2026.
The research claim is intentionally kept open until mid-2027, when it will be finalized with my advisor against the novelty landscape at that time (candidates: controlled comparison of demonstration interfaces × data quality × policy; residual calibration for low-cost servos). Experiments run summer 2027; target venue ICRA 2028 (backups: IROS 2028, RA-L).
One focus per phase. Statuses below are parsed live from the project's working notes — including what is not done yet.
Kinematics done properly (Modern Robotics ch. 2–6 with exercises), SO(3) / quaternions / Lie exp-log, servo-bus electronics, WebXR streaming + One-Euro filtering + damped-least-squares IK.
PyTorch, then Karpathy's Zero-to-Hero in full (micrograd → GPT), hand-rolled behavior cloning to watch compounding error happen, ACT paper + code studied block by block, SmolVLA fine-tuning, minimal statistics (Wilson CI, multi-seed) before any evaluation.
ROS 2 Jazzy (port the teleop stack to a node graph), classical control (PID, trajectory tracking), intro RL (PPO) in MuJoCo — each pillar time-boxed to ~1 month with exactly one artifact.
Statistics and experiment design, run the paper experiments (3 interfaces × 3 tasks + ablations), write.
| # | Deliverable | Definition of done | Status |
|---|---|---|---|
| H0 | Order components | Servos, control board and PSU ordered; STL files sent for 3D printing | planned |
| M0 | Environment + simulation | Ubuntu + LeRobot running; end-to-end phone → virtual-arm teleop and an imitation-learning dry-run, all in simulation | planned |
| M1 | Arm alive | Assembled and calibrated; per-joint jog; forward kinematics verified against a physical ruler | planned |
| M2 | Baseline teleop | End-effector control from keyboard; 5 demos recorded and replayed; logging schema v0 frozen | planned |
| M3 | Phone streaming | WebXR pose + live visualizer; < 50 ms latency to the server | planned |
| M4 | Phone → real arm | Writing letters in the air; clutch, filtering and safety limits running; demo video published | planned |
| M5 | Imitation learning end-to-end | 30 demos → ACT trained on Kaggle → ≥ 50 % success over 20 rollouts, reported with Wilson CI | planned |
| M6 | Paper experiments | Full dataset: 3 interfaces × 3 tasks + ablations | planned |
A skill is only claimed when the milestone that proves it is done. Gray = scheduled, with the phase that will produce the evidence.
Working with Tuan Dang (University of Arkansas). A knowledge presentation every Tuesday — preparation capped at 1 h / 5 slides so building always beats slide-making. First session: 14 Jul 2026.
Plus a fixed 15-minute Sunday review against a per-week checklist.
1 paper read in full (three-pass method), notes kept per paper.
Prior work tracked weekly: Phone2Act, RoboPocket, GELLO, RoboTurk. This watchlist already caught one overlap early (Phone2Act, May 2026) and forced a strategic re-plan — which is exactly why the paper claim stays open until mid-2027.
Every topic ends as a note with checkpoint questions, which feed a spaced-repetition review on a 1-7-30-day schedule.