James Steiner
Robotics & Vision-Language-Action models
I work on robot learning and Vision-Language-Action models, that is teaching robots to act from pixels and natural-language instructions. Below are a few projects; each links to its code.
Projects
Building a V-JEPA-style world model to learn driving dynamics from nuScenes
I implemented a V-JEPA-style latent-prediction world model from scratch (context and target encoders with an EMA teacher, a clip-level predictor, aggressive tube masking), trained it and an MAE pixel baseline with 2-GPU DDP, and probed both for ego-speed. With a parameter-free k-NN probe, JEPA's frozen features read speed (R² about 0.75) and clearly beat the pixel baseline, which keys on appearance: the latent-versus-pixel result the project set out to test. A high-dimensional linear probe overfit the tiny dataset and hid it, a reminder that probe choice matters at small N.
Flow vs. regression vs. diffusion: a controlled comparison of VLA action heads
I trained the same VLA action expert three ways (flow matching, L1 regression, and DDPM diffusion) and changed nothing else, to see how much the objective actually matters.
Can a Transformer Learn Position in a Few Thousand Parameters?
Replacing nanoGPT's 131,072 position-embedding parameters with a 3,456-parameter periodic attention bias on enwik8, a 38× reduction with no measurable loss in performance.