Machine Learning Engineer - Vision Foundation Models
Zoox
- Company
- Zoox
- Location
- Foster City
- Role type
- ML Engineering
- Job family
- Machine Learning Engineer jobs
- Employment type
- Full Time
- Workplace
- Hybrid
- Experience level
- Mid
- Salary
- $254k – $305k
- Posted
- Last verified open
The Perception team at Zoox is responsible for how our purpose-built robotaxi understands the world, including what happens inside the cabin. In this role, you will design, train, and evaluate vision-based in-cabin perception models that understand passengers, objects, and behaviors inside the vehicle. These models support rider safety and the operations that keep each ride running smoothly.
You will independently own a model area end to end, from data curation and training through evaluation and deployment under real-time and compute constraints, working closely with partners across the company. Your work has a direct impact on rider safety and experience as Zoox scales its robotaxi service.
In this role, you will...
-
Design, train, and evaluate vision models that understand passengers, objects, and behaviors inside the vehicle, across detection, pose, and action recognition.
-
Independently own a model or feature area end to end: curate data, define labeling guidelines, train and evaluate models, and take them through deployment.
-
Set the quality bar for the models you own, using rigorous evaluation and error analysis so they are reliable enough to support live operations.
-
Deploy models under real-time and compute constraints shared with the rest of the onboard perception system.
-
Partner across product, operations, safety, and vehicle software to turn requirements into model specifications, metrics, and validation plans.
Qualifications
-
MS or PhD in Computer Science, Machine Learning, Computer Vision, or a related field with 6+ years of industry experience (PhD with 3+ years).
-
Deep expertise in computer vision and deep learning for image and video understanding: object detection, pose and action recognition, and fine-grained classification, with hands-on experience with modern architectures (convolutional and vision-transformer detectors, and self-supervised or foundation-model backbones such as DINOv3).
-
Strong proficiency in Python and PyTorch for model design, training, and evaluation.
-
Experience with vision-language models or foundation-model backbones, and with auto-labeling or synthetic data for long-tail and rare classes.
-
Track record of owning the full ML lifecycle on real-world data: dataset curation and labeling guidelines, training, metric design (precision, recall, and false-positive rate at an operating point), and error analysis.
Bonus Qualifications
-
Experience with in-cabin or occupant monitoring (occupant or driver monitoring systems), ideally including multi-camera or fisheye imagery and low-light conditions.
-
Experience deploying deep learning models to real-time, resource-constrained systems (robotics, automotive, or edge devices) with TensorRT / ONNX or similar.
-
Proficiency in C++ for model integration.
More jobs at Zoox
Contract Student Worker - Machine Learning Engineer
Zoox | Boston, MA
Data Science Intern [Summer 2027]
Zoox | Foster City, CA
Staff / Senior Staff Machine Learning Engineer - Collision Avoidance System
Zoox | Foster City, CA | $291k – $409k
Senior Data Scientist
Zoox | Boston, MA | $225k – $307k
Data Scientist, Fleet Orchestration Algorithms
Zoox | Foster City, CA | $177k – $217k
Similar open AI roles
Contract Student Worker - Machine Learning Engineer
Zoox | Boston, MA
Senior Machine Learning Engineer (Applied Intelligence)
Anduril Industries | Santa Ana, California | $220k – $292k
Principal Engineer, Autonomy Systems Engineering, ODD Definition & Risk
Motional | Boston, Massachusetts, United States; Las Vegas, Nevada, United States; Pittsburgh, Pennsylvania, United States; Remote U.S. | $200k – $275k
Senior/Staff Systems Engineer, Autonomy Verification
Nuro | Mountain View, California
Software Engineer, Model Lifecycle and Signals
Cursor | New York, NY | $140k – $225k
Sr. Model Engineer, Voice
xAI | Palo Alto, CA | $220k – $357k
2027 Summer Intern, PhD, Machine Learning - Generative World Models & Controllable Simulation
Waymo | Mountain View, California
2027 Summer Intern, Machine Learning, MS/PhD, Simulation Realism (World Model)
Waymo | Mountain View, California