Technical Lead Manager, ML Dataset Platform
Waabi
- Company
- Waabi
- Location
- San Francisco
- Role type
- ML Infrastructure
- Job family
- Engineering Manager jobs
- Level
- Senior roles
- Employment type
- Full Time
- Workplace
- Hybrid
- Experience level
- Senior
- Salary
- $200k – $280k
- Posted
- Last verified open
You will…
-
Lead and grow the ML Dataset Platform team as both its technical lead and people manager: set the technical direction, define the roadmap, hire, and mentor engineers.
-
Define the APIs, tools and storage formats for Waabi's ML datasets, balancing maximal reuse of data and tooling against the specific needs of each training workload.
-
Build the tooling to manage datasets at scale, including versioning and efficient incremental updates without full rebuilds.
-
Design the APIs that let teams express training-time transformations along the continuum between fully reproducible ahead-of-time ETL and dynamic transformation within data loaders.
-
Own the common dataloading tools used across workloads, including observability into bottlenecks and the throughput of training jobs.
-
Streamline data access for multi-cloud training so that datasets are available as needed with predictable cost and latency.
-
Make data lineage a first-class, fully automatic property of every dataset, from raw data through to the model trained on them.
-
Partner with the ML Training Platform team on the boundary between data and training infrastructure, and with the ML development teams on transitioning shared-dataset tooling into the new platform.
-
Drive adoption: work directly with model teams to migrate their training data onto the platform and feed what you learn back into your team’s roadmap.
Qualifications:
-
Deep ML dataset experience: You have designed and built large-scale data infrastructure for ML training (storage formats, dataset versioning, ETL, dataloading) and understand the performance characteristics of petabyte-scale sensor or multimodal data.
-
Strong software engineering and implementation: You have very strong Python skills and are comfortable in PyTorch training loops; you have strong software-engineering fundamentals and experience with distributed systems, cloud storage, and data processing frameworks (e.g., Spark, Ray, Beam).
-
Proven technical leadership: You have led a platform or infrastructure effort used by multiple teams, defined its APIs, and driven adoption across an organization.
-
People leadership: You have managed or tech-led a team of engineers, including hiring and mentoring, and can balance hands-on technical contribution with roadmap ownership and stakeholder management.
-
Team player: You communicate clearly with researchers and engineers, build consensus across teams with different needs, and deliver in a fast-paced environment.
Bonus:
-
Experience with ML data infrastructure in autonomous driving or robotics (multi-sensor logs, LiDAR, camera, HD maps).
-
Experience with columnar and tensor storage formats (e.g., Parquet, Lance, WebDataset, Zarr) and dataset versioning tools.
-
Experience with multi-cloud or hybrid training environments and the data-movement challenges that come with them.
-
Experience with data lineage, metadata catalogs, or feature-store systems.
- Competitive compensation and equity awards.
- Health and Wellness benefits encompassing Medical, Dental and Vision coverage (for full-time employees only).
- Unlimited Vacation.
- Flexible hours and Work from Home support.
- Daily drinks, snacks and catered meals (when in office).
- Regularly scheduled team building activities and social events both on-site, off-site & virtually.
- As we grow, this list continues to evolve!
More jobs at Waabi
Safety & Systems Technical Program Manager
Waabi | Remote US | $152k – $232k
Senior / Staff Technical Program Manager, Platform Software
Waabi | San Francisco, CA | $150k – $230k
Senior / Staff Research Engineer, Generative AI Deployment
Waabi | Toronto, ON | $200k – $275k
Lead Technical Program Manager, Robotaxi
Waabi | San Francisco, CA | $235k – $270k
Lead Product Manager, Hardware Platform
Waabi | San Francisco, CA | $190k – $270k
Similar open AI roles
Engineering Manager, Kubernetes Control Plane - Lambda Kubernetes Service
Lambda | San Francisco Office | $297k – $440k
ML Infra Engineer, Compute
Physical Intelligence | San Francisco
Software Engineer, Training Infrastructure
Thinking Machines Lab | San Francisco | $300k – $400k
Data Engineering Manager, Devices
OpenAI | San Francisco | $459k – $536k
Member of Technical Staff - Pre-Training Evals Infra (Inference)
Reflection AI | Global Office
Staff Software Engineer - Data & ML Systems (Behaviors)
Motional | Singapore, Central, Singapore
Member of Technical Staff - Infrastructure Security
Black Forest Labs | Remote | $220k – $280k
Senior Software Engineer II, Inference
CoreWeave | Sunnyvale, CA / Bellevue, WA | $182k – $242k