Software Engineer, Inference
Thinking Machines Lab
- Company
- Thinking Machines Lab
- Location
- San Francisco
- Role type
- ML Infrastructure
- Job family
- Software Engineer jobs
- Employment type
- Full Time
- Workplace
- On Site
- Experience level
- Mid
- Salary
- $300k – $350k
- Visa
- Employer states it sponsors work visas (see all)
- Posted
- Last verified open
About Thinking Machines
The mission of Thinking Machines is to build AI that extends human will and judgment. We are training frontier models with Inkling, developing Tinker to let people make models their own, and crafting interfaces that broaden human-AI communication. We believe the future worth building is human, and we're hiring people who want to build it.
About the Role
We're hiring a Software Engineer, Inference to own the reliability, scale, and efficiency of the systems that serve our models to real users. Our research and inference teams push the limits of model performance and serving efficiency; this role makes sure those gains reach production safely and stay up — powering Tinker's live, multi-tenant serving and the products built on top of our models.
This is a production-facing systems role at the center of the company. You'll be the bridge between cutting-edge inference techniques and the day-to-day reality of serving real traffic: rollouts, capacity, incidents, and everything that keeps a fast-growing platform online.
What You'll Do
Operate and scale the production inference systems that serve live traffic, including Tinker's multi-tenant serving platform
Own the rollout process for new models, model versions, and inference optimizations, ensuring safe, incremental deployment to production
Build and improve observability, alerting, and capacity planning so the team can detect, diagnose, and resolve production issues quickly
Partner with inference and research teams to productionize new serving techniques without compromising reliability
Lead incident response for production inference issues, driving root cause analysis and durable fixes
Design for graceful degradation, failover, and redundancy so that serving stays resilient as usage grows
Manage capacity and cost tradeoffs for serving infrastructure as traffic and model sizes scale
Skills & Qualifications
Minimum Qualifications
Experience operating large-scale, latency-sensitive production systems
Proficiency in Python and Go or another systems language
Experience with observability, monitoring, and incident response for production services
Strong understanding of distributed systems and how they fail at scale
Preferred Qualifications
Experience running production inference for large language models or other large-scale ML systems
Experience with deployment and rollout systems, such as canarying, blue/green deploys, or feature flags
Experience with capacity planning and cost optimization for GPU or TPU infrastructure
Familiarity with inference-specific techniques, such as batching, caching, or quantization, and their operational implications
Comfortable being on-call and leading incident response for critical production systems
Comfortable working with high autonomy in a fast-changing, early-stage environment
Logistics
Location: This role is based in San Francisco, CA.
Compensation: Depending on background, skills and experience, the expected annual salary range for this position is $300,000 - $350,000 USD.
Visa sponsorship: We sponsor visas. While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.
Benefits: Thinking Machines offers generous health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.
More jobs at Thinking Machines Lab
Software Engineer, Training Infrastructure
Thinking Machines Lab | San Francisco | $300k – $400k
Technical Program Manager, Pre-Training
Thinking Machines Lab | San Francisco | $350k – $450k
Research, Tinker, RL Systems
Thinking Machines Lab | San Francisco | $350k – $475k
Technical Program Manager, Supercomputing
Thinking Machines Lab | San Francisco
Web Crawling - Research Engineer
Thinking Machines Lab | San Francisco | $350k – $475k
Similar open AI roles
Engineering Manager, Kubernetes Control Plane - Lambda Kubernetes Service
Lambda | San Francisco Office | $297k – $440k
ML Infra Engineer, Compute
Physical Intelligence | San Francisco
Software Engineer, Training Infrastructure
Thinking Machines Lab | San Francisco | $300k – $400k
Data Engineering Manager, Devices
OpenAI | San Francisco | $459k – $536k
Member of Technical Staff - Pre-Training Evals Infra (Inference)
Reflection AI | Global Office
Staff Software Engineer - Data & ML Systems (Behaviors)
Motional | Singapore, Central, Singapore
Member of Technical Staff - Infrastructure Security
Black Forest Labs | Remote | $220k – $280k
Senior Software Engineer II, Inference
CoreWeave | Sunnyvale, CA / Bellevue, WA | $182k – $242k