200K–400K USD / year
Product Manager, Inference Platform
ProductHybrid — San Francisco, New York
Published on 2026-11-11
These details were extracted automatically from the original listing. They may not be complete or fully up to date — it's worth checking the original job post.
About this role
Lead product work that turns advances in AI inference performance into capabilities customers can confidently use in production. Partner with inference engineers, researchers, and customers to improve speed, reliability, and cost across real workloads.
About the company
Baseten is an inference cloud that helps companies run and scale AI models in production with flexible infrastructure and developer tooling.
The team
EPD, Product
Stack
PyTorchvLLMTensorRT-LLMSGLangNVIDIA Dynamo
What you'll do
- Bring new inference performance capabilities to customers in partnership with engineering and research.
- Translate advances in batching, caching, quantization, speculative decoding, and serving architectures into product features.
- Define benchmarks and evaluations for latency, throughput, quality, and cost.
- Prioritize optimizations based on customer workloads and impact.
- Make performance features easier to configure, evaluate, and adopt in production.
- Help customers assess tradeoffs across hardware, models, autoscaling, and routing.
- Partner with engineering on release readiness, compatibility, and regression testing.
What we're looking for
- Experience building and shipping AI/ML products or infrastructure where performance, scalability, or production behavior mattered.
- Technical background in software engineering, ML engineering, research, or equivalent hands-on experience.
- Strong understanding of model serving architectures, GPU economics, and latency, throughput, quality, and cost tradeoffs.
- Experience working with researchers or infrastructure engineers to bring complex capabilities into production.
- Rigorous approach to benchmarking, experimentation, and performance measurement.
- Strong product judgment and ability to translate technical work into customer value.
Nice to have
- Hands-on experience with PyTorch, vLLM, TensorRT-LLM, SGLang, NVIDIA Dynamo, or similar technologies.
- Experience with open-source inference projects, model optimization, or ML systems research.
Benefits
- Meaningful equity
- Medical, dental, and vision coverage for employees and dependents in the U.S.
- Flexible PTO and company-wide Winter Break
- Paid parental leave
- Fertility and family-building stipend
- Company-facilitated 401(k) in the U.S.