140K–225K USD / year
Software Engineer, Model Routing & Inference
AIOn-site — New York, NY
Published on 2026-11-10
These details were extracted automatically from the original listing. They may not be complete or fully up to date — it's worth checking the original job post.
About this role
Build and operate the inference platform that routes AI requests across providers. The role focuses on making inference faster, more reliable, and cost-effective at high traffic volumes.
About the company
Cursor builds an AI coding editor and agent workflows that help developers write and ship software faster. It serves software teams looking for tighter AI assistance inside the IDE.
What you'll do
- Build and evolve an inference gateway that standardizes provider APIs
- Design cross-provider failover to reduce user impact from outages
- Implement routing backpressure and admission control to handle traffic spikes
What we're looking for
- Experience building high-throughput, low-latency distributed systems
- Bachelor’s degree in a relevant discipline, or at least 2 years of professional software development experience
Nice to have
- Understanding of GPU utilization, provider economics, and capacity planning
- Experience shipping production systems that handle millions of requests
Benefits
- Equity
- Medical, vision, and dental coverage
- 401(k) plan
- Short- and long-term disability insurance
- Life insurance
- Other discounts and perks
Hiring process
- 2–3 short technical interviews
- Onsite project, discussion, and team meetings