245K–295K USD / year
Site Reliability Engineer, Provider Operations
DevOpsRemote — United States
Published on 2026-10-13
These details were extracted automatically from the original listing. They may not be complete or fully up to date — it's worth checking the original job post.
About this role
Own the operational health of OpenRouter’s AI inference providers and endpoints, ensuring they remain fast, correct, and available. Build monitoring, automated failover, quality canaries, and operational tooling, while leading incident response and provider escalations.
About the company
OpenRouter is a unified API that routes requests across large language model providers so developers can access, switch, and scale models without lock-in or complex multi-provider integrations.
The team
Provider Operations team, reporting to the Provider Operations Manager.
Stack
TypeScriptPythonCloudflare WorkersPostgresClickHouseGCPVercel
What you'll do
- Build provider and endpoint monitoring for latency, throughput, errors, uptime, and output correctness.
- Improve degraded-endpoint detection and automated traffic failover.
- Lead provider incident response, communications, postmortems, and follow-up work.
- Create provider scorecards and SLO reporting, and handle technical escalations.
- Build continuous canaries and evaluations to detect inference quality and reporting regressions.
- Automate provider operations such as endpoint changes, capacity adjustments, and rate-limit tuning.
- Develop load-testing tools to assess endpoint readiness before launches.
What we're looking for
- 4+ years in SRE, production engineering, or infrastructure roles supporting high-traffic customer-facing systems.
- Experience with observability, metrics, tracing, logs, SLOs, error budgets, and actionable alerting.
- Software engineering skills, preferably in TypeScript and/or Python.
- Understanding of distributed-systems failure modes such as timeouts, retries, backpressure, and partial outages.
- Able to lead incidents calmly and communicate clearly with external partners.
- Understanding of, or willingness to learn, LLM inference concepts including streaming, tool calling, caching, and latency-throughput tradeoffs.
Nice to have
- Experience at an inference provider, model lab, GPU cloud, API gateway, or CDN company.
- Background in routing, load balancing, or traffic management.
- Experience with evaluations or synthetic monitoring for ML systems.