Senior AI Product Engineer

About this role

Own end-to-end improvements to Oyster AI, a customer and team-member assistant, from problem definition and implementation through evaluation, rollout, and production learning. The role spans agents, retrieval, answer quality, and infrastructure, balancing model quality, latency, reliability, and cost.

About the company

Oyster is a global employment platform that lets companies hire, pay, and manage workers in other countries through employer-of-record and related services.

Stack

What you'll do

• Design and ship production AI improvements from user needs through safe rollout.
• Build reliable agents and tools with robust error handling and appropriate human control.
• Improve knowledge retrieval while preserving data access boundaries.
• Develop evaluation loops, datasets, experiments, regression checks, and calibrated automated judges.
• Diagnose and fix production failures across prompts, retrieval, tools, models, and infrastructure.
• Guide and mentor other product teams adopting reusable AI patterns.

What we're looking for

• Senior-level engineering judgment and experience leading complex production work.
• Hands-on experience shipping and operating LLM-powered product features.
• Depth in retrieval/RAG, evaluation, LLM-as-judge, or agents and tool calling.
• Strong coding, debugging, testing, API integration, and system design skills.
• Sound judgment about sensitive data, access control, model limitations, and operational risk.
• Clear communication, async collaboration, and practical mentoring skills.

Nice to have

• Production Ruby on Rails experience.
• Full-stack delivery of conversational or assistant experiences.
• Slack or in-app assistant integrations.
• Experience operating retrieval pipelines, agent tools, evaluation infrastructure, model routing, or cost controls at scale.

Benefits

• 40 days of paid time off per year, including holidays and vacation, or more where required locally.
• At least three months of paid parental leave.
• Monthly wellbeing allowance and mental health support.
• Home-office equipment stipend.
• Work from anywhere where authorized, with a fully remote, async-friendly culture.