Salary not listed
Software Engineer, Agent Platform
AIOn-site — San Francisco, USA
Published on 2026-09-18
These details were extracted automatically from the original listing. They may not be complete or fully up to date — it's worth checking the original job post.
About this role
We are looking for an AI Engineer to take ownership of model behavior in production systems, ensuring reliability and trustworthiness of AI features. The role involves designing evaluation systems, managing AI behaviors, and working across product and infrastructure teams to deliver dependable results.
About the company
Retool is a software development company that provides a platform for building internal tools rapidly. They serve various industries by enabling businesses to create custom applications tailored to their operational needs.
Stack
TypeScriptNode.jsReact
What you'll do
- Own the behavior of AI-powered features across multiple product surfaces
- Design prompting and tool-use strategies
- Build evaluation systems for model performance
- Diagnose non-deterministic failures
- Define guardrails and fallbacks for AI models
- Partner with product teams to assess model behavior
What we're looking for
- 6+ years of professional engineering experience
- Experience owning AI/LLM behavior and mitigating variance
- Comfort with probabilistic systems
- Designing or maintaining evaluation frameworks
- Strong product intuition with non-deterministic output
- Ability to operate independently in ambiguous settings
Nice to have
- Experience with RAG or agentic systems
- Familiarity with vector databases or retrieval pipelines
- Exposure to fine-tuning or post-training techniques
- Experience building shared AI infrastructure
- Mentoring engineers on designing for non-determinism