170K–170K USD / year
Lead Engineer, AI Quality
AIRemote — Portugal
Published on 2026-09-30
These details were extracted automatically from the original listing. They may not be complete or fully up to date — it's worth checking the original job post.
About this role
The Lead Engineer for AI Quality will build and oversee evaluation frameworks to measure and improve the quality of AI-powered features at Circle. This role involves leading a team while remaining hands-on in engineering tasks related to AI systems.
About the company
Circle is a technology company that provides a platform for building community-driven spaces, focusing on enabling creators and teams to engage and collaborate effectively. They offer tools for marketplace solutions, enhancing user experience through design and advanced technology.
The team
AI Quality engineering team
Stack
Ruby on RailsPython
What you'll do
- Design CI/CD pipelines and datasets for evaluating AI agents
- Diagnose issues in the agent pipeline
- Build annotation workflows and AI-generated conversations
- Run structured experiments across AI models
- Lead and manage the AI Quality engineering team
- Collaborate with AI Core engineering
What we're looking for
- 7+ years of production software experience
- Familiarity with LLM-powered agents
- Experience in Ruby on Rails and Python
- Background in ML/AI evaluation infrastructure
- Experience designing datasets and annotation workflows
- Ability to learn quickly and thrive in dynamic environments
Benefits
- U.S.-benchmarked compensation
- Equity in the company
- 35 days of paid time off