AI Platform Engineer, Applied AI

About the role

The Applied AI team at Circle is at the forefront of leveraging AI to drive innovation through transformative applications of LLMs. We’re looking for a strong AI Platform Engineer to help establish the foundation for robust infrastructure for measuring, diagnosing, and improving AI system speed and performance. This is the first AI platform engineer hire at Circle, focusing on production AI systems rather than ML research.

What you’ll be doing

• Build evaluation infrastructure to measure AI system speed and accuracy (offline and online)
• Create observability tooling and dashboards that surface quality metrics week-over-week
• Diagnose quality gaps in retrieval, ranking, prompting, or other areas
• Experiment with different models and agent configurations using data to guide decisions
• Prototype and validate improvements to our RAG pipeline (chunking, retrieval, re-ranking)
• Analyze customer usage of AI features to identify improvements
• Help the team stay up-to-date with cutting-edge AI research and tools

What you’ll need to be successful

• 6+ years of experience in software engineering, data science, or ML with a foundation in search/retrieval
• Experience building and evaluating production AI systems (RAG, search, LLM apps)
• Strong proficiency in Python for prototyping and experimentation
• Openness to learning Ruby on Rails for production system integration
• Comfortable building infrastructure and tooling (eval pipelines, dashboards, data processing)
• Deep understanding of RAG architecture and context management
• Strong experimentation mindset and data analysis skills
• Desire to work in an environment valuing speed of iteration and autonomy
• Proficiency in English at a CEFR Level C2 / ILR Level 5

Bonus points

• Experience with evaluation frameworks (Braintrust, LangSmith, or similar)

What We Have to Offer

• Fully remote: work from anywhere in the world!
• Autonomy and trust to do your job: we care about outcomes over everything else.
• Paid time away: all employees are given 35 days of PTO annually. We also offer a paid sabbatical after 5 years.
• Generous U.S. benchmarked compensation and startup equity no matter where you are in the world.
• Awesome medical coverage with 100% coverage for you and your family, or medical reimbursement options where applicable!
• Parental leave for parents expanding their family, or just starting one.
• Home office stipend to help you get up and running.
• Learning & development stipend to help you level up your professional skills.
• Company retreats: Twice a year, the Circle team gets together for a fully paid company retreat in incredible places around the world!