Full-Stack Engineer, Agent Evaluation & Reliability

Role details

ABOUT THE ROLE Join a small, technical team building AI agent evaluation and reliability infrastructure for enterprise customers. This backend-leaning full-stack role focuses on making agent performance measurable and improving the systems that support reliable AI products. WHAT YOU'LL DO - Build backend systems and data models for evaluation datasets, experiment runs, agent traces, knowledge versions, and quality criteria. - Develop and operate enterprise integrations, including authentication, data syncing, retries, monitoring, and production support. - Ship features across the stack, from technical design and implementation through deployment, monitoring, and iteration. - Work with customers to diagnose system failures and turn their needs into reusable product capabilities. - Build data pipelines and interfaces for collecting, analyzing, and visualizing evaluation results. WHAT WE'RE LOOKING FOR - At least 3 years of experience building and shipping production systems as a full-stack engineer, ideally including work at a fast-paced startup. - Hands-on machine learning model development experience, along with knowledge of evaluating and testing AI agents or other nondeterministic systems. - Strong backend skills with TypeScript and Node.js or Bun, and experience building production frontend features. - Strong PostgreSQL skills, including query optimization, and practical experience with Redis, job queues, background workers, or workflow systems. - Experience designing APIs and backend services, building evaluation data pipelines, and developing dashboards or visualization interfaces. - Familiarity with observability, monitoring, logging, and production troubleshooting. COMPENSATION & BENEFITS Base salary: $120,000 to $180,000 annually. Visa sponsorship is available. LOCATION On-site in San Francisco, California, United States.