Full-Stack Engineer, Agent Evaluation & Reliability

Role details

ABOUT THE ROLE Join a seed-stage AI infrastructure startup building systems that help enterprise AI agents become more reliable. This backend-leaning full-stack role focuses on the data models, integrations, and workflows that turn agent performance signals into continuous product improvements. WHAT YOU'LL DO - Build and operate backend systems for agent evaluation and continuous improvement. - Design data models for evaluation datasets, experiment runs, agent traces, knowledge versions, findings, and customer-defined quality criteria. - Develop and maintain enterprise integrations, including authentication, OAuth, webhooks, syncs, backfills, retries, and monitoring. - Own features from technical design and implementation through deployment, production monitoring, customer feedback, and iteration. - Work with customers to understand their systems, diagnose failures, and turn specific needs into reusable product capabilities. - Ship full-stack features that make the platform's underlying infrastructure useful to customers. WHAT WE'RE LOOKING FOR - Two to four years of experience building and operating production software, including experience at a fast-paced startup. - Hands-on machine learning engineering and model development experience, with an understanding of how evaluating nondeterministic agent systems differs from conventional testing. - Strong production experience with TypeScript and Node.js or Bun, and the ability to work across the full stack. - Strong PostgreSQL skills and practical experience with Redis, job queues, background workers, or workflow systems. - Experience owning projects end to end, moving quickly, and making sound technical and product decisions. COMPENSATION & BENEFITS Base salary: $120,000 to $180,000 annually. Visa sponsorship is available. LOCATION On-site in San Francisco, California, United States.