ML Infrastructure Engineer

Role details

ABOUT THE ROLE This is a hands-on infrastructure engineering role at an early-stage enterprise AI company building a context layer that makes AI agents reliable, accurate, and secure for mission-critical business operations. You'll own the systems that keep those agents running fast and reliably in production — from design through deployment — working closely with ML and infrastructure teams to scale inference at increasing concurrency. WHAT YOU'LL DO - Own inference and model-serving infrastructure end to end, from architecture design through production deployment. - Build and scale systems that enable AI agents to run reliably and efficiently under high concurrency in production environments. - Collaborate with ML and infrastructure teams to ensure seamless integration and drive performance optimization. - Identify infrastructure bottlenecks and lead the engineering effort to resolve them. WHAT WE'RE LOOKING FOR - 5+ years of experience building and operating machine learning inference systems, model-serving platforms, or ML infrastructure in production environments. - Hands-on experience designing and scaling inference-serving infrastructure using tools such as TensorFlow Serving, TorchServe, Triton, KServe, or equivalent custom systems. - Demonstrated ability to optimize production ML systems for latency, throughput, and reliability at scale. - Strong proficiency with containerization and orchestration technologies — Docker and Kubernetes — for deploying ML workloads. - Experience building or maintaining distributed systems that handle concurrent requests and manage resource allocation under load. - Solid command of monitoring, observability, and debugging tooling for production systems (e.g., Prometheus, Grafana, ELK, distributed tracing). - Experience deploying and managing ML systems on cloud platforms such as AWS, GCP, or Azure. - Proficiency in at least one systems or backend language: Python, Go, Rust, C++, or Java. - Experience with knowledge graphs, semantic search, or graph databases (e.g., Neo4j, Amazon Neptune) is a plus. - Familiarity with real-time or low-latency inference systems, agentic AI pipelines, or enterprise data infrastructure is a plus. LOCATION On-site in San Mateo, California, United States. Visa sponsorship is not available for this role.