Staff Applied AI Engineer, Agentic Systems

Role details

ABOUT BEACON AI We’re a fast-moving team of aviators, engineers, and operators building an AI platform to make flying safer, more efficient, and more capable. Backed by top investors, we’ve secured a dozen Department of Defense contracts and partnered with major airlines to deliver mission-critical systems. We operate without silos or heavy processes. Small, focused teams own what they build, ship quickly, and learn fast, pushing the boundaries of how humans and AI work together in aviation. WHAT YOU'LL DO At Beacon AI, we're building AI-powered systems that help aviation professionals navigate complex, high-stakes operational decisions. This role sits at the intersection of agentic AI, information retrieval, and production software engineering. We build systems that reason over diverse sources of information, use tools to investigate problems, and produce reliable, explainable results. We're looking for engineers who can think critically about how these systems should work, design their underlying information and orchestration architectures, and take them from experimental prototypes to dependable production services. DESIGN AND BUILD AGENTIC SYSTEMS - Design and implement agent architectures involving tool use, planning, orchestration, state management, and memory. - Develop effective context engineering strategies, balancing relevant information, model capabilities, latency, and cost. - Make thoughtful architectural decisions about when to use autonomous agents, explicit workflows, conventional algorithms, or combinations thereof. - Design tool interfaces and structured model interactions that support reliable, inspectable behavior. TURN COMPLEX INFORMATION INTO USABLE KNOWLEDGE - Develop an understanding of complex document collections and data sources: their structure, semantics, authority, relationships, and scope. - Translate domain knowledge into effective retrieval systems, structured representations, and agent-accessible tools. - Apply information retrieval, information extraction, and knowledge modeling techniques to heterogeneous structured and unstructured data. - Design strategies for handling temporal relevance, provenance, conflicting evidence, and incomplete information. EVALUATE AND IMPROVE AI BEHAVIOR - Develop rigorous evaluations for AI-powered systems, including representative datasets, human annotations, automated assessments, and targeted failure analysis. - Investigate system-level failures across retrieval, context construction, reasoning, tool execution, and final outputs. - Experiment with alternative architectures, models, and prompting strategies, using empirical results to guide decisions. - Help establish engineering practices for measuring and improving the reliability of agentic systems. BUILD SYSTEMS THAT WORK IN PRODUCTION - Develop reliable services and asynchronous workflows with attention to concurrency, fault tolerance, observability, and maintainability. - Integrate model inference, retrieval systems, and external data services into production applications. - Build and operate containerized workloads and cloud infrastructure. - Collaborate with engineers and domain experts to translate complex operational requirements into working systems. AREAS OF SPECIALIZATION Applied AI is a broad discipline. We don't expect every candidate to have deep expertise in every part of the stack. We particularly encourage candidates with substantial experience in one or more of the following areas: - Agentic systems and context engineering: Agent orchestration patterns, planning and execution, tool-use architectures, persistent and transient memory, and context management. - Information retrieval and knowledge representation: Semantic and structured retrieval, information extraction, document understanding, entity and relationship modeling, and knowledge-driven tool design. - AI evaluation and reliability: Evaluation methodology, benchmark and dataset construction, human and model-based assessment, statistical analysis, and systematic failure characterization. - Model adaptation and inference optimization: Fine-tuning, model selection, inference serving, quantization, batching, and performance optimization. We value candidates who bring genuine depth to at least one of these areas, along with the curiosity and engineering judgment to work effectively across disciplinary boundaries. WHAT WILL MAKE YOU SUCCESSFUL - Strong software engineering fundamentals. You write maintainable production code, reason about system architecture, and understand testing, debugging, and operational reliability. - Substantive applied AI experience. You've worked on AI systems beyond basic model API integrations and can explain their architecture, tradeoffs, limitations, and observed behavior. - Information modeling instincts. You can understand a new domain, identify its important entities and relationships, and design software abstractions that reflect how its information is actually used. - Experimental discipline. You test assumptions, measure results, investigate failure modes, and use evidence to guide technical decisions. - Independent technical judgment. You can evaluate alternative approaches, recognize unnecessary complexity, and make appropriate tradeoffs. - Clear communication. You can discuss technical decisions with both technical and non-technical stakeholders and domain experts. We care more about demonstrated technical understanding and the quality of your engineering work than familiarity with any particular AI framework. OUR TECHNOLOGY ENVIRONMENT Our systems primarily use Python, LangGraph, BAML, AWS Bedrock, PostgreSQL/pgvector, and event-driven AWS infrastructure. Experience with these technologies is helpful but not required. We're more interested in your understanding of the underlying concepts and ability to work effectively with unfamiliar tools. ESPECIALLY VALUABLE EXPERIENCE - Aviation, aerospace, or another technically complex, safety-critical domain. - Computa