Embedded AI Engineer, On-Device Models

Role details

COMPANY OVERVIEW Deepgram is the leading platform underpinning the emerging trillion-dollar Voice AI economy, providing real-time APIs for speech-to-text (STT), text-to-speech (TTS), and building production-grade voice agents at scale. More than 200,000 developers and 1,300+ organizations build voice offerings that are ‘Powered by Deepgram’, including Twilio, Cloudflare, Sierra, Decagon, Vapi, Daily, Cresta, Granola, and Jack in the Box. Deepgram’s voice-native foundation models are accessed through cloud APIs or as self-hosted and on-premises software, with unmatched accuracy, low latency, and cost efficiency. Backed by a recent Series C led by leading global investors and strategic partners, Deepgram has processed over 50,000 years of audio and transcribed more than 1 trillion words. There is no organization in the world that understands voice better than Deepgram. COMPANY OPERATING RHYTHM At Deepgram, we expect an AI-first mindset—AI use and comfort aren’t optional, they’re core to how we operate, innovate, and measure performance. Every team member who works at Deepgram is expected to actively use and experiment with advanced AI tools, and even build your own into your everyday work. We measure how effectively AI is applied to deliver results, and consistent, creative use of the latest AI capabilities is key to success here. Candidates should be comfortable adopting new models and modes quickly, integrating AI into their workflows, and continuously pushing the boundaries of what these technologies can do. Additionally, we move at the pace of AI. Change is rapid, and you can expect your day-to-day work to evolve just as quickly. This may not be the right role if you’re not excited to experiment, adapt, think on your feet, and learn constantly, or if you’re seeking something highly prescriptive with a traditional 9-to-5. ABOUT THE ROLE Deepgram's speech models are among the fastest and most accurate in the world, and we have deep machinery for running them on NVIDIA GPUs. Our customers need them on everything else: non-NVIDIA accelerators, embedded SoCs, mobile application processors, DSPs and NPUs, and purpose-built devices with tight memory, compute, thermal, and power budgets. When a target platform's standard kernels and runtime can't run a Deepgram model well enough, someone has to go below them. That is this role. As an Embedded AI Engineer on the Partner Platform Engineering team, you work at the lowest layer of our edge stack. You write and optimize custom kernels and operators for specific hardware, collapse models onto device-specific execution units, and do the target-side quantization and assembly-level tuning that standard toolchains can't. You hand what you build up to Applied ML Engineers, who fit Deepgram models to your kernels. Your work is what makes a new hardware platform viable for Deepgram at all. This role is a great fit for a senior embedded engineer who has spent their career close to the metal and wants to point that at speech AI, or a staff-level engineer who wants to define how Deepgram's models get onto new silicon. We'll set the level to your experience. WHAT YOU'LL DO - Write and optimize custom kernels and operators (C, C++, Rust, and platform assembly or intrinsics) for non-NVIDIA accelerators, embedded SoCs, DSPs, and NPUs where the vendor's standard operator set is insufficient for Deepgram models. - Own target-side optimization: collapse models onto device execution units through quantization, operator fusion, memory layout, and architecture-specific compilation to meet latency, memory, power, and thermal budgets. - Integrate with vendor NPU/DSP toolchains and edge inference runtimes, and extend them with custom operators when the graph doesn't map cleanly. - Deliver kernels and runtime components as reusable building blocks that Applied ML Engineers can target when adapting models, with clear interfaces and documented constraints. - Build performance-critical runtime code for embedded environments, including embedded Linux, bare-metal, and RTOS targets. - Establish per-platform benchmarking and validation for latency, accuracy, power, memory footprint, and utilization, and catch regressions before they ship. - Partner with silicon and platform vendors on SDK integration and low-level performance tuning for new chipsets and reference platforms. - Feed hardware constraints back to Applied ML and Research so model designs are easier to land on constrained targets. YOU'LL LOVE THIS ROLE IF YOU - Find deep satisfaction in making a large model run on hardware that was never meant to run it, and still hitting accuracy and latency targets. - Reach for the profiler and the ISA manual before you reach for a bigger chip. - Would rather write the kernel than wait for the vendor to ship it. - Care about the details that don't show up in a cloud benchmark: cold start, power draw, thermals, memory fragmentation, cache behavior. - Prefer hard, constrained, ship-it problems over open-ended research. - Care about the details that don't show up in a cloud benchmark: cold-start time, power draw, thermals, and memory fragmentation. IT'S IMPORTANT TO US THAT YOU HAVE - Experience delivering production systems on resource-constrained hardware — embedded systems, mobile, edge AI, or small low-power devices. - Strong proficiency in C, C++, and/or Rust, with experience writing performance-critical code for constrained environments. - Hands-on experience with model optimization for on-device deployment, including quantization, pruning, knowledge distillation, or architecture-specific compilation. - Familiarity with edge inference runtimes (e.g., ONNX Runtime, TensorRT, TFLite, ExecuTorch) and/or vendor-specific NPU/DSP toolchains. - A strong understanding of hardware-software interaction — CPU/GPU/NPU/DSP architectures, memory hierarchies, fixed-point/integer arithmetic, and power management — and how they affect inference performance. - Experience working close to the metal: