Software engineer, GPU inference
Work closely with researchers, engineers, and product teams to bring cutting-edge AI models into real-world production; Architect and optimize our inference infrastructure to deliver low-latency, high-reliability perf…
- Employer: Soniox
- Location: Slovenia
- Employment type: Full-time
- Work mode: ON_SITE
- Posted:
- Apply before:
Job description:
- Soniox is pushing the boundaries of real-time speech AI, and we’re looking for an engineer to help us scale the world’s most advanced language models across a low-latency, high-throughput, production-grade inference stack.
- In this role, you’ll work at the intersection of deep learning, systems engineering, and performance optimization — helping us squeeze every FLOP out of our GPUs, reduce latency to the millisecond, and keep our systems running at global scale.
- Work closely with researchers, engineers, and product teams to bring cutting-edge AI models into real-world production.
- Architect and optimize our inference infrastructure to deliver low-latency, high-reliability performance across thousands of concurrent requests.
- Identify and eliminate system bottlenecks, improving throughput and GPU utilization across the fleet.
- Introduce and implement tools and techniques to monitor, debug, and improve model inference at scale.
- Tune our VM fleet to maximize compute, memory, and network efficiency — down to the last GPU cycle.
- Support advanced research workflows by building robust, scalable systems that enable rapid experimentation.
Requirements:
- Have a strong intuition for optimizing modern ML architectures for inference performance.
- Are deeply familiar with PyTorch, CUDA, NCCL, and GPU internals — or excited to become an expert quickly.
- Understand HPC fundamentals and have worked with technologies like InfiniBand, NVLink, or MPI.
- Have experience building and scaling distributed systems in production, ideally performance-critical ones.
- Have rebuilt or refactored systems due to 10x+ scale increases — and know what to watch out for.
- Are a self-starter who thrives in fast-moving environments and finds clarity amidst ambiguity.
- Care about reliability, simplicity, and performance — and take ownership from design to deployment.
- Have at least 5 years of professional software engineering experience.
We offer:
- You’ll help build one of the most technically advanced AI platforms in the world — and shape how it reaches and supports users globally.
- You’ll work directly with a world-class team of engineers and researchers solving frontier problems in speech and language AI.
- You'll have a voice in how our company grows, how our customers succeed, and how AI transforms human communication.