Software Development Engineer, AI Runtime
Amazon
- Location
- Onsite (Pune, Maharashtra)
- Employment
- Full-time
- Level
- Mid Level
Posted 2 weeks ago
About the Role
Join the AI Runtime team to build and scale the next generation of infrastructure for Alexa+. You will design large-scale distributed systems that orchestrate expert agent executions and LLM inference pipelines, ensuring reliable, low-latency delivery for millions of customers globally.
Skills
Distributed systems
LLM inference
Software development
System design
Architecture
Latency optimization
Multi-agent coordination
Prompt routing
Operational excellence
Machine learning integration
Code reviews
Technical leadership
Performance tuning
Reliability engineering
Scalability
Full job details
We are looking for a passionate and experienced Software Development Engineer to help us build and scale the next generation of AI runtime infrastructure for Alexa+. You will design and implement Tier-1 distributed systems that orchestrate expert agent executions, manage LLM inference pipelines, and ensure reliable, low-latency delivery across Alexa-enabled devices globally.
This is a high-impact role where your work directly shapes how millions of customers experience Alexa+ every day.
Key job responsibilities
- Design, build, and operate large-scale distributed services that power Alexa+ Expert executions
- Develop and optimize LLM orchestration pipelines — including model serving, prompt routing, and inference optimization
- Build multi-agent coordination systems that route requests to the right expert agents with minimal latency
- Own the operational excellence of Tier-1 services, ensuring 99.99%+ availability at scale
- Collaborate with applied scientists and ML engineers to integrate new AI capabilities into production
- Drive architectural decisions that balance performance, reliability, and cost efficiency
- Mentor teammates and raise the engineering bar through code reviews, design reviews, and technical leadership
About the team
The AI Runtime team is the backbone of Alexa+. We own the infrastructure that powers Expert executions — the multi-agent systems that bring Alexa+'s intelligent capabilities to life for hundreds of millions of customers worldwide. Our services orchestrate LLM inference, coordinate expert agents, and deliver responses at massive scale with low latency. When a customer interacts with Alexa+ and it feels magical, our platform made it happen. Basic Qualifications: - 3+ years of non-internship professional software development experience
- 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
- Bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field
- Experience programming with at least one software programming language Preferred Qualifications: - 3+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
- Bachelor's degree in computer science or equivalent
- System design with AI components and able to evaluate latency, cost, reliability, and accuracy.
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.
This is a high-impact role where your work directly shapes how millions of customers experience Alexa+ every day.
Key job responsibilities
- Design, build, and operate large-scale distributed services that power Alexa+ Expert executions
- Develop and optimize LLM orchestration pipelines — including model serving, prompt routing, and inference optimization
- Build multi-agent coordination systems that route requests to the right expert agents with minimal latency
- Own the operational excellence of Tier-1 services, ensuring 99.99%+ availability at scale
- Collaborate with applied scientists and ML engineers to integrate new AI capabilities into production
- Drive architectural decisions that balance performance, reliability, and cost efficiency
- Mentor teammates and raise the engineering bar through code reviews, design reviews, and technical leadership
About the team
The AI Runtime team is the backbone of Alexa+. We own the infrastructure that powers Expert executions — the multi-agent systems that bring Alexa+'s intelligent capabilities to life for hundreds of millions of customers worldwide. Our services orchestrate LLM inference, coordinate expert agents, and deliver responses at massive scale with low latency. When a customer interacts with Alexa+ and it feels magical, our platform made it happen. Basic Qualifications: - 3+ years of non-internship professional software development experience
- 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
- Bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field
- Experience programming with at least one software programming language Preferred Qualifications: - 3+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
- Bachelor's degree in computer science or equivalent
- System design with AI components and able to evaluate latency, cost, reliability, and accuracy.
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.