Member of Technical Staff - Applied Inference
The Member of Technical Staff - Applied Inference role at xAI involves architecting and implementing scalable distributed infrastructure for model serving, including load balancing, auto-scaling, batch scheduling, and global KVcache systems. The position is based in Palo Alto, California, and is integral to xAI's mission of creating AI systems that accurately understand the universe and aid humanity in its pursuit of knowledge.
Key responsibilities include ensuring the reliability of inference services by targeting 100% uptime and a 0% error rate through proactive monitoring, fault-tolerant designs, and rigorous testing. The role also involves creating custom tools to trace, replay, and fix issues across the entire stack, from cluster orchestration to GPU kernels, as well as benchmarking and fine-tuning inference engines to deliver optimal performance under diverse production workloads. Additionally, the position requires developing robust CI/CD infrastructure to enable seamless endpoint deployment, image publishing, feature rollouts, and inference engine updates.
Candidates should have experience working on large-scale, high-concurrent production serving, GPU inference engines, and the testing, benchmarking, and reliability of inference services. Experience in designing and implementing CI/CD infrastructure is also required.
The annual salary range for this position is $180,000 to $440,000 USD. xAI offers a comprehensive total rewards package, including equity, medical, vision, and dental coverage, access to a 401(k) retirement plan, short and long-term disability insurance, life insurance, and various other discounts and perks.
xAI operates with a flat organizational structure, emphasizing hands-on contributions and direct involvement in the company's mission. Leadership is granted to those who show initiative and consistently deliver excellence. The company values strong work ethic, prioritization skills, and effective communication, fostering an environment where employees can thrive on curiosity and challenge themselves.