Senior Machine Learning Systems Engineer (Training Optimization)
As a Senior Machine Learning Systems Engineer at Canva, you will be a key member of the CORE team within the Generative AI supergroup. This team is dedicated to developing foundational technologies that drive the future of AI-assisted design, focusing on large-scale models and innovative research to enhance Canva's creative intelligence engine. The role is pivotal in shaping the training systems for the next generation of Canva's AI-powered products.
In this position, you will design, implement, and optimize large-scale machine learning systems for both training and inference. Your responsibilities include improving GPU utilization, reducing communication overhead, and enhancing memory efficiency. Collaboration with research and modeling teams is essential to align system designs with algorithmic requirements. You will also evaluate and apply best practices for distributed training using frameworks like Megatron-LM, NVIDIA NeMo, FSDP, and Triton, and engage in low-level optimization tasks, including developing custom CUDA or Triton kernels.
The ideal candidate possesses a strong background in large language models (LLMs), multimodal AI, or diffusion models. Proficiency in Python is required, with additional familiarity in system programming languages such as C++ or Rust being advantageous. Deep knowledge of PyTorch or JAX, along with experience in libraries like Megatron-LM, NeMo, or DeepSpeed, is essential. Familiarity with optimization techniques such as FSDP/ZeRO, gradient checkpointing, or low-precision data types is expected. Hands-on experience in writing custom GPU kernels using CUDA or Triton is also required. Strong communication and problem-solving skills, with full proficiency in English, are necessary for effective collaboration across teams.
Canva offers a dynamic and collaborative work environment where innovation and creativity are highly valued. Employees have the opportunity to work on impactful projects that reach millions of users worldwide. The company provides various benefits, including equity packages, inclusive parental leave policies, and flexible leave options to support personal and professional growth. Joining Canva means becoming part of a team that is redefining how the world experiences design.