Research, Pre-Training Science
The Research, Pre-Training Science role at Thinking Machines Lab is central to advancing the science of how large models learn from data. This position involves exploring new pre-training methods, architectures, and learning objectives to enhance model training efficiency, robustness, and alignment with human goals. Thinking Machines Lab is dedicated to empowering humanity by advancing collaborative general intelligence, aiming to make AI accessible and customizable for diverse needs.
Key responsibilities include researching and developing new pre-training methodologies, working on scaling, architecture, algorithms, or optimization of large-scale training runs, designing data curricula and sampling strategies to improve learning dynamics and model generalization, collaborating with infrastructure and data teams to conduct large-scale experiments efficiently and reproducibly, and publishing and presenting research to advance the AI community.
Required qualifications encompass the ability to design, run, and analyze experiments with demonstrated research judgment and empirical rigor, experience with distributed or high-performance computing environments, proficiency in Python and familiarity with at least one deep learning framework such as PyTorch, TensorFlow, or JAX, a bachelor's degree or equivalent experience in Computer Science, Machine Learning, Physics, Mathematics, or a related discipline with strong theoretical and empirical grounding, and clarity in communication with the ability to explain complex technical concepts in writing.
Preferred qualifications include a strong grasp of probability, statistics, and machine learning fundamentals, prior experience training or analyzing large-scale models or contributing to pre-training or foundation model research, a strong publication record or open-source contributions in representation learning, optimization, scaling laws, or other areas of pre-training, familiarity with curriculum learning, data selection, or active learning techniques, experience designing or maintaining evaluation frameworks for large models, contributions to open datasets, research publications, or data tooling, and a PhD in a relevant discipline or equivalent industry research experience.
Compensation for this role ranges from $350,000 to $475,000 annually, depending on background, skills, and experience. Benefits include generous health, dental, and vision coverage, unlimited paid time off, paid parental leave, and relocation support as needed. Visa sponsorship is available for qualified candidates.