Staff Data Infrastructure Engineer
Peregrine Technologies is seeking a Staff Data Infrastructure Engineer to join our growing team. In this role, you will have deep ownership over the data layer that underpins everything Peregrine does. You will architect and build the systems that ingest, store, and serve massive volumes of real-time operational data, enabling our customers to make critical decisions with speed and confidence.
Key responsibilities include designing and operating a high-throughput, real-time data integration platform across diverse customer environments, architecting a scalable open table format layer for reliable data storage at petabyte scale, and building and optimizing distributed data processing pipelines with Apache Spark and adjacent streaming technologies. You will also drive performance, reliability, and cost efficiency across the full data infrastructure stack, collaborate with platform and product engineering teams to define data contracts, schemas, and integration patterns, and establish best practices, tooling, and patterns that raise the quality bar for data infrastructure across the organization.
The ideal candidate will have 8+ years of experience architecting and operating large-scale data infrastructure systems in production environments. Deep expertise with open table formats, particularly Apache Iceberg, including schema evolution, partitioning strategies, compaction, and time travel, is essential. Extensive hands-on experience with Apache Spark for batch and streaming data processing at scale, a strong background in real-time data integration and stream processing, and solid experience with data pipeline orchestration using Airflow or similar tools are also required. Strong software engineering fundamentals in Python and/or Scala, extensive experience with AWS or comparable cloud platforms, and experience with Kubernetes and containerized deployment of data workloads are necessary. A degree in Computer Science, Engineering, or a related field, or equivalent practical experience, is expected.
The position is located in San Francisco, New York, or Washington, D.C., with an expectation to work in the office. The salary range is $200,000 - $275,000 annually, plus benefits, equity (if applicable), and bonus (if applicable).