Principal Data Engineer
The Principal Data Engineer at Black Duck Software, Inc. will lead the design and development of cross-product data services, creating a unified data plane for multiple product lines. This role involves defining canonical customer identifiers, shared dimensions, and consistent facts to be used across products, ensuring a cohesive data model. The position is based in Belfast, UK, and reports to the VP of Data Engineering.
Key responsibilities include building and operationalizing data ingestion patterns for batch, streaming, and event data, with repeatable onboarding for new sources. The engineer will own the operational playbook for data reliability, encompassing data contracts, quality checks, lineage, monitoring, and incident response. Additionally, the role involves implementing access methods that make data usable, such as curated datasets, secure query interfaces, and product-ready data APIs. Productizing customer-facing data products with versioning, documentation, and clear ownership is also a critical aspect of the job.
The ideal candidate will have significant experience building and operating production data platforms at scale, including on-call and operational ownership. Strong SQL and Python skills are essential, as well as hands-on experience running cloud systems on AWS and Google Cloud. Practical experience with both operational databases and analytics stores, including performance tuning, is required. A strong ability in data modeling, including schema evolution and conformed dimensions, is also necessary. The candidate should have a track record of delivering data products that other teams or customers depend on, with clear contracts and reliability expectations.
Preferred qualifications include experience building self-service data platforms used by multiple teams and experience in regulated or security-sensitive environments, including retention, auditing, and data access controls.
Black Duck Software, Inc. offers a collaborative work environment with flexible collaboration hours overlapping UK and US time zones. The company provides opportunities for professional growth and development, encouraging employees to raise the technical bar through architecture reviews, standards, and mentoring. Minimal travel is required for this role.