Data Engineer

h3Job Description /h3 pAt CluePoints, we’re redefining how clinical trials are run. As the premier provider of Risk-Based Quality Management (RBQM) and Data Quality Oversight software, we harness advanced statistics, artificial intelligence, and machine learning to ensure the quality, accuracy, and integrity of clinical trial data, helping life sciences organisations bring safer, more effective treatments to patients faster. We’re an ambitious, fast-growing technology scale-up with a dynamic and diverse international team. Collaboration, flexibility, and continuous learning are part of our DNA, and we are guided by our values of Care, Passion, and Smart Disruption. /p h3The Role /h3 pThe Data Engineer will build the pipelines, connectors and harmonized data snapshots at the heart of CluePoints’ new data layer. The layer brings together data from different studies and applications so that CluePoints’ consumer applications can use it, with auditability and regulatory compliance designed in from the start. You will work with the platform architect and the AI Engineer to turn the architecture into reliable, production-grade data flows. Success looks like trustworthy, traceable, harmonized snapshots, delivered consistently across studies and data sources. /p h3Job Requirements /h3 ul liStrong practical experience designing and running production-grade ingestion, transformation and orchestration pipelines. /li liAdvanced SQL and strong Python development skills. /li liHands-on experience with Databricks and/or Snowflake and modern lakehouse or warehouse architectures. /li liExperience integrating data from databases, APIs and SaaS applications. /li liStrong data modelling skills, including layered data architectures. /li liExperience with Git-based development, code review, automated testing and CI/CD. /li liExperience implementing data-quality controls, observability, monitoring and alerting. /li liUnderstanding of data governance, metadata, lineage and traceability, and why they matter in regulated settings. /li liAbility to reconcile data with differing structures and variables and to validate it against source systems. /li liHigh autonomy and ownership from requirements through deployment, monitoring and maintenance. /li /ul h3Nice To Have /h3 ul liExperience in clinical trials, life sciences or another regulated environment, or familiarity with clinical data standards (for example CDISC) and data-capture system exports. /li liExperience designing metadata-driven pipelines or harmonizing data across heterogeneous sources. /li liExperience with Delta Lake, Unity Catalog and Databricks Asset Bundles, and with Microsoft Azure. /li liExperience building integrations into customer-managed data environments. /li liPractical use of generative AI in data engineering workflows, and awareness of knowledge graphs and RAG. /li /ul h3Job Responsibilities /h3 h3Data-Layer Pipelines Harmonization /h3 ul liBuild and maintain production pipelines that bring data from different studies and applications into the data layer. /li liImplement harmonization logic that reconciles studies with differing variables, including approaches driven by metadata from data-capture solutions. /li liProduce harmonized data snapshots for CluePoints’ consumer applications, with clear refresh and versioning approaches. /li liValidate new or changed data flows against source systems and business expectations. /li /ul h3Connectors Integration /h3 ul liShape and build connector approaches for common data-capture systems. /li liSupport customer-specific Databricks and Snowflake environments with secure, repeatable integration patterns. /li liMonitor and manage schema changes, missing data, duplication and inconsistencies between sources. /li liEvaluate new data sources and determine the most appropriate extraction and integration approach. /li /ul h3Auditability, Quality Compliance /h3 ul liEnsure snapshots support auditability, traceability and regulatory compliance, including lineage, versioning and reproducibility. /li liImplement automated data-quality controls covering completeness, accuracy, consistency, uniqueness and freshness. /li liSet up monitoring and alerting for pipeline failures, unexpected schema changes and data-quality issues. /li liApply testing, validation and documentation standards appropriate to a regulated environment. /li /ul h3Platform Engineering Collaboration /h3 ul liContribute to technology evaluation and architecture decisions with the platform architect. /li liEstablish version control, code review, automated testing, CI/CD and orchestration practices for the data layer. /li liMonitor and optimize performance, resource usage and cost. /li liWork with the AI Engineer to make well-described, harmonized data available to retrieval and AI components. /li liDocument critical solutions and share knowledge to avoid dependency on a single individual. /li /ul h3Job Benefits /h3 ul liThe chance to build a core data capab