Senior Data Engineer Consultant

Mission Overview:

Keystone Solutions is seeking a Senior Data Engineer for a consultancy mission at a client site. You will be part of a dynamic team working on the Data Exchange Platform (DEP) project, which involves building an on-premise data platform for a governmental organization. This role emphasizes the importance of security, governance, and data sovereignty. Main Responsibilities:

Design and deploy data ingestion pipelines from heterogeneous sources: files, APIs, relational databases. Implement pipelines for incremental and streaming ingestion. Ensure reliability, idempotence, and traceability of each pipeline. Develop transformation models on Bronze, Silver, and Gold layers (Medallion architecture). Implement best practices for data historization (e.g., SCD2, upsert, append only). Develop exports to Delta Lake. Manage and maintain Delta Lake tables. Guarantee transactional consistency of data in a multi-domain environment. Implement and maintain orchestration assets. Monitor runs, manage errors, and automate alerts. Push technical and business metadata to the data platform. Maintain end-to-end lineage. Contribute to the drafting of DUA, SLA, and associated governance documents. Develop endpoints to expose the Gold layer to BI dashboards. Collaborate with reporting teams for BI dashboard feeding. Required Technical Skills:

Proficiency in Python. Proficiency in SQL. Proven experience with dbt-core (models, snapshots, macros, tests, profiles). Good knowledge of DuckDB as an embedded analytical query engine. Experience with Delta Lake (ACID transactions, time travel, OPTIMIZE/VACUUM). Knowledge of relational databases: MSSQL, PostgreSQL (advanced SQL, CDC). Experience with a data orchestrator: Dagster, Airflow, or equivalent. Comfortable with Linux/OpenShift and using PowerShell (Windows dev environment). Familiarity with governance tools: DataHub, OpenMetadata, or equivalent. Experience in BI development. Preferred Skills:

Experience with Kafka/Debezium for real-time data capture. Knowledge of Spark (Spark SQL, Thrift Server, Beeline). Experience with Power BI Report Server (PBIRS) or Apache Superset. Awareness of data security in restricted access environments. Knowledge of Lakehouse concepts (Medallion architecture), DWH, and Datalake. Soft Skills:

Rigorous and autonomous in managing complex multi-domain pipelines. Ability to document technical decisions and simplify for non-technical stakeholders. Team spirit. Curious and solution-oriented mindset. What We Offer:

A stable mission that is long-term. A high-impact project in a secure and technically stimulating environment. A modern and open-source stack: DLT, dbt, DuckDB, Delta Lake, Dagster. An experienced and supportive team focused on quality and best practices. Opportunities for skill enhancement in governance, orchestration, and lakehouse. If you are ready to tackle technical and strategic challenges in a dynamic consultancy environment, apply today at Keystone Solutions Career Portal. Duration:

As soon as possible - 31/12/2026 5 months (full time) Skills required:

Dagster

- Level: Junior - Most recent: Any time Data acquisition (ETL, ELT, ...)

- Level: Confirmed - Most recent: Any time Data architecture

- Level: Junior - Most recent: Any time data engineering

- Level: Confirmed - Most recent: Any time Data Governance

- Level: Confirmed - Most recent: Any time DataHub

- Level: Junior - Most recent: Any time datalake

- Level: Junior - Most recent: Any time dbt

- Level: Junior - Most recent: Any time Kafka

- Level: Junior - Most recent: Any time Metadata management

- Level: Confirmed - Most recent: Any time POWER BI

- Level: Junior - Most recent: Any time Python (from a Data Engineer perspective), Pandas & Apache Spark

- Level: Confirmed - Most recent: Any time SPARK

- Level: Junior - Most recent: Any time SQL

- Level: Confirmed - Most recent: Any time Language requirements:

Dutch Level Active knowledge English Level Active knowledge French Level Native