Senior Data Engineer

Dates: 2026-08-25 — 2026-12-31

Within the DEP (Data Exchange Platform) programme, SMALS is building an on-premise data platform for the Belgian Ministry of Defence. The platform must meet strict security, governance, and data sovereignty requirements. You will join the Sierra Team alongside a Data Architect, a Solution Architect, and a BI developer. The assignment is full-time with a hybrid working arrangement in Brussels.

You will handle heterogeneous data ingestion, incremental and streaming pipelines, and transformations across the Bronze, Silver, and Gold layers. You will work on data historisation practices, Delta Lake exports, ACID transactions, and time travel. You will also manage orchestration, monitoring, error handling, and automated alerts. Pipeline reliability, idempotence, and traceability are integral parts of the role.

The position also covers governance, lineage, and data exposure for BI use cases. You will push technical and business metadata, maintain end-to-end lineage, and contribute to DUA, SLA, and related governance documentation. You will develop endpoints to expose the Gold layer and collaborate with reporting teams. The environment includes Python, SQL, dbt, DuckDB, Delta Lake, Dagster, and other modern open-source technologies.

Responsibilities

• Must have:
  • Mastery of Python
  • Mastery of SQL
  • Confirmed dbt-core experience
  • Good DuckDB knowledge
  • MSSQL PostgreSQL knowledge
  • Dagster Airflow orchestration
• Nice to have:
  • Spark ecosystem knowledge
  • PBIRS or Superset
  • Data security awareness
  • Lakehouse DWH concepts

Additional Responsibilities

• Ensure transactional consistency
• Implement orchestration assets
• Maintain orchestration assets
• Push technical metadata
• Push business metadata
• Develop Gold endpoints
• Collaborate reporting teams
• Rigour and autonomy in managing complex multi-domain pipelines
• Ability to document technical decisions
• Ability to explain technical topics to non-technical stakeholders
• Team spirit
• Curiosity and solution-oriented mindset
• Belgian nationality required
• Dutch: active knowledge
• English: active knowledge
• Awareness of data security in restricted-access environments

Technical skills

• Mastery of Python
• Mastery of SQL
• Confirmed experience with dbt-core: models, snapshots, macros, tests, and profiles
• Good knowledge of DuckDB as an embedded analytical query engine
• Experience with Delta Lake: ACID transactions, time travel, OPTIMIZE/VACUUM
• Knowledge of MSSQL and PostgreSQL, including advanced SQL and CDC
• Experience with a data orchestrator: Dagster, Airflow, or equivalent
• Comfortable with Linux/OpenShift and PowerShell in a Windows development environment
• Familiarity with governance tools: DataHub, OpenMetadata, or equivalent
• Experience in BI development
• Confirmed experience in ETL/ELT data acquisition
• Confirmed Data Governance experience
• Confirmed experience with Python, Pandas, and Apache Spark from a Data Engineer perspective
• Kafka/Debezium experience for real-time data capture
• Knowledge of Spark: Spark SQL, Thrift Server, and Beeline
• Experience with Power BI Report Server (PBIRS) or Apache Superset
• Awareness of data security in restricted-access environments
• Knowledge of Lakehouse concepts, Medallion architecture, DWH, and Datalake