Dbt Analytics Data engineer

Il y a 6 heures

Arrondissement of BrusselsCapital, Brussels, Belgique ThoughtLabs Belgium Temps plein
ulliDesign and development of transformation pipelines for scalable and reliable data workflows /liliOptimisation of execution times of dbt projects /liliDesign, implement and document data quality checks and control tests using the functionalities provided within the dbt landscape /liliSet up automatic regression tests and validation procedures of pipeline outputs /liliIntroduce and share best practices within the team about managing and evolving dbt projects /liliInterpret business analysis documents and translate them into operational transformation pipelines /li /ul h3Specific expertise and technologies /h3 ulliProven hands-on expertise with dbt (preferably core) for designing, maintaining, testing, documenting and extending modular analytics transformation projects in production environments. /liliStrong experience in analytics engineering and modern data modelling, including layered architectures (for example staging, intermediate and marts), reusable macros, data quality controls and CI/CD practices for dbt projects. /liliStrong proficiency in SQL for complex transformations, performance optimisation and incremental processing. /liliPractical experience in developing PySpark transformation scripts for large-scale data processing. Familiarity with optimisation of Spark jobs is a plus. /liliGood understanding of Apache Spark execution concepts, partitioning strategies, file layout optimisation and performance tuning in distributed data environments. /liliProven experience working with Apache Iceberg tables, including partition evolution, schema evolution, snapshot management and time travel. Ability to work with the internals of the technology, such compaction strategies, transaction behaviour and implications for performance is considered a plus. /liliStrong knowledge of data validation, reconciliation and testing approaches to ensure functional equivalence, especially when migrating from legacy proprietary software to dbt and PySpark-based implementations. /liliFamiliarity with version control and collaborative development practices, especially Git-based workflows, code review, release management and automated deployment pipelines. /liliExperience with job orchestration and scheduling tools (e.g. Apache Spark, Dagster) used to run dbt and Spark workloads in enterprise environments. /liliAbility to produce clear technical documentation covering transformation logic, migration decisions, table behaviour, operational procedures and support handover. /liliCapacity to collaborate with data engineers, analysts, business stakeholders and platform teams to translate legacy logic into scalable, well-governed modern data transformations. /liliSolid knowledge of SAS-based ETL and capacity to understand legacy codebases, along with demonstrated ability to migrate legacy processing workflows from SAS to dbt, would be considered a strong plus. /li /ul h3Education /h3 ulliEQF Level 6 or higher in a relevant field such as Computer Science, Information Technology, or Systems Engineering. /li /ul