Senior Data Engineer

We are supporting a major international trading organisation in the continued development of its algorithmic trading platform.

The team designs advanced trading algorithms and integrates Artificial Intelligence and quantitative models to analyse market conditions, generate signals and support real-time trading decisions across multiple markets.

As the platform continues to scale, the team is facing a significant data industrialisation challenge. We are therefore looking for a Senior Data Engineer who will take ownership of the data architecture, pipelines and infrastructure supporting the algorithmic trading desk.

Your role

You will design, deploy and maintain the data infrastructure required by real-time and near-real-time trading algorithms.

You will work closely with three core groups:

• Algorithm and quantitative development teams;
• Software engineers responsible for building and deploying services;
• Research, trading and business stakeholders.

Your objective will be to ensure that market data, pricing information, forecasts, trading signals and model outputs are available, reliable, traceable and processed with the appropriate level of performance.

Key responsibilities

Data architecture

• Design, build and optimise robust, scalable and low-latency data architectures.
• Define the technical foundations required to support algorithmic trading and large-scale market data processing.
• Select and implement the most appropriate storage, streaming and distributed processing technologies.
• Contribute to the definition of architecture standards and engineering best practices across the trading platform.

Real-time data pipelines

• Develop, automate and monitor complex ETL and ELT pipelines.
• Ingest and process real-time and near-real-time data, including:
• market prices and order book data;
• transactions, positions and execution data;
• external market and reference data;
• pricing, forecasting and optimisation inputs;
• signals and outputs generated by quantitative and AI models.
• Ensure the scalability, resilience and performance of the pipelines used by the trading algorithms.
• Implement mechanisms for event replay, backfilling, recovery and reconciliation.

DataOps and MLOps industrialisation

• Work closely with AI Engineers and Quantitative Developers to structure the data workflows used by trading models.
• Support the transition from research prototypes to scalable and reliable production services.
• Build reusable data components and standardise development, testing and deployment practices.
• Contribute to feature pipelines, model data preparation and production monitoring.
• Reduce the time required to move new algorithms and models from research into production.

Data quality, tracking and governance

• Implement automated data quality controls and validation mechanisms.
• Ensure data availability, integrity, consistency and traceability.
• Introduce appropriate tracking, lineage and governance practices.
• Define monitoring, alerting and recovery mechanisms for critical data flows.
• Manage challenges such as late events, duplicated messages, missing data, schema evolution, replay and backfilling.
• Establish clear indicators covering data freshness, pipeline performance and platform availability.

Production and collaboration

• Deploy and operate data services in production environments.
• Contribute to CI/CD, Infrastructure as Code and automated deployment practices.
• Investigate production incidents and improve platform observability and reliability.
• Work closely with Software Engineers, Quantitative Developers, Researchers and Traders.
• Act as the technical interface between Technology, Quantitative Research and Trading teams.
• Promote a strong production and engineering culture across the desk.

Technical environment

The platform relies on a modern real-time data and cloud stack, including:

• Programming: Python, with at least one additional performance-oriented langua