(Medior) LLMOps
Il y a 23 heures
Auderghem Oudergem, Brussels-Capital, Belgique
AETHER
Temps plein
Gratuit avec email ou Google
Enregistrez cette offre et organisez votre recherche
Créez un compte gratuit pour enregistrer des offres d'emploi, créer des alertes et revenir à cette liste depuis votre tableau de bord.
Gratuit avec email ou Google
En continuant, vous acceptez nos Conditions d’utilisation & Politique de confidentialité.
Context
AETHER aims to be the key partner for intrapreneurial projects in Belgium, and we want to reinforce our partnerships with key actors in the services sector, who are facing interesting challenges regarding the reliable operation of AI services for employees and customers.
Projects are ongoing to ensure that employees and customers can use AI assistants and agents with reliable answers and dependable connections to business systems.
If you're keen to support intrapreneurs and see this opportunity as a way to grow your own intrapreneurial or entrepreneurial leadership, we are looking for a medior AI Engineer to strengthen our team of Project Partners.
Task Description
As an AETHER Project Partner embedded within the project team, your objective will be to take over AI services from the engineers who built them and keep them reliable, measured, secure and cost-efficient in production.
The services include retrieval-based assistants and AI agents that use tools and business systems on users' behalf. Initial services enter production before the assignment starts, with further releases planned during 2027.
You will work alongside an internal AI Platform Architect, two Senior AI Engineers responsible for development, business and content owners, and IT operations teams.
The platform combines managed Azure services with self-hosted open-source components on AKS. It uses Microsoft Foundry with the OpenAI model family for LLMs and embeddings, Azure AI Search, Azure API Management and Azure AI Content Safety. Agent orchestration uses Python and LangChain, alongside an ingestion pipeline. MLflow on Databricks supports evaluation and tracing with OpenTelemetry. The delivery team manages Infrastructure-as-Code for application-scoped Azure resources using Terraform and Azure DevOps pipelines.
Key Responsibilities
Might Include Activities will vary from service to service. Typically you will:
• Operate AI services in production: monitor availability, response times and errors, handle incidents and service requests, analyse root causes, then fix or escalate issues.
• Keep answer quality under control: run and maintain evaluation sets, or golden sets, with business content owners; track answer quality, non-answer rates and user feedback; detect and investigate quality regressions.
• Operate ingestion pipelines: manage incremental refreshes, failed documents, index health and the addition of new content sources or business domains through existing connectors.
• Operate agent integrations: monitor tool calls and connections to business systems, handle failures and timeouts, and review action logs and exceptions for agents acting on users' behalf.
• Manage change safely: validate model version upgrades and retirements, prompt and configuration changes, and dependency and security updates against evaluation sets before release.
• Follow operating costs: track token consumption and cost per service, set alerts for unexpected changes and propose optimisations.
• Maintain Terraform, Helm and Azure DevOps artefacts, and keep DEV, TST, ACC and PROD environments consistent.
• Apply data protection rules: manage trace retention, erasure and access restrictions as defined in each service's data protection impact assessment.
• Maintain runbooks and documentation, and regularly report service health to service owners and the AI governance team.
• Deliver small enhancements and fixes; structural changes and new features remain with the development team. Profile
• Medior AI Engineer able to take ownership of production operations for AI agents and RAG services.
• Intermediate proficiency in vector databases, Azure DevOps, Azure Kubernetes Service (AKS), Azure OpenAI Service, RAG (Retrieval-Augmented Generation), Terraform, Python, LangChain and LLMs (Large Language Models).
• Native or bilingual proficiency in Dutch and French, and full professional proficiency in English.
• Available for an assignment in Brussels, Belgium, from 4 January 2027 to 31 December 2027, lasting approximately 11 months.
• An entrepreneurial experience or ambition is a plus, including extra-professional or extra-academic activities. Our Vision Success in industry often relies on key individuals who demonstrate intrapreneurial leadership — those who continuously initiate, transform, and pursue excellence. The best way to develop this strength is through ad interim projects, i.e. working as a consultant with established intrapreneurs in international companies, to support their major expansion or improvement initiatives regarding governance, studies, or operations. Since 2014, AETHER has been managing these opportunities for outstanding consultants in key international groups, as these offer greater learning opportunities with multi-million impacts. Please share your CV and we will check if this demand is the current best opportunity for your career as a consultant. INSPIRE | LEAD GROWTH | CREATE IMPACT
Key Responsibilities
Might Include Activities will vary from service to service. Typically you will:
• Operate AI services in production: monitor availability, response times and errors, handle incidents and service requests, analyse root causes, then fix or escalate issues.
• Keep answer quality under control: run and maintain evaluation sets, or golden sets, with business content owners; track answer quality, non-answer rates and user feedback; detect and investigate quality regressions.
• Operate ingestion pipelines: manage incremental refreshes, failed documents, index health and the addition of new content sources or business domains through existing connectors.
• Operate agent integrations: monitor tool calls and connections to business systems, handle failures and timeouts, and review action logs and exceptions for agents acting on users' behalf.
• Manage change safely: validate model version upgrades and retirements, prompt and configuration changes, and dependency and security updates against evaluation sets before release.
• Follow operating costs: track token consumption and cost per service, set alerts for unexpected changes and propose optimisations.
• Maintain Terraform, Helm and Azure DevOps artefacts, and keep DEV, TST, ACC and PROD environments consistent.
• Apply data protection rules: manage trace retention, erasure and access restrictions as defined in each service's data protection impact assessment.
• Maintain runbooks and documentation, and regularly report service health to service owners and the AI governance team.
• Deliver small enhancements and fixes; structural changes and new features remain with the development team. Profile
• Medior AI Engineer able to take ownership of production operations for AI agents and RAG services.
• Intermediate proficiency in vector databases, Azure DevOps, Azure Kubernetes Service (AKS), Azure OpenAI Service, RAG (Retrieval-Augmented Generation), Terraform, Python, LangChain and LLMs (Large Language Models).
• Native or bilingual proficiency in Dutch and French, and full professional proficiency in English.
• Available for an assignment in Brussels, Belgium, from 4 January 2027 to 31 December 2027, lasting approximately 11 months.
• An entrepreneurial experience or ambition is a plus, including extra-professional or extra-academic activities. Our Vision Success in industry often relies on key individuals who demonstrate intrapreneurial leadership — those who continuously initiate, transform, and pursue excellence. The best way to develop this strength is through ad interim projects, i.e. working as a consultant with established intrapreneurs in international companies, to support their major expansion or improvement initiatives regarding governance, studies, or operations. Since 2014, AETHER has been managing these opportunities for outstanding consultants in key international groups, as these offer greater learning opportunities with multi-million impacts. Please share your CV and we will check if this demand is the current best opportunity for your career as a consultant. INSPIRE | LEAD GROWTH | CREATE IMPACT