Staff Data Engineer – Data Platforms & AI

Staff / Principal Data Engineer – Data Platforms & AI Location: Europe – Remote Experience: 7+ years Level: Staff / Principal / Technical Lead About the Role my clientfor a highly experienced Data Engineer to take a leading technical role in the evolution of a modern data platform. This is an opportunity for a hands-on technical leader to shape architecture, improve how data is delivered across the business, and embed AI-driven capabilities into data engineering practices. You will work across data infrastructure, real-time processing, analytics and AI enablement, balancing long-term technical strategy with practical engineering delivery. The ideal candidate combines deep technical expertise with the ability to influence engineering teams, engage senior stakeholders and turn business challenges into scalable technical solutions. Key Responsibilities Define and drive the technical architecture and long-term roadmap for a scalable data platform spanning ingestion, batch processing, streaming, warehousing and data services. Lead the adoption of AI-enabled engineering practices, including LLM-based tools and agents to support pipeline development, testing, documentation, data validation and operational support. Build the foundations for enterprise-wide AI adoption by making data accessible, well-documented, governed and suitable for use by AI applications, LLMs and machine learning models. Develop and evolve real-time data capabilities to support time-sensitive business decisions, customer-facing services, operational workflows and marketing use cases. Establish robust data modelling, quality assurance, observability and governance practices to improve the reliability, consistency and usability of data. Evaluate existing systems and technical approaches, identifying opportunities to modernise infrastructure, simplify architecture and improve performance, scalability and cost efficiency. Drive platform transformation initiatives, including architectural changes, technology migrations and improvements to engineering processes. Promote self-service analytics by enabling business teams to access trusted, clearly defined data without unnecessary engineering dependencies. Work closely with engineering, data, analytics, product, infrastructure and business stakeholders to align platform priorities with organisational objectives. Provide technical leadership through architecture reviews, design discussions, mentoring and the development of engineering standards. Remain hands-on with code, prototyping, troubleshooting and resolving complex data processing, performance and integration challenges. Translate business requirements into clear technical proposals, explaining trade-offs, delivery plans, risks and costs to both technical and non-technical audiences. Essential Skills and Experience 7+ years of professional experience in data engineering, data platform engineering or a closely related discipline. Demonstrable experience operating at Staff, Principal, Lead or equivalent senior technical level, with responsibility for architecture, technical direction or cross-team engineering initiatives. A proven track record of delivering significant data platform improvements, modernisation programmes or production migrations. Strong hands-on programming experience in Python, Scala or Java, alongside advanced SQL skills. Extensive experience with data streaming and event-driven architectures, ideally using Apache Kafka or equivalent technologies. Experience with real-time analytical databases or OLAP platforms, such as ClickHouse, Apache Druid or StarRocks, or comparable solutions. Strong knowledge of data warehousing, data modelling and transformation frameworks, including dbt or equivalent technologies, and platforms such as Amazon Redshift or similar. Experience designing and operating data platforms on AWS or another major cloud provider. Practical experience applying AI or LLM-based capabilities to data engineering workflows, or building and serving data for AI/ML applications. Strong understanding of data quality engineering, automated testing, monitoring, observability, governance and data ownership. Experience with CI/CD, containers and infrastructure-as-code practices for data platforms and workloads. The ability to influence technical decisions, mentor engineers and lead complex initiatives without relying solely on formal management authority. Excellent communication and stakeholder management skills, with the ability to connect engineering decisions to business outcomes. A pragmatic, product-oriented approach to problem-solving, with a focus on delivering measurable value rather than adopting technology for its own sake. Professional proficiency in written and spoken English. Desirable Experience Experience in high-volume consumer technology, online entertainment, financial services or other complex, regulated environments. Building real-time data integrations with CRM, customer engagement or marketing automation platforms. Experience supporting personalisation, customer segmentation, fraud detection, risk analytics or similar data-driven use cases. Familiarity with MLOps, LLMOps, semantic layers, vector databases or AI-ready data architectures. Experience with business intelligence and self-service analytics platforms such as Looker. Previous experience mentoring engineers or coordinating cross-functional technical teams. What Makes This Opportunity Interesting? A significant opportunity to influence data architecture and engineering strategy. The chance to combine hands-on software engineering with platform-level technical leadership. Scope to introduce practical AI capabilities into engineering workflows and business data services. The opportunity to improve how teams access, understand and use data across an organisation. A collaborative environment where technical decisions have a direct impact on business capabilities and product development. If you are an experienced data engineering professional who enjoys solving complex platform challenges, shaping technical direction and building the next generation of AI-enabled data capabilities, we would be keen to hear from you.