Limited Time - 20% Off Online Job Postings - Click Here to Post a Job

Lead Data Engineer

Competition & Markets Authority

Belfast, Cardiff, Edinburgh, London or Manchester

Application Deadline
£55,133 - £62,417

Salary: £55,133 - £62,417 pa

Hours: Full time / part time / flexible working

Contract: Permanent

Location: Belfast, Cardiff, Edinburgh, London or Manchester

This is an exciting opportunity to build your data engineering expertise and join the Data, Technology and Analytics (DaTA) team at the Competition and Markets Authority (CMA) to play a key role at the forefront of technology, investigating and regulating digital firms and markets to make a positive impact on society.

The CMA is a world-leading competition authority at the forefront of tackling pressing public policy questions, including the regulation of the biggest digital markets. Our mission is to make markets work well in the interests of consumers, businesses, and the economy.

The Data, Technology and Analytics (DaTA) unit is an established multidisciplinary team that brings together data engineers, data scientists, technologists, behavioural scientists and digital forensics and eDiscovery experts.

The data science and engineering team

As a Lead Data Engineer in the DaTA unit, you will join an established team of around 15 data engineers and data scientists who are critical in designing and managing the CMA’s analytical infrastructure and supporting the delivery of our most technically challenging projects.

Some of our recent work in Data Engineering includes:

  • Designing pipelines featuring parallelization, workflow orchestration, data version control and MLOps to process large, complex data sources
  • Securely translating large corpora of documents submitted by third parties using a containerised architecture on our cloud infrastructure
  • Building end-to-end services, defined as Infrastructure as Code, to run jobs, including web scraping at scale and regular ML prediction on large datasets

The role

Your responsibilities will span the entire tech stack, with an emphasis on backend and infrastructure development, data orchestration and handling. This will include:

  • Providing leadership and engineering expertise on projects
  • Creating and implementing new services, pipelines and data driven tools within our cloud platform
  • Driving the development and maintenance of our cloud-hosted analytics platform (JupyterHub, AWS EC2s) and existing services through Infrastructure as Code (AWS CDK)
  • Promoting and producing high quality code in Python (and other languages, as needed)
  • Building the CMA’s coding and data management standards and helping others to improve their coding and data handling capabilities
  • Identifying opportunities to support CMA cases with data engineering expertise

You will be expected to make architectural decisions and lead projects, along with managing internal stakeholders to make sure we are building the most impactful tools possible for the CMA.

You will be given time and support for study and development and will be expected to spend most of your time working on hands-on technical work.

What we’re looking for

You will be an experienced Data Engineer, Developer, or DevOps specialist, with experience in leading and delivering data projects and a good understanding of service architecture and design best practices.

Significant experience of writing, documenting, testing and managing code, including Python, to work with large and messy datasets is essential.

A proven interest in learning about and understanding the latest methods and technologies within machine learning and data engineering is equally important.

You will also need experience of overseeing and writing both architectural, design and technical documentation, and presentations and reports for non-technical audiences.

What we can offer you

The CMA is a hugely diverse, rewarding and stimulating place to work – an opportunity to work for the public good and to stretch and develop yourself in doing so. We offer generous benefits, including an excellent pension scheme with an average employer contribution of 27%, at least 25 days leave (increasing to 30 over five years), and generous maternity/paternity leave provision. You can also take advantage of interest-free season ticket loans, a cycle to work scheme, our employee assistance programme, and our extensive flexible working options.

To find out more about this opportunity and about working for the CMA, please click on the APPLY button.

Closing date for applications: 11:55 pm on 9th October 2023.

Like many other organisations, we operate a hybrid working model with a mix of office and home working. We believe this supports us to deliver our best work while allowing the flexibility to maintain balanced and healthy lives. We work at least 40% of our time from our office locations across the UK.

The CMA are an accredited Disability Confident Employer, and we are an equal opportunities employer, welcoming suitably qualified applicants from all backgrounds. We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles.

Not what you're looking for?

Use our search to find your ideal career opportunity

Search term is required
No jobs were found
Total of [[ totalJobs ]] jobs were found

[[ ]]

[[ jobs.location ]]
[[ jobs.posted_time_friendly ]]

Featured Jobs

Competition & Markets Authority

Belfast, Cardiff, Edinburgh, London or Manchester

October 09, 2023


London or Manchester, flexible home-working arrangements

October 11, 2023

Financial Ombudsman Service

London, UK

October 19, 2023

The Institute of Science and Technology Austria (ISTA)

Klosterneuburg, Austria

October 25, 2023

Lehigh University

Bethlehem, PA, USA

October 19, 2023

Competition and Markets Authority

Belfast, Cardiff, Edinburgh, London or Manchester

October 02, 2023

Lehigh University

Bethlehem, PA, USA

October 19, 2023

The Alan Turing Institute

Cheltenham, London, UK

October 01, 2023

The Institute of Statistical Science of Academia Sinica

Taipei City, Taiwan

December 01, 2023

Competition & Markets Authority

Belfast, Cardiff, Edinburgh, London or Manchester

October 09, 2023

Metropolitan Police

Westminster, London, UK

October 11, 2023

Our Partners

Logo for Logo University Of Manchester
Logo for Yougov
Logo for Ministry
Logo for Ons Logo
Logo for Un
Logo for Office Depot
Logo for Mit Logo
Logo for Fca

Like what you see?

Post a job