Data Engineer
£60,000 - £90,000
Permanent | London
Consultant: Alfie Slade | Job ID 689

This is a unique opportunity  combining elements of data science, software engineering and Automation for an industry leading organisation.

Day to day you'll be building/maintaining a Python-based platform for data analysis. This includes creating scaleable and resilient data pipelines, developing new and better analytical capabilities and leveraging a range of technologies and methods to enable our dynamic business to make better decisions.

This is an ideal role for an enthusiastic data engineer with a strong Python development skills, and who wants to work in a greenfield project. The team is expanding its responsibilities across the organisation and scaling the existing data platform using cloud technologies. The data engineering team will innovate through exploration, design and bench-marking. The goal is to make technology recommendations and implement a scaleable data analysis platform in the cloud. The role is also tasked with the development and implementation of a range of infrastructure and automation projects.   

 The key responsibilities of the Data Analytics team are:

  1. To build and maintain a scalable data platform.
  2. Using the platform to build scalable data pipelines and ETL workflows.
  3. Support other areas of the business with analysis, visualisations, automation, and performance bottlenecks.

Essential Skills and Experience

  • Strong Python development skills including experience working with Numpy and Pandas.
  • Understanding and experience working with data driven architectures.
  • Proficient using Linux and Bash scripting.
  • Experience using/building cloud-based solutions using AWS.
  • Docker and Docker Compose.
  • Experience working with efficient data storage formats such as Parquet, Avro or ORC.
  • Management of virtual environments using anaconda and pip.
  • Interest and willingness to teach and learn from others.

Beneficial Skills and Experience

  • Experience working with C/C++.
  • Hands-on experience with Nifi, Airflow and Kubernetes.
  • Familiarity with visualisation tools such as Plotly or Bokeh.
  • Understanding of data platforms and resilient architectures.

 

Apply for this job * Mandatory fields