Back to search
We're thrilled to be supporting a well-known organisation in Liverpool that specialises in insurance services. As they continue to grow their technology function in line with meeting their increasing demands, they look to hire a Lead Data Engineer. Read on for more information.
Lead Data Engineer
Liverpool
£80,000
Lead Data Engineer duties:
- Lead the design development and optimisation of scalable data pipelines in a cloud-based environment.
- Implement and manage data warehouses on Azure Synapse, ensuring data integrity and security.
- Build, deploy, and manage ETL processes for both real-time and batch data processing using Azure tools, Databricks, PySpark, and SQL.
- Oversee data storage management across relational and non-relational databases to ensure efficient data retrieval.
- Design and implement robust data security protocols to protect sensitive information.
- Collaborate with DBA's, cloud architects, and end users to ensure smooth integration between data platforms and business requirements.
- Manage the ingestion, transformation, and processing of large datasets using big data technologies.
- Optimise the performance of data pipelines and infrastructure with cloud services like Azure and AWS S3.
- Lead a team of data engineers, providing technical guidance and promoting a culture of continuous learning and improvement.
As Lead Data Engineer you will:
- Substantial experience within Data Engineering
- Expertise in Azure Data Warehouse (DWH) and (AWS Databricks).
- Strong programming skills in Python/PySpark or other relevant languages for data manipulation and ETL Processes.
- Proficient in SQL with experience in both relational databases and non relational databases.
- Experience working with AWS S3 and other AWS services for big data solutions.
- Excellent communication skills, with the ability to document processes and present technical concepts effectively.
- Relevant Azure or AWS cloud certifications are a plus.
In return you will receive:
- Competitive market pay.
- Flexible/Hybrid working.
- Amazing career progression.
- Sick pay, maternity/paternity paid leave.
- A wide range of discounted vouchers with selected retailers.
Robert Walters Operations Limited is an employment business and employment agency and welcomes applications from all candidates
