Data Engineer - MS Fabric Expertise - Outside IR35

Data Engineer - MS Fabric Expertise

Brand new vacancy this morning for 2 Data Engineers with Microsoft Fabric expertise to join a leading international events, intelligence, and scholarly research group.

Length: 6 monthsLocation:London 2 days per weekIR35 Status: Outside

The Data Engineer will be joining the Technology Data Management project, building a scalable enterprise data platform that processes Technology Meta Data from various sources through a medallion architecture. This role involves the ongoing development and maintenance of the platform, supported by the project team and existing consultants (during a hand over period). The purpose of the data platform to enhance Business Intelligence and Analytics within the technology department for Informa - allowing for better strategic and operational decision making.

Key Areas of Responsibility/Accountability

• Platform Development: Lead the development of the enterprise data platform, ensuring it meets the needs of the technology department.
• Data Processing: Designing and implementation of ELT pipelines to enable the effective handling of Technology Data from various sources using medallion architecture.
• Maintenance: Oversee ongoing maintenance and improvements of the platform to ensure its reliability and performance.
• Collaboration: Work closely with project team members and consultants to achieve project goals.
• Be hands-on, and guide teams to improve technology across product portfolios, infrastructure, and processes.

Key Outputs and Outcomes

• Enhanced Data Platform: A robust and scalable data platform that processes Technology Data efficiently.
• Build of Data Assets: Continue to develop / build of data assets from multiple source systems ensuring that these are of quality to be used by the reporting teams
• Continued Implementation of Data Quality tooling: Leverage the existing frameworks/tooling to continue Data Quality measurement across the platform.

QualificationsRequired Skills

• Microsoft Fabric expertise: Demonstrated experience in developing and maintaining complex data pipelines with Microsoft Fabric
• Data Engineering foundations: Solid experience in data warehousing, ETL/ELT processes, and dimensional data modelling
• Azure: Working knowledge of Azure services, including Data Factory, Synapse Analytics, and Azure SQL Database
• Coding fluency: Proficiency in Pyspark, SQL and Python for data scripting, transformation, and automation

Desirable Skills

• Github Repository Management: Specifically the development and maintenance of Github Actions for CI/CD
• IaC Development: Terraform
• Nutter Testing Framework
• Soda Data Quality Testing
• Certifications related to Microsoft Azure Data Engineering and Fabric
• Proficient in T-SQL programming

Additional informationDemonstration of Success

• Project Milestones: Successful completion of project milestones within the specified timeline.
• Performance Metrics: Achievement of key performance metrics, such as data processing speed, accuracy, and reliability.
• Stakeholder Feedback: Positive feedback from stakeholders, indicating satisfaction with the platform's performance and impact.

Carbon60, Lorien & SRG - The Impellam Group STEM Portfolio are acting as an Employment Business in relation to this vacancy.