Data Engineer

Teqniq is searching for a Data Engineerto work in the public sector.

35 hours per week.

Permanent Role (Hybrid)

09:00-17:00

The post holder will join our Data Hub team as a Data Engineer. You will help to create business and user value from data by undertaking significant engineering projects which improve and combine data from back office and customer-facing systems. You will make sure our data, and data shared by others, is of the highest quality, highly available, usable, and shareable by rigorous standard setting and quality control. You will work with data professionals to build products and services which deliver continuous insight and value to our staff and residents

Essential experience:

• Experience in using Microsoft Azure for data storage, extraction, transformation, troubleshooting, and maintaining the data platform.
• Proficiency in both SQL and NoSQL databases is essential, along with hands-on experience with Azure resources such as ADLS, Synapse, Databricks, Stream Analytics, SQL DW, COSMOS DB, Analysis Services, and Azure Functions.

Main duties and responsibilities

To undertake all responsibilities listed below:

1. Provide knowledge and expertise in Microsoft Azure for data engineering.

2. Build pipelines and APIs to connect operational systems, data for analytics and business intelligence (BI) systems from hybrid environments i.e., locally hosted, supplier hosted, and Cloud based sources.

3. Work across the organisation to extract, clean, combine and store data in a way that allows it to be exploited and as an asset to help service delivery and decision making. Anonymise data wherever necessary, working within the principles of UK GDPR (General Data Protection Regulation) and with suitable data protection and sharing agreements in place.

4. Contribute to the achievement of the Council’s vision, values, and strategic objectives by providing technical knowledge in Microsoft Azure within the service and across the Council, specifically regarding data storage, extraction and transformation.

5. Using data from various service systems, databases, data marts and external sources to prepare clear, accurate data models and automated performance reporting services, briefing papers, and reports for Directors and senior managers.

6. Make major contributions to our data strategy, writing good standards and practice for data engineering.

7. Build and link data sets so that data professionals/analysts can easily derive insights, including accessing sensitive data in back-office systems, extracting, and loading into appropriate technology.

8. Lead a series of data engineering projects in partnership, managing both more junior members and external partners and suppliers to reach our goals. Set direction, lead planning, manage risks and deliver on time and to budget.

9. Manage relationships with key stakeholders at project level including elected Members and Board level Directors.

10. Any other duties commensurate with the grade of the post and as requested by the line manager.