Back to search
I am working with a client who is looking for a Data Engineer to take ownership of all things data for their feasibility product. This role is essential in building and maintaining data stores, automation, and stream consumers, enabling Data Scientists and Analysts to develop effective algorithms, processes, and reports. As a bridge between software engineering and data science, you'll work within the tech team to develop scalable solutions that meet business needs.
THE ROLE AND RESPONSIBILITIES- Develop applications to consume and transform production data streams (Kafka) for analytical and ML use.
- Build and optimize ETL/ELT workflows to support feasibility models.
- Automate model training, evaluation, and deployment pipelines.
- Design and maintain cloud-based data stores using AWS Redshift and other cloud tools.
- Implement monitoring solutions to ensure data integrity and model performance.
- Develop FastAPI-based RESTful APIs and microservices to expose feasibility data to other teams.
- Integrate APIs with CI/CD pipelines to enable automated testing and deployment.
- Work closely with cross-functional teams to gather requirements and deliver data-driven solutions.
- Follow best practices for clean, modular, and testable code, conducting code reviews to ensure quality.
- Identify opportunities to improve the performance, maintainability, and scalability of existing systems.
*** Please only apply if you have experience developing FastAPIs ***
- Strong experience with Python or similar languages (e.g., R).
- Experience developing FastAPIs
- Hands-on experience with SQL databases (PostgreSQL preferred).
- Familiarity with AWS services (S3, SageMaker, RDS, EC2).
- Experience with CI/CD tools (AWS CodePipeline, GitHub Actions).
- Proficiency with infrastructure as code tools (Terraform, CloudFormation).
- Experience with distributed data processing tools (Spark, Dask, or similar).
- Proficiency in Git version control and collaborative coding practices.
- Familiarity with Kafka is an advantage.
If all of the above aligns with your experience, please apply using the link below.
