Das ist der Job
Optimize and refactor existing ETL pipelines to improve efficiency, scalability, and cost-effectiveness.
Darum lohnt es sich
About the job Intermediate Data Engineer
Job Purpose
We are seeking a talented and experienced Data Engineer to join our MLOps team which drives critical business applications.
As a key member of our team, you will play a crucial role in designing, building, testing, deploying, and monitoring end-to-end data pipelines for both batch and streaming use cases.
You will work closely with data scientists, actuaries, software engineers, and other data engineers to contribute to architecting our Client's modern Machine Learning ecosystem. Re-architecting data pipelines for a modern data stack leveraging modern data tools to support actuarial, machine learning, and AI use cases.
Areas of responsibility may include but not limited to:
Design, build, and maintain ETL pipelines for both batch and streaming use cases. Data visualization and report building. Utilize expertise in Python and SQL for data pipeline development. Using Linux and shell scripting for system automation.
Hands-on experience working with Docker and container orchestration tools is advantageous. Knowledge of Spark is advantageous. Platforms and Tools:
Experience working with ETL tools such as Azure Data Factory, dbt, Ai
