Responsibilities
- Design, develop, and implement scalable and reliable data solutions on the Microsoft Azure platform.
- Collaborate with cross‑functional teams to gather and analyze data requirements.
- Design and implement data ingestion pipelines to collect data from various sources, ensuring data integrity and reliability.
- Perform data integration and transformation activities, ensuring data quality and consistency.
- Implement data storage and retrieval mechanisms, utilizing Azure services such as Azure SQL Database, Azure Data Lake, and Azure Blob Storage.
- Monitor data pipelines and troubleshoot issues to ensure smooth data flow and availability.
- Implement data quality measures and data governance practices to ensure data accuracy, consistency, and privacy.
- Collaborate with data scientists and analysts to support their data needs and enable data‑driven insights.
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related field.
- 5+ years of experience with Big Data technologies
- Strong knowledge and experience with Microsoft Azure cloud platform
- Proficiency in SQL and experience with SQL-based database systems.
- Experience with batch and data streaming.
- Proficiency in data processing frameworks such as Apache Spark, Apache Hadoop, or cloud-native data processing services (Azure Data Lake, Azure Data factory, Azure Databricks, Azure Synapse, Snowflake, CosmosDB)
- Hands‑on experience with Azure data services, such as Azure SQL Database, Azure Data Lake, and Azure Blob Storage.
- Experience using Azure Databricks in real-world scenarios.
- Experience with data integration and ETL (Extract, Transform, Load) processes.
- Strong analytical and problem-solving skills.
- Good understanding of data engineering principles and best practices.
- Experience with programming languages such as Python or Scala.
- Relevant certifications in Azure data services or data engineering are a plus.
Core Competencies
Demonstrates expertise in designing and implementing scalable data solutions on the Microsoft Azure platform, with a strong focus on data ingestion, integration, and quality assurance. Proficient in utilizing Azure data services and frameworks to support data-driven insights and ensure data integrity.
