Distributed Cloud | Azure Data Engineer

We are seeking an experienced and dedicated Azure Data Engineer to join our data team. You will be responsible for designing, building, and optimizing robust, scalable, and highly available data pipelines and ETL/ELT solutions exclusively within the Microsoft Azure ecosystem.

Requirements

• Design, build, and maintain scalable data pipelines and ETL/ELT processes using core Azure data services, primarily Azure Data Factory (ADF) and Azure Databricks.
• Develop and optimize data infrastructure on Azure, leveraging Azure Data Lake Storage (ADLS) for scalable data ingestion and storage.
• Implement and manage data warehousing solutions using Azure Synapse Analytics (SQL Pool or Spark Pool) to support analytics and reporting needs.
• Implement data quality checks, monitoring, and validation to ensure accuracy and integrity of data across all Azure systems.
• Collaborate closely with Data Scientists and Data Analysts to ensure data readiness for reporting, analytics, and Machine Learning initiatives.
• Automate deployment, monitoring, and testing of data infrastructure and pipelines using Infrastructure as Code (IaC) tools like Terraform or ARM Templates and CI/CD practices.
• Provide technical guidance and recommendations on data architecture and technology choices within the Azure ecosystem.