We are seeking an experienced Senior Data Engineer for a contract position to design, develop, and optimize data solutions using Microsoft Fabric.
Requirements
- Microsoft Fabric Expertise
- Hands-on experience with Fabric notebooks, pipelines, and workspace configuration
- Fabric Data Agent implementation and orchestration
- Fabric CLI and CI/CD deployment practices
- Programming & Development
- Python (advanced proficiency)
- PySpark for distributed data processing
- Pandas and Polars for data manipulation
- Experience with Python libraries for data engineering workflows
- REST API development and integration
- Data Platform & Storage
- Delta Lake and Iceberg table formats
- Delta table optimization techniques (partitioning, Z-ordering, liquid clustering)
- Spark performance tuning and optimization
- SQL query optimization and performance tuning
- Development Environment
- Visual Studio Code
- Azure DevOps for CI/CD and deployment pipelines
- Experience with both code-first and low-code development approaches
- Data Modeling
- Data warehouse dimensional modeling (star schema, snowflake schema)
- Slowly Changing Dimensions (SCD Type 1, 2, 3)
- Modern lakehouse architecture patterns
- Metadata driven approaches