Databricks Data Engineer/Solution Architect (On site or remote)

We are looking for a Databricks Data Engineer / Solution Architect to lead the design, implementation, and optimization of our data platform and pipelines.

Requirements

• Lead the end-to-end design, development, deployment and maintenance of data pipelines using Databricks
• Architect solutions for ingestion, transformation, storage and serving layers
• Collaborate with stakeholders to understand business needs and translate them into technical requirements and data models
• Define and enforce data governance, data quality, metadata management, security, access controls, lineage
• Work with cloud infrastructure to provision, monitor and optimize data compute, storage, and networking resources
• Implement CI/CD, testing, monitoring, alerting for data pipelines
• Mentor and coach more junior engineers, define best practices, code reviews, knowledge sharing
• Stay current with developments in Databricks, distributed systems, data engineering trends - propose innovations
• Support data architecture decisions: what tools, frameworks, storage formats, etc. are appropriate
• Troubleshoot performance issues, data discrepancies, failures; ensure high availability, disaster recovery

Benefits

• Generous Paid Time Off
• 401k Matching
• Retirement Plan
• Visa Sponsorship
• Four Day Work Week
• Generous Parental Leave
• Tuition Reimbursement
• Relocation Assistance