A skilled Data Engineer is seeking a candidate with a strong background in PySpark and extensive experience with AWS services, particularly Athena and EMR. The role involves designing, developing, and optimizing large-scale data processing systems, ensuring data integrity and reliability, and collaborating with stakeholders.
Requirements
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field
- 5+ years of experience as a Data Engineer or in a similar role
- Proficiency in PySpark
- Extensive experience with AWS services (Athena, EMR)
- Strong knowledge of SQL and database technologies
- Experience with Apache Airflow
- Familiarity with S3, Lambda, and Redshift
- Proficiency in Python
- Excellent analytical and problem-solving skills
- Strong communication skills
- Agility