Data Engineer with Python/PySpark

Primary Responsibilities

Develop data pipelines in a Python/PySpark environment to transform disparate data into business ready data sets.

Partner with developers and testers in an Agile Scrum framework to communicate, refine, and validate requirements and solutions.

Analyze and document new and existing data interfaces within the Surest data platform.

Understand vendor and Surest capabilities and data to create data definitions, transformation logic, and data storage options based on business requirements.

Collaborate with product owners, analysts, and vendors to build and document complex business logic across both responsibility areas and data integration across the platform.

Translate complex concepts to messaging that can be understood by broader audiences.

Required Qualifications

• 2+ years PySpark & Python data development and testing experience
• 2+ years of SQL or NoSQL query experience
• Experience in a big data environment building data processing pipelines
• Undergraduate degree or equivalent experience

Preferred Qualifications

• Excellent interpersonal and communication skills, both written and verbal
• Familiarity with database design principles
• Experience attending and participating in scrum agile ceremonies
• Experience working with healthcare data
• Expertise in troubleshooting complex data questions
• Ability to work independently to identify, solve and communicate data processing, storage and structural challenges
• Experience with Cloud/AWS environments preferred