Job Description

Role: Python Data Engineer

Location: 100% remote (EST/CST candidates only)

Duration: 12 months

Required Skills

Strong foundation in Python, PySpark, and SQL
Using python and pyspark for only back end, ETL skills
Spark is also good for streaming as well
Boto3 is the standard, a lot of other libraries they will use from a database perspective - depends on the use case which library they will use
SQL is very important for the role - don’t need to write complex SQL, but should be able to read SQL and read them to the business users, which will be who they communicate with, reading queries and giving feedback to the business
Fast APIs will be helpful - re writing java APIs to python, new enhancements will be need be in python
Substantial experience with AWS services
Proficiency in Lambda, SNS, SQS, S3, Step Functions, and Glue
DynamoDB, redshift
Streaming and batch
Hands-on experience in working with APIs using Python and API Gateway
Looking more on the ETL side as opposed to front end Python side
Should have AWS services from an ETL perspective - EMR, Glue, lambdas, DynamoDB, S3, SNS, SQS, step functions
Experience with streaming services like Kinesis and Glue streaming - will be using this more over the next two years for real-time and near real-time streaming
Glue environment has already been built out - has been doing this for the past two years - wants this person to enhance the environment, new services
Java experience would be nice to have - have java and python in their applications
Communication is important - they need to coordinate with business, so they need to be able to communicate well