AWS Data Engineer

Job Description

Data Tech team is looking for a Data Engineer to join a diverse team dedicated to providing best in class data services to our customers, stakeholders, and partners.

As a part of our CDO organization, you will work with our ML Engineers, Data Scientist, and various BUs to define solutions for operationalizing data-driven decision making in a cost-effective and scalable manner.

Qualifications

• Bachelor's degree in Computer Science, Software Engineering, MIS, or equivalent combination of education and experience
• Experience implementing, supporting data lakes, data warehouses, and data applications on AWS for large enterprises
• AWS Solutions Architect or AWS Big Data Certification preferred
• Programming experience with Java, Python/ Scala, Shell scripting

Solid Experience with AWS includes:

• CloudFormation
• S3
• Glue
• EMR/Spark
• RDS
• Redshift
• DynamoDB
• Lambda
• Step Functions
• IAM
• KMS
• SM etc.

Additional Qualifications

• Experience implementing solutions on AWS based data lakes
• Experience with metadata solutions leveraging AWS non-relational data solutions
• Experience in AWS data lake, data warehouse, and business analytics
• Experience in system analysis, design, development, and implementation of data ingestion pipeline in AWS
• Knowledge of ETL/ ELT
• Working experience with Hadoop, HDFS, SQOOP, Hive, Python, and Spark is desired
• Experience working on Agile projects

Requirements

• 3 - 5 years of experience as Data Engineer
• Experience developing business applications using NoSQL/ SQL databases
• Experience working with Object stores (S3) and JSON is a must-have

Experience with AWS Services

• API Gateway
• Lambda
• Step Functions
• SQS
• DynamoDB
• S3
• ElasticSearch

Responsibilities

• Designing, building, and maintaining efficient, reusable, and reliable architecture and code
• Ensure the best possible performance and quality of high scale web applications and services
• Participate in the architecture and system design discussions
• Independently perform hands-on development and unit testing of the applications
• Collaborate with the development team and build individual components into complex enterprise web systems
• Work in a team environment with product, frontend design, production operation, QE/ QA and cross-functional teams to deliver a project throughout the whole software development cycle
• Responsible to identify and resolve any performance issues
• Keep up to date with new technology development and implementation
• Participate in code review to make sure standards and best practices are met

Plus

• Experience with business intelligence tools such as Tableau, Power BI or equivalent
• Aware of Machine learning algorithms (supervised/unsupervised)
• Nice to have AWS developer certificate

AWS Data Engineer(Python, Pyspark)