Data Engineer

Technical/Functional Skills

• Frameworks: Spark, PySpark, Python, Apache Airflow and Cassandra, Sprint Boot Micro Services, Apache Camel, Angular 6 UI Framework
• DB: Postgres DB, Oracle
• DevOps: GITLAB, Jenkins, Maven, Confluence, Jfrog Artifactory, SONARQUBE, JIRA

Experience Required

• Must have strong experience in Spark, PySpark, Python, Apache Airflow and Cassandra.
• Must have strong experience in Apache Flink, Python, Apache Kafka and Cassandra

Roles & Responsibilities

• Ability to translate application storyboards and use cases into functional applications
• Build near real-time data pipeline with Apache Spark, PySpark & Python to extract data from NoSQL DBs and Load it into Cassandra Data Enterprise Graph
• Build real-time data pipeline with Apache Flink, Apache Kafka, and Python to extract data from Relational and NoSQL DBs and Load it into Cassandra Data Enterprise Graph
• Design, build, and maintain efficient, reusable, and reliable Java code
• Ensure the best possible performance, quality, and responsiveness of the applications
• Help maintain code quality, organization, and automatization
• Strong debugging, problem-solving, and investigative skills
• Experience in Agile/Scrum methodology
• Proven ability to create innovative solutions
• Strong Agile/Scrum development experience
• Expertise developing web applications and APIs using Micro Services, with demonstrable production-scale experience
• Web applications development using Java, HTML5, CSS3, JavaScript, jQuery, AngularJS, Spring Boot
• Skilled in Version control tools e.g., GIT, SVN, etc. to maintain common source
• Create testing and validation scripts for the features developed
• Develop the Automation Script for end-to-end verifications
• Create, enhance, debug, and execute the Test cases
• Writing unit/Integration tests using JUnit/Mockito for code coverage as per the standards and Log4j 2 for logging
• Expert in database scripts, stored procedures, functions, triggers etc

Data Engineer- Python, PySpark, Java