Job Description
Dice is the leading career destination for tech experts at every stage of their careers. Our client, Recru, LLC, is seeking a Senior AWS Data Engineer skilled in Python open to working hybrid in Houston! This role is contract-to-hire. Apply via Dice today!
Purpose
Establish data feeds to the digital platform and manage ETL processes. Design raw file data structures to be used in data science applications. Work with Data Analysts to ensure enterprise data is cataloged in accordance with data governance standards. Develop system test and integration plans, execute test procedures, support user community, and obtain approvals for change management. Provide hands-on technical configuration, application development, integration, and testing. Implement cloud data integration strategies between cloud providers. Provide technical expertise to the business and internal IT with respect to digital solutions. Design, develop, and maintain data solutions that leverage the power and flexibility of AWS.
Essential Job Functions
- Developing, testing, and deploying various data solutions on AWS services such as S3, Glue, Athena, Redshift, EMR, Kinesis, Lambda, etc.
- Implementing data ingestion, processing, transformation, and delivery using batch, streaming, and real-time methods.
- Applying best practices for data quality, security, reliability, and performance using AWS services.
- Collaborating with different teams to understand data requirements and provide solutions following the AWS Well-Architected Framework.
- Evaluating and adopting new AWS technologies to improve data capabilities and optimize data costs.
- Troubleshooting data issues, ensuring data availability and integrity, and documenting data solutions and standards.
- Designing and building web applications, REST APIs, and GraphQL APIs using various AWS services.
- Applying software engineering rigor and best practices to machine learning, including AI/MLOps, automation, etc.
- Developing and deploying proof-of-concept machine learning systems using AWS services.
- Architecting digital data solutions for Commercial Operations.
- Implementing ETL for enterprise data and working with other IT operations support resources.
- Identifying deficiencies, recommending solutions, and providing automation where applicable.
- Assessing and optimizing processes and applications, mentoring support resources, and communicating effectively with stakeholders.
- Coordinating with other application teams for data integration and developing ETL processes.
- Keeping up-to-date with the latest AWS technologies and industry trends.
- Assisting in training other team members and business users.
Requirements
- Knowledge of energy market data.
- In-depth experience with AWS, Dataiku, and Databricks.
- Proven software engineering experience using Python and machine learning knowledge.
- Designing and building scalable data architectures on AWS.
- Proficiency in programming languages such as Python, Java, Scala, SQL, Spark, and Unix shell scripting.
- Automating data processing workflows using AWS services and scripting languages.
- Experience with DevOps tools and methodologies.
- Knowledge of ETL tools and frameworks, database technologies, and data security.
- Strong analytical, problem-solving, and communication skills.
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- Data visualization tool experience such as Power BI.
- Experience with data warehouses, SQL databases, and data formats.
Education/Experience
Bachelor's degree in a technical field or 15+ software development experience. 10-15 years of information technology experience. Minimum 5 years supporting the needs of an Energy Trading or Asset Management business. Proficient with Microsoft Office and operating systems. Knowledge of ISO Market systems. Familiarity with modern cloud technologies.