Data Engineer

Job Description

About the Role: We are seeking a skilled mid to senior level Python Data Engineer to join our clients’ team in Houston, TX. The ideal candidate will have 5+ years of experience in data engineering with a focus on Node.js, Python, SQL, and Java. Experience with Slurm and Kubernetes are highly preferred. Responsibilities include data pipeline development, data integration, database management, software development, Linux job scheduling, containerization, streaming data, data quality, performance optimization, and documentation.

Responsibilities

• Data Pipeline Development: Build and maintain efficient data pipelines for collecting, processing, and storing large volumes of data.
• Data Integration: Collaborate with cross-functional teams to integrate data from various sources into our systems.
• Database Management: Design, optimize, and maintain databases using SQL and NoSQL technologies.
• Software Development: Write clean and efficient code using tech stacks such as Node.js, Python, Java, and more.
• Linux Job Scheduling: Utilize Slurm or similar Linux job scheduling systems for resource management and job scheduling.
• Containerization: Experience with Kubernetes (Kube) is preferred for container orchestration.
• Streaming Data: Work with real-time data processing and streaming technologies, including MQTT and Kafka.
• Data Quality: Ensure data quality, accuracy, and consistency by implementing data validation and cleansing processes.
• Performance Optimization: Identify and resolve bottlenecks in data processing, storage, and retrieval.
• Documentation: Maintain comprehensive documentation for data pipelines, systems, and processes.

Requirements

• Bachelor's degree in Computer Science, Information Systems, or a related field.
• At least 5 years of data engineering experience, with a strong track record of designing and implementing data solutions.
• Proficiency in data engineering technologies, including Node.js, Python, SQL, and Java.
• Experience with Linux job scheduling, particularly with Slurm, or similar job scheduling systems.
• Familiarity with containerization technologies, with a preference for Kubernetes experience.
• Excellent knowledge of streaming data technologies, including MQTT and Kafka.
• Strong problem-solving skills and the ability to work in a collaborative team environment.
• Strong communication skills and the ability to translate complex technical concepts to non-technical stakeholders.
• A commitment to best practices in data engineering, including data security and privacy considerations.
• Strong problem-solving and analytical skills.
• Excellent communication and collaboration skills.

Benefits

• Profit Sharing
• 401k with match
• 100% paid medical premiums
• Career development/training opportunities