Data Engineer

Role: Data Engineer (Python/Pyspark, Azure)

Location: Mississauga, ON (hybrid/3 days in a week)

Type: Fulltime

Overview

Looking for an Integration engineer responsible for daily support and project based development of credit risk management systems. ETL developers are responsible for designing and creating the data warehouse and all related extraction, transformation and load of data functions. This is an opportunity to gain experience in risk management processing using new technologies.

Responsibilities

• Strong understanding of Snowflake on Azure Architecture.
• Design, implementation and operationalization of large-scale data and analytics solutions on Snowflake Cloud Data Warehouse.
• Hands-on development experience with Snowflake features such as Snow SQL, Snow Pipe, Python, Tasks, Streams, Time travel, Zero Copy Cloning, Optimizer, Metadata Manager, data sharing, and stored procedures.
• Experience in Data warehousing - OLTP, OLAP, Dimensions, Facts, and Data modeling.
• Need to have working knowledge of MS Azure configuration items with respect to Snowflake.
• Developing EL pipelines in and out of data warehouse using a combination of Data bricks, Python, and Snow SQL.
• Developing scripts UNIX, Python etc. to Extract, Load and Transform data, as well as other utility functions.
• Provide production support for Data Warehouse issues such data load problems, transformation translation problems.

Requirements

• Minimum 10 years of designing and implementing an operational production grade large-scale data solution on Microsoft Azure Snowflake Data Warehouse.
• Including hands-on experience with productionized data ingestion and processing pipelines using Python, Data bricks, Snow SQL.
• Excellent understanding of Snowflake Internals and integration of Snowflake with other data processing and reporting technologies.