-
Job Description
Intermediate Data Engineer/Developer to design, develop, and maintain robust Databricks streaming using Pyspark.
Job Type: Contract
Positions to fill: 1
Start Date: Jul 08, 2024
Job End Date: Dec 31, 2024
Pay Rate: Hourly: Negotiable
Job ID: 138473
Our downtown Calgary Oil & Gas client is seeking an Intermediate Data Engineer/Developer to design, develop, and maintain robust Databricks streaming using Pyspark.
Contract would be starting ASAP until December 31, 2024; with high probability of extension.
Must haves
- 3-5+ years experience with Databricks, Pyspark, and data warehousing
- Technical proficiency in Python and SQL
- Experience with Apache Airflow and Git
Nice to haves
- Experience with Azure DevOps and/or Atlassian products (Jira, Confluence) for streamlined development and collaboration
- Technical experience with Power BI
- Experience with cloud platforms, preferably Microsoft Azure
- Bachelor’s degree in computer science, engineering, or related field
Requirements
- Design, develop, and maintain robust Databricks streaming and batch jobs using Databricks and Pyspark
- Collaborate with cross functional teams to understand data requirements and translate them into efficient, reliable, and high quality data solutions
- Optimizing Databricks jobs for performance and cost efficiency
- Manage data ingestion process is of digital platform, handling high frequency and large volumes of data