Junaid M. Email & Phone Number

Canada

• Designed and developed full end-to-end ETL data pipelines in the cloud Cloud using Python/Spark, Azure Data Factory, Azure Data Lake and Azure SQL for robust data ingestions while keeping processing times low and.
• Debugged and updated Databricks notebooks when needed to reduce infrastructure usage and Azure cloud computing costs
• Created a template for Git branching strategies and pull requests for seamless collaboration amongst Data Engineers and Analysts
• Orchestrated data pipeline ETL/ELT jobs using Apache Airflow with various operators
• Frequently performed data ingestion with Python/Spark through REST APIs and worked with JSON data to be ingested and transformed as per business requirements
• Utilized Azure Key Vault to ensure data security, protect sensitive information and prevent potential breaches.

Toronto, Ontario, Canada

• Implemented an ETL data pipeline orchestration solution by creating a Docker container with Airflow, optimizing scriptscheduling and execution
• Spearheaded Git practices, significantly reducing development times by implementing efficient Git branching strategies
• Utilized DBT for proper data testing, version controlled SQL Scripting, and documentation to ensure data quality
• Displayed a high proficiency in SQL by performing complex querying (INTERSECTs, UNIONs, complex joins andaggregations) as well as using SQL for database development by developing stored procedures and triggers upon.
• Automated data transformations for reporting purposes through the creation of stored procedures and triggers in variousdatabases such as SQL Server, MySQL and Postgres, reducing manual workload by ~70%
• Aided coworkers in refactoring SQL scripts to reduce querying time

Mississauga, Ontario, Canada

• Designed and Developed end-to-end automated, scalable ETL/ELT data pipelines in AWS Cloud Platform, utilizing services such as AWS EMR, S3, Glue, Python, SQL, PySpark etc. to add to existing data infrastructure and.
• Utilized big data technologies such as Spark and Hadoop for high-volume data processing and solutions while using data quality checks to ensure data integrity and usability
• Displayed Python programming and Apache Spark proficiency (Pyspark, SparkSQL) by creating data pipelines from both structured and semi-structured data such as CSV, JSON, SQL database tables etc.
• Used Python scripting to perform data ingestions from API Calls, using functional programming frameworks for readability
• Performed data transformations using Python (Pandas/Polars/Pyspark) and SQL and loaded final tables in Snowflake data warehouse
• Expanded the existing Snowflake data warehouse by including new dimension tables using appropriate database design techniques for data integration that enabled new angles of data analysis and business intelligence for.

• Engineered automated, scalable ETL data pipelines using Python programming, SQL and ApacheSpark/PySpark/SparkSQL in the Cloud using Azure Databricks, streamlining the ingestion process into Azure Data Lakestorage and.
• Performed governance by using data quality checks in both Python/Spark scripts as well as SQL queries by using test cases
• Developed data pipelines using Pandas and pure Python for smaller ETL tasks that did not require a Spark Cluster,reducing overall costs for our data platform
• Orchestrated and scheduled Python and PySpark scripts using Apache Airflow to handle both simple and complexorchestration requirements
• Performance tuned and optimized SQL databases in SQL Server and scripts by using practices such as indexing, shardingand partitioning, as well as making changes in scripts to reduce querying time by 70%
• Utilized Git for maintaining code repository and followed best practices when branching, creating pull requests

Ontario, Canada

• Created visualization dashboards in Tableau to automate reporting, reducing reporting time by several hours per week whiledelivering key insights
• Developed, maintained and optimized data infrastructure in SQL Server, reducing query time by over 80% througheffective indexing and query optimization practices
• Applied data modeling techniques when creating database designs for seamless data integration with new sources
• Utilized various tools such as Python, Pandas and SQL to perform transformations, exploratory data analysis, as well asother forms of data analytics to create data-driven actionable insights from high volumes of data
• Performed data ingestion into Python from various sources such as CSVs, databases and APIs in order to wrangle data andcreate final needed products via ETL data pipelines
• Followed Python programming best practices such as using functional programming paradigms to ensure ETL scripts arereadable