Joel P Email & Phone Number

Toronto, Ontario, Canada

● Responsible for implementing end to end cloud data platform on AWS & Azure to leverage modern cloud data warehousing solution.● Configured s3 buckets and implemented bucket level policy to store the raw data which flows from the source system.● Scheduled AWS step functions for the orchestration of the daily ingestion job’s and configured a json payload that send the log details of the ingestion runs to the cloud watch log group and log streams.● Deployed AWS resources such as AWS… Show more ● Responsible for implementing end to end cloud data platform on AWS & Azure to leverage modern cloud data warehousing solution.● Configured s3 buckets and implemented bucket level policy to store the raw data which flows from the source system.● Scheduled AWS step functions for the orchestration of the daily ingestion job’s and configured a json payload that send the log details of the ingestion runs to the cloud watch log group and log streams.● Deployed AWS resources such as AWS VPC, Redshift cluster and Redshift serverless using terraform which acted as the cloud data warehousing solution.● worked on implementing the IAM roles and policies required for the resources to interact with each other and also ensured lowest level of permission are allowed.● Created a dashboard in the open search kibana where the log data would be visualized in the for of reports which stored all the details regarding the job runs and the failures.● Deployed Lambda functions and provided python as input to it which contains Redshift API call’s which will be called using the boto3 library to run every 2 hours to trigger the daily extraction job’s● Consulting with various Business Stakeholders and participate in requirements elicitation and user story mapping with the business and technical SMEs to understand their data use case and reporting needs.● Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL and U-SQL Azure Data Lake Analytics. Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in In Azure Databricks.● Developed Bash Scripts (UNIX) to traverse and extract the necessary text data from large files.● Data loading to Data Lake from different sources using Azure Data Factory.● Create Pipelines and Scheduling jobs using Azure Data Factory● Data Loading to Datamart and other destinations from Data Lake using Data Bricks. Show less

Hyderabad, Telangana, India

● Translated business requirements to technical definitions to help develop a solution for mapping scenarios.● Participated in meetings with Users, Business analysts in understanding the business requirements and analyzed use cases to develop detailed map strategy.● Worked independently in troubleshooting the issues, data mining, data analysis and provide estimates for planned activities.● Worked in migration project to migrate customers from legacy rate plans to new rate plans… Show more ● Translated business requirements to technical definitions to help develop a solution for mapping scenarios.● Participated in meetings with Users, Business analysts in understanding the business requirements and analyzed use cases to develop detailed map strategy.● Worked independently in troubleshooting the issues, data mining, data analysis and provide estimates for planned activities.● Worked in migration project to migrate customers from legacy rate plans to new rate plans using Alteryx tool● Designing Azure Data warehouse and using Azure Data Factory● Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform, and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards.● Scheduled Alteryx modules and monitored the out coming data results.● Provided various mapping solutions based on the business need, troubleshoot the reasons for unmapped customers.● Used Teradata utilities fast load to load data.● Wrote views based on business reporting requirements, performance tuning and various complex SQL queries optimization.● Extensively used SQL Analyzer and wrote complex SQL Queries using joins, sub queries and correlated sub queries● Scheduled and maintained routine jobs, tasks, and alerts. For instance, an alert would be sent to the Dev. Team if any stale data or ETL failed.● Developed Power BI reports using Power Query from SharePoint& different Data sources.● Monitor, Triage, Track and resolve issues with the Excel Power BI reports.● Interacted with various upstream and downstream customers in interfacing various systems for data extractions, ETL, Analytics and Reporting needs.● Used different Control Flow Tasks and Data flow Task for Creating SSIS Packages. Used different types of Transformations for Data Conversion, Sorting, and data cleansing from different Sources into Company Formats. Show less

Hyderabad, Telangana, India

• Worked with Business Users and Business analyst in gathering requirements and preparing the functional and technical documentation. • Developed complex T-SQL code such as Stored Procedures, functions, triggers, Indexes, and views for the application.• Used Temporary tables, table variables in Stored procedures to simply complex queries. • Experience in building and architecting multiple Data pipelines, end to end ETL and ELT process for Data ingestion.• Written complex SQLs… Show more • Worked with Business Users and Business analyst in gathering requirements and preparing the functional and technical documentation. • Developed complex T-SQL code such as Stored Procedures, functions, triggers, Indexes, and views for the application.• Used Temporary tables, table variables in Stored procedures to simply complex queries. • Experience in building and architecting multiple Data pipelines, end to end ETL and ELT process for Data ingestion.• Written complex SQLs using joins, sub queries and correlated sub queries. Expertise in SQL Queries for cross verification of data.• Employed best practices in creating a complete schema including tables, relationships, stored procedures, views, clustered and non-clustered indexes, and triggers, and implemented the import and export of data using SSIS. • Created ETL packages that reads Product, Vendor, and Order data from an FTP server into a SQL Server database. The solution also validated incoming Data and performs notifications when jobs complete. • Created SSIS packages to load data from Heterogeneous sources like Flat files, CSV files and Excel. • Implemented custom logging to keep track of the packages being run automatically. Configured the packages to run on different servers using Package Configurations. • Created packages in SSIS with error handling using Event handlers. • Experience in fact dimensional modeling (Star schema, Snowflake schema), transactional modeling and SCD (Slowly changing dimension)• Devised PL/SQL Stored Procedures, Functions, Triggers, Views, and packages. Made use of Indexing, Aggregation and Materialized views to optimize query performance. Using C#, visual basic.Net in SSIS script tasks and script components with variables Show less