Ganesh C Email & Phone Number

Toronto, Ontario, Canada

Develop dashboards and visualizations to help business users analyze data as well as providing data insight to upper management with a focus on Microsoft products like SQL Server Reporting Services (SSRS) and Power BI.Perform the migration of large data sets to Databricks (Spark), create and administer cluster, load data, configure data pipelines, loading data from ADLS Gen2 to Databricks using ADF pipelines.Create various pipelines to load the data from Azure data lake into Staging SQL… Show more Develop dashboards and visualizations to help business users analyze data as well as providing data insight to upper management with a focus on Microsoft products like SQL Server Reporting Services (SSRS) and Power BI.Perform the migration of large data sets to Databricks (Spark), create and administer cluster, load data, configure data pipelines, loading data from ADLS Gen2 to Databricks using ADF pipelines.Create various pipelines to load the data from Azure data lake into Staging SQL DB and followed by to Azure SQL DB.Create Databrick notebooks to streamline and curate the data for various business use cases and also mounted blob storage on Databrick.Work on Azure Data Factory to integrate data of both on-prem (MY SQL, Cassandra) and cloud (Blob Storage, Azure SQL DB) and applied transformations to load back to Azure Synapse. Show less

Hyderabad

Performed end- to-end Architecture & implementation assessment of various AWS services like Amazon EMR, Redshift, S3.Implemented the machine learning algorithms using Python to predict the quantityUsed AWS EMR to transform and move large amounts of data into and out of other AWS data stores and databases, (Amazon S3) and Amazon DynamoDB.Used Spark SQL for Scala & amp, Python interface that automatically converts RDD case classes to schema RDD.Import the data from different… Show more Performed end- to-end Architecture & implementation assessment of various AWS services like Amazon EMR, Redshift, S3.Implemented the machine learning algorithms using Python to predict the quantityUsed AWS EMR to transform and move large amounts of data into and out of other AWS data stores and databases, (Amazon S3) and Amazon DynamoDB.Used Spark SQL for Scala & amp, Python interface that automatically converts RDD case classes to schema RDD.Import the data from different sources like HDFS/HBase into Spark RDD and perform computations using PySpark to generate the output response.Coded Teradata BTEQ scripts to load, transform data, fix defects like SCD 2 date chaining, cleaning up duplicates.Created Lambda functions with Boto3 to deregister unused AMIs in all application regions to reduce the cost for EC2 resources. Show less

Hyderabad, Telangana, India

Worked on developing ETL processes (Data Stage Open Studio) to load data from multiple data sources to HDFS using FLUME and SQOOP, and performed structural modifications using Map Reduce, HIVE.Developed Spark scripts, UDFS using both Spark DSL and Spark SQL writing data back into RDBMS through Sqoop.Developed Impala for manipulating the dataDeveloped ETL pipelines in and out of data warehouse using combination of Python and Snowflakes SnowSQL Writing SQL queries against… Show more Worked on developing ETL processes (Data Stage Open Studio) to load data from multiple data sources to HDFS using FLUME and SQOOP, and performed structural modifications using Map Reduce, HIVE.Developed Spark scripts, UDFS using both Spark DSL and Spark SQL writing data back into RDBMS through Sqoop.Developed Impala for manipulating the dataDeveloped ETL pipelines in and out of data warehouse using combination of Python and Snowflakes SnowSQL Writing SQL queries against SnowflakeUsed DataStax Spark connector which is used to store the data into Cassandra database or get the data from Cassandra databaseWorked on implementation of a log producer in ScalaDeveloped Spark Streaming programs to process near real time data from Kafka, and process data with both stateless and state full transformations. Show less