Dev J Email & Phone Number

 10+ years of expertise in Big Data Ecosystem - Data Acquisition, Ingestion, Modeling, Storage Analysis, Integration, and DataProcessing. Experience working with Azure Cloud, Azure Data Factory, Azure Lake Storage, Azure Synapse Analytics, Azure AnalyticalAdministrations to Ingest, Transform and consolidate Structured and Unstructured Data for downstream Use cases. Experience in Building Data Pipelines using Azure Data Factory, Azure Data bricks and loading data to Azure Data Lake, Azure SQLDatabase/Datawarehouse to control and grant user level access. Experienced in building data ingestion pipelines on Azure HDInsight spark cluster by using Azure Data Factory and Spark SQL services. Experienced in building applications with AWS services like S3, EMR, Amazon Redshift, Amazon Elastic Cloud Balancing, IAM, AutoScaling, Cloud watch, Cloud Front, SNS, SQS, SES, and Lambda. Experienced in using PowerBI, Tableau, AWS Quicksight for visualization of data and in building dashboards/reports. Experience in creating, managing, analyzing, and reporting the internal business client data using AWS services like Athena, Redshift,EMR and QuickSight. Responsible for storing data on S3 using Lambda functions and AWS Glue using PySpark. Experience working with Batch Ingestion into the platform for Snowflake consumption. Worked on distributed frameworks such as Apache Spark and Presto in Amazon EMR, Redshift and interact with data in other AWS datastorage services such as Amazon S3 and Amazon DynamoDB. Involved in the automation of daily and weekly ETL jobs using Apache Airflow. Experience in working with Microsoft SQL Server database programming and as ETL Developer using SSIS, SSRS, SSAS Experience in working on python libraries like Numpy, Pandas, and MatlabLib. Skilled in System Analysis, E-R/Dimensional Data Modeling, Database Design and implementing RDBMS specific features.  Good knowledge in converting Hive/SQL queries into PySpark transformations using Data Frames. Experience working with different file formats like Json, Avro, Parquet, CSV etc. Experience in developing applications in Spark using Python/Scala to compare the performance of Spark with Hive. Good working experience with Hive and HBase/MapRDB Integration. Experienced in developing shell scripts and Python scripts to automate Spark jobs and Hive scripts. Experience in Incident Tracking and Ticketing systems such as Jira, Service Now, and Remedy, used Git and SVN for version control.