Mohammad Sameer Email & Phone Number

Irving, Tx, Us

Weehawken, New Jersey, United States

• Extensively collaborated with the Azure cloud platform, showcasing proficiency in HDInsight, Data Lake, Databricks, Blob Storage, Data Factory, Synapse, SQL, SQL DB, DWH, and Data Storage Explorer.• Executed data ingestion into multiple Azure Services, including Azure Data Lake, Azure Storage, Azure SQL, and Azure DW, followed by processing the data within Azure Databricks.• Demonstrated expertise in designing and configuring Azure Cloud relational servers and databases, aligning with current and future business requirements.• Implemented medallion architecture for the event and batch data leveraging databricks Lakehouse and unity catalog for data governance.• Implemented DABs as part of implementing framework for deploying jobs and integrate with Githubworkflow actions.• Configured Input & Output bindings of Azure Function with Azure Cosmos DB collection for seamless data read and write operations.• Developed robust ETL pipelines in Azure Data Factory (ADF), employing Linked Services from various sources and loading them into Azure SQL Data Warehouse.• Established Elastic pool databases and scheduled Elastic jobs to execute T-SQL procedures.• Proficiently performed ETL operations in Azure Databricks, connecting to different relational database source systems using JDBC connectors.• Hand-on experience working on stream data processing, checkpoint migration and handling catalogs and volumes.• Engineered scalable ETL applications for Medicaid and Medicare data, seamlessly integrating Azure Data Lake and Data Warehouse using Azure Databricks.• Automated data engineering ETL pipeline on Snowflake DB using Apache Spark, consolidating data from various sources with Python APIs like PySpark and organizing them in a data mart (Star schema).

Missouri, United States

• Engaged in the design and deployment of multi-tier applications, leveraging a comprehensive suite of AWS services such as EC2, Route53, S3, RDS, Dynamo DB, SNS, SQS, and IAM. Focused on ensuring high availability, fault tolerance, and auto-scaling through AWS CloudFormation.• Crafted AWS Data pipelines, utilizing AWS API Gateway to receive responses from AWS Lambda, extracting and transforming data into JSON format, and storing it in AWS Redshift.• Developed scalable AWS Lambda code in Python, specifically designed for handling nested JSON files, encompassing tasks such as conversion, comparison, and sorting.• Implemented Spark Applications using Scala, executing an Apache Spark data processing project to manage data sourced from various RDBMS and Streaming platforms.• Enhanced the performance and efficiency of existing Spark jobs, transitioning from MapReduce scripts to Spark SQL.• Proficiently collected real-time data from an AWS S3 bucket using Spark Streaming, executed necessary transformations and aggregations, and persisted the data in HDFS.• Implemented AWS Glue catalog with a crawler to extract data from S3, facilitating SQL query operations.• Developed robust and scalable data integration pipelines, orchestrating the transfer of data from the S3 bucket to the RedShift database using Python and AWS Glue.• Managed and maintained a Hadoop cluster on AWS EMR, leveraging AWS services like EC2 and S3 for processing and storing small data sets.• Created Python code for various tasks, dependencies, and time sensors to facilitate workflow management and automation using the Airflow tool.• Scheduled Spark/Scala jobs using Oozie workflow within the Hadoop Cluster, providing detailed design documentation for source-to-target transformations.• Designed reports and dashboards in Tableau, aligning with business requirements to harness data for interactive and insightful dashboards.

Oak Brook, Illinois, United States

• Employed Azure Data Facto ry, T-SQL, Spark SQL, and U-SQL in Azure Data Lake Analytics to extract, transform, and load data from various source systems into Azure Data Storage services. This included data ingestion into multiple Azure services like Azure Data Lake, Azure Storage, Azure SQL, and Azure DW, followed by data processing in Azure Databricks.• Successfully implemented proof of concepts for both SOAP and REST APIs and harnessed REST APIs to retrieve analytical data from diverse data feeds.• Developed pipelines within Azure Data Factory (ADF) using Linked Services, Datasets, and Pipelines to extract, transform, and load data from sources such as Azure SQL, Blob storage, Azure SQL Data Warehouse, write-back tools, and more.• Showcased proficiency in developing SQL scripts for automation purposes, streamlining data processes.• Utilized PY-Spark and Spark-SQL to build Spark applications for data extraction, transformation, and aggregation from various file formats, enabling comprehensive analysis and insight discovery.• Took responsibility for estimating cluster sizes and meticulously monitored and troubleshot the Spark Databricks cluster for optimal performance.• Demonstrated expertise in performance tuning for Spark Applications, including the adjustment of Batch Interval times, optimal parallelism levels, and memory management.• Crafted JSON scripts for deploying pipelines in Azure Data Factory (ADF), enabling data processing through SQL Activities.• Created Build and Release processes for multiple projects/modules within a production environment, leveraging Visual Studio Team Services (VSTS).• Responded to user requests for system upgrades and changes within local area networks (LANs) and wide area networks (WANs). Monitored network performance and employed diagnostic equipment and status indicators to identify and resolve issues.

India

• Extracted diverse datasets, encompassing customer behavior, sales and revenue data, as well as supply chain and logistics data, from HDFS.• Utilized Apache Nifi, an open-source data integration tool, to efficiently transfer data to AWS S3, ensuring powerful and scalable dataflows.• Implemented Python scripts for data validation and cleansing before storing the refined data in S3.• Employed PySpark, a distributed computing framework with a Python API, to process and transform the data.• Loaded the transformed data into AWS RedShift data warehousing for comprehensive data analysis.• Scheduled the entire pipeline using Apache Oozie, a workflow scheduler system designed to manage Apache Hadoop jobs.• Took charge of developing and maintaining a library of custom Airflow DAG templates and operators, enhancing consistency and code quality within the team.• Utilized Apache Hive, a data warehouse software, for data analysis on HDFS, adapting Hive queries into PySpark transformations via PySpark RDDs and Data Frame API.• Monitored the data pipeline and applications using Grafana, ensuring seamless performance.• Configured Zookeeper to support distributed applications, enhancing system reliability.• Applied functional programming concepts and Scala's collection framework to efficiently store and process complex data.

India

• Participated in the assessment of both functional and non-functional requirements for data processing tasks.• Installed and configured Hadoop MapReduce HDFS, developing multiple Java-based MapReduce jobs for data cleaning and pre-processing.• Configured and utilized Pig, writing Pig Latin scripts, and also integrating Pig Latin into MapReduce jobs.• Managed and reviewed Hadoop log files to ensure system integrity and performance.• Imported data from MySQL to HDFS using Sqoop on a regular basis, establishing efficient data transfer processes.• Developed scripts and batch jobs for scheduling various Hadoop programs, optimizing workflow management.• Authored Hive queries for data analysis, creating and working with Hive tables to meet specific business requirements.• Utilized Sqoop for importing and exporting data into HDFS and Hive, ensuring seamless data integration.• Gained experience with NOSQL databases, particularly SOLR and HBase, broadening data storage capabilities.• Contributed to defining job flows and orchestrating complex data processing tasks.• Implemented a custom Filesystems plug-in for Hadoop, enabling direct access to files on the Data Platform for applications such as MapReduce, HBase, Pig, and Hive.