Sohail M Email & Phone Number

California, United States

 As a Sr. Data Engineer designed and deployed scalable, highly available, and fault tolerant systems on Azure. Lead the estimation, review the estimates, identify the complexities, and communicate to all the stakeholders. Involved in complete SDLC life cycle of big data project that includes requirement analysis, design, coding, testing and production. Defined the business objectives comprehensively through discussions with business stakeholders, functional analysts and participating in requirement collection sessions.  Implemented end-to-end systems for Data Analytics, Data Automation and integrated with custom visualization tools. Migrated on-primes environment on Cloud using MS Azure.  Designed the business requirement collection approach based on the project scope and SDLC (Agile) methodology. Moved data to Azure Data Lake to Azure data warehouse using PolyBase.  Created external tables in ADW with 4 compute nodes and scheduled. Extensively used Agile Method for daily scrum to discuss the project related information. Worked with data ingestions from multiple sources into the Azure SQL data warehouse Transformed and loading data into Azure SQL Database. Configured Spark streaming to receive real time data from the Kafka and store the stream data to HDFS.

Hyderabad, Telangana, India

• Responsible for providing data analysis that focused on improving user experience and optimizing user retention.• Wrote complex ad-hoc MySQL queries involving correlated subqueries, window functions and common table expressions to track user activity metrics such as retention rate and daily active user. Optimized existing queries to run faster on large dataset.• Designed and maintained MySQL databases and created pipelines using user-defined functions and stored procedures for daily reporting tasks.• Developed dashboards in Tableau and present data to track KPIs and product performance using data from various sources such as MS Excel, AWS S3, AWS RDS, AWS Redshift, JSON and XML.• Spotted and analyze trends in user activity data, identify underperformed user segmentation report insight to stakeholders.• Designed metrics for A/B testing, creating dashboards in Tableau to monitor test processes, analyzing test results, interpret and give recommendations to stakeholders.• Performed statistical analysis such as hypothesis testing causal inference and Bayesian analysis.• Communicated between the business, product manager and data engineers to ensure the data quality.• Built ETL pipelines to retrieve data from NoSQL databases and load aggregated data into the analytical platform.

Hyderabad, Telangana, India

• Involved in loading data from UNIX file system to HDFS. Imported and exported data into HDFS and Hive using Sqoop. • Evaluated business requirements and prepared detailed specifications that follow project guidelines required to develop written programs. • Devised procedures that solve complex business problems with due considerations for hardware/software capacity and limitations, operating times and desired results. • Analyzed large amounts of data sets to determine optimal way to aggregate and report on it. Provided quick response to ad hoc internal and external client requests for data and experienced in creating ad hoc reports. • Responsible for building scalable distributed data solutions using Hadoop. Worked hands on with ETL process. • Responsible for cluster maintenance, adding and removing cluster nodes, cluster monitoring and troubleshooting, manage and review data backups, manage and review Hadoop log files. • Handled Imported of data from various data sources, performed transformations using Hive, MapReduce, and loaded data into HDFS.