Sumathi M Email & Phone Number

Texas, United States

Dallas-Fort Worth Metroplex

• Worked on Delta Lake to read and write the large sets of data which is on top of Azure Data Lake Store (ADLS Zen2)
• Built the frameworks and modules to process large data sets using the concepts of spark SQL, data frames and load them into use case extraction layer, develop the complex queries including aggregation grouping logic.
• Performed the data analysis and profile the data using SQL, Spark SQL and other cloud technologies to resolve the business-related issues for product owners and data scientists. The data consumed for this analysis is.
• Worked on segmentation and cleaning of various data sources related to the Model for better data analysis and to increase the final output performance of the model.
• Analyzed the data in ADLS and DB2 to come up with the missing data sources in ADLS and helped solution architect in gathering the requirements.
• Worked on Visualizing the final model results in Power BI by connecting the Power BI to ADLS and import the model output tables.

• Experience with batch processing of data sources using Apache Spark.
• Experience with real time processing of data sources using Apache Spark Streaming.
• Used Spark-Streaming APIs to perform necessary transformations and actions on the fly for building the common learner data model which gets the data from Kafka in near real time and persists into HBase.
• Worked on a product team using Agile Scrum methodology to design, develop, deploy and support solutions that leverage the Client big data platform.
• Experienced in handling large datasets using Partitions, Spark in Memory capabilities, Broadcasts in Spark, Effective & efficient Joins, Transformations and other during ingestion process itself.
• Optimized MapReduce Jobs to use HDFS efficiently by using various compression mechanisms.

• Implemented and configured big data technologies to tune and optimize processes for performance at scale.
• Developed Spark applications for ETL transformations using python.
• Used Spark-Streaming APIs to perform necessary transformations and actions on the fly for building the common learner data model which gets the data from Kafka in near real time and persists into HBase.
• Worked on a lot of ETL script developed in Hive and Spark SQL.
• Worked on a product team using Agile Scrum methodology to design, develop, deploy and support solutions that leverage the Client big data platform.
• Developed Spark code for faster testing and data processing.

Master Of Science - Ms, Computer Technology/Computer Systems Technology

Bachelor Of Technology - Btech, Computer Science