Mohan Krishna P Email & Phone Number

Chicago, Il, Us

Dublin, California, United States

Developed custom python program including CI/CD rules for AWS data CatLog for metadata management. Design and develop spark job with Scala to implement end to end data pipeline for batch processing.Do fact dimensional modelling and proposed solution to load it.Processing data with Scala, spark, spark SQL and load in hive partition tables in parquet file format.Develop spark job with partitioned RDD (like hash, range, custom) for faster processing. Design conceptual data… Show more Developed custom python program including CI/CD rules for AWS data CatLog for metadata management. Design and develop spark job with Scala to implement end to end data pipeline for batch processing.Do fact dimensional modelling and proposed solution to load it.Processing data with Scala, spark, spark SQL and load in hive partition tables in parquet file format.Develop spark job with partitioned RDD (like hash, range, custom) for faster processing. Design conceptual data models by analyzing the data requirements needed to support the business processes. To manage and store the data we are working on cloud-based technology like AWS which includes all the services from AWS like Ec2 instances, lambda, EMR and S3 services. Developed Automated scripts to do the migration using Unix shell scripting, Python, Oracle/TD SQL, TD Macros and Procedures. Show less

Charlotte, North Carolina, United States

Analyze the client needs and design the implementation to the product.Worked with Azure BLOB and Data Lake storage and loading data into Azure SQL Synapse analytics (DW).Planning and design the solution to implement, using Azure Files Share.Worked on Big Data Integration &Analytics based on Hadoop, Spark, Kafka clusters and web Methods. Composed python scripts to parse XML and JSON reports and load the information in database. We built a pipeline (True Source) to handle huge… Show more Analyze the client needs and design the implementation to the product.Worked with Azure BLOB and Data Lake storage and loading data into Azure SQL Synapse analytics (DW).Planning and design the solution to implement, using Azure Files Share.Worked on Big Data Integration &Analytics based on Hadoop, Spark, Kafka clusters and web Methods. Composed python scripts to parse XML and JSON reports and load the information in database. We built a pipeline (True Source) to handle huge amount of data which comes through our internal and external vendors were pushed mainly Retail bank data to load directly to the cloud. Developed multi cloud strategies in better using GCP (for its PAAS) and Azure (for its SAAS). Analyzed source data coming different sources like SQL Server tables, XML files and Flat files then transformed according to business rules using Informatica and loaded the data into target tables. Show less

New York, United States

Worked effectively on SQL Profiler, Index Tuning Wizard, Estimated Query Plan to optimize the performance tuning of SQL Queries and Stored Procedures. Developed tools using python and Shell Scripting to automate some of menial tasks. Designed and developed Security Framework to provide fine grained access to objects in AWS S3 using AWS Lambda, DynamoDB. Integrated Apache Airflow with AWS to monitor multi-stage ML workflows with the tasks running on Amazon Sage Maker.Experienced… Show more Worked effectively on SQL Profiler, Index Tuning Wizard, Estimated Query Plan to optimize the performance tuning of SQL Queries and Stored Procedures. Developed tools using python and Shell Scripting to automate some of menial tasks. Designed and developed Security Framework to provide fine grained access to objects in AWS S3 using AWS Lambda, DynamoDB. Integrated Apache Airflow with AWS to monitor multi-stage ML workflows with the tasks running on Amazon Sage Maker.Experienced in developing Web Services with Python programming language. Used Spark and Scala for developing machine learning algorithms that analyze clickstream data. Implemented a CI/CD pipeline using Jenkins, Airflow for Containers from Docker, and Kubernetes. Working directly with Development and Data team to validate fields from Source DB to Target DB using the Data Mapping document. Worked on complex SQL Queries, PL/SQL procedures and convert them to ETL tasks. Involved in development of Web Services using SOAP for sending and getting data from the external interface in the XML format. Support current and new services that leverage AWS cloud computing architecture including EC2, S3, and other managed service offerings. Show less

Irving, Texas, United States

• Performed Exploratory Data Analysis using R Also involved in generating various graphs and charts for analyzing the data using Python Libraries. • Worked with internal architects and assisting in the development of current and target state enterprise data architectures. Designed and developed the UI for the website with HTML, XHTML, CSS, Java Script, and AJAX. Created SQL scripts using OLAP functions to improve the query performance while pulling the data from large tables… Show more • Performed Exploratory Data Analysis using R Also involved in generating various graphs and charts for analyzing the data using Python Libraries. • Worked with internal architects and assisting in the development of current and target state enterprise data architectures. Designed and developed the UI for the website with HTML, XHTML, CSS, Java Script, and AJAX. Created SQL scripts using OLAP functions to improve the query performance while pulling the data from large tables. Applied various machine learning algorithms and statistical modeling like decision tree, logistic regression, Gradient Boosting Machine to build predictive model using scikit-learn package in Python. Developed SQL queries to perform data extraction from existing sources to check format accuracy. Co-developed the SQL server database system to maximize performance benefits for clients. Creating Unix scripts for automation. Responsible to tune ETL procedures and Star Schemas to optimize load and query performance. Worked with project team representatives to ensure that logical and physical data models were developed in line with corporate standards and guidelines. Wrote and executed various MySQL database queries from Python-MySQL connector and MySQL dB package. Working with business users on the new Tableau versions features and explaining self-service capabilities. Designed & developed logical & physical data model using data warehouse methodologies. Show less

India

Developed workflow in Oozie to automate the tasks of loading data into HDFS and pre-processing with Pig and Hive.Configured Fair Scheduler to allocate resources to all the applications across the cluster.Performed maintenance, monitoring, deployments, and upgrades across infrastructure that supports all Hadoop clusters.Optimized and tuned ETL processes & SQL Queries for better performance. Used Zookeeper and Oozie for coordinating the cluster and scheduling… Show more Developed workflow in Oozie to automate the tasks of loading data into HDFS and pre-processing with Pig and Hive.Configured Fair Scheduler to allocate resources to all the applications across the cluster.Performed maintenance, monitoring, deployments, and upgrades across infrastructure that supports all Hadoop clusters.Optimized and tuned ETL processes & SQL Queries for better performance. Used Zookeeper and Oozie for coordinating the cluster and scheduling workflows.Involved in designing the star schema and populating the fact table and associated dimension tables. Managed jobs using Fair Scheduler to allocate processing resources. Show less