Bala Krishna Email & Phone Number

Missouri, United States

 Orchestrated secure data management within the AWS environment, ensuring compliance with SEC guidelines, leveraging AWS services such as EC2, S3, and Aurora. As Data Engineer in to drive projects using Apache Spark, SQL, and AWS cloud environment. Hands on Experience in migrating datasets and ETL/ELT workloads with Python from On-prem to AWS Cloudservices Proficient in utilizing AWS Glue, a fully managed extract, transform, and load (ETL) service, for building and managing data pipelines to process and transform large volumes of data. Developed and maintained PySpark applications for big data processing and analysis. Integrated ECS with AWS services such as EC2, ECR, IAM, and CloudWatch to streamline container management and monitoring. Managed and configured AWS resources through the AWS Management Console, ensuring optimal performance and cost-efficiency. Created ETL/ELT pipelines in and out of the data warehouse using a combination of Python and Snowflakes SnowSQL Writing SQL queries against Snowflake. Designed and developed SSIS packages to extract, transform, and load (ETL) data from various sources into SQL Server databases, improving data accuracy and reliability. Working on building visuals and dashboards using Power BI reporting tool.

Michigan, United States

 Involved in ingestion, transformation, manipulation, and computation of data using kinesis, SQL, AWS glue and Spark  Migrated data from RDBMS to a NoSQL database, and gives the whole picture for data deployed in various data systems.  Designed and implemented end-to-end data solutions (storage, integration, processing, and visualization) in AWS.  Hands on Experience in migrating datasets and ETL workloads with Python from On-prem to AWS Cloud services. Developed PySpark based pipelines using spark data frame operations to load data to EDL using EMR for jobs execution & AWS S3 as storage layer. Enterprise Data Lake was designed and set up to enable a variety of use cases, covering analytics, processing, storing, and reporting on large amounts of data. Collaborated with the clients and solution architect for maintaining quality data points in source by carrying out the activities such as cleansing, transformation, and maintaining Integrity in a relational environment. Create and setup self-hosted integration runtime on virtual machines to access private networks.  Working on building visuals and dashboards using Power BI reporting tool.  Built Apache Airflow with AWS to analyze multi-stage machine learning processes with Amazon SageMaker tasks. Developed streaming pipelines using Apache Spark with Python.  Designed and developed Security Framework to provide fine grained access to objects in AWS S3 using AWS Lambda Created PL/SQL packages and Database Triggers and developed user procedures and prepared user manuals for the new programs.  To meet specific business requirements wrote UDF’s in Scala and PySpark.  Worked on Informatica tools such as power center, MDM, Repository manager and workflow monitor.

 Worked in AWS environment for development and deployment of Custom Hadoop Applications. Designed and Developed ETL Processes in AWS Glue to migrate Campaign data from external sources like S3, ORC/Parquet/Text Files into AWS Redshift Engaged in solving and supporting real business issues with your Hadoop distributed File systems and Open-Source framework knowledge.  Used AWS EMR to move large data (Big Data) into other platforms such as AWS data stores, Amazon S3 and Amazon Dynamo DB. Developed AWS lambdas using Python & Step functions to orchestrate data pipelines. Responsible for data governance rules and standards to maintain the consistency of the business element names in the different data layers.  Involved in various phases of development analyzed and developed the system going through Agile Scrum methodology.  Performed detailed analysis of business problems and technical environments and use this data in designing the solution and maintaining data architecture.  Built a program with Python and Apache beam and execute it in cloud Data flow to run Data validation between raw source file and big query tables.  Developed Python, PySpark, Bash scripts logs to Transform, and Load data across on premise and cloud platform. Built the data pipelines that will enable faster, better, data-informed decision-making within the business.  Loaded and transformed large sets of structured, semi structured and unstructured data using Hadoop/Big Data concepts.

 Worked on Cloudera distribution and deployed on AWS EC2 Instances. Used Amazon Web Services (AWS) which include EC2, S3, Cloud Front, Elastic File System, RDS, VPC, Direct Connect, Route53, Cloud Watch, Cloud Trail, Cloud Formation, and IAM which allowed automated operations. Developed, Deployed and Troubleshooted the ETL workflow using Hive, Pig, and Sqoop. Developed ETL pipelines to extract, transform, and load data from various sources into Snowflake using Python and SQL. Created external and permanent tables in Snowflake on the AWS data Built the web application by using Python, Django, AWS, PostgreSQL, MySQL, and MongoDB with continuous deployment using AWS. Designed automated testing scripts using Python and SQL to perform comprehensive data validation across distributed computing environments. Built Daily loads to copy from Salesforce objects to Dim/Fact tables in AWS.

Newport, Nj

 Implemented Spark Scripts using Scala, and Spark SQL to access hive tables into Spark for faster processing of data Developed various automated scripts for DI (DataIngestion) and DL(Data Loading) using Python map reduction. Built and maintained ETL pipelines to ingest, transform, and load data from various sources into Databricks using Apache Spark and Python. Having experience with AWS Connect to sync Salesforce data. Exposure to callouts from salesforce to call external processes on AWS. Developed and maintained automated monitoring and alerting systems to proactively identify and address potential issues, ensuring high availability and reliability of Cosmos DB clusters. Collaborated closely with cross-functional teams to design and implement data models, migration strategies, and disaster recovery plans utilizing Cosmos DB

Santa Clara, Ca

 Responsibility mainly included collecting and organizing specific ad-hoc and annual third-party data to analyze from different teams. Keep track and update operational and financial data with statistical report weekly, monthly and yearly. Perform statistical analysis and research on meter date data from the annual report and compare the company’s standpoint with other companies. Worked as Big Data Engineer in the team dealing with platform issues. Providing data analysis for the team as well as developing enhancements

Mumbai, Maharashtra, India

 Data migration (import & export - BCP) from Text to SQL server.  Developed SSIS package to get data from different sources, consolidate and merge into one single source.  Created datasets for reports using T-SQL and stored procedures.