Sai Ganesh Email and Phone Number

• Enhanced Data Processing Efficiency: Managed intricate ETL systems utilizing Python, SQL, and AWS services such as Step Functions, EMR, Lambda, S3, and Redshift, achieving a 30% increase in data processing efficiency. • Rapid Issue Resolution: Engineered comprehensive platform issue resolutions using MongoDB, AWS Redshift, and Sumo Logic; ensuring stable system performance slashed downtime by 40%, and fortified system dependability.• Executed Scalable Data Warehousing Solution: Engineered a high-performance data warehousing solution using AWS Glue and Step Functions, enhancing data storage and retrieval processes resulting in a 25% decrease in data processing time.• Managed Complex Data Migration Projects: Led the successful execution of data migration projects, ensuring smooth integration of new data sources with minimal disruption; expanded data analytics capabilities by 60%, enabling enhanced insights.• Utilized MongoDB and implemented SCD Type 2 in Snowflake for personalized healthcare communications and historical tracking and leveraged CDC and transformed OLTP data into OLAP format for real-time updates and efficient data analysis• Implemented Airflow for Workflow Orchestration: Utilized Airflow to orchestrate complex data workflows, ensuring efficient execution and monitoring of data pipelines. Developed custom Airflow operators using Python to handle specific data processing tasks.• Developed and optimized complex SQL queries and Scala scripts for data transformation and analysis, collaborating with data scientists and business stakeholders in an Agile environment to deliver actionable insights and drive data-driven strategies.• Implemented CI/CD pipelines using GitHub and Jenkins, automating the build, test, and deployment processes for data pipeline code, and ensuring code quality and reliability through rigorous testing and code reviews.

•Worked on a project that uses AWS lambda functions, DynamoDB, S3 Bucket, SQS, Secrets Manager, IAM, and Cloud Watch to automate the process of synchronizing all files from storage to AWS S3 bucket on a nightly basis.•Python was used to extract, transform, and aggregate data from multiple source systems for business use cases.•Sqoop was designed and implemented for an incremental job to read data from DB2 and load it into Hive tables, and it was linked to Tableau to generate interactive reports using Hive server2.•ETL/ELT processes were designed and developed to handle data migration from multiple business units and sources, including Oracle, Postgres, and Informix.•Enterprise Data Lake was designed and set up to support a variety of use cases, including analytics, processing, storing, and reporting on large amounts of rapidly changing data.•Created an API for using AWS Lambda to manage servers and run code in AWS.•ADF Pipelines were created to load data from on-premises storage to Azure cloud storage and databases.•On the AWS cloud, S3 buckets were used for file storage and backup.•Used cloud shell SDK in GCP to configure the services Data Proc, Storage, BigQuery.•Created end-to-end data processing pipelines that begin with data receiving via distributed messaging systems Kafka and end with data persisting into Cassandra.•Data Integration Processes were demonstrated/presented to business stakeholders, and work was done on developing standards and processes for integration initiatives.•Pipelines and analytical workloads were implemented using big data technologies such as Hadoop, Spark, Hive, and HDFS.•Integrated Tableau with a Hadoop data source to create a dashboard that provides various insights into the organization's sales.•Importing data to Data Lake Raw Zone using Sqoop from various source systems such as mainframes, Oracle, MySQL, DB2, and so on.

• Designed, developed, and implemented Hadoop applications using Sqoop, Spark, Hive and MapReduce• Responsible for loading structured, unstructured, and semi-structured data into Hadoop by creating static and dynamic partitions.• Strong knowledge of various Data warehousing methodologies and Data modeling concepts.• Developed Hadoop streaming Map/Reduce works using Python.• Improved query performance and space utilization by 40% by creating Hive partitioned tables using Parquet & Avro format.• Responsibilities include Database Design and Creation of User Database. Moving ETL pipelines from SQL server to Hadoop Environment.• Extensively worked on NGAP for managing clusters and accessing them.• Scheduling mechanism in Airflow for developing jobs for scheduling DAG’s.• Developed complex SQL queries and managed roles in Snowflake.• Worked on ingestion data from multiple relational databases including Snowflake into S3 buckets.• Designed, developed and implemented Hadoop applications using Sqoop, Spark, Hive and MapReduce• Developed Spark applications using Pyspark and Spark-SQL for data extraction, transformation, and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.• Implemented a CI/CD pipeline using Jenkins, Airflow for Containers from Docker, and Kubernetes.• Used Python scripts, Spark Applications for ETL Operations to create data flow pipelines and involved in transforming data from legacy tables to Hive, HBase tables, and S3 buckets for handoff to business and Data scientists to create analytics over the data.• Support current and new services that leverage AWS cloud computing architecture including EMR S3, and other managed service offerings.• Implemented data quality checks using Spark Streaming and arranged bad and passable flags on the data.• Used advanced SQL methods to code, test, debug, and document complex database queries.

• Proficient in writing efficient SQL queries, developing stored procedures, and optimizing Snowflake SQL for enhanced performance and ELT/ETL processes.• Expertise in Snowflake data modeling, user management, and Role-Based Access Control (RBAC).• Developed and scheduled automated processes, including creating jobs, using SQL Mail Agent, and setting up alerts and DTS/SSIS packages. • Provided technical guidance and best practices for access controls, data security, and Snowflake resource optimization.• Implemented and optimized ETL processes using SQL, PL/SQL, and stored procedures for data integrity and quality.• Developed automated processes and managed databases, ensuring optimized performance and reliable deployments.• Documented technical and management processes and tracked and resolved defects for effective issue management.• Maintained users, roles, and permissions within the SQL Server.

• Demonstrated experience in Agile Scrum methodology, actively participating in daily stand-up meetings and collaborating with cross-functional teams for successful project delivery.• Proficient in utilizing Visual SourceSafe for version control and code management in Visual Studio 2010, coupled with adept usage of project tracking tools like Trello for monitoring and facilitating project progress.• Expertise in designing and optimizing data warehouse solutions, leveraging relational databases (Oracle, SQL Server) to efficiently store and process large-scale datasets for analytics and reporting.• Created interactive reports in Power BI with drill-through and drill-down capabilities, drop-down menus, sorting, and subtotals, enabling comprehensive data exploration.• Leveraged the data warehouse to develop a Data Mart, feeding downstream reports and designing a user access tool for ad-hoc reporting and query analysis, empowering self-service data insights.• Deployed SSIS packages and orchestrated efficient job scheduling for streamlined data integration processes.• Experienced in architecting and constructing Cubes and Dimensions with diverse data sources and architectures, utilizing MDX scripting to enhance Business Intelligence capabilities.• Developed SSIS jobs for report automation and cube refresh packages.• Proficient in deploying SSIS packages to production and utilizing package configurations for environment independence.• Experienced with SQL Server Reporting Services (SSRS) for authoring, managing, and delivering paper-based and interactive web-based reports.• Developed stored procedures and triggers to ensure consistent data entry into the database.