Designed and implemented data ingestion pipelines using Azure DataFactory, ensuring efficient data flow from multiple sources to storage solutions. Utilized Azure Databricks for data processing and transformation tasks, enhancing the performance of data analytics operations. Managed Azure DataLake storage systems to optimize data storage and retrieval processes, supporting large-scale data operations.

• Developed ETL jobs to migrate data from Oracle, MySQL, MS SQL Server into HDFS, optimizing data processing speed by 25%.• Utilized Python scripts to automate data loading from multiple sources into a centralized system, setting up database clusters and backup processes.• Implemented data pipelines using Apache Spark and Hive to ingest, store, and process large datasets efficiently.

In my role as a Jr. Data Engineer at Annamraju Designs & Technologies in India, I developed machine learning algorithms using Python for predictive analytics, designed scripts for analyzing large datasets on AWS EC2 instances, and implemented a Security Framework for fine-grained access control in AWS S3 using Lambda and DynamoDB.

Managed and analyzed data using MS SQL Server 2008, both client and server versions, tosupport software development projects and operational reporting.Designed and optimized cubes and tabular models for efficient querying and analysis of largedatasets.Designed and implemented dimensional data models for data warehouse solutions, optimizingdata structures for efficient reporting and analysis.Implemented calculated measures, KPIs (Key Performance Indicators), and hierarchies to enhanceanalytical capabilities and usability.Developed complex SQL queries and scripts to extract, transform, and load data, enhancingdata-driven decision-making processes.Automated data cleansing and validation tasks within SSIS packages, ensuring data accuracy andintegrity

Developed user interfaces utilizing HTML5, CSS3, JavaScript, JQuery, AJAX, AngularJS. Provided technical support for applications that use Microsoft SQL Server databases. Optimized web application performance by making database queries more efficient. Deployed web applications on various cloud platforms such as Azure and AWS. Participated in the full software development life cycle (SDLC) from requirements gathering through system implementation. Generated technical documentation for developed software components. Implemented agile development practices across multiple projects, ensuring timely delivery of high-quality solutions within budget constraints.