Sagarika Kandrapu Email & Phone Number

Decatur, Ga, Us

St Louis, Missouri, United States

Working as Senior Data Engineer, I am responsible for designing a comprehensive approach for business requirement collection, aligning with project scope and SDLC methodology. Developed Apache Spark applications in Java for efficient data processing from various streaming sources. Implemented a Spark job to index data into Elasticsearch from external Hive tables in HDFS.Responsible for developing data pipelines on Amazon AWS, extracting data from weblogs, and storing it in HDFS. Utilized Sqoop extensively for importing metadata from Oracle. Created and configured AWS Lambda functions in Python and Java for event-driven processing. Developed Python scripts for seamless data ingestion from diverse sources into the data pipeline.Developed MapReduce programs and Hive UDFs in Java. Migrated MapReduce jobs to Spark for enhanced performance. Implemented data quality measures in Talend ETL tool. Introduced security measures in PostgreSQL, including user roles, permissions, and encryption. Designed visually appealing reports and dashboards in Power BI and Tableau for actionable insights.Implemented comprehensive monitoring solutions using Python, ensuring proactive issue identification in the data pipeline. Worked on designing MapReduce and Yarn flows, performance tuning, and debugging. Installed applications on AWS EC2 instances, configured storage on S3 buckets, and stored data like HDFS. Leveraged AWS Data Pipeline for scheduling Amazon EMR clusters and processing web server logs in S3.Developed end-to-end data processing pipelines using Spark, Scala, and Apache Kafka. Worked on Spark Databricks cluster monitoring and troubleshooting. Utilized NoSQL databases including HBase, Cassandra, and MongoDB, with hands-on experience in application development. Demonstrated proficiency in DevOps practices and tools, contributing to continuous improvement.

Mumbai, Maharashtra, India

As a proficient Senior Data Engineer at UBS, I bring a wealth of experience in designing and implementing robust data solutions that span across diverse technologies and platforms. My expertise includes working with NoSQL databases like HBase, where I've successfully created HBase tables to handle large sets of semi-structured data from various sources.One of my key achievements is designing and implementing ETL processes using Informatica to seamlessly load data from flat files and Excel files into the target Oracle Data Warehouse database. This involves understanding and translating complex business requirements into practical and efficient data workflows. My ability to optimize data loading operations is demonstrated by the Python utility I developed to validate HDFS tables against source tables and to implement data loading into S3 buckets using PySpark and AWS Glue.In the cloud computing realm, I've played a crucial role in configuring AWS Identity and Access Management (IAM) Groups and Users, enhancing login authentication for improved security and access control. My experience extends to working with Snowflake, where I've applied best practices for data warehousing, including clustering, partitioning, and indexing, to enhance query performance and reduce latency. In the realm of visualization, I design and develop interactive dashboards using PowerBI. A key aspect of my role involves the development of Python wrapper scripts to extract specific date ranges using Sqoop, incorporating custom properties required for workflow execution. I've also been responsible for filtering data stored in S3 buckets using Elasticsearch and loading it into Hive external tables, showcasing my versatility in working with diverse data storage and retrieval mechanisms.

Hyderabad, Telangana, India

Working at Tech Mahindra as a Data Engineer, I bring a rich background in designing and implementing robust data solutions across diverse platforms, specializing in Hadoop and Azure ecosystems. Proficient in installing and configuring Hadoop components such as Hive, Pig, Sqoop, Flume, and Oozie, I have a proven track record of recreating application logic and functionality in Azure Data Lake, Data Factory, SQL Database, and SQL Data Warehouse environments.I've successfully imported and exported data from Snowflake, Oracle, and DB2 into HDFS and HIVE using Sqoop, enabling detailed analysis, visualization, and report generation. Utilizing Apache Kafka, I've populated HDFS and PostgreSQL with extensive datasets, employing PowerBI to develop data analysis prototypes and visualize reports.In the realm of business intelligence, I have built impactful reports and dashboards on Snowflake databases using Tableau. Engaging in large-scale BI solutions on Azure, I've utilized Azure Data Platform services like Azure Data Lake, Data Factory, Data Lake Analytics, Stream Analytics, Azure SQL DW, HDInsight/Databricks, and NoSQL DB.In the realm of ETL, I've developed robust pipelines using Python, extracting data from diverse sources, transforming it according to business requirements, and loading it into target data warehouses. Leveraging tools like Pig Latin for data preparation and Azure Data Factory for ETL automation, I've orchestrated the movement and transformation of data across various sources.My expertise includes UNIX scripting for defining use case workflows and automating jobs, migrating objects from Teradata to Snowflake, and developing end-to-end ETL processes in Azure. I've also harnessed Spark scripts using Python on Azure HDInsight for data aggregation and validation, with a focus on performance optimization over MR jobs.

Hyderabad, Telangana, India

As a Data Analyst, I bring a diverse skill set and a wealth of experience in designing and implementing robust data solutions that bridge on-premises and cloud environments. My expertise encompasses the seamless integration of data from various sources, including OLEDB, flat files, and AWS services such as Amazon S3 and Amazon RDS, into SQL Server databases using SSIS packages.A key aspect of my role involves documenting Python code, data models, and ETL processes to facilitate knowledge sharing and collaboration within cross-functional teams. This commitment to clear documentation ensures compatibility with both on-premises SQL Server and AWS services, fostering efficient data management and future maintenance.In the realm of big data, I've demonstrated proficiency in Hadoop, Pig, and Hive. Expanding my skills to AWS, I've created Hive tables and optimized job flows within Amazon EMR for efficient data processing in a cloud-based big data environment. In the Hadoop ecosystem, I've written MapReduce jobs using Pig Latin, actively participating in ETL, data integration, and migration processes. Importing and exporting data into HDFS and Hive using Sqoop has been a routine aspect of my responsibilities. I've also been involved in creating Hive tables, loading data, and writing Hive queries that run internally in a MapReduce way. Automation is a cornerstone of my approach, and I've created batch jobs and configuration files to establish automated processes using SSIS. This not only ensures the efficiency of data workflows but also contributes to the organization's overall data management strategy.In addition to my data integration and visualization skills, I've deployed and scheduled reports using SSRS to generate daily, weekly, monthly, and quarterly reports. This ensures that decision-makers have access to timely and relevant information to drive strategic initiatives.

Masters, Management Information Systems And Services, A

Activities and Societies: Participated in International/Global Initiatives to enhance my cultural awareness and work with colleagues.

Bachelor'S Degree, Civil Engineering, A

Activities and Societies: I actively engaged in various extracurricular activities and societies, enhancing both my leadership.