Revathi B Email & Phone Number

Dallas, Tx, Us

• Creating Batch Pipelines in Azure Data Factory (ADF) by configuring Linked Services/Integration Runtime to Extract, Transform and load data from different sources into Azure Data Lake.• Creating real-time streaming pipelines using Azure IOT Hub, Azure EventHub, Azure Stream Analytics and Spark structured streaming. • Implemented security controls and data governance practices within Snowflake. Configured role-based access controls (RBAC), granted and revoked privileges, and enforced data encryption mechanisms to protect sensitive data.• Designed, developed, and deployed SSIS packages to automate data extraction, transformation, and loading processes, resulting in a 20% reduction in data processing time.• Provisioning Hadoop and Spark Clusters on the Azure Databricks, to run the spark jobs for processing the data and provide the datasets to the data scientists, data visualization teams.• Planning and Developing roadmaps and deliverables to advance the migration of existing solutions on-premises systems/applications to Azure cloud and proposing architectures considering cost/spend in Azure and develop recommendations to right-size data infrastructure.• Assessed and validated data quality using Ab Initio features. It enables users to define data quality rules and implement data profiling to identify data issues, such as inconsistencies, duplicates, or missing values. • Created data lineage reports in Power BI which helps in tracking the movement of data from source to destination.• Developed email notification alerts framework using Azure Logic Apps to send the emails alerts to business teams and orchestrated azure services using Azure Logic Apps.• Using G-cloud function with Python to load data into Big Query for on arrival csv files in GCS bucket.• Involved in analyzing the different cluster types like Interactive Cluster, Job Cluster and Pool clusters and decreased the cluster usage by setting up the appropriate cluster type to the scheduled jobs.

Texas, United States

• Developed Spark Applications by using Python and Implemented Apache Spark data processing Project to handle data from various RDBMS and Streaming sources.• Responsible for building scalable distributed data solutions using Hadoop.• Developed interactive and visually appealing dashboards and reports within Tableau, utilizing Snowflake as the underlying data source. Implemented intuitive filters, parameters, and visualizations to facilitate data exploration and storytelling.• Build data pipelines using airflow in GCP for ETL related jobs using different airflow operators.• Used Apache airflow in GCP composer environment to build data pipelines and used various airflow operators like bash operator, Hadoop operators and python callable and branching operators.• Conducted performance tuning and optimization activities to enhance data processing and query performance within Azure Data Lake and Snowflake. Utilized partitioning, indexing, and caching strategies to improve data access and retrieval speed.• Built NiFi dataflow to consume data from Kafka, make transformations on data, place in HDFS and exposed port to run spark streaming job.• Experience building cloud-native applications and supporting technologies practices including AWS, Docker, CI/CD and microservices.• Worked on cloud deployments using maven, docker and Jenkins.• Used Maven extensively for building jar files of MapReduce programs and deployed to clusters.• Migrated an existing on-premises application to Aws using various services.• Design and Develop ETL Processes in AWS Glue to migrate Campaign data from external sources like S3, ORC/Parquet/Text Files into S3 AWS Redshift.• Developed file cleaners using Python libraries and utilized Python libraries Boto3, Pandas.• Used Amazon EMR for map reduction jobs and tested locally using Jenkins.• Created Azure SQL database, performed monitoring and restoring of Azure SQL database. Performed migration of Microsoft SQL server to Azure SQL database.

Massachusetts, United States

Created Function as a service is a category of cloud computing services that provides a platform allowing customers to develop, run, and manage application functionalities without the complexity of building and maintaining the infrastructure typically associated with developing.Implemented a 'serverless' architecture using API Gateway, Lambda, and DynamoDB and deployed AWS Lambda code from Amazon S3 buckets. Created ECS container, Network Load Balancer, API Gateway and other AWS services using AWS cloud formation templates in Dev, QA and prod environments.Created Docker container and used docker images to deploy the machine learning models in different environments and automated the process using Jenkins.Created a Lambda deployment function and configured it to receive events from your S3 bucket.Designed the data models to be used in data intensive AWS Lambda applications which are aimed to do complex analysis creating analytical reports for end-to-end traceability, lineage, definition of Key Business elements from Aurora.Writing code that optimizes performance of AWS services used by application teams and provide Code-level application security for clients (IAM roles, credentials, encryption, etc.)Design and Develop ETL Processes in AWS Glue to migrate Campaign data from external sources like S3, ORC/Parquet/Text Files into S3 AWS Redshift.Developed file cleaners using Python libraries and utilized Python libraries Boto3, PandasUsed Amazon EMR for map reduction jobs and test locally using Jenkins.Data Extraction, aggregations, and consolidation of Adobe data within AWS Glue using PySpark.

Hyderabad, Telangana, India

Involved in Sprint planning sessions and participated in the daily Agile SCRUM meetings.Worked on the project from gathering requirements to developing the entire application.Worked on the Anaconda Python Environment. Created, activated, and programmed in the Anaconda environment. Wrote programs for performance calculations using NumPy and SQLAlchemy.Used python modules of urllib, urllib2, Requests for web crawling. Used all these ML techniques: clustering, regression, classification, graphical models.Used with other packages such as Beautiful Soup for data parsing.Involved in development of Web Services using SOAP for sending and getting data from the external interface in the XML format. Used with other packages such as Beautiful Soup for data parsing.Worked on development of SQL and stored procedures on MYSQL.Analyzed the code completely and have reduced the code redundancy to the optimal level.Designed and built a text classification application using different text classification modelsDesigned MySQL table schemas and implemented stored procedures to extract and store customer purchase and session data.Queried data from MySQL database and validated and detected inconsistent data using Python packages Pandas and Numpy.Actively involved in designing A/B tests, defining metrics to validate new user interface features, calculating sample size and checking statistical assumptions for tests.