Deepika T Email & Phone Number

• Designed and set up an Enterprise Data Lake to support various use cases, including the integration of healthcare-specific data such as Electronic Health Records (EHR) and clinical data for analytics, processing, storing, and reporting of patient information.• Maintained quality reference data by performing cleaning, transformation, and ensuring data integrity in a relational environment.• Developed a Security Framework to provide fine-grained access to objects in AWS S3 using AWS Lambda and DynamoDB, ensuring adherence to healthcare data privacy regulations.• Set up Kerberos authentication principals to establish secure network communication on the Hadoop cluster and tested HDFS, Hive, Pig, and MapReduce for new users.• Performed end-to-end architecture and implementation assessment of various AWS services like Amazon EMR, Redshift, and S3, leveraging Hadoop ecosystems for big data processing.• Implemented ML algorithms to analyze patient data and predict health outcomes, utilizing Kinesis Firehose and the S3 data lake for real-time data ingestion and storage.• Used AWS EMR and Apache Spark to transform and move large amounts of data into and out of other AWS data stores and databases, applying Spark SQL for efficient data processing and querying.• Created Lambda functions with Boto3 to deregister unused AMIs across application regions, optimizing EC2 resource usage and reducing costs.• Developed a reusable framework to automate ETL processes from RDBMS systems to the Data Lake, using Spark Data Sources and Hive data objects for streamlined data integration.• Conducted data blending and preparation using Alteryx and SQL for Tableau consumption, and published data sources to the Tableau server for visualization.• Integrated Apache Airflow with AWS to orchestrate and monitor multi-stage ML workflows, with tasks running on Amazon SageMaker, and used AWS CloudWatch for comprehensive monitoring and logging of pipeline performance and system health.

• Implemented a scalable and efficient real-time fraud detection system to enhance security and prevent financial losses for a major banking institution, focusing on protecting customer accounts and transaction integrity.• Utilized GCP Pub/Sub as a messaging backbone for real-time transaction data ingestion. Implemented Dataflow pipelines for stream processing to enrich and analyze incoming transaction data.• Deployed machine learning models on AI Platform for continuous evaluation of transactional patterns and fraud detection. Continuous streaming of transaction data from multiple banking channels into Pub/Sub.• Real-time data processing and transformation using Dataflow pipelines to derive actionable insights for banking operations.• Integration of custom ML models for fraud detection and anomaly detection using TensorFlow/Keras. Automated alerting and reporting mechanisms for suspicious activities, directly notifying banking security teams.• Leveraged Kubernetes Engine for container orchestration to ensure scalability and fault tolerance.• Designed auto-scaling policies to handle varying transaction volumes and computational demands effectively.• Achieved low-latency processing to enable real-time decision-making on banking transactions.• Implemented encryption mechanisms for data at rest and in transit using GCP's security features.• Ensured compliance with industry standards (e.g., PCI DSS) for handling sensitive financial data.• Configured Stackdriver for monitoring system health, performance metrics, and logging of transaction activities.• Set up alerts for critical events and anomalies detected during real-time processing, including integration with the bank's existing risk management systems and regulatory reporting frameworks.

• Designed and implemented a real-time customer insights and fraud detection system for a leading bank using Google Cloud Platform (GCP).• Utilized Google Cloud Pub/Sub to capture and ingest real-time transaction data, including deposits, withdrawals, and transfers from various banking channels.• Implemented Apache Beam pipelines on Google Cloud Dataflow to process and transform streaming data in real-time, ensuring data was cleansed, enriched, and aggregated for fraud analysis.• Stored processed data in Google BigQuery for scalable, high-performance querying and analysis, and utilized Google Cloud Storage for cost-effective storage of raw and processed financial data.• Developed real-time analytics dashboards using Google Data Studio, powered by BigQuery, to provide actionable insights into customer transaction behaviors and detect potentially fraudulent activities.• Built and deployed machine learning models using Google AI Platform to predict and detect fraudulent transactions, ensuring the security and integrity of customer accounts.• Created interactive dashboards and reports using Google Data Studio, enabling stakeholders to visualize key performance metrics related to transaction volumes and fraud detection.• Implemented Stackdriver for real-time monitoring and logging of data pipelines and application performance, ensuring high availability and rapid troubleshooting of issues.• Ensured data security and compliance with financial industry standards by implementing IAM roles and policies, VPC Service Controls, and encryption for data at rest and in transit.• Set up continuous integration and continuous deployment (CI/CD) pipelines using Google Cloud

Hyderabad, Telangana, India

• Created and deployed custom PySpark scripts to preprocess and transform data, mapping it to tables within the CIF data warehouse.• Implemented AWS Glue ETL jobs to automate the extraction, transformation, and loading of data from diverse sources into S3 and Redshift.• Developed and managed shell scripts for Sqoop jobs to perform periodic incremental imports of structured data from various RDBMS to AWS S3.• Utilized Kafka to ingest real-time website traffic data into HDFS, ensuring seamless data flow and availability for analysis.• Translated complex code into Informatica logic, creating ETL design documents and developing mappings using transformations like Source Qualifier, Expression, and Lookup.• Enhanced ETL pipeline performance by tuning SQL queries and implementing data partitioning techniques, reducing processing time and resource usage.• Scheduled and automated ETL processes to extract, transform, and load data into the data warehouse based on business requirements using scheduling tools like AWS Lambda.• Managed data extraction, transformation, and loading into landing, staging, integration, and semantic layers of the data warehouse using Informatica and Teradata.• Created and updated independent data marts from the existing data warehouse on a bi-weekly basis to meet specific application requirements.• Reduced billing costs by transitioning from Redshift storage to Hive tables for unpaid services and improving query performance through partitioning and bucketing. • Utilized various tests on sources, targets, and transformations to optimize the performance of ETL mappings, ensuring efficient data processing and storage.Environment: PySpark, Apache Spark, CIF Data Warehouse, AWS Glue, AWS S3, Amazon Redshift, Apache Sqoop, Shell Scripting, MySQL, PostgreSQL, Apache Kafka, HDFS, SQL, Teradata, AWS Lambda, Apache Airflow, Snowflake, Informatica, JIRA, Confluence, Slack.