Divya K Email & Phone Number

United States

Seattle, Washington, United States

• GCP MIGRATION: Led the migration of IKEA's digital infrastructure to GCP ensuring seamless integration with minimal disruption to business operations.• ETL PIPELINES: Architected and optimized ETL pipelines using "Google DataFlow" and "Cloud Composer" on GCP to streamline the data integration from transactional databases and CRM systems into a centralized data warehouse using "Informatica" and "Apache Kafka".• Architecting the data pipelines: Utilized "Google Cloud Storage" for storing raw data from various customer touchpoints, such as in-store purchases, online transactions, and customer feedback and "Google Pub/Sub" for real time data ingestion.• Enhancing data Quality using "Google Dataflow" and "Google cloud DLP (Data Loss Prevention)" to protect sensitive data and quality of data through "Big Query".• Created reports and dashboards using "Data Studio" and "Looker" to visualize customer insights, key performance indicators and exploration metrics.• Utilized Snowflake architecture to separate storage and compute for independent scaling based on resource demand for cost efficiency. • Managed “Kubernetes” clusters directly from “Cloud Shell” using built-in tool for deploying and monitoring containerized applications on GKE.• Handled data extraction, transformation, and loading (ETL) using "Ab initio", integrating with "Kafka" for custom processing and monitoring tasks.

Boston, Massachusetts, United States

• Designed and developed ETL pipelines using AWS Glue, enabling efficient data ingestion and transformation into Amazon Redshift.• Implemented real-time data processing with Amazon Kinesis and Apache Kafka, handling high-velocity data streams from network sensors.• Leveraged Amazon EMR and Apache Spark for large-scale data processing and advanced analytics.• Automated AWS infrastructure provisioning with Terraform, ensuring consistent and repeatable deployments.• Created interactive dashboards with Amazon QuickSight and Tableau, providing real-time insights into network performance.• Led the development of the Advanced Fiber Network Data Analytics Platform (AFNDAP), optimizing fiber network performance through big data analytics.

Boston, Massachusetts, United States

• Designed and implemented ingestion pipelines using Azure Data Factory, Azure EventHub, and Snowflake to seamlessly ingest Adobe Experience Platform (AEP) data into centralized repositories.• Configured real-time Adobe data collection using Azure EventHub and Kafka, enabling high-throughput ingestion and processing of streaming data for customer insights.• Engineered scalable Data Lakes in Azure using Azure Blob Storage and Data Lake Gen2 to store and process large-scale Adobe Experience Data Model (XDM) datasets for advanced analytics.• Developed integration workflows to synchronize Adobe Data Usage (DU) data with Snowflake through Azure, leveraging Azure Data Factory and Data Pipelines to ensure consistent data flow.• Engineered Customer Journey Analytics (CJA) connectors in Azure to enable real-time data integration between Adobe Experience Platform and Azure-based data lakes, improving analytics.• Built custom API and connector-based data integrations between Adobe platforms and non-Adobe systems like Salesforce Marketing Cloud (SFMC), ensuring seamless data synchronization and enriched customer profiles.• Utilized Azure Functions and Azure Logic Apps to optimize the ETL process for AEP data ingestion into Azure Synapse Analytics, improving data pipeline efficiency and reducing processing time.• Integrated Adobe data with non-Adobe platforms using Azure resources (e.g., Azure EventHub, Logic Apps), facilitating cross-platform data sharing and enhancing overall data flow efficiency.• Implemented secure data ingestion and storage protocols using Azure Security Center, Azure Key Vault, and Azure Policy to ensure Adobe data compliance with enterprise and regulatory standards.• Developed Azure-based solutions for cross-cloud Adobe data integration, leveraging Azure Data Factory to connect with GCP and AWS environments, ensuring unified data pipelines and consolidated reporting.

Illinois, United States

• Designed and developed ETL pipelines using AWS Glue, enabling efficient data ingestion and transformation into Amazon Redshift.• Implemented real-time data processing with Amazon Kinesis and Apache Kafka, handling high-velocity data streams from network sensors.• Leveraged Amazon EMR and Apache Spark for large-scale data processing and advanced analytics.• Automated AWS infrastructure provisioning with Terraform, ensuring consistent and repeatable deployments.• Created interactive dashboards with Amazon QuickSight and Tableau, providing real-time insights into network performance.• Led the development of the Advanced Fiber Network Data Analytics Platform (AFNDAP), optimizing fiber network performance through big data analytics.

Bengaluru, Karnataka, India

• Collaborated with data engineers to integrate data from various sources, including in-store sales, online transactions, and CRM Systems, and external financial data providers, ensuring data consistency and quality across the platform. Monitored and logged data ensuring high availability and rapid incident response.• Developed and maintained interactive dashboards and reports using Tableau and Power BI to provide real-time insights into customer behavior, enabling stakeholders to monitor key performance indicators (KPIs) such as customer satisfaction, purchase patterns, and feedback trends.• Ensured data quality by developing and executing data validation and cleansing processes, addressing data inconsistencies and inaccuracies proactively.• Utilized advanced SQL queries to extract and manipulate data from databases such as MySQL, PostgreSQL, and Snowflake, supporting in-depth analysis and reporting.• Automated data workflows using Python and SQL scripts to streamline data preparation and reporting processes, reducing manual effort and improving efficiency.• Developed real-time alerting and notification systems within BI tools to keep stakeholders informed of critical events and trends in customer behavior.• Automated platform health monitoring scripts with the Qlikview team, reducing manual intervention by 30%.• Utilized ETL (Extract, Transform, and Load) operations, facilitating the seamless integration of data from various sources into a centralized data warehouse. Supported the development and deployment of CI/CD pipelines using AWS Code Pipeline and Jenkins, ensuring efficient and reliable code deployment across environments.• For server less data processing, allowing real-time data ingestion and transformation. For creating interactive dashboards and reports, providing real-time insights into key performance indicators (KPIs) and operational metrics.