Albertsons Inc| Pleasanton, CA | Nov 2020 – till dateSr. Data Engineer/Architect• Collaborated closely with data scientists, business analysts, and business stakeholders to understand business requirements and deliver actionable insights.• Implemented data ingestion pipelines using Azure Data Factory to extract data from disparate sources of structured/unstructured data from Kafka stream, transform it, and load it into Snowflake Data Warehouse hosted in Azure.• Developed scalable data pipelines using Azure Databricks, python, and SQL to transform large datasets with daily volume of 10TB in Snowflake warehouse on the Azure cloud. • Implemented and maintained to ensure data privacy, security, and classification of PII/PCI data. Also maintained pharmacy data aligned with HIPA regulations.• Developed relational data models (Conceptual, Logical and Physical) for Pharmacy/HealthCare and Merchandise Business Unit in Azure Cloud SQL database.• Implemented governance processes to ensure data security and integrity, which led to a 98% accuracy rate.• Implemented auto-monitoring system in the data pipelines to proactively address data quality issues, which improved data quality and completeness by 36%.• Collaborated with data science team to help implement complex algorithms that provide unique insights and ready to consume by the Analytics team.

• Gathered Business data by conducting the JAD sessions with the Business Stakeholders.• Designed and implemented data pipelines using python and SQL in Dataflow to ingest batch (80%) and near real-time (20%) data from enterprise Kafka into BigQuery.• Designed and deployed dimensional schemas involving 150 tables and 8500+ attributes in BigQuery DW on the GCP.• Created and managed DAGs in Apache Airflow, including task dependencies and monitoring with as low as 0.2% of downtime.• Designed data models for efficiency storage that reduced duplication by 21%• Collaborated with cross-functional teams to design, deploy, and adapt scalable data solutions, resulting in improved data availability and quality.• Worked with product owners for end-to-end documentations of ETL workflows. Prepared presentations to educate Business Analytics groups on data transformations.

• Developed Target state architecture for Alation BI Analytics data mart.• Reviewed validated the data management components like metadata management, data governance, data warehousing, and data catalog.• Developed Conceptual/Logical/physical data models for data mart.• Migrated report/queries to the newly developed model/database.• Developed technical documentation like BRD, data dictionary and data catalog.• Validated the ETL Architecture, wrote quires to load the star schema.• Developed and implemented data governance policy and procedures.

• Migrated on-premises data Warehouse to BigQuery in Google Cloud Platform replacing existing Informatica workflows by python DAGS and Airflow running on Google Cloud Composer.• Created and maintained Data Pipelines using DAGs written in python using various Airflow libraries, ETL plugins and operators for Bigquery.• Led initiative to move ETL pipelines running in on-prem Hadoop clusters to Cloud native ETL in Airflow. Deprecates existing platform with ~220 nodes clusters, 8K tables in 50+ schemas < 90 days with new codes and pipelines.• Led the performance tuning activities of the critical workflows included design changes, splitting mappings into multiple mappings, changing cache values, having multiple sessions, using parallel partitions at different stages of the mappings at session level and using only fields needed at the target level from the source and implement the equivalent strategies while writing the new pipelines.• Supported multiple business teams to understand specifications of various fields, prepare mapping of the fields to be stored into the Cloud databases for internal/external reporting.

• Developed the ODS data architecture for Marketing and Sales subject area.• Gathered the Business data by conducting the JAD sessions with the Business Stake Holders.• Developed a relational model sourcing from multiple sources (Conceptual, Logical, Physical models)• Reviewed, conformed the dimensional modeling standards, naming conventions, modeling methodologies.• Identified confirmed dimensions and facts across multiple subject areas.• Created the dimensional model.• Created ELDM to Dim mapping document for ETL loading and Lead the ETL team.

Developed Enterprise data warehouse architecture(Hybrid), combined with master data architecture, governance framework. Designed, Implemented the security data model for EDW including masking, encryption and Pseudonymization techniques.  Created conceptual, logical and physical data models in 3rd normal form to support the EDW. Developed star and snowflake models to support the business intelligence and dashboard requirements. Developed data lake architecture with Hadoop storage and spark processing engine Developed lambda architecture for batch and streaming data/reporting

Developed data mart architecture with metadata architecture. Completed STAR schema/snowflake (conceptual, Logical and Physical data models (using the tool Erwin 9.X)) for the Data Mart, Fact less- Fact table.

Designed BI architecture for multiple business groups, working collaboratively to understand needs and builds solutions. Created star and snowflake data models that supported the dashboard analytical reports, and operational reports. Developed architecture for in memory technologies like SAP HANA, HBase, and MongoDB. Provided OLAP services and intelligent cube design for advanced analytics and Trend analysis reporting.

Developed reports for various departments in the company, mapping the reporting requirements from business terms into Business Objects. Developed business object universe and full thin client reports for finance, marketing, and sales. Extracted Level 0 data from Essbase 6.5.1 cube to star schema database in Oracle 8i, and developed Dashboard reports in Business Objects for billing, booking, backlog, sales unit analysis and inventory.  Maintained, monitored, and optimized Scripts for extraction of ROW and EMEA data into data warehouse merged and populated into Essbase Cubes, scheduling the Essbase Load rules and calculation using an in-house developed tool.

Developed an integrated data mart to collect the various information from the legacy systems for producing the analytical reports and Ahom query, which included developing ETL workflows to extract the source systems data from the distributed environment and load into the data mart database.  Designed the Star-Schema based warehouse after understanding the business logic.  Designed and developed Powerplay cubes and reports in Cognos.