Yash Narayan Email & Phone Number

Toronto, On, Ca

Hyderabad, Telangana, India

• Led the architecture, design, and development of end-to-end data pipelines on GCP for data integration, ingestion, transformation, and visualization.• Collaborated with stakeholders to gather and document business requirements. Developed technical specifications and worked with IT teams to implement solutions.• Conducted system testing and troubleshooting to ensure optimal performance. Managed project timelines and budgets, ensuring on-time delivery of solutions.• Built… Show more • Led the architecture, design, and development of end-to-end data pipelines on GCP for data integration, ingestion, transformation, and visualization.• Collaborated with stakeholders to gather and document business requirements. Developed technical specifications and worked with IT teams to implement solutions.• Conducted system testing and troubleshooting to ensure optimal performance. Managed project timelines and budgets, ensuring on-time delivery of solutions.• Built real-time streaming and batch data pipelines using Cloud Dataflow and Py Spark for efficient data processing.• Utilized Big Query (SQL) for data modeling, optimization, and querying to support business intelligence needs.• Defined and implemented disaster recovery strategies for data pipelines and GCP resources using GCP Backup & Restore.• Utilized Terraform (Infrastructure as Code) to automate infrastructure provisioning and management on GCP.• Implemented containers (Docker) orchestration, networking and deployment in Kubernetes(K8s), improving scalability and reliability.• Utilized the Cloud Data Catalog integrated with Dataflow for automatic lineage capture for data transformations occurring within the pipelines.• Streamlined data processing pipelines by 45% using Spark and Hadoop, handling over 100 GB of daily data ingestion• Developed scalable stream (Kafka, Pub/Sub) & batch pipelines (5 TB/day) using Dataflow (Python) & Data Proc (Py Spark)• Maintained, automated and orchestrated pipelines in Google Cloud Composer for reliability, scalability, and workflow coordination• Optimized retail analytics with Big Query, analyzing sales trends, identifying key growth areas, leading to 15% faster insights• Automation of ETL pipeline, extracted Millions of records, performing transformation in Big Query and load it in the Cloud SQL• Developed real time and 10+ KPI Tableau dashboards, enhancing report accuracy by 30% and reducing reporting time by 40% Show less

Hyderabad Area, India

• Collaborated with cross-functional teams to implement business requirements, translating business requirements into technical specifications, to design and implement technology solutions• Developed and optimized SQL queries, stored procedures, and database schema to ensure efficient data retrieval, manipulation, and storage, therefore reduced query execution time by 20% through optimization• Utilized Git for version control, collaborating effectively with team members and tracking… Show more • Collaborated with cross-functional teams to implement business requirements, translating business requirements into technical specifications, to design and implement technology solutions• Developed and optimized SQL queries, stored procedures, and database schema to ensure efficient data retrieval, manipulation, and storage, therefore reduced query execution time by 20% through optimization• Utilized Git for version control, collaborating effectively with team members and tracking project progress. Investigated and resolved issues related to Python applications, SQL databases, and data integrity.• Developed RESTful APIs and web applications using Flask for microservices, to efficiently handle HTTP requests and responses. • Improved Snowflake and Star schema database performance by 30% via SQL optimization, aggregation, indexing, and partitioning• Automation of pipeline using Airflow (DAGs), extracted Millions of records, transformation in Talend and load it in MySQL DB• Developed and deployed 10+ MapReduce jobs, reducing ETL processing time by 35% across distributed Hadoop systems• Optimized data processing by 40% using Spark SQL and HiveQL on 5 TB Hadoop clusters for real-time analytics• Created interactive dashboards using Power BI to visualize KPIs for National Gas, increasing user engagement with reports by 30%• Designed and implemented robust shell scripts to orchestrate data processing pipelines, integrating with various data sources and systems to ensure seamless data flow.• Contributed to the development of a new web application using a Python Django framework• Integrated PostgreSQL database for data storage, ensuring robustness, reliability, and scalability.• Developed advanced database management strategies, including schema design, query optimization, and data migration, leveraging PostgreSQL.• Employed agile methodologies to iterate quickly, adapt to changing requirements, and deliver value incrementally. Show less

Postgraduate Degree, Business Analytics, 4.0

Bachelor Of Technology, Computer Science