- Spearheading data-build and engineering processes; providing leadership in development and execution of data engineering and architecture initiatives; transforming semi-structured data into structured formats using Snowflake technology.- Utilizing expertise in large language models, Python, Java, and the Bigdata stack.- Overseeing the design, implementation, and maintenance of scalable, secure, and reliable data infrastructure, including data warehouses, data lakes, and so on.- Defining and implementing data architecture principles, standards, and best practices to ensure maintainability across systems.- Designing data models, schemas, and data flow diagrams to support both operational and analytical use cases.- Fostering collaboration with cross-functional teams including software engineering, product management, and business operations to drive data-driven decision-making and innovation.- Ensuring efficient production load monitoring and support for Datawarehouse and data marts.- Leading and mentoring a team of data engineers, architects, and analysts to ensure high performance and productivity.- Optimizing team performance and fostering a culture of high motivation and productivity.

Led the development of the data warehouse from the ground up. This involved the end to end setup including various pipelining tools (such as Airflow, AWS Glue), also the streaming mechanism to bring data into EDW and the development of customised Data Engineering frameworks.

- Built a CI/CD-enabled Scala-Spark framework; utilized Python, Scala-Spark, and Data Ingestion frameworks.- Saved $1 million annually by adhering to best development and QA practices for data engineering products.- Supported the implementation of data solutions and optimized data pipelines.- Engineered dynamic ingestion frameworks for actionable insights and analysis.- Developed and implemented a data quality monitoring system that reduced errors, ensuring data accuracy and reliability.

- Developed a versatile data ingestion framework integrating PySpark and ScalaSpark.- Engineered dynamic ingestion frameworks to derive actionable insights and facilitate analysis.- Utilized Python for HIVE LLAP to enable actionable insights for profitability.

- Designed and implemented ETL jobs for various data warehousing operations.- Leveraged custom Java libraries, including Apache POI, for intricate use cases.- Managed complex ETL requirements and design; collaborated with product development teams and senior designers to develop architectural requirements to ensure client satisfaction.- Developed full SDLC project plans to implement ETL solutions and identified resource requirements.- Shaped and enhanced the overall ETL Informatica architecture; optimized SQL queries to improve data retrieval performance.