Krishna J Desai Email and Phone Number

A Data Engineer with proficiency in working with multiple technologies. My expertise spans across PySpark, Databricks, AWS (S3, Redshift, DynamoDB, RDS) Databases and messaging services empowering me to architect and optimize ETL and data pipelines for seamless data processing.Technical Proficiency:Leveraging PySpark, I seamlessly handle dataframes and diverse file formats (parquet, csv, avro, json), implementing advanced techniques such as repartitioning, streaming, and batch processing for optimal ETL workflows.Adept at crafting efficient SQL queries using CTE, window functions, and subqueries, ensuring data precision and reliability.Solution Architecture:In data integration projects, I design and implement robust transaction management, event-driven architecture, and reliability patterns. My expertise extends to securing APIs with policies like JWT, OAuth, Rate limiting, and HTTP Caching.Integration:Proficient in RESTful, file-based (SFTP), and message-based integrations, I utilize Pub/Sub patterns for sourcing data, creating seamless connections in complex data landscapes.Specialized in building ETL pipelines using AWS services like DynamoDB, RDS, Redshift, EC2, and S3, ensuring efficient data read and load.Data Analytics:My skill set includes exploratory data analysis and pre-processing in PySpark, enabling me to design holistic solutions that handle both historical and daily loads.Code Development & Lifecycle Management:From writing API specifications in RAML to deploying APIs to production, I manage the entire API lifecycle with precision.Proven track record in writing reusable APIs and developing PySpark transformation logic for enhanced analytics.Passionate about pushing the boundaries of data engineering, I am open to new challenges and collaborations that drive innovation in the ever-evolving landscape of technology. Let's connect and explore opportunities to make a meaningful impact together!