Data Engineer
Current• Expertise in building robust infrastructure for efficient extraction, transformation, and loading (ETL) of data from diverse sources , with a focus on US healthcare data• Developed advanced data transformation scripts, achieving 100% optimization in data delivery from on-premise sources.• Proficient in orchestrating data workflows using Azure Data Factory Pipelines, performing data transformation in Databricks with PySpark/SQL, and managing Azure Blob Storage and Azure Data Lake.•… Show more • Expertise in building robust infrastructure for efficient extraction, transformation, and loading (ETL) of data from diverse sources , with a focus on US healthcare data• Developed advanced data transformation scripts, achieving 100% optimization in data delivery from on-premise sources.• Proficient in orchestrating data workflows using Azure Data Factory Pipelines, performing data transformation in Databricks with PySpark/SQL, and managing Azure Blob Storage and Azure Data Lake.• Specialized in optimizing data storage, retrieval, and processing techniques to ensure high performance and cost efficiency.• Collaborated with cross-functional teams to define and implement data requirements, ensuring alignment of data solutions with business objectives.• Skilled in monitoring and troubleshooting data pipelines to identify and resolve performance issues and bottlenecks.• Managed Machine Learning pipelines using Azure ML Studio as part of the ML Ops Global team for various ad-hoc projects. Show less
