As a Data Engineer, I developed and maintained data pipelines using Azure Data Factory, Databricks, and Azure Synapse Analytics and Implemented Azure SQL Database and Azure Data Lake Storage for large-scale data warehousing, enhancing accessibility and performance. Designed and automated ETL workflows with Azure Data Factory improving efficiency. Established CI/CD pipelines with Azure DevOps for seamless integration and deployment. Created dynamic Power BI dashboards, translating complex data… Show more As a Data Engineer, I developed and maintained data pipelines using Azure Data Factory, Databricks, and Azure Synapse Analytics and Implemented Azure SQL Database and Azure Data Lake Storage for large-scale data warehousing, enhancing accessibility and performance. Designed and automated ETL workflows with Azure Data Factory improving efficiency. Established CI/CD pipelines with Azure DevOps for seamless integration and deployment. Created dynamic Power BI dashboards, translating complex data into actionable insights. Developed custom connectors and APIs for external data integration. Show less

Developed data pipelines and ETL processes using Spark, improving data flow management. I supported SQL database optimization for efficient data retrieval and storage, ensuring high data integrity. I assisted in maintaining data systems and learning system performance optimization. I conducted prescriptive, predictive, and diagnostic analytics using Spark-based ETL solutions, aligning data solutions with business requirements. I collaborated with stakeholders to translate business needs into… Show more Developed data pipelines and ETL processes using Spark, improving data flow management. I supported SQL database optimization for efficient data retrieval and storage, ensuring high data integrity. I assisted in maintaining data systems and learning system performance optimization. I conducted prescriptive, predictive, and diagnostic analytics using Spark-based ETL solutions, aligning data solutions with business requirements. I collaborated with stakeholders to translate business needs into technical solutions and created technical documentation for ETL processes. I improved database reliability and scalability by 30% through advanced RDBMS tasks and implemented real-time data ingestion and stream-analytic solutions with Spark and Kafka. Show less

Developed and automated ETL workflows using AWS Glue, enhancing data integration efficiency by 20%. Utilized AWS services (S3, EC2, Redshift, Lambda) for data storage and processing. Created data visualizations and dashboards with AWS QuickSight. Developed Hive tables and used Flume for log data loading on Cloudera Hadoop Cluster. Automated ETL pipelines from DynamoDB to Redshift using Python. Configured AWS Data Pipeline for S3 to Redshift data loading. Integrated data from multiple sources… Show more Developed and automated ETL workflows using AWS Glue, enhancing data integration efficiency by 20%. Utilized AWS services (S3, EC2, Redshift, Lambda) for data storage and processing. Created data visualizations and dashboards with AWS QuickSight. Developed Hive tables and used Flume for log data loading on Cloudera Hadoop Cluster. Automated ETL pipelines from DynamoDB to Redshift using Python. Configured AWS Data Pipeline for S3 to Redshift data loading. Integrated data from multiple sources into Snowflake, handling nested JSON. Automated workflows with Apache Airflow and AWS Step Functions. Ensured data quality through profiling and validation. Maintained data security and compliance. Provided analysis on client requests and collaborated on data solution projects. Designed efficient data models for an AWS-based warehouse. Show less

Developed data pipelines and ETL processes using Spark, enhancing data flow management. Supported SQL database optimization tasks for efficient data retrieval and storage. Ensured high data integrity and reliability through data quality assurance tasks. Assisted in maintaining data systems and learning system performance optimization. Conducted prescriptive, predictive, and diagnostic analytics using Spark-based ETL solutions. Aligned data solutions with business requirements and organizational… Show more Developed data pipelines and ETL processes using Spark, enhancing data flow management. Supported SQL database optimization tasks for efficient data retrieval and storage. Ensured high data integrity and reliability through data quality assurance tasks. Assisted in maintaining data systems and learning system performance optimization. Conducted prescriptive, predictive, and diagnostic analytics using Spark-based ETL solutions. Aligned data solutions with business requirements and organizational goals. Collaborated with stakeholders to translate business requirements into technical solutions. Created technical documentation and training materials for ETL processes. Improved database reliability and scalability by 30% through advanced RDBMS tasks. Implemented real-time data ingestion and stream-analytic solutions with Spark and Kafka. Show less