Played a pivotal role in enhancing data accessibility, reliability, and security across various projects.Successfully orchestrated the migration of data from diverse sources, including legacy systems, to AWS Data Lake and RDS, utilizing AWS DMS and robust data processing techniques with Python and PySpark.Automated cloud-based data pipelines that support real-time reporting and analytics using advanced visualization tools like Power BI. Implemented event-driven architectures with AWS services like EventBridge and Lambda, ensuring efficient data handling.Led Agile ceremonies and streamlined processes to boost team productivity and deliver projects on time, establishing a culture of collaboration and continuous improvement.Implemented advanced data security measures, including PII data handling protocols and encryption techniques, ensuring compliance with regulatory standards.Designed auto-deployment processes for data management and established CI/CD practices using Git and CodePipeline, enhancing operational efficiency and reliability.Guided and mentored a team of developers, promoting best practices and fostering a collaborative environment that drives project success.

Developed a robust Data Lake pipeline insurance client, integrating and processing data from upstream systems like Headless Manufacture for downstream accessibility. Conducted data cleaning and transformations across multiple channels, generating over 100 tailored reports to fulfill diverse business needs.Managed JSON data in ingestion pipelines, converting it to Parquet format for optimized storage and processing.Utilized AWS Step Functions to streamline workflows, enhancing operational efficiency.Created APIs for testing teams to fetch entity status and generate ad-hoc reports.Established Docker environments for efficient querying of Parquet data.Developed automated email solutions to disseminate client reports via AWS Glue jobs.Restructured existing jobs using design patterns and OOP principles without altering their external behavior.Leveraged AWS Textract for processing images, PDFs, and files aimed at report generation.Created dashboards on Amazon QuickSight, delivering insights from various data sources.Conducted reconciliation to identify inaccuracies in daily data transmissions.Managed client group products, collaborating directly with clients for new developments and issue resolution.Developed a Kinesis pipeline for logging data flow, ensuring relevance and integrity throughout the process.Utilized AWS services such as Lambda, Glue, SQS, SNS, Kinesis, DynamoDB, Aurora, and S3 to effectively address data processing and integration needs.

Designed and developed a cloud-based data lake to facilitate reporting and analytics, showcasing my expertise in architecting scalable and efficient data storage solutions on AWS. Developed real-time Serverless solutions to enhance sales performance, effectively leveraging AWS services for immediate data processing.Implemented Python-based Lambda functions for efficient data processing, demonstrating my proficiency in serverless computing.Executed PySpark jobs on Amazon EMR for historical data loads, highlighting my expertise in big data processing and analysis.Integrated ElasticSearch for audit logging and reporting in Kibana, showcasing my ability to utilize AWS Elasticsearch Service for comprehensive log analytics.Created a UI-based testing automation framework for Data Lake projectsLeveraged Python libraries such as Pandas, NumPy, and Boto3 for efficient data processing and seamless interaction with AWS services.

Contributed in Drupal-based website development, integrating Apache Solr to implement advanced search functionalities such as faceting, highlighting, and stop-word removal. Assisted in managing Solr configurations in a clustered environment with ZooKeeper to ensure high performance and reliability.Contributed to the internal portal developments.Gained hands-on experience with Apache Solr, focusing on configuring and optimizing search features to improve user experience.Completed comprehensive full-stack training.Utilized a variety of technologies, including Apache Solr, AWS, Drupal, Java, JavaScript, and ZooKeeper, to contribute to the development of comprehensive solutions.