- Enhanced Data Workflow & Processing:Developed and maintained scalable applications using Node.js within AWS, utilizing Lambda functions and DynamoDB.Migrated legacy Python codebase to JavaScript, ensuring smooth data flow and processing efficiency.Extensively worked with AWS components (Airflow, EMR, Athena, Snowflake) for optimal data workflows.Developed scripts for loading data into S3 buckets, managing data structures, and optimizing large datasets on HDFS and S3.Deployed and configured scalable Hadoop clusters on AWS, ensuring optimal performance.Optimized and integrated Hadoop jobs with AWS services using EMR for efficient processing.Implemented best practices for importing data with SQOOP and optimized Hive, Pig, and SQOOP processes.- Full-Stack Development:Designed and developed potential future UI components leveraging JavaScript expertise.Developed analytical components using Scala and Kafka for data processing.Designed forms, modules, and templates using Django and Python.Built Cloud-based applications utilizing Java/J2EE, Spring Boot, and RESTful services.Redesigned legacy Python modules to deliver specific data formats.Developed applications in Linux environments and utilized various Linux commands.- Deployment & Operations:Managed Continuous Integration services (Jenkins, Nexus) for streamlined development.Created dynamic UI designs using HTML5, CSS3, Javascript frameworks, and AJAX.Utilized Spark Data Frames for data validation and analytics on Hive data.- Additionally:Demonstrated strong problem-solving skills and innovative thinking to propose improvements in technologies and workflows.Exhibited excellent communication and teamwork skills, effectively collaborating with cross-functional teams.

- Full-Stack Development:Led the entire software development lifecycle (SDLC) for web-based open-stack applications built with Python and Django to analyze large datasets.Designed and developed user interfaces using HTML, CSS, Javascript frameworks (JQuery, AngularJS, Bootstrap), ensuring user-friendliness and responsiveness.Developed views and templates using Django for dynamic and user-friendly web interfaces.- Data Analysis and Automation:Extensively utilized Python's core functionalities (lambda, map, reduce) for efficient data processing and analysis.Utilized NumPy for advanced numerical analysis, particularly in insurance premium calculations.Developed Spark code in Python for efficient data processing on Hive (Hadoop) and implemented data cleaning and transformation logic.Automated repetitive tasks using Shell Scripting and Python, improving efficiency and reducing manual effort.Created custom User Defined Functions (UDFs) in Python for data manipulation within Hadoop ecosystems (Hive and Pig).Connected and manipulated data from SQL databases through Spark drivers.- Deployment and Operations:Successfully configured and deployed projects using Amazon EC2 on AWS for cloud-based accessibility.Established continuous integration and continuous deployment (CI/CD) pipelines using Jenkins and Puppet, streamlining the development and deployment process.Implemented unit and regression tests using appropriate frameworks to ensure code quality and stability.Utilized version control tools (Git with Jenkins) for collaborative development and tracking.Embraced Agile methodologies (Scrum, JIRA) for efficient project management and communication.

- Data Ingestion & Processing:Utilized Sqoop for data extraction from Linux systems and RDBMS databases.Leveraged cloud services (AWS) for data retrieval.Implemented real-time data processing in the cloud using Spark APIs and Scala programming.Employed Kafka and RDDs to process live streaming information.Designed Hive join queries for efficient data integration into Elasticsearch.Automated data loading and pre-processing tasks using Oozie and Pig.- Web Development:Developed user interfaces and backend functionalities using Python, PHP and MySQL.Created an online logo library and standardized presentation templates.- Technical Support:Performed system and software research, installation, and troubleshooting.Collaborated with third-party software support teams.