• Designed and developed data pipelines with AWS Services using S3, Event Notifications, Lambda, Step Functions, Event Bridge, Cloud Formation, Athena and Glue.•Build and maintained infrastructure as code using AWS Cloud Formation.• Designed and developed ETL processes in AWS Glue to migrate campaign data from external sources like S3, Parquet/ Avro files into Redshift and involved with Data Extraction, aggregations, and consolidation of data within AWS Glue using PySpark.• Developed highly complex Python and SQL code, which is maintainable, easy to use, and satisfies application requirements, data processing and analytics using inbuilt libraries.• Configured AWS Glue Connections with AWS RDS(PostgresDB).• Convert into different data formats for user/business requirements by streaming data pipeline from various sources.•Designed and managed ETL pipelines utilizing AWS Glue and Snowflake, enhancing data processing efficiency by 40%.• Automated data ingestion and transformation processes using AWS Lambda and Snowflake tasks.

• Implemented Data collection and transformation in AWS cloud computing platform using S3, Athena, Glue, Quicksight.• Built S3 buckets and managed policies for S3 buckets and used S3 bucket for storage and backup on AWS.• Established Amazon S3 bucket to create a data lake consisting historical and current records from 2012 onwards for a central reporting solution.• Configured AWS Glue Crawlers to scan data sets (Raw Data) and populate Glue Data Catalog, which served as a central metadata repository.• Created ETL Scripts using AWS Glue and triggered the job to automate using PySpark.• Configured AWS Glue Job for transformation of Raw data into Refined data.• Utilized Python and Amazon Athena to perform data exploration and data analysis on data cataloged in Glue.•Configure Amazon Athena to query the data from Amazon Glue Data Catalog.

• Extensive Involvement in Requirement Analysis and system implementation.• Actively involved in SDLC phases like Analysis, Design and Development.• Responsible for developing modules and assisting in deployment as per the client’s requirements.• Developed internal tools for text mining, and web crawler for collecting data by Python and SQL • Developed shell scripting and Python programs to automate the data flow on day-to-day tasks• Analyzed the SQL scripts and designed the solution to Developed a custom ETL pipeline by using Python• Understanding Python files in an open stack environment and making necessary changes if needed.• Involved in Designing DB Schema for the application.• Involved in Tool development, Testing and Bug Fixing.

• Conducted tutorials and lab sessions for OOPS and probability theory courses, instructing undergrad freshmen and sophomores