Pavan Kumar Email & Phone Number

United States

Boston, Massachusetts, United States

Involved in analyzing business requirements and prepared detailed specifications that follow project guidelines required for project development.Designed and implemented configurable data delivery pipeline for scheduled updates to customer facing data stores built with PythonUsed Pyspark for data frames, ETL, Data Mapping, Transformation and Loading in.

India

Working with AWS services like Lambda function, Athena, DynamoDB, Step functions, SNS, SQS, S3, IAM etc.Developed the PySpark code for AWS Glue jobs and for EMR.Created data pipeline for different events of ingestion, aggregation and load consumer response data in AWS S3 bucket into Hive external tables in HDFS location to serve as feed for tableau.

India

Migrated an existing on-premises application to AWS. Used AWS services like EC2 and S3 for small data sets processing and storage.Worked on setting up and configuring AWS's EMR Clusters and Used Amazon IAM to grant fine-grained access to AWS resources to usersCreated reports in TABLEAU for visualization of the data sets created and tested Spark SQL.

India

Transformed the data using AWS Glue dynamic frames with PySpark; cataloged the transformed the data using Crawlers and scheduled the job and crawler using workflow featureDeveloped data pipeline programs with Spark Scala APIs, data aggregations with Hive, and formatting data (JSON) for visualization, and generating.Worked on developing ETL processes (Data.

Hyderabad, Telangana, India

Involved in creating Hive tables, loading with data and writing hive queries on top of data present in HDFS.Worked on tuning the performance Pig queries. Involved in Developing the Pig scripts for processing data.Written Hive queries to transform the data into tabular format and process the results using Hive Query Language.By using Apache Flume loaded.