Surendra Pal Email & Phone Number

Mumbai, Mh, In

Mumbai, Maharashtra, India

Mumbai, Maharashtra, India

•AWS Cloud practitioner certified •AWS Solution architect associate certified •Working with redshift for a reputed company we maintain there end to end data transaction from souce to destinatation with GLUE as an etl tool, for historical data we use spectrum as the data storage. We process the data to report using ReportManager2(Apache Airflow). We use S3, Glue, Redshift, Lambda, Spectrum, Python, Pyspark for data process and data saving & we are using periscope for graphs monitoring.•We have Optout process which is monthly jobs and has the retention policy for 30 days and All the older historical data get store in spectrum and history Optout cluster. Historical data more than 2 years get permanently deleted from cluster and spectrum.•Coordinate with Datascience team (client side) and work on Query Optimization & logics of the scripts. Setup cronjobs for new ETL jobs for those scripts •Creating new data pipeline code in python to execute SQL script and setting up independent jobs in crontab hourly,daily, monthly process on EC2 instance for redshift cluster.•Working with Databrick for the same project as above we have jobs which run as the ETL and popluated the data hourly, weekly, monthly & yearly. The data is based on use watch session over TV.•Automated the manual data reconciliation process between the different sources, increasing the data accuracy and efficiency by 90%Automated ETL processes, making it easier to wrangle data and reducing time by as much as 40%Increased the efficiency of the data fetching by approximately 30% using query optimization and indexing

Mumbai, Maharashtra, India

Working on an Citrix virtual app and investigating customer and providers SSN details and the services for those customers. Following proper procedure flows to take appropriate action for the same. manage to work on multiples IBM AS400 emulator and testing the automation tools for processing those claims in a proper format and in appropriate way.

Master'S Degree, Computer Science

Graduate, Computer Science