• Developed spark applications in PySpark on distributed environment to load huge number of CSV files with different schema in to Hive ORC tables.• Developed ETL pipelines in and out data warehouse using combination of Python and SparkSQL.• Developed wrapper in Python for instantiating multi-threaded application.• Hands on experience on developing UDF, Data Frames and SQL queries in Spark SQL. • Built real-time streaming data pipelines with Kafka, Spark structured streaming.• Involved in developing Spark application using PySpark as per business requirement.

a) Griffin Project:Working on ingesting the files from various source systems to the required cluster and developed the respective ingestion jobs for daily loading of data in the cluster.Worked on Collibra platform for metadata creation for the ingestion jobs.Developed the SQL logic for the corresponding tables in the Sparkola platform and loaded the data into the underlying alluxio locations

a) Cheque Analytics:Cheque Frequency Prediction and distribution in a bank to make a user and bank journey easier.Worked on Cheque Volume Forecasting module and developed a seasonal Arima model in time series analysis based on historical data for forecasting the number of inward and outward cheques for the next 14 days.b) HK Regionalisation:Used Teradata Platform for developing and running the history migration scripts.Worked on Hadoop and hive for loading data from different source systems.Worked on stabilizing the data in the large datasets while loading the data.

Automatic Toll Collection using Image Processing :Using the web camera, the number plate of the vehicle is captured and processed using Optical Character Recognition and the number is extracted.The extracted number is compared and searched in the centralized database. License Plate is linked with the owner's bank account in the database.After checking the database toll amount is deducted from the owner's account and the user is notified by a message.

Performed simulation on mobile crowdsensing by using CrowdSenSim Simulator.Simulation was done using Java, by taking the benchmark data from a research paper on Mobile Crowd Sensing. This was done under the guidance of Prof. Atul Negi from the University of Hyderabad