Sandeep Kurdagi Email & Phone Number

Amsterdam Area

I am responsible for developing and maintaing platform capabilities to enhance data accessibility andperformance for business users. - Building and managing the Airflow DAGS to orchestrate the data pipelines. - Managing end-to-end AWS Infra and setting up best practices on the cloud to improve performance, security and and reduced costs by approximately 40%. - Automated reporting processes using Airflow and AWS services such as S3 and Athena.- Implemented and managed Apache Superset and Tableau dashboards on EKS, integrated with Okta SSO.

Bengaluru, Karnataka, India

Bengaluru, Karnataka, India

I am responsible for building highly available and robust systems/apps on Hybrid/Multi-cloud. Managing the AWS ecosystem and infrastructure end to end. My main focus areas here are- Building distributed systems/app on Hybrid/Multi-cloud - AWS, Azure, GCP.- Developing streaming apps and large-scale data pipelines -Apache Kafka, Athena, Qubole, AWS eco-system- Log Management and application monitoring - Elasticsearch, fluent and kibana- Improving the reliability and performance of systems- Cost reduction across multi-cloud eco-systems- Performance optimization across multiple modules and projects- Developing generic workflows using Apache Airflow, Python Pandas.

Bengaluru, Karnataka, India

Building apps on the cloud. Large-scale distributed web crawling. Serverless, Scaling data pipelines. - AWS managed Services - S3, EC2, Spot instances, Cost optimization, Lambda, RDS, Elasticcahe, Time Series Redis, Grafana, EFS, Dockers- Developed distributed systems and scheduler for large-scale data aggregation engine.- Custom Auto-scaling groups for managing very high non-uniform workloads.- Enabled systems to work seamlessly on Hybrid and Multi clouds.- Built Stateless applications to enable cost optimization using spot instances and brought down the cost by almost 50%- Generating insights on the web aggregation engine metadata using Google Bigquery, Redash, and Google Data-studio

Bengaluru Area, India

I was responsible for building and scaling the data platforms and the following are the areas and contributions.- Enabling the tools and services to reduce the manual work and improve the usability of the tools- Developed and managed large-scale distributed web-crawling, scheduler and batch processing.- Building Web Apps using Python API Frameworks like Cherrypy, Tornado, Django. Celery, Rabbitmq, Redis- Proxy Management for large-scale web crawling and aggregating the data from the public web efficiently. - Setting up deployments(CI/CD) using python fabric, bitbucket pipelines, and AWS code deploy

Bengaluru Area, India

- Worked on improving the Web crawling aggregation engine. - Involved in Developing the Web scrapers for Telecom/Retail Data - python-requests, selenium- Extracting meaning full data from the semi-structured raw data (HTML pages). - Regular expressions, Beautiful soup- Developed Rest API's for multiple internal tools and applications- Documentation for applications and services.

Bangaon Area, India

Sorting Hat - Task clustering for efficient Automatic Hybrid Routing

Bengaluru Area, India

Completed masters in Data Science. Worked on research projects and broad areas of databases, analytics, modeling, machine learning, information retrieval, usability.

Master Of Technology (M.Tech.), Data Science, 3.43

Activities and Societies: Open Systems Lab

Be, Information Technology, 9.09

Activities and Societies: IEEE, RISE, Techtrix