Ankur Chopra Email & Phone Number

Toronto, On, Ca

Toronto, Ontario, Canada

➼ Developed scalable and maintainable data pipelines using Apache Spark, Hadoop, and other Big Data technologies, resulting in improved data processing efficiency and performance.➼ Successfully transformed complex analytical models using PySpark, Python, and SQL into scalable, production-ready solutions, enhancing the overall data processing capabilities of the organization.➼ Led the development of scalable Databricks pipelines, leveraging Delta Lake for improved datareliability in a 10TB lake, enhancing query performance by 40%.➼ Extensive experience in ETL/ELT automation of data extraction, data cleaning and data preparation using Pyspark, Python, Spark SQL, AWS Glue, Airflow.➼ Collected, cleansed, and provided modeling and analysis of structured and semi-structured data used for business initiatives which resulted in $1M+ annual savings.➼ Developed a complete automation framework using Python that led the migration of over 500+ Spark jobs from Spark 2.4 to Spark 3.2, ensuring compatibility, optimizing performance, and overall savings of over $1.5 million.➼ Developed a comprehensive automation framework utilizing Python, designed to dynamically optimize 3000+ Spark Jobs across multiple parameters. ➼ Excelled in debugging complex data pipelines, resolving issues that led to a 50% reduction in downtime and significantly enhancing data flow efficiency between production and analytics environments.➼ Developed a complete automation framework using Python on Databricks that led the migration ofover 500+ Spark jobs from Spark 2.4 to Spark 3.2, ensuring compatibility, optimizing performance,and overall savings of over $1.5 million.

Toronto, Ontario, Canada

Pune, Maharashtra, India

• Implementing the Data pipeline architectures in big data for multi-vendor datasets with the help of Pyspark and Cloudera HDFS services.• Extracting, cleaning and filtering the required data from the raw formations into the warehouse.• Writing Pyspark scripts for the transformation and ingestion loads of warehouse data, along with scripts for creation of datamart/API tables. • Performed end-to-end ETL automation of data extraction, data cleaning and data preparation using Pyspark, Python, and Spark SQL.• Increased the efficiency of data pipelines that processed over 100 TB of data daily.• Refining the existing workflows with optimization techniques and best code practices.• Wrote scripts in Hive SQL for creating complex tables with high performance metrics like partitioning, clustering and skewing. • Using Git 2.x for maintaining the code versioning with data scientists, data engineers and analysts’ teams• Getting & preparing requirements from Business Users.• Implementing the Data pipeline architectures for multi-vendor datasets with the help of pyspark.• Extracting, cleaning and filtering the required data from the raw formations into the warehouse.• Refining the existing workflows with optimization techniques and best code practices.• Understanding the input list of accounts provided by Business.• Writing acceptance criteria for analysis and development stories. • Preparing high level design documents and low-level documents for ETL.• Preparing complete end to end data flow architecture & design. • Engagement with Application Support and administrators to validate the design for architectural approval.• Suggest alternative solutions to learn that involve best practices, development and performance standard.• Mapping the source and target hive tables against the columns. • Generation of analytics-based reporting for tableau dashboards. • Involved in building reports in multiple views, Dashboards and Storyboard using Tableau

Pune Area, India

• Translated business needs into technical specifications to deliver customized reporting systems• Designed, built, and maintained 100% of automated systems for reporting, analysis, and analytics.• Used Python and pyspark scripts to reconcile statement and ledger files. • Getting & preparing requirements from Business Users.• Automated the manual process of reconciliation which saved 25 hours of manual work. • Writing acceptance criteria for analysis and development stories. • Preparing high level design documents and low-level documents for ETL.• Preparing complete end to end data flow architecture & design. • Engagement with Application Support and administrators to validate the design for architectural approval.• Suggest alternative solutions to learn that involve best practices, development and performance standard.• Generation of analytics-based reporting for tableau dashboards.• Writing end-to-end pyspark scripts that consists of loading data from a file to hive tables or staging table to partitioned hive table, transforming Data frames into the intended results, application of all the Business Validations and dump the data Frames into the hive tables. • Mapping the source and target hive tables against the columns. • Configuring the property file and configuration file with the pyspark file in order to make the functionality flexible with respect to the different environments and input arguments. • Developing data pipelines for various data sources.

Pune Area, India

This project deals with the client onboarding process in Morgan Stanley firm. Whenever any new client onboard, it should evaluate against certain rules. Rules are applied on the basis of various norms applied by Client.Environment: Java, Xml, Unix, SybaseWork:1) Analyzing Business Requirement for Rules project.2)Creating and implementing Rules by analyzing and inspecting the requirements given. Coding whole rules by Rule3) Authoring internal tool with the help of Unix and Database4) Supporting and maintaining the whole UI by using angular and Java skills efficiently.

Hyderabad Area, India

PROJECT: Mapple (Training Project)Brief: Mapple was basically an application to depict Mobile store just like Apple store where you can select and buy anyproduct from the store.Environment: Unix for UI, C++ for backend code, Oracle (Storage)Work:1) Designed an application to create mobile store for buying any phone.2) Requirement gathering3) Lead and actively participated in team discussion

Master Of Business Administration - Mba, Information Technology, 1St Division

Activities and Societies: Case Study Competitions: Participated in case study competitions, collaborating with teammates to analyze.

Bachelor Of Technology (Btech), Electrical, Electronics And Communications Engineering, 8.15

communication skills development. Tech Fest Organizer: Took a leadership role in organizing the annual tech fest at Guru Nanak Dev.