Binay Kumar Email & Phone Number

Bentonville, Arkansas, Us

• Leading the data engineering team in delivering scalable solutions for handling trillions of Walmart data using latest technologies such as GCP, Big Query, Dataproc, Kafka, Spark-Scala, CICD, Python, Hudi, Automic, Airflow. Extensively involved in design and development of data pipeline.• Designed and developed Gen AI based automated framework for assisting data model work using python.• Performed end to end Airflow setup from scratch for data pipeline execution. Built multiple CICD pipeline for package installation and airflow upgrade activity.• Designed and developed generic spark Scala data pipeline along with airflow Dag for loading billions of Walmart store history data to Hudi for fill the Walmart Data Lake initiative.• Developed a generic services internally at Walmart for migrating high volume big query table to Hudi, which was utilized by the entire organization.• Implemented a data quality framework across all datasets to proactively identify issues and raise alarms accordingly.• As the sole data engineer, I took full ownership of the end-to-end system, from gathering requirements and collaborating with stakeholders to working with the source team, and ultimately designing, developing, and implementing reporting solutions.

Teaneck, New Jersey, Us

• Worked as ETL/Big Data/DevOps/Cloud Data Engineer for client Capital one.• Work with Project Manager, Business Leaders and Technical teams to finalize requirements and create solution design & architecture.• Perform end to end analysis, design, development and testing activities in below technologies:o Spark Scala Python Sqoopo Abinitioo Amazon Web Services(AWS)o Unix Shell scriptingo Databases like Oracle, Teradatafor migrating legacy ETL applications to open source technology under the cloud-based platforms.• Responsible for studying existing Abinitio ETL graphs, Unix shell scripts, SQL, Teradata queries and propose efficient migration solution using open source, big data and cloud technologies like Spark, Scala, python, AWS. Also, provide estimation for the proposed development, testing and production deployment activities.• Automate execution of manual Data quality (DQ) check SQL queries through batch process using Unix shell script and Python script. Perform scenario as well as boundary level unit testing. Schedule batch jobs using scheduler tools like Control M and Arow.• Design and develop Hadoop ETL solutions to move data to the data lake using big data tools like Sqoop, Hive, Spark, HDFS.• Design and develop Spark code using Scala programming language & Spark SQL for high speed data processing to meet critical business requirements.• Develop Unix Shell scripts to perform Hadoop ETL functions like Sqoop, create external/internal Hive tables, initiate HQL scripts etc.• Design and develop Abintio graphs to meet critical business requirements.• Spin AWS infrastructure for running spark, sqoop and unix shell script jobs. Environment: Big data Hadoop, HDFS, AWS Lambda,AWS S3, AWS Redshift, AWS EMR, AbInitio,OFSAA, Hive, Spark, Scala, Sqoop, ctrl-M,Python, Pyspark, MySQL, UNIX, Putty, JIRA.

Mclean, Va, Us

Mumbai, Maharashtra, In

• Worked as Ab-Iintio Developer for the client Citi group.• Involved in gathering user requirements, design and development of Abintio graphs doing ETL.• Worked on different Ab Initio Components such as reformat, rollup, join, scan, normalize, gather, merge,Partitioning of data, multi file concept, pset etc.• Created Ad-hoc Ab-inito graph creation for dealing issues in production.• Extensively involved in Reconciliation Activity i.e. sync up between master files and database.• Extensively involved in Reusable Components Development for batch/job automation.• Extensively involved in Development of automated reports with Ab-intio graphs and Unix scripts for clients.• To provide the Value-add point and try to improve current system's performance.Suggesting solution for enhancing the performance of the application.• Doing the root cause analysis and working for the permanent fix at code level and application configuration level.• Worked as internal auditor of the team, to ensure team adheres to the defined process.• Involved in Onshore and offshore team management.• Gained experience on Data warehousing, Data modeling(Relational, dimensional, Star Schema, Snowflake Schema Operational Data Source (ODS) and Slowly Changing Dimensions), Data integration, Data Migration, ETL process and Business Intelligence.

New York, New York, Us

Bachelor Of Technology - Btech, Computer Science And Engineering

10+2, Computer Science, Pcm

High School