Jay Patel Email & Phone Number

Chicago, Il, Us

Chicago, Illinois, United States

● Designed and implement ETL pipelines from multiple different sources to a centralized destination in a Snowflake database ● Extracted over a 100 TB of sales, shipping, material and warehouse data from 1000s of flat files, and many external stages ● Leveraged Snowflake’s tasks, streams, and SnowQL to create a 24-hour automated pipeline to transform structured and non-structured data into tables ● Developed and maintained post-hooks written in Python and Bash for handling tasks between Azure Blob and SFTP folders ● Analyzed application data to assess performance and detect problems ● Wrote, maintained and tested complex SQL queries to transform more than 100,000 of daily data points for the purpose of getting data prepared to be used by applications ● Maintained data reliability and validity across multiple different applications and databases ● Wrote and Performed test scripts on code releases to ensure that new features, and bug fixes to the environment were working as expected ● Used Spark, SQL and AWS services to create an ETL Pipeline for the business use of improving guest experience and increasing revenue. ● Ingested data from S3 cloud storage using Apache Spark to do batch processing of the data on an AWS EMR cluster. ● Submitted Spark job on an EMR cluster consisting of one master node and three worker nodes ● Transformed data using Spark SQL to persist a normalized database on an AWS RDS database● Utilized Apache Spark and Python to do batch processing of the data on an AWS EMR cluster ● Used HDFS to store pre-processed data files to leverage reliability and data locality in the cluster ● Wrote python and bash scripts that fully integrated different parts of the pipeline from data clean-up to persistence on an Apache Hive database

Atlanta, Georgia, United States

Assessed hardware devices such as body cameras and instant messaging communications systems for vulnerabilities and IDA Pro to introduce malware in an attempt to find any potential exploits. Implemented a function that would securely encrypt messages and get the communication devices ready to receive and transmit the messages properly.

Collaborated with a team to pipeline the ML infrastructure from data mining phase to production phase as a Shopify Plugin. Worked on customer acquisition and communicated with end users about incidents regarding final product