As a Data Engineer with Ads.com, my responsibilities included developing ETL orchestration with DAGster, creating SQL BI dashboards, and ingesting OLAP data with Python to gather business insights.• Optimized SQL queries and database performance to optimally render data dashboards for analysis by business stakeholders• Integrated data from multiple sources into the AWS environment to reduce overall cloud storage costs• Developed Python scripting to ingest data from the reporting database and output a CSV file to deliver business insights that boosted sales• Developed DAGster assets to automate replication between databases and reduce errors in database replication

As a Data Engineer with Truist, my primary responsibility was to develop ETL and batch-streaming pipelines with PySpark, and to optimize cloud storage.• Implemented solutions for data ingestion and processing using PySpark todeliver business insights across non-technical teams.• Worked with Spark, Python, AWS, GCP, and SQL to bring more efficientdata solutions to a legacy system.• Migrated virtual machines from GCP to AWS using Cloud Endure Migrationin order to lower the cost of data storage.• Integrated data from multiple sources into AWS environment in order to gaindiscounts within the AWS storage system.• Configured data with various ETL tools and instantiated Data lake with AWS S3for secure, effective, and easy data storage.• Orchestrated workflows with AWS Step functions and Lambda to automate datatransfer between database storage and reporting layers.

As a Data Engineer with Georgia Pacific, my responsibilities included developing efficient ETL pipelines with Spark, Python and Kafka.• Developed Python scripts for database updates and file manipulation to deliver meaningful insights.• Configured Spark streaming applications and developed Sqoop techniquesto instantly transfer data.• Designed Kafka brokers and consumers for data processing integration.• Implemented data pipelines and prototypes using Spark ecosystem to efficiently extract, transform, and load data from APIs, Relational Databases, NoSQL Databases, and flat files• Managed data in Hadoop data lakes and implemented Jenkins CI systems to store and process large amounts of data while keeping costs low

At Ford Motor Company, my role as a Data Engineer included transfer between relational databases, increasing parallelization with streaming pipelines, and optimizing data storage.• Implement Incremental Import into Hive tables to allow parallel database calls during data importing to reduce data transfer and processing time• Instantiate Hive tables to hold transformed results in a tabular format to decrease query load times• Design and amend Hive queries with Hive QL so that business analysts proficient in SQL could provide business insights• Program Hive UDFs and oversee ETL to HDFS to maintain data integrity• Extensive practice in importing real-time logs to HDFS with Flume for real-time monitoring• Construct UNIX shell scripts to automatize the build process, and conduct regular jobs like file transfers to reduce manual intervention in routine tasks• Utilize Cloudera Manager for instituting Cloudera Cluster and performance analysis to identify bottlenecks and optimization opportunities• Committed incremental imports to Hive with Sqoop to efficiently transfer data between relational databases and Hadoop

In my role as a Data Engineer with Suncoast, my role revolved around working with Hive and Tableau to effectively store and display big data.• Wrote shell scripts to synchronize workflows in order to pull data from various databases into Hadoop• Transfer consumption purposes into Hbase and Hive tables• Responsible for carrying out upgrades, patches, and bug fixes in Hadoopcluster within a cluster environment• Perform Hive queries for studying data in Hive warehouse with Hive Query Language• Coordinate with devops team to work with Hadoop in order to provide large-scale solutions• Utilize Hadoop Administration tools for installation and management of single-node and multi-node Hadoop cluster• Pull metadata of Hive tables with Hive QL• Amend the Hive views to the top of the source data tables, along with configuring a secured provisioning framework for users to access the data through Hive based views• Launch shell scripts to aid the process of loading data• Collect log data from multiple sources and synchronize it with HDFS using Flume and stage data within HDFS for further examination• Define Hive scripts to take data from• Formulate Hive queries to notice emerging trends by comparative analysis between Hadoop data and historical numerical markers• Implement and create Tableau Desktop to connect to Hortonworks Hive Framework containing structured and unstructured data• Form Hive tables with data along with drafting Hive queries in HiveQL