David Ellington Email & Phone Number

Vancouver, Washington, US

Phoenix, AZ, US

• Developed ETL pipelines with Fivetran, Airflow, AWS Lambda, AWS Glue, and Snowflake.
• Built and maintained dbt models for streamlined data transformation.
• Introduced Airflow and assisted team members in its use.
• Replaced expensive Fivetran pipelines with PySpark, Lambda, Airflow solutions, reducing costs.
• Introduced PyTest for unit testing Python code.

Vancouver, Washington, US

• Deploying AWS data platform infrastructure using Terraform Cloud
• Developing Jenkins pipelines to deploy code into AWS platform
• Developing Airflow utils
• Developing ETLs using Airflow, AWS, and Snowflake
• Designed and contributed to the implementation of an in-house dbt Core solution running on AWS Batch (orchestrated and triggered by Airflow)

Vancouver, Washington, US

• Playing the role of a player-coach engineering manager: managing a data engineering Scrum team and doing engineering work along with the rest of the team
• Conducting code reviews and working with my reports on goals, training, and performance
• Working with the team to continually improve our code, architecture, and processes to provide better reliability, quality, performance, and cost-efficiency
• Developing a Scala Spark library to plug into a PySpark codebase that allows for reading of HBase SequenceFiles and fast deserialization of protobuf serialized data
• Working closely with the product owner to coordinate work with other technical teams and to understand stakeholder requirements and how best to meet them with a technical solution
• Developing ETL to load star schemas in Snowflake

Vancouver, WA, US

• Providing tech leadership on projects to stand up Apache Airflow for ETL pipeline orchestration; to move ETL code from Databricks notebooks into Python eggs; to move Spark ETL processing from Databricks to AWS EMR and.
• Loading high-volume complex JSON from S3 data lake sources into Snowflake data warehouse via Spark on EMR/Databricks Light - orchestrated by Airflow
• Making over 6 billion (and counting) JSON records queryable by developing ETL to load into a partitioned, z-ordered Delta Lake table
• Using SNS/SQS for managing dependencies between Airflow DAGs and Slack for Airflow failure notifications
• Developing an automated scheduled Python process to correct Snowflake permissions, which is driven by YAML templates
• Working with stakeholders to quickly generate various prototype options for ETL targets in Snowflake

• Performing as tech lead for a supply and materials planning analytics squad
• Designing and implementing data pipelines and structures in AWS/Airflow/Snowflake based on requirements from product owners, analysts, viz developers, and business partners within an Agile / Scrum methodology
• Initiating Spark-based data ingest (on-prem Teradata/Oracle to AWS)
• Writing ETL jobs in both Scala Spark and PySpark
• Initiating new patterns for PySpark – more modular/testable/reusable code as opposed to scripting; Python 3.4 instead of 2.7; unit testing Spark ETL code
• Writing Airflow DAGs and configuring Airflow to run jobs that spin up/down EMR clusters and run Spark jobs

• Performing as Tech Lead for a supply planning dashboard team
• Working in an Agile /SAFe (Scaled Agile Framework) development environment
• Designing star schema design on Teradata supporting Cognos and Tableau reporting
• Working with large datasets – some exceeding 1 billion rows
• Designing temporal tables on Teradata for capturing changes in data over time
• Tuning Teradata SQL queries (Teradata is an MPP (massively parallel processing) RDBMS)

• Designing star schemas on Teradata supporting Cognos and Tableau reporting
• Completing a Cognos Framework project
• Working with large datasets – some exceeding 1 billion rows
• Serving as a go-to resource for diagnosing/troubleshooting reporting/data issues and suggesting appropriate fixes
• Writing a white paper on data architecture/DB design
• Participating in a two-day hackathon to grow awareness of potentially security vulnerabilities

Greenwich, CT, US

• Developing, maintaining, and debugging SQL Server 2005/2012 databases, including stored procedures, functions, triggers, views, SQL Server Agent jobs, permissions, etc. and including light DBA work.
• Designing, maintaining, and debugging ETL processes using SSIS, including.NET script tasks.
• Designing and maintaining end-user reporting using SSRS (SharePoint) and Excel, as well as writing queries for ad-hoc data requests.
• Gathering requirements from internal and external users and creating technical and other documentation and diagrams.
• Enhancing and supporting existing.NET Winforms applications using both VB.NET and C#.
• Developing ASP.NET MVC web apps utilizing technologies/modules such as Entity Framework Code First Migrations, Castle Windsor IoC, Elmah, AutoMapper, Bootstrap, jQuery, AJAX, JSON, toastr, etc.

Greenwich, CT, US

• Developing, testing, and implementing new trading partner setups for both inbound (204, 211, 820) and outbound (210, 214, 990) ANSI X12 EDI standards – primarily version 4010. Working daily with external customers and.
• Creating and/or updating and maintaining several Access database tools – EDI setup work queue, EDI reject work tool, customer specific tools for matching shipments (customer reports to Con-way reports) that are missing.
• Using and maintaining an SQL Server 2008 Express database with an Access frontend to load daily inbound 204/211 data (over 3 million records), which I created for searching for shipments as well as for.

Master Of Science (Ms), Library And Information Science

Bachelor Of Science (Bs), Psychology