Dan Coates Email & Phone Number

Hawthorne, California, US

• Built data pipelines to efficiently ingest telemetry flows of hundreds of billions of data points per second into HDFS to be queried and transformed by Spark applications
• Updated structured data pipelines to a Delta Lake-based solution and built and configured a set of Spark clusters to enable SQL data analysis

Mclean, VA, US

• Seamlessly transitioned from a product to an engineering role, becoming one of the top committers to the enterprise streaming data platform for Capital One
• Built and managed a service responsible for storing 100 billion records a day from Kafka to an AWS S3 data lake, including creating the infrastructure code and enhancing the scheduler and Apache Spark application code.
• Developed solutions and architecture proposals for new features of the streaming platform driven by customer needs
• Created SDKs to facilitate users’ submitting data to the platform via JSON records, multipart large file submissions, and Apache Spark
• Wrote multiple blog posts describing features and functionality of the streaming data platform, including tutorials, detailed feature explanations, and technical deep dives
• Managed a team of four summer interns and mentored new and recent college hires on engineering best practices

Mclean, VA, US

• Served in a hybrid Product/Tech role, leading strategy for Spark-based framework and platform products while also giving technical direction to the team's engineers and customers.
• Integrated several large Enterprise services into the framework, coordinating between Enterprise service teams, customers, and the framework team.
• Designed and led the creation of a new Python/Scala library to expose reusable Apache Spark and data transformation capabilities to the Enterprise in a user-friendly, extensible, well-documented product.
• Demoed and promoted new product capabilities to hundreds of internal associates through internal conferences and lunch & learns with presentations and live demos.
• Returned to people management, managing an intern and hiring and managing a new product manager, bringing him up to speed to lead the team.
• Provided thought leadership around platform and data refinement strategy.

Mclean, VA, US

• Served as product owner for one of Capital One's largest inner-source data processing frameworks, leveraging Apache Spark to enable engineers to build dozens of production use cases from customer campaigns to ETL.
• Coordinated between a highly technical team of data engineers, teams of engineers throughout the enterprise using the framework, and business stakeholders to enable teams to easily build data pipelines with faster.
• Led a team of engineers to modernize critical regulatory monitoring from a Teradata/Excel-based platform to a cloud-based solution leveraging Apache Spark.
• Embedded as a developer with Pivotal Labs for six months creating a custom data analysis and reporting web application leveraging React, Flask, Pandas, and Docker, while practicing Extreme Programming Agile.
• Worked with business and technical teams to manage transitions from legacy, on-premise servers and databases to AWS-based cloud solutions.
• Designed and performed primary coding in Python for a data conversion tool which leverages and contributes back to open source Python libraries.

Mclean, VA, US

• Under strict regulatory timelines and requirements, executed SAS and SQL coding and developed business intent for repricing several million accounts after the Capital One acquisition of HSBC, involving significant.
• Led an offshore team to identify and implement significant additions and changes to OCC and FRB regulatory reports for Capital One’s acquired HSBC card portfolios.
• Managed two associates and one contractor in a team providing data analysis support to the Capital One Upmarket card business team.

London, GB

• Coordinated business logic changes in SAS, data infrastructure changes in DB2, and production process implementation to migrate 80+ million accounts to a new accounts receivable system.
• Piloted Tier 3 storage on Unix for end-user SAS processing, resolving issues with the vendor and drastically increasing space available to the business. Developed custom processes to monitor and report resource usage.
• Administered SAS production environments on the mainframe and Unix.

London, GB

• As part of HSBC Card Services’s Data Infrastructure team, regularly assisted business colleagues with accessing data infrastructure, interpreting data, and using tools like SAS, SQL, and NDM.
• Created an account-level data feed for several million accounts to meet OCC regulatory requirements.
• Administered SAS production environments on the mainframe and Unix.

London, GB

• Rationalized and re-engineered processes to improve efficiency and shut down antiquated processes, savings tens of thousands of dollars a year in CPU processing and storage costs
• Ran adhoc reporting for HSBC Card Services’s Finance, Marketing, and Risk teams

• Used SAS software and IBM MVS mainframe system to manage data and perform tasks related to SBC/AT&T’s payphone data.
• Created interactive websites and interactive SAS applications to access payphone compensation payment data, internal reports, etc.
• Reverse-engineered and re-implemented Oracle/UNIX reporting processes to run on IBM MVS Mainframe systems.
• Handled subpoenas requesting call detail for law enforcement and assisted in requests for data from SBC’s legal and finance operations.
• Collaborated with remote team members to create automated systems for retrieving and emailing subpoena requests.

Bachelor’S Degree, Computer Science

Education record