Dan Coates Email & Phone Number

Hawthorne, California, Us

• Built data pipelines to efficiently ingest telemetry flows of hundreds of billions of data points per second into HDFS to be queried and transformed by Spark applications• Updated structured data pipelines to a Delta Lake-based solution and built and configured a set of Spark clusters to enable SQL data analysis

Mclean, Va, Us

• Seamlessly transitioned from a product to an engineering role, becoming one of the top committers to the enterprise streaming data platform for Capital One• Built and managed a service responsible for storing 100 billion records a day from Kafka to an AWS S3 data lake, including creating the infrastructure code and enhancing the scheduler and Apache Spark application code to achieve high reliability and zero-downtime deployments• Developed solutions and architecture proposals for new features of the streaming platform driven by customer needs• Created SDKs to facilitate users’ submitting data to the platform via JSON records, multipart large file submissions, and Apache Spark• Wrote multiple blog posts describing features and functionality of the streaming data platform, including tutorials, detailed feature explanations, and technical deep dives• Managed a team of four summer interns and mentored new and recent college hires on engineering best practices

Mclean, Va, Us

• Served in a hybrid Product/Tech role, leading strategy for Spark-based framework and platform products while also giving technical direction to the team's engineers and customers.• Integrated several large Enterprise services into the framework, coordinating between Enterprise service teams, customers, and the framework team.• Designed and led the creation of a new Python/Scala library to expose reusable Apache Spark and data transformation capabilities to the Enterprise in a user-friendly, extensible, well-documented product.• Demoed and promoted new product capabilities to hundreds of internal associates through internal conferences and lunch & learns with presentations and live demos.• Returned to people management, managing an intern and hiring and managing a new product manager, bringing him up to speed to lead the team.• Provided thought leadership around platform and data refinement strategy.

Mclean, Va, Us

• Served as product owner for one of Capital One's largest inner-source data processing frameworks, leveraging Apache Spark to enable engineers to build dozens of production use cases from customer campaigns to ETL.• Coordinated between a highly technical team of data engineers, teams of engineers throughout the enterprise using the framework, and business stakeholders to enable teams to easily build data pipelines with faster speed-to-market.• Led a team of engineers to modernize critical regulatory monitoring from a Teradata/Excel-based platform to a cloud-based solution leveraging Apache Spark.• Embedded as a developer with Pivotal Labs for six months creating a custom data analysis and reporting web application leveraging React, Flask, Pandas, and Docker, while practicing Extreme Programming Agile.• Worked with business and technical teams to manage transitions from legacy, on-premise servers and databases to AWS-based cloud solutions.• Designed and performed primary coding in Python for a data conversion tool which leverages and contributes back to open source Python libraries.

Mclean, Va, Us

• Under strict regulatory timelines and requirements, executed SAS and SQL coding and developed business intent for repricing several million accounts after the Capital One acquisition of HSBC, involving significant discovery of pricing terms and conditions across three accounts receivable systems.• Led an offshore team to identify and implement significant additions and changes to OCC and FRB regulatory reports for Capital One’s acquired HSBC card portfolios.• Managed two associates and one contractor in a team providing data analysis support to the Capital One Upmarket card business team.

London, Gb

• Coordinated business logic changes in SAS, data infrastructure changes in DB2, and production process implementation to migrate 80+ million accounts to a new accounts receivable system.• Piloted Tier 3 storage on Unix for end-user SAS processing, resolving issues with the vendor and drastically increasing space available to the business. Developed custom processes to monitor and report resource usage by users to ensure resource availability.• Administered SAS production environments on the mainframe and Unix.

London, Gb

• As part of HSBC Card Services’s Data Infrastructure team, regularly assisted business colleagues with accessing data infrastructure, interpreting data, and using tools like SAS, SQL, and NDM.• Created an account-level data feed for several million accounts to meet OCC regulatory requirements.• Administered SAS production environments on the mainframe and Unix.

London, Gb

• Rationalized and re-engineered processes to improve efficiency and shut down antiquated processes, savings tens of thousands of dollars a year in CPU processing and storage costs• Ran adhoc reporting for HSBC Card Services’s Finance, Marketing, and Risk teams

• Used SAS software and IBM MVS mainframe system to manage data and perform tasks related to SBC/AT&T’s payphone data. • Created interactive websites and interactive SAS applications to access payphone compensation payment data, internal reports, etc. • Reverse-engineered and re-implemented Oracle/UNIX reporting processes to run on IBM MVS Mainframe systems.• Handled subpoenas requesting call detail for law enforcement and assisted in requests for data from SBC’s legal and finance operations. • Collaborated with remote team members to create automated systems for retrieving and emailing subpoena requests.

Bachelor’S Degree, Computer Science

Education record