Daniel Collins Email & Phone Number

Oakland, Ca, Us

New York, New York, Us

⦁︎ Architected Chainalysis's Next Generation Enterprise Data Warehouse on the Databricks platform, optimizing performance and scalability for enterprise use⦁︎ Extended DBT functionality to meet enterprise-level compliance requirements and support complex use cases⦁︎ Developed custom SCD2-inspired materialization to enhance efficiency and data management: ∘︎ Implemented checksum caching for optimized snapshotting on large tables ∘︎ Designed intelligent schema change handling to avoid unnecessary version invalidation ∘︎ Integrated with Databricks and Snowflake Time-Travel for high-precision versioning, enabling retroactive loading of historical data and recovery from failed jobs ∘︎ Extended metadata tracking in version rows, including field-level changes, CDC operation types, and version lineage linking ∘︎ Built in primary key uniqueness checks and other health validations to ensure data integrity across the snapshot lifecycle ∘︎ Developed macros for snapshot table recovery and rollback, safeguarding against data loss or corruption⦁︎ Enhanced other DBT materializations and ref functions: ∘︎ Utilized Databricks table tagging and regex scanning of SQL content to enforce data integrity and model expectations⦁︎ Automated DBT layer file generation by creating scripts that scan SOR landing tables to bootstrap SQL, YAML, and markdown files for rapid source onboarding⦁︎ Designed EDA and data quality scripts for SOR landing tables, providing health reports to maintainers and improving overall data governance

New York, New York, Us

⦁︎ Architected DailyPay’s data warehouse ecosystem: ∘︎ Built a comprehensive DBT ELT workflow on Snowflake using the Lakehouse model ∘︎ Developed a near-real-time CDC pipeline integrating over a dozen SORs ∘︎ Designed a growth analytics and financial reporting pipeline, compliant with public company readiness, leveraging SCD2 for accuracy⦁︎ Scaled a two-person project into an enterprise-grade solution: ∘︎ As the senior engineer, led the evolution of a small project into a platform now supported by a 20-person team⦁︎ Provided technical leadership to a team of 5-7 analytics engineers, ensuring best practices and high-performance output across the team⦁︎ Extended core DBT functionality to support enterprise-scale volume and performance, delivering key features needed for optimal results⦁︎ Developed automated routines for regulatory compliance: ∘︎ Created workflows to anonymize PII for specific users, ensuring adherence to CCPA, GDPR, and other legal requests⦁︎ Authored algorithms to analyze usage patterns, enabling early detection of system health issues and proactive issue resolution⦁︎ Led numerous data initiatives and projects over a span of 4+ years, contributing to the platform's success and ongoing development

Mclean, Va, Us

New York, Ny, Us

San Francisco, California, Us

Round Rock, Texas, Us

Boston, Ma, Us

⦁︎ EC500: High Performance Computing. 2 Sem.⦁︎ EC413: Computer Architecture. 1 Sem.⦁︎ EC330: Algorithms and Data Structures. 1 Sem.

Amsterdam, Noord-Holland, Nl

Masters, Data Science

Bachelors, Computer Systems Engineering