David Callaghan

David Callaghan Email and Phone Number

Databricks Leader / Data Privacy Innovator, Trainer and Speaker @ Perficient
555 Maryville University Dr. Suite 600, St Louis, MO 63141, MO 63141, us
David Callaghan's Location
Miami, Florida, United States, United States
About David Callaghan

I have over 30 years of professional experience architecting, coding and implementing enterprise data-driven solutions. I code extensively in Java, Scala, Go, Rust and Python. I have designed and implemented enterprise solutions in the cloud using Databricks, Snowflake, Neo4J, Hadoop, MongoDB and DataStax as well as cloud-specific solutions such as Redshift, CosmoDB and BigQuery.I concentrate in bringing practical DataGovOps solutions to regulated enterprises to release value. I had contributed to the Apache Atlas project, which brings data lineage and governance to Hadoop. I have developed internal solutions for Databricks Delta Lake and Snowflake to augment their data governance using machine learning for automated classification. I also have developed tools to provide for data deidentification for compliance with CCRA and GDPR among others.

David Callaghan's Current Company Details
Perficient

Perficient

View
Databricks Leader / Data Privacy Innovator, Trainer and Speaker
555 Maryville University Dr. Suite 600, St Louis, MO 63141, MO 63141, us
Website:
perficient.com
Employees:
1
David Callaghan Work Experience Details
  • Perficient
    Senior Solutions Architect
    Perficient Jun 2017 - Present
    St Louis, Mo 63141, Mo, Us
    Pfizer : Created Databricks Accelerator to meet international pharmaceutical trial data privacy regulations. Implemented GenAI solution using ChatGPT-licensed product on internal safety data to augment adverse event reporting process. FNBO: Architected CCRA compliance initiative by developing custom privacy preserving algorithms BCBSMI: Migrated Cloudera to Databricks running on AWS. Migrating parquet data to Delta lake, rewrote custom spark code Bancolombia: Developed a stablecoin in Go as a proof-of-concept for a possible offering.Ferguson: Rewrote Java Camel code running locally in Kubernetes to run as Azure Functions in Azure Service Bus/KEDA Broadridge: Used Go on AWS to build wealth management system that provided an OpenAPI front for performance reporting, order management, trade routing, etc using microservices on the backend to connect to client systems.StorageMart: Migrated analytics data from SQL Server to Snowflake on Azure performing all tasks from setup to migrating data with Talend to building more effective queries.TCF Bank: Implemented custom SparkML solutions for risk and marketing including a dynamic model for mobile check deposit thresholds.BCBS MA: Participated in the strategy and build-out of new big data platform analytics platform using Cloudera running on AWS. Led five person team involved in implementing predictive analytics in PySpark. Wrote AWS Glue code to make data available in Athena instead of Hive.Mastercard: Built Spring Data framework for Cassandra implementation. Built RESTful API implementation for Single View of Customer using Spring Boot. Prototyped blockchain solution in Go for managing consumer opt-out requests.Publix: Built data platform on Azure with CosmosDB with Spark to enable ML for recommendation engine.Volkswagen: Built AWS IoT platform to manage electric car stations using Java.Ford Motor Company: Built DataStax back-end (Cassandra, Spark, Solr) on Azure to support Facebook AI chatbot.
  • Bank Of America
    Hadoop / Spark Developer
    Bank Of America Feb 2016 - May 2017
    Charlotte, Nc, Us
    Responsible for developing and implementing an Instrumentation as a Service platform processing 10K records per second from 1K applications using 30+ analytic models. Business partners are able to send logging and other asset monitoring data to the system in json as a file, through a web service or directly to Kafka. This data is processed through our lambda architecture with HBase as the speed layer and both Hive and Impala (using Parquet) as the data layer. Kamanja servers as the real-time decisioning support engine, implementing the models on data at rest and in real time.Introduced and implemented a POC for using blockchain for data governance using Hyperledger for the chaincode and HBase for the off-blockchain storage, extending the use of HBase's cell-level tagging to implement an eventual trust pattern. Completed POCs to replace our current system with a home-grown Spark implementation built with Scalding using the Akka/Play framework. Additional POCs involved Cassandra for faster ingestion and easier reporting access to the real time layer as well as MemSQL to provide a single data repository for speed and batch. I've also implemented potential alternatives to the enterprise platforms using Docker, Kubernetes and Mesos. Worked with the process of evaluating and onboarding FinTech companies.Using Spring Boot and MongoDB, I built an early detection and response system that monitored Zookeeper, Kafka, HDFS, HBase, Hive, Spark and Kamanja. These microservices formed a unified platform called Level Zero to detect and resolve issues before they were raised by Level One support, substantially lowering the cost of ownership for these systems.
  • 2C Data
    Big Data Solution Architect
    2C Data Apr 2013 - May 2017
    Charlotte, Nc, Us
    Responsible for developing and implementing Big Data solutions. Current projects include:Orphans In The DesertCluster sufferers of distinct rare diseases into groups large enough and with a sufficiently well-defined set of symptoms and markers to allow for pharmaceutical companies to develop drugs and for health care providers to develop protocols for treatment using IBM Watson, Cloudant and Bluemix.Professional Investors NetworkA financial services marketing company specializing in building real estate investment entrepreneurs through training, networking and access to financing. • Educational element include articles and training material, including video, is stored in MongoDB and searchable using Lucene/Solr. Document analysis is done in MapReduce with Hive.Business Layer DSLs developed in Scala, back-end development in Java, financial analysis algorithms coded in R.QUIOBO.COBig (Fast) Data PAAS for Communication Service Providers (CSPs) operating in Colombia, SA using AWS.• Store Call Detail (xDRs and CDRs), network and customer data using Flume into Cassandra and provide Business Intelligence using MapReduce and custom and Mahoot-based algorithms for fraud detection (subscription and superimposition fraud), customer profiling, network fault isolation • Business Intelligence reporting using Tableau, querying available through Hive
  • Sparks Ignite, Inc.
    Bringing Big Data And Blockchain To The Cloud
    Sparks Ignite, Inc. May 2015 - May 2016
    Mint Hill, Nc, Us
    Responsible for helping companies successfully implement and realize measurable value from their Big Data, Fast Data and NoSQL implementations.
  • Wells Fargo
    Big Data Developer
    Wells Fargo Jul 2014 - May 2015
    San Francisco, California, Us
    Responsible for developing and implementing Big Data solutions.• Built Regulatory Compliance engine using Kafka, Storm, Drools and Hive to analyze MS Exchange emails. Developed analytic algorithms in R and Mahout as well as text analytic tools using Lucene and Pig.• Built Flume to HBase workflow for Lending Grid to increase their storage capacity and provided Phoenix to integrate into existing Apache Camel applications.• Developed solutions for new regulatory reporting requirements for Treasury Analytics using Hbase, Hive and SAS. • Introduced standards and practices at the development (TDD) and operations (Continuous Deploy) level.
  • Bank Of America
    Big Data Developer
    Bank Of America Feb 2014 - Jul 2014
    Charlotte, Nc, Us
    Responsible for developing applications in Java using Hadoop, Hive and Hbase to support the Intrusion Detection Team. • Built secure Java platform on JAAS for Kerberos authentication and authorization at the job level for MapReduce applications• Implemented bi-directional integration of Hive, HBase and Giraph • Developer native Java wrapper to implement existing R algorithms in Map Reduce• Introduced standards and practices at the development (TDD) and operations (Continuous Deploy) level
  • Duke Energy
    Architect / Senior Java Developer
    Duke Energy Mar 2007 - Mar 2013
    Charlotte, North Carolina, Us
    Senior Java Developer/Architect responsible for evaluating/implementing enterprise-wide technology initiatives including Big Data (Hadoop, HBase and Cassandra), Enterprise Application Integration (Spring Integration), rules engine (Drools) and a standard Java framework using Spring, Hibernate with Maven and Jenkins. Smart Grid• Architected Initial phase of Smart Grid implementation targeting 1.5M smart meters using Oracle Meter Data Management 2.0• Developed Java app using Spring Integration to integrate Oracle MDM 2.0 w/ proprietary customer billing solution. • Prototyped Hadoop map-reduce algorithms and Hive for precalculating data before MDM import.Smart Energy • Developed a Java framework for multiple ongoing prototypes of new and experimental clean energy technologies including electric vehicles and smart meters.• Prototyped Cassandra solution for producing social graph from smart meter data (smartenergycharlotte.com/how-are-we-doing)Pole Inspection • Developed Java app to dynamically create Junit code and xml files from a Drools rules engine in order to test 70K+ inspection use cases.Work Management • Developed web-based call center mgmt app using Drools for complex rule management. Gas Odor Correlation • Developed web-based Java app for real-time analysis of data extracted from call center operations, pipeline structural data, population statistics and maps to predict potential gas leaks in densely populated areas for crew dispatch prioritization using user-configurable spherical trigonometry equations.Mobile Work Management System • Developed SOA system for taking messages from various new and legacy apps across companies from MQ using jms then interpreting/routing the work request to the appropriate system. Enterprise Solution Architecture Framework • Prototyped smart-phone and tablet web interface using JSF 2.0 and modifications to enterprise CSS.• Prototyped Android application for scanning smart meter tags.
  • New York Board Of Trade
    Senior Java Developer
    New York Board Of Trade Jun 2005 - Jan 2007
    Us
    Developed Market Surveillance applications in Java as well as an electronic trading platform NYBOT needed to move from open outcry floor trading, which it used exclusively since 1870, to an electronic platform as a result of relocation to NYMEX after 9/11. Electronic Trading• Developed Struts/Hibernate web application front end for external trading partners and Swing desktop application for floor traders. • Developed the model objects that represented standard commodity contracts as well as esoteric derivative instruments like crack spreads. • Developed secure account creation and management and secure transaction management including auditing and nonrepudiation components.Compliance Applications• Developed market, position and financial surveillance desktop apps in Swing.
  • Li & Fung Usa
    Senior C# Developer
    Li & Fung Usa Jun 2003 - May 2005
    C# and BizTalk developer responsible for developing international logistics management system as well as developing a business process integration system between New York, Istanbul and Hong Kong offices with a nine member team.Logistics Management System• Developed a logistics management system in C# to handle the movement and storage of goods from the point of origin, typically southeast Asia, to major market retailers.• Developed a business process integration system using BizTalk and Sterling Gentran to integrate EDI transmissions with the logistics management system.• Developed supply chain management intelligence system using Cognos’ OLAP and BI. Won 2005 Wired 40 Award
  • Skillsoft
    Senior Test Analyst
    Skillsoft Oct 2002 - Apr 2003
    Nashua, New Hampshire, Us
    I worked for Thomson/NetG (now Skillsoft) putting together a unified testing system for compliance with international security, privacy and accessibility regulations.• Created a unified system of software testing protocols using TestDirector, Mercury Load Runner and RequisitePro to measure compliance in these areas:o accessibility testing procedures for Section 508 (US), Web Content Accessibility Guidelines (EU) complianceo USA Government Information Security Reform Act, UK Data Protection Acto P3P implementation and Privacy Policy complianceo Security Testing, Forensics w/ ISS RealSecure & various Unix tools on Sun, BSD for OWASP compliance
  • The Chiva Group
    Senior Php Developer
    The Chiva Group Mar 2001 - Sep 2002
    Owner and CTO marketing my own web-based platform for the Spanish-language call-center management of satellite services and managed a five member PHP team in Bogota, Colombia.
  • Dell Services
    Senior Java And Php Developer
    Dell Services May 2000 - Feb 2001
    Round Rock, Texas, Us
    Lead developer of 15-member team to build first international trade finance auction site.• Developed a secure, internationalized web-based auction site using PHP.• Developed the back end processes to accurately manage negotiable paper such as bills of exchange, promissory notes and other securities for trade in Java using Oracle.• Developed the secure exchange with Deutsche Bank using Java web service.• Assisted Deutsche bank in implementing the receiving web services. Won Global Finance's Best Trade Finance Software (2001)
  • Revelwood
    Vb Developer
    Revelwood May 1999 - Apr 2000
    Florham Park, New Jersey, Us
    Develop SmartSite Builder, a 5GL domain specific language sold as a commercial programming language for OLAP• SmartSITE Builder is essentially a 5GL domain specific language. Visual Basic was used to create the IDE for the 5GL interface while Java was used to create the domain specific language, which was essentially an abstraction of the core API’s of the leading OLAP manufacturers Applix and Hyperion.  Won a Computerworld Top 100 Software Award in 2000
  • D. Proctor Engineering, Inc.
    Delphi Developer
    D. Proctor Engineering, Inc. Apr 1998 - May 1999
    Fair Oaks, California, Us
    Contracted to develop front end for commercial CheckMe! expert system for HVAC efficiency.
  • Severson & Werson
    Vb Developer
    Severson & Werson May 1998 - Apr 1999
    San Francisco, Ca, Us
    Automated creation of legal documents related to replevin and bordereau-creation.
  • Corporate Property Investors
    Vb Developer
    Corporate Property Investors Mar 1997 - Mar 1998
    IT Manager w/ 6 employees reporting directly to CIO tasked with creating new hardware and software infrastructure for property management of 30+ malls.
  • Societe Generale
    Vb Developer
    Societe Generale Jan 1997 - Mar 1997
    Paris, Fr
    Contracted by international banking division to fix VB, Sybase content management system.
  • Cigna Healthcare
    Software Tester
    Cigna Healthcare May 1996 - Dec 1996
    Bloomfield, Ct, Us
    Managed testing in Integration Lab during migration of one million PCs from OS/2 to WinNT
  • Aamco Transmissions
    Vb And Cobol Developer
    Aamco Transmissions Feb 1995 - Mar 1996
    Us
    Responsible for full lifecycle of migration from AS/400 to client/server environment.
  • Broker Resource Center
    Foxpro Developer
    Broker Resource Center Jul 1992 - Dec 1994
    Built financial analysis tools for General American Life Insurance agents using FoxPro and Access

David Callaghan Skills

Java Hadoop Databases Big Data Oracle Sql Xml Software Development Web Services Soa Hbase Business Intelligence Integration Apache Pig Unix Javascript Solution Architecture Yarn Mysql R Apache Storm Apache Kafka Spring Framework Flume Mongodb Maven Falcon Java Enterprise Edition Cassandra Android C# Blockchain Jms Spring Hyperledger Jsf Nosql Hive

David Callaghan Education Details

  • Indiana University Of Pennsylvania
    Indiana University Of Pennsylvania

Frequently Asked Questions about David Callaghan

What company does David Callaghan work for?

David Callaghan works for Perficient

What is David Callaghan's role at the current company?

David Callaghan's current role is Databricks Leader / Data Privacy Innovator, Trainer and Speaker.

What is David Callaghan's email address?

David Callaghan's email address is mr****@****ail.com

What is David Callaghan's direct phone number?

David Callaghan's direct phone number is +170424*****

What schools did David Callaghan attend?

David Callaghan attended Indiana University Of Pennsylvania.

What skills is David Callaghan known for?

David Callaghan has skills like Java, Hadoop, Databases, Big Data, Oracle, Sql, Xml, Software Development, Web Services, Soa, Hbase, Business Intelligence.

Who are David Callaghan's colleagues?

David Callaghan's colleagues are David Greer, Ntasha Sharma, Palmer Phillips, Vivek Vepa, Akash Chavhan, R Anupriya, Stacy Beine.

Free Chrome Extension

Find emails, phones & company data instantly

Find verified emails from LinkedIn profiles
Get direct phone numbers & mobile contacts
Access company data & employee information
Works directly on LinkedIn - no copy/paste needed
Get Chrome Extension - Free

Aero Online

Your AI prospecting assistant

Download 750 million emails and 100 million phone numbers

Access emails and phone numbers of over 750 million business users. Instantly download verified profiles using 20+ filters, including location, job title, company, function, and industry.