Working on getting new product out the door. Exciting!AWS, Athena, Lambda, CloudFormation/CDK, Terraform, CloudWatch, S3, EMR, Pyspark, Airflow, Postgres, APIs, Python, Flask, ElasticSearch, Snowflake, Kubernetes, Cosmos DB, Finops, ETL, pipelines, Cloud and Software Architecture, Data Quality, Product Integration.

https://spot.io/products/eco/

I built pipelines using Airflow, Spark, APIs, and Python. I also worked with customers and product management to get data the way they need it.

I helped Digital Safe meet client requirements to validate the processing of billions of records, upgraded safe components (e.g., ZooKeeper, firewall) to meet business priorities, and collaborated with clients on continuity of business tests. I also continued to support eDiscovery business as data loading and database design subject-matter expert.* I personally designed and implemented Live Capture Reconciliation system that compared SMTP logs for hundreds of millions of messages daily to identify individual messages that failed to archive and raise alerts before SLA violationsOfficial job title: Application Management Service Delivery Consultant Specialist

I created and improved ETL processes in SaaS eDiscovery business to improve loading throughput and accuracy. I also created tools to analyze and remediate data issues for small and large data sets.* I bridged the gap between clients' structured data loading (SDL) needs and the functionality delivered in a major upgrade of our eDiscovery platform. This was accomplished by writing robust and indispensable scripts and utilities and defining processes around them that data loaders could follow to normalize data while identifying and handling exceptions. * I successfully moved the transformation and loading logic I created for SDL into the product by collaborating with product managers/product owner and development team. * I performed multiple ad hoc audits of billions of records to validate data and remediate known data issues, particularly deduplication problems.

I created custom workflow stored procedures in SQL Server that enabled efficient document reviews by our clients; performed client and internal requests to make ad hoc changes to data on live production systems; designed applications that met department's automation and quality control needs.* I designed a columnar CSV and eDiscovery loadfile editor as a personal passion project that became a core part of multiple internal processes. This replaced the use of ad hoc scripts, Microsoft Excel, and text editors, leading to more accurate data, clearer standard operating procedures, less anxiety among colleagues, and more satisfied customers.* I automated the loadfile conversion/transformation process that previously took up significant time of multiple senior resources running ad hoc solutions. This application drastically improved transformation and loading turnaround times, as well as data quality, while moving the responsibility to junior resources.* I authored learning resources for department, included reference cards to the SQL Server backend and a database schema training. This training became standard for all department members and reference cards were still being used daily a decade later.