Michael Collado Email & Phone Number

Seattle, Washington, United States

Seattle, Washington, United States

At Astronomer via Datakin acquisition, I'm the technical lead for data observability and data lineage in the Astronomer product. I'm a primary contributor to the open source Marquez Project and to the OpenLineage open standard and associated integration projects. I've focused largely on improving the Airflow integration with OpenLineage and adding support for processing that data within the Astronomer project to offer better insight into why failures happen within data ecosystems and where engineers should focus attention.In addition to technical contributions, I've worked with multiple developers to improve the engineering culture, helping to establish the design review standards and protocols, driving improvements to the CI/CD workflow to keep teams moving quickly and independently, and helping to design a mentorship program to enhance the quality of engineers throughout the company.

Seattle, Washington, United States

I was the technical lead for the Datakin engineering team, responsible for the initial data lineage product built on the open source OpenLineage and Marquez projects. I am a primary contributor to the OpenLineage standard and the various integration projects, including the integration with Apache Spark and Apache Airflow. I built most of the framework for the Spark integration, including the extension points that have been used by companies like Microsoft and Apple to integration OpenLineage into their own data ecosystems. I'm also a contributor to the Marquez project, particularly involved in its consumption and storage of OpenLineage data. I helped build much of the Datakin project as an extension of Marquez, building tools that offer data engineers and scientists insights into the operations of their data pipelines and highlighting the impact that upstream systems have on their pipelines. In addition to technical contributions, I helped establish the engineering culture at Datakin, defining the software development lifecycle, expectations around testing and documentation, and establishing a culture of design review. I also contributed multiple technical blog posts advocating and detailing the OpenLineage standard and Marquez.

Greater Seattle Area

Sr Software Engineer working in Data Infrastructure. I owned the services that get vehicle data to the analysts and ML systems that make autonomous vehicles reality. I designed and delivered the service that more than half the company uses to stream PBs of ROS data to our Data Warehouse, ML training, simulation, and QA systems. This included clients working Apache Beam, Apache Spark, and custom command line tools for AV engineers.My team owns the data lake, managing billions of files, and the catalogs clients use to access that data. I also own tools and libraries that provide access for clients performing batch analytics on road and simulation data. I work primarily in Go, but also work with services in Python and analytics libraries in Scala.

Greater Seattle Area

In Global Learning and Development, I'm helping to revolutionize education and learning. We're using machine learning and big data analytics to change how people progress on their learning journeys. I work with a hodgepodge group of machine learning experts, learning/cognitive science academics, and top notch engineers to build products that maximize learners' ability to find the right learning content to achieve their goals.

Seattle

On the Clickstream/Nexus team, I'm the technical lead, working to build the next generation platform for ingestion, storage and processing of large-scale web and mobile device data. I've led the project from conception to design, prototype, and successful launch- used by over a hundred teams to collect and analyze hundreds of data sets- hundreds of billions of records per day. We've made traffic data available sooner, to serve more use cases, more accessible, and cheaper to store and query. Our tools are used by software engineers and data scientists throughout the company. We support real time and batch processing of hundreds of billions of events per day using the latest AWS services and open source frameworks, like Hadoop and Spark. We serve customer behavior data from web, mobile, and IoT devices that are processed by hundreds of teams across Amazon. Our primary systems include streaming ingestion of real time events (millions per second) from web and digital platforms, storage and metadata management, and client libraries that work with Hadoop-oriented technologies, including Hive and Spark. In addition to our own processing, we maintain tools that enable teams throughout the company to process customer behavior data for things like recommendations, ad targeting, robot detection, A/B testing, and more.

Seattle

I was the technical lead on Amazon's A/B testing platform team. I worked with two teams of developers and a team of statisticians responsible for building and maintaining the big data analytics platform that detects the effects of changes on Amazon websites. We processed some of the largest data sets in the world- hundreds of billions of triggering events and website/mobile customer events daily. Over the course of my tenure, we migrated the platform from a custom, home-grown solution to an AWS cloud-based, map-reduce system taking advantage of the latest analytical tools. We enabled custom metrics so that website and mobile developers could innovate freely, while knowing that important business metrics were always monitored. We also implemented distributed algorithms for various types of metrics, including percentile calculation for data sets with billions of records. In addition to the analytics platform, we also maintained the UI that surfaced the analysis results and the high traffic service that randomized users in running experiments. I also had 3 software patents granted during my time on this team.

I was the technical lead on Amazon's A/B testing platform team. I led a team of ten developers responsible for building and maintaining the big data analytics platform that detects the effects of changes on Amazon websites as well as the UI that surfaces the analysis results and the high traffic service that randomizes users in running experiments.

Springfield, Missouri Area

Designed and implemented CMS platform features and enhancements as well as customer and in-house web applications.• Accomplished major upgrades to legacy Struts-based platform core using modern industry-standard technologies including Spring framework, Spring Security and Grails's GORM Hibernate Layer while maintaining backward compatibility for existing applications.• Created and maintained complex task and time tracking system used in-house by Account Managers, Project Manager and Development staff.• Maintained numerous client applications, dealing directly with customers via phone and e-mail regarding bug-fixes and application enhancement requirements.

Springfield, Missouri Area

Responsible for application design, implementation and maintenance of Java codebase for product websites, supplemental projects, including traffic logging, affiliate functionality, and internal administration applications. Directly applied design patterns and principles maximizing code reuse, resulting in a decrease in development time as high as 80% for some projects.• Primary designer and implementor of a complete overhaul of legacy application codebase, using Spring and Hibernate to implement data access and service layers, and Spring MVC and Tiles for web layer.• Sole designer and programmer of traffic logging and analysis application using Spring and Hibernate for data access and service layers, Spring MVC, DWR and Prototype JavaScript library for web layer, and JAX-RS.• Primary designer and programmer of Multivariant Testing framework using Hibernate and custom Tiles integration.• Completely overhauled core shared libraries, increasing flexibility and reuse and added advanced application configuration, JMX for remote management, and CMS functionality for database managed Velocity templates and application messages for customizable affiliate content.• Trained and supported programming staff in usage of new APIs, toolkits, and application configurationDesigned and programmed Groovy/Grails application for automated website testing, including custom Groovy DSL for test specifications and integration with existing JUnit reporting tools.• Trained QA staff in usage of Groovy DSL to write automated testing scripts, including more advanced topics such as XPath, regular expressions, and custom HtmlUnit tests• Worked extensively on JavaEE server setup and configuration, including Tomcat 5.0, 5.5 and 6.0 and Glassfish Application Server 2.0 and 2.1• Maintained legacy applications running on Apache Struts framework and servlets• Worked on JSP templates, including custom tag libraries and wrote HTML and JavaScript for web frontend.

Springfield, Missouri Area

Responsible for web site design and programming for company web site as well as client projects.• Created custom web framework in PHP for client project, implementing e-commerce module for credit card transactions and custom product management interface• Designed and implemented web sites for company clients.• Modernized company website, using Ajax, PHP and MySQL for dynamic functionality.• Met directly with clients aiding in the sales process, gathering site requirements and discussed progress throughout development.