On August 23, 2017 Alluvium turned two years old. When I think back to founding the company those 29 months ago, it is truly a distant memory. This is primarily because of how much we have learned and built since then. Yet, unless you follow me on social media, or happen to have gone to one of a handful of meetups or conferences I attended, it would be difficult to know much about what we have learned and built. Today – I am very happy to report – that has changed.
Today we are re-launching our website, and in doing so finally bringing to light the hard work of our team. Through their collective efforts, and with the help of the incredibly creative and thoughtful team at Little & Co., I am extremely proud of our new public face. I encourage you to look around, but I also want to take some time to discuss some of the most important learnings we have collected over the past two-plus years.
There is a platitude in building data products the goes something like, “Never use machine learning for machine learning’s sake.” This sentiment was brilliantly elaborated in Google’s now seminal paper, Machine Learning: The High Priced Credit Card of Technical Debt. Yet, despite deep recognition of this problem, many people still ignore it. I am sympathetic. It is fun to experiment with new methods and then imagine a product and use case to fit.
Alluvium’s mission has always been to build tools that support men and women working in complex industrial settings. One of the first things we learned as we began to meet and observe these men and women in their work, and in particular observe how they interact with data and use it to make decisions, is that a unique tension exists between data discovery and data reasoning.
Take, for example, the data generated by a typical process control system. Most of the time that data looks identical from one moment to the next, and when things change, the change is consistently predictable. The challenge lies in discovering meaningful changes in these data without dedicating a tremendous amount of operator focus, which is arguably the most valuable resource in a manufacturing environment. Moreover, discovering these changes is not the core competency of the operators and engineers working with process control systems. Their training and experience equips them to reason about these data, but because this tension is pervasive, the potential value of that data is greatly diminished. Now, scale that problem up from a single control system to an entire plant, and from an entire plant to an organization’s global operation.
There are many potential ways that artificial intelligence can be used to begin addressing this problem, but this can also be a trap. Throwing machine learning techniques at this data creates a maze of product paths that can swallow a startup. Critically, we sought to seek firsthand knowledge, and after listening to operators and observing their workflows, it became clear the best application was dimensionality reduction. In moving the cognitive load of data discovery to an AI, we can relieve some of this tension and allow users to focus on reasoning about data, rather than working to discover novelty. This is the essence of Alluvium’s Stability Score, which is at the core of all our work. We have taken both a deliberate and opinionated approach to applying machine learning to industrial operations because we believe reducing the complexity of what operators must observe to be able to reason about their work is the most the best way we can support their work.
Soon after Alluvium was founded, I penned a short post describing my motivations for starting the company. A central focus of that post is the challenge of dealing with data from the physical world, and the unique opportunity this presented for leveraging human expertise in the data generating process to build novel data products. Our early intuition was to build products that could provide relief at the earliest point in the data discovery process. The term of art for this in industrial operations is “edge analytics,” analyzing data in real-time as production systems are in operation to provide the most timely insights.
In pursuing this intuition, we endeavored to build a core software platform for performing real-time Stability Score analysis from industrial data streams. As the principle engineering moved forward and we began to look for early proof-of-concept (POC) partners, we discovered an uncomfortable truth about the industrial market. While many potential partners were sold on the value, and even aspired to use sophisticated artificial intelligence tools in their production systems, the institutional barriers to actually accomplishing this were steep. The very idea of integrating a new tool into the critical workflow of an industrial operations was a tall order that was further complicated by the fact that we were a startup no one had ever heard of using techniques completely foreign to the industry. Not surprisingly, our initial efforts to build POCs at “the edge” fell flat.
While these early efforts were largely unsuccessful, we observed a distinct pattern. Potential partners would note that while edge deployment of a POC inside a production center was challenging, they were still eager to test the technology. In fact, we heard a similar story from several early customers. For years, many of them had been collecting and storing massive troves of production data that could be used to “replay” a production scenario as a POC. We started with a single one-off POC to support this discovery-through-replay paradigm, and eventually it became clear that we had discovered the market’s point of adoption. We found ourselves well-positioned to build a product to support it.
That realization became our flagship product: Alluvium Primer. I could not be prouder of the work the team has done in building Primer, and more thankful to our early pilot customers for their patience and conviction.
I do not like using growth as a metric for company success, particularly in the early days. That said, our team has more than doubled in the last year, and I am honored to work with such a deeply talented team. The breadth of this talent, and diversity of their collective life experiences, speaks to the unique challenges of our customers and the strength of our shared vision.
If you have made it this far, you may be curious as to how to join our team. I am happy to say that we are looking to fill many open positions, and to continue to increase our team’s diversity and the depth of our talents.
What excites me most about our website relaunch is this blog. I am eager to getting back to writing, and in particular writing about the education and experience of building Alluvium. I am also eager for my colleagues to have a chance to share their ideas and experiences, from the technical to the personal.
I will be starting with a topic that I am often asked about: recruiting. I suspect that everything I have to say on the topic will not fit into a single post, but be sure to check back soon.