Do you need a machine learning proof of concept?
Machine learning & artificial intelligence solutions keep gaining popularity among business leaders, and the hype finally wears off. As McKinsey's "State of AI 2021" report findings state, AI adoption is still on the rise: in 2021, 56% of respondents reported AI adoption in at least one function, an increase from the previous year's 50%.
Having made itself at home in the business world, ML is now more widely adopted for the right reasons rather than its cool factor - and that's excellent news. More companies no longer invest in projects with an unclear business value and opt for business-oriented solutions. As leaders become better educated about the opportunities and challenges posed by ML projects, they remember that tech is a means of getting closer to achieving their objectives. Having that in mind, they often decide to start with a machine learning proof of concept. In this article, we’re having a look at what a PoC is, why it might be beneficial, and how it fits into the ML development process.
What is a Proof of Concept?
A Proof of Concept, or PoC for short, is a method to evaluate the feasibility and potential of a given concept - a project, product, or design. It's short-term and cost-efficient and is used to determine whether the idea is attainable before being put into the system development process.
While PoC may be seen as an unnecessary expenditure, it is a crucial element of product development: every idea has to be reviewed before it hits production.
The proof of concept's goal is to validate whether the product will solve the problem at hand. According to Failory, 34% of businesses that fail do so because they lack a product-market fit. Uh-oh, somebody skipped the PoC stage here! So before you put your money into a project check: Is it workable? Is it profitable?
Why do you need an ML proof of concept?
Just as in any software-related project, a machine learning PoC is a reasonable first step into bringing the product to life.
Why would you jump head-first into any project when you're not yet sure what the exact results will be? Same goes for ML projects, or perhaps even more so - after all, an ML project is a research and development project in nature, so it entails some risk - in ML projects, you can face issues including the misalignment between the problem and solution.
With machine learning, not all solutions will be obvious. Naturally, there are cases when you know exactly what you want done, say, a recommender system, and it's now a well-known fact that ML excels at product & content recommendations, so your engineers will immediately know what to build. However, not every project is that simple. In many cases, the work being done with ML is experimental, it's a research and development project, which means that the path from the problem to the answer is not a short, straight line.
In the process of building an ML PoC, you focus on how ML can add value to your processes. As stated in Carnegie Mellon University's "AI Engineering: 11 Foundational Practices" white paper, the most essential rule of artificial intelligence projects is that they can and should be solved with AI:
"Ensure you have a problem that both can and should be solved by AI. Start with a well-defined problem, understanding what you want to accomplish and the outcomes you need, while ensuring you have data available to infer those outcomes. Once you know that you have a potential AI problem, verify that other, simpler options— which could be better solutions—do not exist. AI is not a panacea and is often a more complex, less efficient solution for problems where other solutions may already exist."
|_source: AI Engineering: 11 Foundational Practices by Software Engineering Institute at Carnegie Mellon University_
The benefits of a proof of concept
We're now well aware that the PoC is a phase when you validate the idea and explore possible solutions to your business problem - in the case of ML projects, it's assessing what machine learning model (or models) will be the best fit for your use case. But what are the benefits of starting with the PoC?
Validation of the idea
That's the main reason to do the PoC in the first place, and it's also the most crucial advantage of doing it. It's already been covered in parts in the previous paragraphs, but let's do a short recap: the proof of concept helps you look critically at the project idea and assess the ways to solve it. No big commitments are needed at this point.
Risk mitigation
Since you analyze and evaluate your project first, you start strategically and minimize risk of failure. How so? First of all, you can cross unfeasible ideas off the list, so you don't invest money in senseless projects. Then, a PoC helps you set a roadmap of what needs to happen next, so you're moving towards your goal at a steady pace, making sure that you're always going in the right direction. Remember that in every project, the right metrics need to be in place - and checked regularly.
Implementing good practices for data collection and storage
Data-related issues are among the most burning problems that prevent companies from going into ML. When you want to build an ML model for your use case, you need to have the right data. The right data means a big amount of it and the right type. The ML engineers or data scientists involved in your PoC will help you understand how you can make use of the data you have, whether you need to obtain more data and how you can do it, and how you can store your data in an appropriate, secure way.
Team alignment
ML projects require the involvement of people with diverse skills: data engineering, ML modeling, software development, and domain knowledge - so if it's a model for the medical industry, a person with the relevant expertise will be a necessary addition to the team. However, matching all these different worlds isn't always that easy. Sometimes, you might need an intermediary - a person who can grasp the perspectives of all crew members and help them find common ground. The PoC is the right stage to do that and learn to collaborate well.
Setting the foundation
The PoC is a starting point for bigger projects, helping set the right goals, plan the work, assess the solutions, and explore ideas that have not come to your mind before. After that, there's a clear path ahead of you!
Business-oriented machine learning
When I use the phrase "business-oriented machine learning", I refer to ML projects that focus on the end goal: the solution to the problem.
The issue with machine learning projects is that they need to combine business, science, and technology. That also means they require interdisciplinary collaboration among team members with different skills: data science, ML engineering, software development, and domain expertise. Connecting all these can be challenging, as is finding the balance between the fields: so it can happen that promising projects never get productionized when missing the business element, or expectations are not realistic when the business dominates the tech.
Despite the growing popularity of ML-based systems, many companies still face challenges implementing them. In O'Reilly's report "AI adoption in the Enterprise 2022", it is shown that the main bottlenecks to adopting AI solutions have remained very similar throughout the past 2 years; the respondents stated that the biggest obstacles on the way to them adopting ML were:
- lack of skilled people - 20%
- lack of data or data quality issues - 20%
- difficulties in identifying appropriate business use cases - 16%
Bottlenecks to AI adoption, source: "AI Adoption in the Enterprise 2022" by Mike Loukides, O'Reilly
Given that organizations considering adopting machine learning face these difficulties at the very beginning of their journey, to overcome these issues, they must make a good start, no matter if it's their 1st ML project ever or perhaps they've done dozens before, so a Proof of Concept seems just about right. So how is it done?
Developing a proof of concept
The elements of a PoC may vary depending on what you're planning to build, so you need to take into consideration that the below is not a universal recipe for a successful multi-purpose PoC. It is, however, a battle-tested way to deliver a quality PoC for ML projects.
ML workshops first
No ML project indeed starts with coding. It just can't. At the beginning of the way, we sit down, all together, to discuss what the PoC is supposed to prove. During the workshops, we cover your needs and requirements, opportunities and limitations posed by possible solutions, success criteria, and evaluation methods together with appropriate metrics. The ML workshops create space for business and scientific ideas to collide so we can verify whether ML is a viable solution to your business problem, identify the scope of the project, agree on the next steps of the process, and more. The workshops are a starting point for your ML project after which you can decide to move forward or stop.
Science as a Service
The PoC is a mini version of our Science as a Service process, which includes the following steps:
Exploration of solutions via available data
That's a stage when you discuss your business profile with our engineers, identify your pain points and the problem to be solved with ML, and define available data.
Data analysis and preprocessing
The ML team analyzes the data and searches for supplementary datasets. They suggest other relevant data sources and perform feature engineering.
Machine learning model training
ML engineers select base models and modify them to suit the problem, train the chosen models and iterate to define the final configuration.
Machine learning model deployment
ML engineers and MLOps work on integrating the models with your systems, define the required infrastructure, implement the model, and present the results.
The PoC is a lighter and cheaper version of the target software that will show you whether the created model proves profitable and validate the assumptions. Thanks to that, you quickly learn whether the ML solution meets your expectations and decide on the next steps.
Summary
The Proof of Concept may be an underestimated element of project development, but I hope that this article has brought closer why it matters and how it can help your organization start with machine learning. If you're unsure how to start or have any questions considering ML adoption, drop us a line and we'll help you with that.