How a portfolio approach to AI helps your ROI

Instead of computing the success or failure of AI initiatives on a project-by-project basis, companies using the portfolio approach compute the ROI for all their AI initiatives. A portfolio approach works in other areas of business, and the same principles apply here. Take a look at three relevant examples and the lessons for AI.

Test and learn: Pharma’s drug discovery and approval process

In the pharmaceutical world, developing a new drug takes an average of at least ten years and costs over $2.6 billion. Literally thousands and even millions of molecules and investigative drugs are studied during the initial drug discovery and preclinical trial phases of the R&D process. Successful drug candidates then go through three phases of clinical trials, followed by a lengthy Food and Drug Administration (FDA) review process before a drug is potentially approved. Even after it’s been approved, a drug may need to go through Phase 4 trials for research and monitoring purposes. Ultimately, only 12% of all drug candidates that enter preclinical trials win final approval from the FDA.

Given this extensive process, it is very difficult for pharmaceutical companies to determine whether a particular drug candidate will be successful, so they take a portfolio approach. While a large proportion of candidate drugs fail clinical trials, a few succeed, and some even become blockbusters that more than compensate for the rest of the drugs in the portfolio. 

There are a number of similarities between the drug discovery process and the machine learning (ML) model development process. First, there is a high level of uncertainty for individual initiatives, whether you’re looking at how effective a drug candidate will be or how accurate an ML model performs. That requires companies to try different approaches to development, and, in the case of AI, that might mean considering statistical models, deep learning approaches, rule-based approaches and so on. Ultimately, you’ll need to use an ensemble of different approaches to improve the robustness of results. 

AI development also mirrors drug development even after initial success. As with a drug that requires continuous post-approval monitoring and adverse drug reaction reporting, AI must be monitored continuously. Is the model performing to its original specifications? Is it time to revise or retire a model? Unfortunately, many models don’t take this into account when they’re initially developed — or even at the stage when they are being deployed in production.

Risk-return tradeoff: Allocating portfolios and optimizing financial assets

Arguably, the single most salient invention of twentieth century finance is modern portfolio theory (MPT), which earned Harry Markovitz a Nobel Prize in economics. MPT helps investors efficiently construct portfolios of assets that either maximize the return for a selected level of risk or minimize the risk for a given level of expected return. 

In addition to the risk-return tradeoff, MPT allows portfolio managers to allocate different classes of assets, such as stocks, bonds and cash. Each of these asset classes has a different risk-return profile, as well as other attributes, such as liquidity, management fee, etc. As a result, portfolio managers don’t look at the returns of individual stocks or bonds. Instead, they review the overall portfolios. 

To manage AI initiatives, companies can adopt three salient features from MPT: a portfolio approach to measuring returns, the introduction of risks and the risk-return tradeoff as an important consideration, and the presence of different asset classes with different risk-return profiles. Rather than looking just at accuracy and selecting the most accurate algorithm to deploy and generate benefits, AI managers should look at a broad range of characteristics, including fairness, explicability, robustness and safety of models. 

These other characteristics can pose a number of risks that should be balanced with the benefits of the models. Similar to the asset classes with different risk-return profiles, there are classes of data science techniques — statistical techniques, deep learning techniques, genetic algorithms and so on — that have different risk-return profiles that can be used to help create a portfolio of models. 

Companies can apply the portfolio approach at both the model level (which models and how many to build) and the technique level (which techniques to use when building a single model and using a variety of techniques based on the tradeoffs across accuracy, fairness, explicability, etc.). 

Competitiveness and investment tradeoff: Balancing an innovation portfolio

The third type of portfolio approach, discussed by futurist and PwC Exchange fellow Chunka Mui, is a tradeoff between competitiveness and investment strategies that look at innovation initiatives. In this approach, the competitive impact is categorized as disadvantage, parity or advantage. The investment categories include stay in business (SIB), ROI and option-creating initiatives (OCI). 

Typically, organizations that want to have a competitive advantage in a specific area are overweight in OCI, those that want to maintain parity are typically overweight in ROI initiatives and those below parity typically are overweight in SIB initiatives. 

This competitiveness-investment tradeoff can be a good tool for AI initiatives. Companies trying to build a competitive advantage with AI should select a number of OCIs and explore cutting-edge techniques in deep learning, digital twins and quantum machine learning. Companies that want to stay at parity with their competitors are likely to select more ROI projects, and firms that are below parity may choose to have a number of SIB projects. 

As organizations become more proficient in machine learning and natural language processing (NLP) projects, they tend to move from OCI to ROI projects. Eventually, these techniques would become table stakes or SIB projects. Many data transformation, data quality and BI projects are now becoming SIB projects. Analyzing your AI portfolio along these three major categories adds another dimension to how you manage your AI initiatives.

Demonstrating AI payoffs

A portfolio approach to ROI in AI can be very powerful. Companies can borrow from the test-and-learn dimension of pharmaceutical companies, the risk-return tradeoff from modern portfolio theory in financial services and the competitiveness-investment tradeoff for balancing the innovation portfolio. Applying these principles can help improve your chances of demonstrating how AI is delivering the payback your business expects.