What does it mean to independently validate an AI?
The use of AI is becoming more widespread in mobility. And, as more organizations look to use it for measuring risk, pricing insurance and managing safety, it becomes increasingly important to be able to demonstrate that it can be trusted.
Anyone can make claims about the capabilities of their AI. Independent validation is how those claims get tested – and proven.
Here, Anders Lindelöf explains why independent validation of AI is so important, what is involved in AI validation and what the validation of Greater Than’s AI found.
Why does AI validation matter in risk management?
When any type of technology is used to make decisions that affect road safety, the stakes are high – we are talking about human lives. Therefore, it’s important that organizations looking to use AI can feel confident about what it is doing, and the impact it will have in the real world.
Independent validation provides that confidence. It means an external, non-biased expert has examined the methodology behind the technology and confirmed that the results provide a true representation of the AI’s capabilities.
Why did Greater Than choose to have their AI independently validated?
With more AI solutions entering the mobility market, we felt it was important to go beyond internal testing. More organizations are using our AI to improve public safety, so we wanted the evidence behind it to be independently verified by someone outside the company.
We commissioned an independent review by Anders Arpteg, Ph.D., an AI expert with no prior affiliation with Greater Than. His role was to evaluate the scientific rigor of our approach and assess whether the results we report are correctly calculated and supported by the underlying data. Read more about the validation.
What did the independent validation of Greater Than’s AI involve?
The validation assessed the methodology behind Greater Than’s crash probability insights as well as its predictive capabilities.
Performance was evaluated using a percentile-based framework. Within different fleets, drivers were ranked by our AI in order of crash probability. Those predictions were then tested against real claims data – examined both in total and by severity – to assess whether the drivers identified as higher risk were genuinely associated with more serious outcomes.
Crucially, both at-fault and not-at-fault claims were included, avoiding bias related to fault determination and reflecting the probabilistic nature of crash risk.
What were the main findings of Greater Than’s AI validation?
The review confirmed that drivers identified by our AI as highest risk were consistently associated with a disproportionate share of claims – across different geographies, vehicle types and driver categories.
Two findings stood out the most. First, the AI isolates a 25x differential in crash probability between the safest and riskiest driver profiles. Second, drivers identified as highest risk are 91 times more likely to be involved in a crash associated with a severe or major claim vs the drivers identified as lowest risk.
What do the 25x and 91x findings mean in practice?
For those working with risk, the findings have clear practical implications. They show that risk is not evenly distributed across a driving population – and that AI can identify where risk is concentrated.
By prioritizing the highest risk drivers, organizations can focus interventions where they have the greatest impact. The 91x finding is particularly significant, because it relates specifically to the most severe and costly outcomes: the crashes that are most important to prevent.
Why does independent validation matter when choosing a technology partner?
If you’re thinking of using an AI solution, it’s worth considering whether it’s been tested by someone with no vested interest in the outcome.
While internal testing is valuable, and work with customers produces meaningful use cases, independent validation goes a step further – evaluating the methodology behind the technology, as well as its outcomes.
For organizations making long-term decisions about which technology to use for risk management, independent validation is a clear signal of credibility, demonstrating that any claims made have been tested by someone qualified to test them.
