What does it mean to independently validate an AI?
The use of AI is becoming more widespread in mobility. And, as more organizations look to use it for measuring risk, pricing insurance and managing safety, it becomes increasingly important to be able to demonstrate that it can be trusted.
Anyone can make claims about the capabilities of their AI. Independent validation is how those claims get tested – and proven.
Here, Anders Lindelöf explains why independent validation of AI is so important, what is involved in AI validation and what the validation of Greater Than’s AI found.
Why does AI validation matter in risk management?
When any type of technology is used to make decisions that affect road safety, the stakes are high – we are talking about human lives. Therefore, it’s important that organizations looking to use AI can feel confident about what it is doing, and the impact it will have in the real world.
Independent validation provides that confidence. It means an external, non-biased expert has examined the methodology behind the technology and confirmed that the results provide a true representation of the AI’s capabilities.
Why did Greater Than choose to have their AI independently validated?
With more AI solutions entering the mobility market, we felt it was important to go beyond internal testing. More organizations are using our AI to improve public safety, so we wanted the evidence behind it to be independently verified by someone outside the company.
We commissioned an independent review by Anders Arpteg, Ph.D., an AI expert with no prior affiliation with Greater Than. His role was to evaluate the scientific rigor of our approach and assess whether the results we report are correctly calculated and supported by the underlying data. Read more about the validation.
What did the independent validation of Greater Than’s AI involve?
The validation assessed the methodology behind Greater Than’s crash probability insights as well as its predictive capabilities.
Performance was evaluated using a percentile-based framework. Within different fleets, drivers were ranked by our AI in order of crash probability. Those predictions were then tested against real claims data – examined both in total and by severity – to assess whether the drivers identified as higher risk were genuinely associated with more serious outcomes.
Crucially, both at-fault and not-at-fault claims were included, avoiding bias related to fault determination and reflecting the probabilistic nature of crash risk.
What were the main findings of Greater Than’s AI validation?
The review confirmed that drivers identified by our AI as highest risk were consistently associated with a disproportionate share of claims – across different geographies, vehicle types and driver categories.
Two findings stood out the most. First, the AI isolates a 25x differential in crash probability between the safest and riskiest driver profiles. Second, drivers identified as highest risk are 91 times more likely to be involved in a crash associated with a severe or major claim vs the drivers identified as lowest risk.
What do the 25x and 91x findings mean in practice?
For those working with risk, the findings have clear practical implications. They show that risk is not evenly distributed across a driving population – and that AI can identify where risk is concentrated.
By prioritizing the highest risk drivers, organizations can focus interventions where they have the greatest impact. The 91x finding is particularly significant, because it relates specifically to the most severe and costly outcomes: the crashes that are most important to prevent.
Why does independent validation matter when choosing a technology partner?
If you’re thinking of using an AI solution, it’s worth considering whether it’s been tested by someone with no vested interest in the outcome.
While internal testing is valuable, and work with customers produces meaningful use cases, independent validation goes a step further – evaluating the methodology behind the technology, as well as its outcomes.
For organizations making long-term decisions about which technology to use for risk management, independent validation is a clear signal of credibility, demonstrating that any claims made have been tested by someone qualified to test them.
FAQ
What is independent AI validation?
Independent AI validation is a process in which an external expert evaluates an AI model’s methodology, data handling and performance results, confirming that reported findings are accurate and scientifically sound.
Who validated Greater Than’s AI?
Greater Than’s AI was independently validated by Anders Arpteg, Ph.D., an expert in artificial intelligence.
What did Greater Than’s AI validation find?
The validation confirmed that drivers with identified as higher risk by the AI are consistently associated with a disproportionate share of claims. Key findings include a 25x differential in crash probability between the safest and riskiest driver profiles, and a 91x greater likelihood of the highest-risk drivers being involved in severe or major claims compared to the lowest-risk drivers.
Does Greater Than’s AI work across different vehicle types and geographies?
Yes. The validation was conducted across multiple continents, different vehicle types and different driver categories. Performance was consistent across all of these, confirming that the AI is not optimized for a single market or fleet type.
How is independent validation different from internal testing?
Internal testing assesses how a model performs on the data and benchmarks chosen by the company that built it. Independent validation involves an external expert examining the methodology and results using real-world outcomes, with no stake in what the results show.
How does Greater Than’s AI measure driver risk?
Rather than flagging isolated events, such as harsh braking or speeding, Greater Than’s AI continuously analyzes driving behavior throughout entire trips, identifying patterns that it compares against a database of billions of real-world driving profiles linked to known outcomes. This pattern-based approach is what makes it possible to predict crash probability rather than simply react to events. Watch how this approach works in our short video.
