Aurora AI's Accuracy and Speed
When it comes to selling and designing PV systems, accuracy and speed are imperative. To deliver top-notch accuracy and speed, Aurora AI is trained on nearly half a million site models and generates 3D models for Aurora projects using the best satellite imagery and LIDAR data available for a given location. Similar to a human designer, Aurora AI generates higher quality designs when higher quality data is available. Aurora AI’s speed and accuracy has improved over time, and how we measure it has improved, as well.
How We Measure Accuracy and Speed
Aurora AI has improved since its initial release with additional training data and better modeling techniques. How we measure and track Aurora AI’s performance has also improved. We’ve updated how we measure accuracy in an attempt to better understand how Aurora AI does with complex roofs in addition to simple roofs. We also adjusted how we assess our accuracy based on customer feedback that knowing how many panels can fit on a roof generated by Aurora AI is a better real-world measurement that takes into account the installable roof space available. (see Table 1). The new metric is called relative accuracy.
Table 1: The number of panels on an AI-generated design were compared to the panels of two different human-generated designs.
Speed is another critical requirement for solar sales and designs. Across the solar sales and design market, 3D models can be drawn manually, purchased as a service, or — for Aurora customers — generated automatically through AI. Solar design services vary in turnaround times from 24 hours to 30 minutes, depending on the service and how much a customer is willing to spend per project. Our engineers closely monitor how quickly Aurora AI can generate a roof model when a customer uses the feature.
How Well Aurora AI Performs
When measuring for relative accuracy, Aurora AI was able to generate a roof model that was better, as good, or within a narrow margin of a human designer ~76% of the time for all roof types grouped together. Interestingly, when looking at relative accuracy by roof types, Aurora AI performed as well or better than a human 63% of the time for simple roofs, 83% of the time for moderate roofs, and 76% of the time for complex roofs (see Table 2). The relatively higher performance on more complex roofs compared to simple was an unexpected finding and will be further investigated in the near future.
Table 2: The breakdown of simple, moderate, and complex roofs included in the study with respective relative accuracy results.
With regards to speed, based on our latest analysis which included approximately 8500 runs completed by our customers over one week in January of 2024, the average time it took for Aurora AI to generate a roof model was 10.05 seconds* (see Table 3).
Table 3: Industry average design service turnaround times compared to Aurora's.
*Aurora AI run times are specific to how long it takes our internal servers to process the request. Customers may see slightly longer delays specific to their hardware, software, and internet connections.
In Summary
The accuracy data from this study provides insight into when and how Aurora AI can be used to create designs. We continue to recommend that all Aurora AI designs be checked by a human before going to permitting, but Aurora AI provides suitable accuracy to enable lead generation, provide major time savings, and generate a proposal-ready design for sales teams.
Aurora AI is the fastest possible method available to generate a 3D roof model when designing PV systems. This means that when you consider speed and accuracy at the same time, Aurora AI is best suited for sales teams that prefer speed upstream in the project lifecycle or designers who prefer a head start, especially for moderate or complex roofs. Next, we will take a look at some of our customer utilization data to see how Aurora AI is being applied.