The Abyss Extract case study was designed to trial and assess Abyss Solutions’ machine learning algorithm, and its ability to reduce the time required to review footage and its potential to automatically identify and code defects. The trial was conducted in two phases: Phase 1 – Urban Utilities contributed 3 kilometres (1.86 miles) of CCTV footage of AC and VC sewer reticulation network where Abyss applied their machine-learning algorithm to this footage to produce a condensed view of only the segments containing defects. And Phase 2 – Where the ability of the machine learning algorithm was tested to automatically classify/code defects.
Time: The manual review 3 hours of footage took 2.5 hours to annotate (83% of the video time), while the Abyss extract tool 6.5 hours of footage took 2 hours to annotate (31% of the video time). The time taken comparison for Phase 1 shows an efficiency gain of 63%.
Accuracy: 3 hours of footage set aside for evaluation. The algorithm detected 26 faults and missed 14 faults. It was noted that 50% of the videos in the validation batch were heavily overexposed due to the unusually bright light on the camera. It is acknowledged that the over-exposed footage did impact accuracy. Based on the outcome and feedback, Urban Utilities directed Abyss Solutions to proceed with Phase 2. Urban Utilities also agreed on an amendment to Phase 2, asking Abyss Solutions to focus resources on further training of the algorithm to improve the accuracy in detection and ability to automatically code some of the more common defects.
“Overall, Dan [the auditor] and I [Hugo Salmon] have found the interface effective and intuitive. It only took a few hours of practice for us to get used to it. The warp function was very natural and kept the video at a viewable speed and skipping through only the relevant parts of the video was easy and definitely saved time. The interface also substantially improved reviewing efficiency.”
Phase two sought out to reduce the time a person is required to review footage and allow resources currently used for review to be reallocated. While unable to quantify the benefits, the outcomes of the pilot have demonstrated the capacity the machine learning algorithm to produce a higher degree of accuracy over a shorter period. This will increase the efficiency of the current CCTV review process, and allow comparison of results over numerous review cycles.