Toolbox Spotter: A Computer Vision System for Real World Situational Awareness in Heavy Industries

Abstract: The majority of fatalities and traumatic injuries in heavy industries involve mobile plant and vehicles, often resulting from a lapse of attention or communication. Existing approaches to hazard identification include the use of human spotters, passive reversing cameras, non-differentiating proximity sensors and tag based systems. These approaches either suffer from problems of worker attention or require the use of additional devices on all workers and obstacles. Whilst computer vision detection systems have previously been deployed in structured applications such as manufacturing and on-road vehicles, there does not yet exist a robust and portable solution for use in unstructured environments like construction that effectively communicates risks to relevant workers. To address these limitations, our solution, the Toolbox Spotter (TBS), acts to improve worker safety and reduce preventable incidents by employing an embedded robotic perception and distributed HMI alert system to augment both detection and communication of hazards in safety critical environments. In this paper we outline the TBS safety system and evaluate its performance based on data from real world implementations, demonstrating the suitability of the Toolbox Spotter for applications in heavy industries.