The German-Austrian joint project FLORIDA developed legally compliant tools for the visual and auditory analysis of large volumes of heterogeneous image and video data. The focus is on material collected after a criminal offence or terrorist attack, for example footage provided by witnesses or obtained from different video sources, which must be evaluated promptly by law-enforcement authorities.

The University of Konstanz subproject focused on interactive visual-analytics methods and the spatio-temporal reconstruction of complex scenes. Image and video material is integrated into a shared spatial context, enabling investigators to examine relevant events step by step, understand relationships and critically review analytical results. Complementary methods support, among other things, the prioritisation of video data, the search for freely definable objects and the analysis of audio tracks.

The aim is not automated surveillance or automated decision-making. Instead, the tools support experts in evaluating extensive material relating to specific events in a structured manner, identifying missing contextual information and recognising possible misinterpretations. Questions concerning the legally compliant and ethically responsible collection, processing and analysis of image and video data were explicitly addressed within the joint project.

In Germany, FLORIDA was funded by the Federal Ministry of Education and Research (BMBF) as part of the programme “Research for Civil Security”, with a focus on “Aspects and Measures for Countering Terrorism”.

Selected Results

A research prototype enables the interactive reconstruction of a crime scene in a three-dimensional environment. Automatically detected objects from multiple image and video sources can be spatially positioned and examined together. Integrated annotation functions support investigators in reconstructing the sequence of events before and after the offence in a transparent manner and in identifying potential gaps or inconsistencies in the available data.