Smart Search: A Proactive Approach
Video surveillance can capture significant levels of critical information, but unless that information can be quickly and efficiently accessed, customers’ investments in surveillance will often fall short in terms of the expected returns. Searching video accurately and effectively has always been a challenge, but adopting technologies such as Avigilon’s Appearance Search makes a difference.
Video surveillance is unquestionably a valuable tool in the fight against crime, offering protection to businesses and organisations against a wide range of risks. It also allows users to reap additional benefits via business intelligence, site management, automation and data sharing. In the 21st century, video is acknowledged as a great enabler across a wide range of business functions.
Video data allows a high degree of intuition to be applied to situations by users. Because it captures information that humans easily understand – visual information – it enables operators to apply logical decisions. This is also true when it comes to deciding the type of information users require from a system. For example, if a camera is surveying a corridor, the user can immediately understand what they want to achieve. If someone loiters in the corridor, or if they travel in a certain direction, or if the enter of exit from a specific room, that might constitute an important event. With this level of clarity, it is possible to set up a series of alarms or events based upon known exceptions.
Security is not always about the expected. Unanticipated events will need to be addressed, and modern video solutions include a variety of features which help operators to manage these. Functions such as timeline search and motion search allow users to retrospectively find the moment when an asset disappears or when an incident starts, usually by working backwards from a specified event. For example, if someone is spotted lying on the floor, the video can be reviewed to see what occurred prior to that moment, thus allowing an investigation to take place. Equally, tracking back to find out when an asset disappeared is also relatively simple.
Because of humans’ intrinsic understanding of visual information, video becomes an important, and many will argue pivotal, part of any security system. However, despite the ability of operators and others to quickly comprehend visual information, video surveillance has one downfall, especially after an incident has occurred: the sheer volume of data that is captured.
Big data issues
Video surveillance systems are primarily tasked with data collection. The data being collected is predominantly visual, but may also include metadata, system status information and audio capture. A single video camera pushes a significant amount of data into the system at a constant rate. This might be 50 frames per second of HD (or higher resolution) video, along with accompanying data, and it is transmitted around the clock.
Every additional camera added to the system multiplies the volume of data being archived. When systems with dozens (or hundreds or thousands) of cameras are considered, the amount of data being captured can become overwhelming. This creates problems when the urgent analysis of information is required.
Because the video data flow is constant, it means that every time an operator reviews an event or has to clear down a series of notifications (or simply carries out another non-surveillance task), a backlog of footage is being created. Even the best trained operators cannot watch multiple streams of video during a shift with unwavering attention. As a result, the majority of data captured by video surveillance systems goes unused. This is seen as no hardship by many security teams, because the vast majority of that data contains little or nothing of interest for them. However, sometimes the data that is missed can be vital to an investigation.
While only a small percentage of video surveillance footage contains information of use to the security team, in many applications only a small percentage of the useful data is ever used. This is because finding important data can be a hit-and-miss affair. Often security teams are forced to find a balance between the time invested in searches versus the results achieved. An additional pressure is that operators know during the time spent searching for relevant data, new recordings are piling up.
The volume of captured data in video surveillance systems is both a blessing and a curse. On one hand, it means that when an event occurs, the system will usually contain a lot of information that can help assess what has happened, enabling appropriate action to be taken, resolving the situation or providing critical evidence. On the other hand, finding those important ‘needles’ in the haystack can be labour-intensive, frustrating and fraught with error.
Often opportunities are missed because it is not viable to keep searching for relevant footage. When a search is abandoned because it is too time-consuming, the system has failed for the customer.
The potential power of visual information is huge, if the right footage can be quickly found. For this example, consider a shopping centre with 100 cameras. These would be situated inside the centre in the malls, stairwells, lifts and store entrances. The system would also cover the external areas as well as car parks. All of the captured data is brought back to a central control room.
A shopper reports that they have been the victim of a pickpocketing offence, and from information provided by the victim, the control room is able to find footage of the event. They now have a physical description of the suspect, but what other information can video footage provide?
Firstly, video footage could show if the suspect is working alone or part of a gang. It might identify if other victims have suffered at his hands, including some who might be unaware they have been targeted, or who might not report incidents because they’re not sure if they’ve lost items rather than being robbed. Video footage can also identify whether the suspect is still on site, and if so, where they are. It can provide information about how they arrived at the shopping centre, how they left and whether they have visited before. Video analysis can reveal patterns in behaviour and activity which could be used to apprehend the criminal.
This degree of detailed information, which can also be used as evidence, will be available from most systems, but the efficiency of the search mechanisms means that it is hardly ever used proactively. The process for locating the right data simply doesn’t allow it.
If the suspect is a tall middle-aged male wearing a red shirt, the operators will need to scrub through hours of footage, from each individual camera source, trying to find someone with a red shirt, as this is the most obvious identifier available to them. Then they have to examine each occurrence of red clothing to see if it is the suspect or not. They may identify the suspect at earlier times, or maybe after the event, if they’re very lucky.
Of course, if he has changed his shirt or put on a jacket, then opportunities to find earlier or later footage will probably be missed. The odds are that if the process needs to be repeated across numerous cameras, even if the operators do find where he went after the event, he will be long gone by the time any action can be taken.
In the vast majority of cases, unless the incident is very serious and the police are involved in the video search, the result will be the search will be abandoned. Little more than the circulation of a snapshot of the suspect will take place. Given that the data, which can provide a high degree of information, is in the system, the inability to find it quickly and easily is a significant flaw.
A smart approach
Increases in processing power and the use of GPUs in surveillance systems have allowed video search to move forward significantly in recent times. The use of new technologies allows intelligent searches to be carried out. These have an element of ‘self-learning’ which enables the system’s search engine to filter results based upon feedback from the operator. The intelligent algorithms are fast and accurate, thus redefining how incidents can be managed. An example of this is Avigilon Appearance Search technology.
Appearance Search can be used to accurately detect the presence of people and vehicles in video footage, across multiple cameras. It allows many hours of footage to be searched, with targets identified and verified, in minutes.
In the case of the pickpocket suspect, Appearance Search can check across all video streams – live and recorded – when searching for other incidents involving the suspect. The technology will present a series of snapshots and the operator can either accept or reject these. The results are then used to further refine the search and as the system ‘learns’ more about the suspect, so it can find appearances across wider timescales, where the suspect may be dressed differently.
As a starting point, Appearance Search can either use a captured image, or filtering can be applied. Initial filters are simply selected from clear and concise menus. Options include gender and clothing colour. The latter has options for upper and lower body colour.
Additionally, the technology also incorporates the characteristics of a person’s face, enabling the operator to search for the same person, even if items such as their clothing change over time.
Appearance Search examines footage across all selected cameras and returns the results. The process is fast.
The operator is presented with a series of matches as image thumbnails. Clicking on the thumbnail plays the related video clip, allowing an assessment to be made as to whether it is relevant. The clips are either rejected or accepted, and the system uses the accepted results to further refine the search.
Accepting images as relevant is simply a case of clicking on a star icon. Starred footage is highlighted in the timeline, making it very easy to find, and the system even presents a graphical bar graph highlighting scenes where the presence of the target is more dominant.
Once a suspect is identified, the system can find them in later footage to track their progress, and could help to locate them if they are still on site. Alternatively, by tracking them through the site it may be possible to identify their vehicle in a car parking area or ascertain how they travel to and from the site.
With fast and accurate searching, aided by the operator’s input via the simple ‘one click’ selections, Appearance Search allows all captured video to be examined, ensuring a complete evidence trail can be created. It also allows additional information to be discovered following an event, allowing a rapid response by users.
Video surveillance is a powerful tool, and when its full potential is exploited, it delivers a wealth of benefits to users. The smart automation of searches via Appearance Search elevates video from being a good crime detection tool to being one that enables users to proactively manage video data with ease, providing critical footage where and when it is needed.
By utilising smart technologies, Appearance Search ensures that users can maximise the value of their investment in video surveillance. It eliminates the typical ‘hit and miss’ nature of searches and increases the protection on offer, creating smarter solutions that meet the requirements of users without compromise.