Surveillance systems are capable of capturing high quality images, providing end users with critical data when incidents occur. However, many solutions struggle when the end user needs to urgently access that information. Searching video effectively is vital, and a smart solution is Avigilon Appearance Search. Benchmark assessed the technology to see what it delivers.
Previously Benchmark considered how the use of smart search options can ensure end users receive an increased return on investment from video surveillance solutions. One significant issue exists where sites have a high camera count, often recording video footage on a constant basis. When an incident occurs, the associated footage can usually be found using time and date information or a combination of search criteria. The challenge is how the operator can then expand the insights gained by searching through additional footage to locate other video instances of the perpetrator.
This additional video footage can reveal a depth of information, such as identifying whether the target is still on site, how they arrived or left, their movements and behaviours before and after the incident, whether they are alone or have accomplices, if they have been on site previously, etc..
With many video solutions, system operators are required to ‘scrub’ through timelines from numerous cameras to try and find additional relevant footage, often using little more than very basic information.
This is not only a cumbersome approach, but also requires a lot of man-hours, especially where there might be tens or even hundreds of video streams to search.
Often, the success of such a venture is dependent upon two factors: available time and luck. Given that end users will have made a sizeable financial investment in their video surveillance solution, this isn’t an acceptable situation.
While the security solution will have captured vital data which could assist in the resolution of incidents, accessing that data is often difficult for operators; so difficult that in many cases they simply give up.
It must also be remembered during the time spent manually searching, other incidents may be occurring that require operator intervention. At some point, the pressure on time resources will dictate that the searches have to be ended.
Appearance Search is a technology from Avigilon that uses AI and deep learning techniques to provide smart searching. It speeds up the process by accurately filtering hours of video, identifying matches across multiple streams in a few seconds.
Implementation
Avigilon Appearance Search is a feature of the ACC Enterprise (Avigilon Control Center) VMS (version 6 and upwards), running on an Avigilon NVR. Legacy NVR 3 and NVR 2 models do require a GPU upgrade kit, but the new NVR 4 series makes use of the latest processors and so includes Appearance Search pre-configured without any need for upgrades.
In the past, it was also necessary for sites using Appearance Search to deploy Avigilon’s cameras with self-learning analytics. However, the recent launch of the Avigilon AI Appliance now allows systems with legacy third party cameras to leverage the power of Appearance Search. Available in two variants, the AI Appliance support either 30 or 60 video streams.
Because Appearance Search using self-learning, it doesn’t require any configuration by the system integrator or security installer during installation. It is listed in the VMS search menu, and is integrated into the intuitive GUI, making use simple.
The Avigilon NVRs feature a customised BIOS with optimised hardware. Because of the way the video is managed for storage and review, the latency seen on some VMS options when carrying out intensive system-wide searches doesn’t materialise. This gives the operational performance a smooth and consistent feel.
Search criteria
There are several ways to use Appearance Search, ensuring that system operators have a high degree of flexibility when managing incidents and compiling evidence. ACC uses an object classifier, and this detects people and vehicles within every scene. Each detected ‘target’ has a bounding box applied, and these boxes are key to the analytics which power Appearance Search, adding benefits when an operator is interrogating the video data.
Video searches are generally triggered by one of three events. An operator might see something suspicious in live or replay video and want to view associated footage to better understand what is happening. Alternatively, a search based on flagged alarm events (through IVA, motion detection, inputs or other methods) might identify people or vehicles of interest. Finally, a verbal or written report – often vague in nature – will be passed to the operator for investigation.
All such instances can benefit from a smart search using Appearance Search. Instigating a search is simple and can be activated with minimal data input from the operator. This ensures the process is fast, efficient and intuitive.
For example, when viewing a live or recorded stream, if an operator sees a person or vehicle of interest, clicking into the bounding box brings up a simple contextual menu. This has options to initiate a search either before or after the point in time of the viewed footage.
Where flagged events are being used, the operator simply finds the IVA, motion detection or input-triggered event of interest. This can be done by filtering alarms, carrying out a motion search, using time and date information or using the thumbnail search. These standard search options work in a similar way to most VMS searches, and can use criteria not based upon the presence of people or vehicles.
For example, if an object has been removed from a storeroom, the operator can scroll through the video to define start and finish times, identifying points where the item is present and then missing. The search will deliver a sequence of thumbnails, allowing the moment the item was removed to be identified. From the relevant clip, the operator can click into the bounding box of the person removing the item, and by selecting the appropriate search, other footage of that person can be found, quickly and efficiently, even across multiple cameras.
The process changes slightly where a search is being made from a ‘cold’ start. However, it is still fast and intuitive. Using the Appearance menu, the operator enters known information about the target. This includes gender, hair colour, clothing colour (upper and lower body colours can be specified, and multiple colours can be selected), approximate time and date and the cameras to be searched. If a location is known, specific cameras can be selected, or a search across all cameras can be carried out.
The best approach is for an operator to start out with a short time window and a few cameras, if the information is available. This allows simple filtering of results to allow the algorithm to ‘learn’ the target before expanding the search.
Initiating the search is, as explained, very simple, and filtering the results to achieve high levels of accuracy is equally quick and easy using the Appearance Search GUI.
Filtering results
When an Appearance Search is run, the system presents a series of video thumbnails based upon the search criteria. This is gleaned either from the person or vehicle inside the selected bounding box, or from the data entered by the operator. The thumbnails contain instances which the algorithm flags as containing the target.
Where the thumbnail contains a person in the background, the presented image zooms in on the facial details to assist the operator. From the suggested thumbnails the operator can view the linked footage with a single click, and if the target is verified as being included, the thumbnail can be ‘starred’, again by a single mouse click. This action informs the search algorithm that the target is valid. As more thumbnails are starred, the algorithm has more instances to refine the search criteria, and subsequently the accuracy of the search increases.
During the Benchmark assessment (carried out in a busy office environment), simply starring three or four thumbnails was usually enough to deliver highly accurate results. Even with just a few thumbnails starred, the time window and camera count can be expanded, and the results remain very precise.
As thumbnails are starred, an icon appears in the timeline at the bottom of the operator screen, identifying the various clips. This enables an ‘at a glance’ understanding of the chronology of an event. If the clips are replayed, the system will show them in the correct chronological order.
Another useful tool is that under the time line, a bar graph appears as the search is carried out. The length of the bars indicates the likelihood of a valid target being included in that specific time window.
Searches are fast, and as thumbnails are starred the GUI dynamically presents thumbnails with a higher likelihood of containing the target.
Added benefits
Appearance Search makes managing the search results straightforward. For example, numerous starred clips can be selected and archived to create a continuous storyboard of an event. The archived footage includes a standalone player which generates an ACC interface, delivering features designed to make Appearance Search results easy to interrogate. This ensures that those without access to ACC, such as the police, satellite company offices and other site management teams, can enjoy the same degree of flexibility as ACC operators.
Another area where Appearance Search excels is with regard to the identification of accomplices when an incident occurs.
For example, search results might show a target with another person. From the clip it might not be possible to ascertain whether that person is involved. By simply clicking on their bounding box, a new search can be initiated. However, if the operator is still involved in the initial search, the secondary investigation can be passed to another operator using a collaboration mode.
Other ACC features
ACC, the Avigilon VMS, includes a number of other functionalities alongside Appearance Search. It is a flexible and feature-rich video management system, and as such includes advanced recording and playback options, as well as smart alarm and event handling through a sophisticated rules engine.
The VMS also supports UMD (unusual motion detection) which utilises AI technology to learn about the typical activity in a scene, flagging up any exceptions.
The VMS includes access control integrations via Access Control Manager, ANPR functionality and video intercom support.
In summary
Appearance Search is currently one of the most comprehensive search tools available to video surveillance users, and solves a very real problem for sites with higher camera counts. As such, it not only adds benefits but also ensures a superior return on investment for many surveillance systems.
For the system integrator and security installer, Appearance Search represents an easy-to-implement differentiation. The technology is a fully integrated part of the Avigilon VMS, so requires no activation or configuration to run. The latest generation NVRs are equipped with GPUs to ensure operational performance is as expected, and even legacy systems can be upgraded to support the advanced functionality.
In the past, the need to use Avigilon’s self-learning analytics cameras may have been a stumbling block for some where legacy systems existed. However, with the launch of the Avigilon AI Appliance, this is no longer an issue.
Where end users rely on finding vital video evidence quickly and easily, Appearance Search represents a cost-effective solution. The technology used is complex, but for the integrator and end user, it is very simple and highly effective.