In recent times, an increasing number of camera manufacturers have made bold claims with regard to low light performance. These go beyond the normal sensitivity figures into numerous decimal places! The current trend is to suggest that proprietary technologies allow real-time colour images to be captured in previously difficult low light conditions. With so many established brands – and many of them credible ones – making such claims, Benchmark thought it worthwhile to take a closer look at the cameras on offer.
Click for larger image
[dropcap]B[/dropcap]enchmark has carried out many low light tests in the past. In fact, it has become something of an annual ritual. People ask why we test for low light performance in the summer months, and the reason is because traditionally cameras have struggled with twilight conditions. They work well in daylight, and darkness can be dealt with by adding illumination. It’s the bit in between that often causes problems.
Despite what manufacturers claim, we’ve never had to work too late when testing low light performance. Most cameras – even the good ones – tend to struggle between 10 and 5 lux, before the image deteriorates and switching is required. Because of the increased pixel density on HD and megapixel models, the issues can occur towards the higher end of that range.
Despite the recent trend for adding proprietary ‘low light technologies’ and claiming that cameras will deliver real-time colour images in near darkness, we’ve yet to see this become a reality. Without giving too much away, we’re certainly getting closer to it. Are they really capable of delivering smooth colour footage in poor conditions? During the test, one tester remarked, ‘I hate these cameras; we’re going to be here all night!’
Many of the box cameras in the test were supplied with the manufacturers’ ‘recommended’ lens. Many of these are not supplied with the cameras. To deliver consistency in the results, all low light measurements were taken with control lenses, and all cameras were optimised to deliver an HD1080p stream.
Testing was completed using an open platform VMS, but the ease of installation ratings were based upon initial configuration, connection and set-up of the cameras using the supplied tools, processes and software.
The features and functions of the devices was also assessed. Whilst Benchmark has a policy of not publishing pricing information, the available features and functions are considered against the trade price of the units. This ensures that higher priced cameras are judged fairly against economical models.
D-Link: DCS-3716
The DCS-3716 is a networked day/night 3 megapixel camera which can also deliver HD1080p streams. It utilises H.264, MPEG-4 and M-JPEG compression. The camera features a 1/2.8 inch sensor and sensitivity is quoted as 0.5 lux. Other features include motion detection, wide dynamic range and SD-based edge recording. Power is PoE or 12V DC.
When the DCS-3716 was first launched, it would be fair to say that most HD cameras delivered pretty similar low light performance. Much of the processing power available in cameras at that time was used to handle video streaming, and few if any had credible low light enhancement features.
As cameras have advanced, so additional processing power has been added, and today a growing number employ advanced algorithms to better handle poor levels of ambient illumination. Indeed, this factor significantly contributed to cameras in the first part of the test (published in the September 2015 issue of Benchmark) delivering performance hitherto unseen from HD and megapixel surveillance cameras.
Unfortunately, the DCS-3716 is still very much as it was when launched a few years ago, and that makes it a tough task to stand toe-to-toe with today’s leading units.
Admittedly the camera carries a lower price tag than some of the units on test, but that reflects that it does lack some punch in the engine-room.
Initial installation is simple. The camera is supplied with a utilities CD which loads an installation wizard. Alternatively you can use the camera’s static IP address and simply allow the unit to load its ActiveX element.
Once logged in, the set-up menu allows most configurations to be handled. These include basic camera set-up.
There are a few quirks – you need to set the aspect ratio to 16:9 and save that before you can select HD1080p resolution, for example – but generally the process is simple. The set-up menu is home to the gain and noise reduction settings, but no other low light options.
When it comes to day/night settings, you have to move to the Advanced menu. The option is to set switching to automatic or scheduled, or force the unit into day or night mode. That’s it. There’s no real control over the switching point and no option to enhance low light performance. It does somewhat remind you that this camera is getting on a bit.
One plus point is that the camera does feature a 12V DC output for peripheral devices which can be synched with the ICR switching.
With the camera configured, some optimisation was required to reduce noise, even at around 30 lux. Noise reduction (or Denoise, as D-Link call it) can be set at 255 levels! Latency was low, at less than one half second, and there was no sign of motion blur.
With no low light enhancement technologies, you have to rely on the camera’s default processing for performance in harsh conditions. As light levels fall past 10 lux, you’ll either need to crank up the Denoise setting or accept degradation; you also see the first signs of motion blur. Colour fidelity also starts to drop away at around 8 lux.
By the time the ambient illumination drops to 5 lux, noise becomes an issue, and blur is obvious with any fast motion. At 4 lux, the camera switches to night mode.
Two or three years ago, the DCS-3716 would have been considered a good option with regard to low light performance. Indeed, 12 months ago it wouldn’t have been too far behind the pack. However, it shows its age when considered against the newer models.
Once switched to monochrome mode, the image is clean and stable, and retains a high level of definition. Contrast is decent, and fast motion is handled well.
Mobotix: MX-M15DNight
The MX-M15DNight is a networked day/night camera capable of delivering a 6 megapixel video stream. The camera differs from the others in the test in that it uses twin camera modules. One is a colour unit for daytime coverage, while the other is a monochrome camera for use at night. Alternatively, the unit can stream from both modules simultaneously.
The camera utilises the proprietary Mobotix MxPEG compression codec, although it can also deliver M-JPEG streams. It also features MxLEO image optimisation. The LEO part of the designation refers to Lowlight Exposure Optimisation, and is claimed to eliminate motion blur in low light applications, as well as offering reduced noise and application-based exposure control.
Sensitivity is quoted as 0.1 lux, and the modules feature 1/1.8 inch sensors. Other features include intelligent motion detection and SD-based edge recording. Power is PoE.
Mobotix products tend to divide opinion on a number of levels. There are people who love them and others who loathe them. There are also a number of people who appreciate what they’re offering. If you’re not a dyed-in-the-wool Mobotix fan, then at first glance the installer interface can feel a little odd.
The set-up is relatively straightforward. You can either use a static IP address (Mobotix has default addresses in the 10.xxx.xxx.xxx range; the specific address is on a label on the rear of the camera) or you can use a free utility (MxEasy). The latter is not included with the camera, and if you want to download it you’ll need to register with the Mobotix website. Another alternative is to use MxControlCenter, which is a Mobotix-based VMS package.
With the unit connected, the initial image appeared to be a touch noisy, even in average indoor conditions. Using the manual option we switched between the camera modules, and noticed slight noise in the monochrome image too. However, the menus offer an image program wizard to optimise set-up for high quality images, and once this was activated the images were visibly improved.
As we’ve already said, if you’re not familiar with the Mobotix GUI, you will spend a bit of time searching for the various settings. It’s not as intuitive as some cameras, but it’s nowhere close to being bad either!
There are a couple of irritations. In the menus, each time we set the parameters from a particular screen, it was then sent to the back behind the viewing window. This meant we then had to retrieve the screen to move on the next menu layout. There must be a way to stop this, but it’s not intuitive.
The menu screens are split into Image Control and Event Control. The first group includes General Image Settings, Exposure Settings, Colour Settings, JPEG Settings (this covers compression of all types), Text and Display Setting and Virtual PTZ Settings. The Event Control pages include General Event Settings, Event Overview, Action Group Overview and Recording.
It was whilst looking through the Exposure Settings that we found what was probably the culprit for the slight noise in the initial images. On both camera modules, gain was set to full, plus a feature called Night Improvement was enabled. The latter includes a warning that noise may be generated!
Once you’ve been through the process of maximising the camera, image quality is very good.
Day/night switching is set by default at 10 lux. Using the old fashioned method of measuring light at the viewed scene, the camera switched at 9.79 lux, so it seems the settings are pretty accurate in the real world. The switching level can be adjusted, from 0.1 lux to 320 lux!
In colour mode, the first signs of noise appear at around 5 lux, and some fast motion will show blur. The image holds on to a degree, but once you are below 3 lux both noise and blur are a distraction. You do get the occasional brightness ‘kick’ from the gain, but in truth the degree of degradation below 3 lux does make you realise that the factory default switching point is set at 10 lux for good reason.
Once switched, the image shows a high degree of sharpness and detail. Performance is good in terms of visual quality and motion remains smooth.
During the test we did notice one anomaly. The camera was supplied with the video modules factory-fitted. However, both showed rounded black edges at the extreme corners of the image where the sensors were obviously marginally obscured by the edges of the camera module housing. At the six megapixel setting, these were visible on all four corners, but at HD1080p they were only visible on the top corners, as the image was cropped.
Panasonic: WV-SPN631
The WV-SPN631 is a networked day/night HD1080p camera with a frame rate of up to 50fps. It utilises H.264 and M-JPEG compression and supports multiple streams. It features a 1/3 inch MOS sensor to deliver improved low light performance, and sensitivity is quoted as 0.04 lux.
The camera makes use of the UniPhier LSI platform, and includes enhanced Super Dynamic and Adaptive Black Stretch technologies. Other features include intelligent video, variable image quality by zone, selectable light control modes and SD-based edge recording. Power is PoE or 12V DC.
Unlike many of the new breed of cameras which feature headline proprietary low light algorithms, Panasonic comes at the issue from a slightly different angle. Instead of one low light feature, the camera brings together a host of proprietary functions which can be used to maximise low light and adverse illumination performance. From the chipset through to a host of processing options, the approach allows the installer or integrator to maximise the device for any given application.
The camera is supplied with a CD containing a configuration utility and the manuals. There is no actual documentation included.
The utility is the Easy IP Set-Up Tool, and this finds Panasonic cameras on the local network, even if on a different network segment. Once the units are found and identified by the displayed MAC address, there are options to either adjust the network settings or connect to the camera.
It is worth noting that the network settings for a camera can only be changed via the utility for the first 20 minutes that the device is powered up. After this period, you’ll need to restart the unit to use the utility. If you’re adding a couple of cameras this isn’t an issue, but with large deployments it’ll be best to work in batches.
With the network configurations complete, the connection to the camera can be made, and you’re prompted to download the viewer element to allow set-up.
The process was much slower than we’d expected, and on a few occasions we did wonder whether the camera or the server had crashed. However, we left it alone and eventually it completed the task, allowing access to the set-up menus.
The configuration menus will be familiar to anyone who has worked with a recent Panasonic camera. There is a choice of operation modes. These are HD1080p at 30 or 60fps, 2 megapixel 4:3 at 30fps and 3 megapixel 4:3 at 30fps. For the test we opted for HD1080p at 30fps.
The camera is very feature-rich, and as such the process of set-up will take slightly longer if the unit is be optimised for the specific environment. As with any camera of this type, care must be taken to not overload the processing engine. There are basic modes, and these can be used as a starting point. There’s unlikely to be a situation where the camera can’t be tweaked to some extent to offset any issues with image quality.
With regard to low light performance, there are three options to control day/night switching. The first is for ‘normal’ switching, based upon luminance at the chipset. The second is optimised for use with infrared lighting, and the third uses SCC (super chrominance compensation). This allows the capture of colour images in low light.
There are two switching points: High and Low. These switch to monochrome mode at 2 lux and 1 lux respectively.
First off, a word about the SCC function. This does hold a colour image, even down to 0.2 lux. However, once ambient levels have fallen below 2 lux motion blur becomes an issue, and at 1 lux or lower it becomes debatable how useful the image will be if an incident occurs. Some applications might be able to make use of this function, but our guess is that most installers and integrators will be happier to switch to monochrome mode and retain detail of movement and activity.
In the normal day/night mode, the video processing has no detrimental effect on the quality as light levels fall. Indeed, noise only becomes visible at around 2 lux, and even then it is not detrimental to the overall image. There is also slight motion blur. The High level mode should see the camera switch at this point. It hangs on until 1.3 lux to actually change modes, which is a tad too long. We’d recommend using the input switching to counter this.
The camera does deliver a very good degree of low light performance, albeit with a touch of noise and blur in the most challenging conditions. If there was a bit more adjustment for the switching point, then the camera would be hard to beat!
Once switched and with IR illumination deployed, the camera retains a detailed and clean image with smooth motion and no visible artefacts.
Pelco: IXE31
The IXE31 is a networked day/night 3 megapixel camera which can also deliver HD1080p resolution. It utilises H.264 and M-JPEG compression and supports multiple streams. It features a 1/3 inch sensor and makes use of Pelco’s SureVision 2.0 processing platform to deliver improved low light performance.
SureVision is claimed to deliver simultaneous advanced low-light performance with wide dynamic range and anti-bloom technologies. Sensitivity is quoted as 0.1 lux.
Other features include integral intelligent video and SD-based edge recording. Power is PoE.
The camera is supplied with a Resources CD. This autoruns and allows the manuals to be loaded; the camera is not supplied with any printed documentation. For initial set-up, there is a device utility. The CD does have a folder for software, but this is empty. Instead, it links through to the Pelco website from which you need to download the file.
The utility finds attached cameras, and once the settings are confirmed you can connect. In order to carry out set-up, the camera requires a media player. The pop-up on the camera’s web page provides a link to the Pelco website, or suggests you use the resources CD (which does not contain the media player). In order to download the media player, you will need to be registered on the website.
We accept that drivers do get updated, and that using the latest version is always preferable, but the vast majority of manufacturers manage to get through the process in a more user-friendly way.
The menus are relatively straightforward and offer a good degree of control over the camera functions. The Imaging menus are divided into pages for General, Exposure, Focus, Tone Map and Window Blanking. Stream definitions are made on the video configuration page of the A/V Streams menu.
Low light performance is pretty much controlled via the Exposure page. Pelco’s proprietary SureVision technology boosts low light performance, but there’s no deep-level adjustability; it’s a background technology.
The priority for video can be set to either reduce noise or to preserve frame rate. Maximum exposure time and gain can also be set. Day/night switching is configured as automatic or manual, with three levels for automatic use. A detection time can also be specified to prevent the camera from hunting.
During set-up, detail was good and colour fidelity was accurate. However, the camera did seem prone to rebuild artefacts. The issue was only noticeable during set-up and our feeling was that the media player used with the camera’s web pages was a little feeble if high bitrates were used.
The other concern we had was that the camera runs hot. The housing was noticeably hotter than any other camera on the test by some significant amount.
Latency is a little higher than the other cameras on test, at over one half of a second, but it is consistent so shouldn’t present any problems.
As light levels fall, from around 5 lux downwards, colours start to lose vibrancy and look slightly muted. At around 3 lux noise appears, as does slight motion blur. There are three switching levels: lighter, default and darker. Our test camera tended to switch at around 3.5 lux when set to the ‘Lighter’ level, and we felt this was about right.
It is possible to configure the Pelco camera to deliver relatively noise-free images, switching at around the right point before degradation is an issue.
Under infrared lighting, the image is clean and motion is smooth. There are no issues with image quality and performance is as expected.
Sony: SNC-VB635
The SNC-VB635 is a networked day/night HD1080p camera with a frame rate of up to 50fps. It utilises H.264 and M-JPEG compression and supports multiple streams. It features a 1/2 inch Exmor sensor powered by the Ipela Engine EX to deliver improved low light performance; sensitivity is quoted as 0.04 lux (50 IRE).
Other features include DEPA advanced intelligent video analytics, View DR dynamic range and edge recording. Power is PoE or 12V DC/24V AC.
Installation is straightforward. The camera is supplied with the SNC Easy IP Set-up utility. This is included on a CD which also contains a manual for the utility. There is also a printed quick installation guide, but there is no operator manual for the camera. This, we feel, won’t impress many users.
The utility quickly found the camera and allowed the network configurations to be made. Once a connection was created, the viewing software plug-in loaded and configuration could be carried out.
The menus are straightforward, and much of the proprietary processing functionality is automated, which makes the set-up process relatively simple.
With regard to optimisation for low light, many of the exposure settings reside in the Picture menu, although the day/night control has its own menu. It has a few options: threshold (1-12), hold time and an option to trigger the Easy Focus automatic back focus function whenever the unit switches. The latter is a good choice if a camera is experiencing focus shift.
The camera delivers a sharp, detailed and colour-accurate image. As light levels fall, it continues to impress. Colours remain faithful, although if we really wanted to split hairs we did feel that reds lost a bit of their fidelity as the light level went down past the 5 lux mark.
The image is relatively noise-free, even in conditions down to 2.5 lux, but once light drops to below 2 lux noise creeps in, as does blur on fast motion. At 1 lux or below, the degradation becomes obvious.
The camera switching threshold can be adjusted from 0-12. In order to achieve the best balance between colour footage and a clean image, we found that switching at level 7 was the right option. This meant switching occurred at around 2.5 lux, when the video stream was still clean and crisp with no obvious signs of degradation.
Because of a high degree of flexibility with regards to day/night configurations, installers and integrators will be able to find a balance which meets the end user’s needs in the majority of applications. It depends if noise of frame rate is their goal!
Once the camera switches, the monochrome detail is sharp, and motion is smooth and free of artefacts.
Verdict
Previous low light tests with HD and megapixel cameras have typically seen noise become evident at around 5-8 lux, with most cameras either switching or delivering poor quality images shortly after. While manufacturers’ claims have indicated significant improvements were afoot, this is the first test where we’ve seen evidence of it in the real world.
It is also interesting that all but the oldest of the cameras achieved Recommended status, and that is because the new breed of devices can allocate spare processing capacity to tackle low light performance. Manufacturers have certainly raised the bar. From a technology point of view, it is hard to imagine that they’ll be able to squeeze much more in terms of low light performance from the processing in the future. However, in today’s industry, the performance has taken a significant step fowards.
The DCS-3716 from D-Link is over four years old, and whilst it still delivers the specified functionality, the competition has moved on. There is no additional functionality to specifically address low light performance, and whilst it does carry a lower price tag than some devices in the test, if you’re after the best low light performance it does fall short. That said, it still represents a decent budget choice for many applications.
The MX-M15DNight from Mobotix departs from the norm and features two camera modules: one colour and one monochrome. Image quality is good, and the processing options deliver a high degree of flexibility. In terms of low light performance, it might not hit the figures of some of the other cameras, but it does what it does well, and that will be more than enough for many installers and integrators.
The WV-SPN631 from Panasonic is impressive with regards to low light performance. If it has one slight negative it’s that it hangs on to a colour image until noise and blur are present. The image does not become unusable, but if it switched 1 lux earlier it would be a better camera for it!
Pelco’s IXE31 is a good performer, and does cope well with adverse lighting conditions. It is slightly behind the latest cameras in terms of performance, but not by much! The demo unit we received ran very hot, and whilst we accept the media player plug-in is only for set-up, it didn’t cope well with high bandwidths. However, the problems didn’t appear when using a VMS.
The Sony SNC-VB635 is an impressive device. It is certainly feature-rich, and in terms of low light performance it does what it does very well, and with the right set-up it is more than capable of giving any camera on the test a run for its money. It offers a good range of flexibility with regard to the day/night switching point. It might not hit the specs that the highest rated models manage, but it gets close and so represents a good option for many applications.