Arribada just returned from our first thermal camera field trials in Greenland, where we tested the image quality and detection abilities of our chosen thermal sensors in an arctic climate, our prototype camera design and looked at real installation conditions and areas for monitoring. We are testing these thermal cameras as a winner of the WWF and WILDLABS Human Wildlife Conflict Tech Challenge and hope to build an early animal detection and alert system to warn communities when dangerous animals have entered a community. We’re specifically working on polar bear and elephant detection now but hope our system can help with all types of animals. You can read our previous progress reports on WILDLABS.net and stay tuned for our next update about field test results with elephants in India.

As a quick re-cap, we chose a ULIS Micro80 Gen2 shutterless microbolometer sensor with 80×80 pixel resolution for work in the Arctic, with polar bears, or any climate below -10°C, and a FLIR Lepton 2.5 with 80 x 60 pixel resolution for work in India with elephants, or any climate above -10°C. After reviewing various thermopile and microbolometer thermal sensors, we chose these two microbolometers because we believe they will give us the necessary image quality for a computer to identify a target animal at the lowest cost. These two sensors have similar capabilities and differ mostly in presence of shutter. A shutter is essential to removing noise from final images taken with a microbolometer and producing a clean image. However, mechanical shutters will freeze in temperatures below -10°C, making a shutterless sensor necessary for work in the Arctic. We will go into the implications of using shutterless sensors in more detail later but it’s important to know that removing noise and producing clean images is more difficult with a shutterless sensor. 

Arctic camera prototype

Our arctic camera prototype consisted of:

During our arctic trial, we wanted to investigate the image quality Micro80 Gen2 sensor in cold temperatures and specifically the amount of noise in the final images. We also wanted to test our ability to remove noise from the images using a shutterless correction method. Removing noise from the image is critical to having a clear enough image for a computer algorithm to identify a polar bear as a polar bear.

Additionally, it was equally important to test the power consumption and the PIR sensor trigger. In our prototype camera, the Micro80 Gen2 was always on, but no images would be saved unless the PIR sensors sensed movement. Getting false triggers was concern, as sometimes changing temperature conditions are enough to mimic the change in IR radiation caused by something warm moving past the sensor. Ittoqqortoormiit, Greenland also experience heavy winds that blow snow through the air which we feared might cause false triggers.

Arctic Testing

We went to Ittoqqortoormiit, Greenland to test our arctic prototype cameras. Polar bears have been visiting the town’s garbage dump, its sanitation workers, and sometimes wandering further into town. We positioned our cameras on the side of buildings near the dump site to monitor activity at the dump and the nearby shoreline, since polar bears come ashore from the sea ice in the adjacent bay and move toward the smell of garbage.

Site challenges

Arctic weather and conditions posed some challenges for installation and maintenance. Polar bear season is during the winter, when snow is heavy, temperatures are coldest and day light hours are few. High winds after snowfalls can create snow drifts against buildings taller than a person. The cameras had to be placed high enough to avoid snow drifts, but low enough to see an animal walking close to the lens. Short day light hours and low light angle meant a solar panel could not supply extra power making the cameras reliant on batteries alone. Cold arctic temperatures reducing battery life meant performing more frequent maintenance or a perhaps purchasing a larger most costly battery. The low light angle also had to be considered when positioning the cameras, as the thermal sensors cannot be pointed straight at the sun or they become over-saturated and cannot produce images. This is not a problem when the sun is high over head during the day, but during the winter in the arctic, the sun stays near the horizon and a camera facing the horizon would put the thermal sensor directly in front of the sun all day.

All microbolometers have an inherent problem with the sensors responding non-uniformly to incoming radiation. This uneven response creates noise inside the sensor that shows up in any image of a scene, person or animal taken by the microbolometer. This noise degrades the image, obscuring object or animals and making detection and identification difficult or impossible. Image corrections are applied to remove this noise from the final image. With shuttered microbolometers, a shutter is closed over the lens giving the sensor a uniform temperature scene. Any temperature variation detected is therefore being generated by the focal plane array as excess noise. This pattern of noise is isolated and removed from subsequent images of people or animals.

With shutterless microbolometers, the image correction must be applied using an algorithm that anticipates the amount of noise in sensor at a certain temperature. These corrections can be performed in real-time while the image is taken, or after the image has been captured. Real-time algorithms require a lot of up-front calibration of the thermal sensor to isolate noise patterns at different temperatures. They then use the temperature of the sensor at the time of image capture to apply to correct noise removing algorithm. These real-time images are the best at removing noise but take a lot of time and effort to create. Algorithms used in post-processing are already created but must assume the same temperature of the sensor for every image capture. Because the actual temperature of the sensor changes with outside temperature, these algorithms cannot remove all the noise from an image.

We used a post-processing algorithm to remove noise from our arctic images to get a sense of the image quality we could expect without spending months developing a real-time algorithm. This resulted in some noisier images than would be expected if we used a real-time processing algorithm, but we still had some good quality results.

Results

We deployed two prototype cameras at the dump site in Ittoqqortoomiit were we knew we would pick up sanitation workers and hoped to see polar bears. Unfortunately, no polar bears visited the dump site while our cameras were up, so we were unable to see the image quality of a polar bear in arctic conditions. We did have success capture people working at the dump site with good image quality.

Amount of noise varied between photos depending on the weather conditions and temperature range of objects in the scene. The closer the sensor temperature to that assumed by the post-processing algorithm, they better noise removal in the image. Range of temperatures, or contrast, in the scene also affects noise levels in an image. A scene with low contrast will always make noise more visible than a scene with high contrast.

Images were taken seconds apart from each other and show me moving in front of the camera. We had no obvious false triggers, even on days with heavy wind. The few blank images we had, such as the right image above, were due to a larger field of view on the PIR sensor trigger than thermal sensor causing movement outside the thermal sensors field of view to trigger the PIR sensor. Battery life was a challenge. Not only did the cold temperatures shorten battery life, but thermal sensor drained more power than it did during testing in the UK. Consequently, the battery lasted only 1/3 of the time predicted, which meant we couldn’t monitor continuously all day and night without charging the battery. This will be an obstacle to overcome for a final camera build.

What’s next?

Overall, we’re happy with the results of our first arctic test, but we would still like to test our the Micro80 Gen2 on a polar bear in arctic conditions. We hope to be able to simulate some of these conditions in zoos in cold climates that have polar bears. This will also be a good way to test different shutterless noise removal corrections to have the clearest image possible of a polar bear. The next step gathering enough images to create test algorithm which can automatically identify the thermal image of a polar bear and send an alert to the community. We hope we can remove enough noise and get a clear enough image with the Micro80 Gen 2 thermal sensor. However, because of the difficulties of noise removal with shutterless cameras and the low thermal contrast of polar bears, it might be necessary to increase the pixel resolution and thermal sensitivity of the sensor. To test this possibility, we are going to collect some initial images of polar bear using a ULIS Pico160, with a pixel resolution of 160 x 120 and higher thermal sensitivity for more detailed images. 

Stay tuned for our next update with results from our field testing in India!