Research, Build, Break, Tinker: Developing a Thermal Sensor Solution

The COVID-19 pandemic has upended much of our world’s traditional operations. The challenges brought about have forced organizations to quickly adapt to new ways of doing business in order to keep essential services running, while simultaneously developing plans to bring schools, offices, restaurants and more back online. The disruption brought on by COVID-19 has given us an opportunity to bring our most creative, innovative minds together to solve one of the pandemic’s first challenges: how to effectively using temperature screens to help keep students and employees safe as they return to the classroom and workplace.

Yesterday, we posted Scott Laliberte’s thoughts on this project and in this blog, we detail the technology behind this imperative: Could we quickly build an inexpensive thermal imaging system, using available and commodity parts? Our love of researching, building, breaking and tinkering was about to be put to the test. This project represented the same type of emerging tech expertise Protiviti brings to every client looking to grow their business by investing in connected devices (IoT), artificial intelligence, blockchain, quantum computing or the next emerging solution.

The problem statement focused on solving a critical issue organizations such as healthcare providers, schools and other organizations employing essential personnel are facing right now: how to screen personnel to successfully identify those that might be sick and put protocols in place to reduce the risk of infecting others. Many of these organizations are already facing budget constraints and we believed the funds they did have could be better used in helping people rather than buying expensive, difficult-to-deploy equipment.

Initial Goals and Constraints

Our initial project objectives included:

1. Create a thermal imaging system for preliminary temperature screenings; reliably identify people with elevated body temperature while following CDC recommended guidelines for social distancing
2. Procure readily available parts/components
3. Write an application that allows anyone to easily determine a normal or elevated temperature range or failure to measure temperature; software would need to be free and available to the public – with a preference for Open Source
4. Target cost: up to $500.

The Build

Base System: Our project objectives requiring components to be both inexpensive and readily available led us to evaluating both Raspberry Pis ($35) and Feathers ($20) as possible base systems.

Thermal Sensor: We wanted to use a sensor that wouldn’t break the bank, but would be accurate enough for the target skin temperature ranges ( 96 °F – 99 °F (35.6 °C – 37.2 °C) for typical skin temperature or 99.4 °F – 104 °F (37.5 °C – 40 °C).

We targeted several thermal sensors for evaluation:

The only readily available sensor was the AMG8833, which was also substantially less expensive than most of the other sensors we considered. The tradeoff for this economy was sensor resolution and expected accuracy. An 8×8 resolution was not a major concern for us at the distances we were envisioning, but the ∓2.5 °C precision on the datasheet (Pansonic, 2020) was identified early on as a concern. We chose to move forward with the AMG8833 to investigate whether we could achieve better accuracy under controlled conditions, so we bought one sensor on a generic PCB breakout board and one sensor on a Featherwing form-factor board.

Optical Sensor: The optical sensor (camera) did not need to be high resolution. We simply needed a camera that would allow us to layer an optical image over the thermal image we would compose from the thermal sensor. To do this, we chose the Raspberry Pi Camera Module V2. This camera uses a Sony IMX219 8-megapixel sensor, which provides relatively good quality 1080p video at 30 frames/sec and costs just $25. It is connected directly to the MIPI camera module port on the Raspberry Pi, providing more than enough throughput for that component.

Software: The software needed for this project could be broken up into a few phases. In the first software phase, we followed the basic Adafruit documentation (Miller, 2018) to obtain the raw data that the AMG8833 sensor was providing. For the purposes of visualizing that data, we also needed to be able to convert that raw data stream into a grid that would provide the user with a thermal image that could be recognized as a person standing in front of the sensor.

In phase two of software development, we would write an application to overlay the optical image on top of the thermal image and validate that everything in-frame conforms to required environmental conditions. This way, the observer would be able to make sure the subject getting thermally measured was positioning themselves correctly for the sensor to get an accurate measurement.

In phase three of software development we would provide a simple user interface with red/yellow/green indicators so the observer would be able to know if the person should be sent for a secondary no-touch infrared thermometers reading (red), the sensor reading could not be measured accurately (yellow) or the person had an acceptable skin temperature and would be allowed to proceed (green).

Prototyping

The first step with the Raspberry Pi was loading and updating the operating system and physically connecting the thermal sensor and optical camera.

We validated that the thermal sensor and camera were communicating and working with the Raspberry Pi. We installed the python packages needed for processing and rendering the data from the thermal sensor. At this point, we were able to obtain a blurry, but discernable thermal image from the sensor at about 10 frames per second.

The last test of the basic imaging was to ensure the Raspberry Pi had enough horsepower to render/capture a thermal and optical image at the same time.

The good news was that everything seemed to be working. But things were about to take an undesired turn.

Precision

With the basics for getting the hardware working complete, we needed to calibrate the sensor and make sure it was as accurate as needed for our tests. We conducted three separate tests in a temperature-controlled, low-humidity environment, and with the “patient” standing at distances of two feet and another at six inches from the sensor. A third test, with the patient six inches from the sensor, but allowing the sensor to run for 30 minutes before testing, was also administered. All three tests resulted in low temperature readings ranging from 27.5 °C (81.5 °F) to 33.5 °C (92.3 °F). Considering the possibility we had defective equipment, we replaced each part of the systems (Raspberry Pi/AMG8833 Thermal Sensor/Optical Camera/Wiring) and rebuilt the operating system and software packages. The measurements were the same. At this point, we knew we had a significant issue.

Reassessing Our Options

We wanted to use a heat source that would have a constant and controllable temperature to determine if the measurements from AMG8833 were consistently off by the same amount. If the AMG8833 was off by the same number of degrees Celsius every time, we could write our screening application to account for this difference and still flag people as having an elevated temperature accurately. So, we performed a series of tests to determine the AMG8833’s precision and accuracy, which confirmed our issues with the thermal sensor.

We had conversations with our Artificial Intelligence and Machine Learning group to see if there was a way to create an adaptive method of leveling out our temperature readings to something more reliable, but this wasn’t realistic given the constraints we had. With accuracy and precision problems even in these highly controlled conditions, we expected that readings would be even less predictable on real people, at less precisely measured distances.

We decided to add an additional sensor (a no-touch infrared thermometer) that might be able to help augment the readings.  Again, after performing a series of tests, the setup still produced the same inconsistent temperature readings and required to be almost touching the person to get semi-consistent readings.

A New Plan

With Plan A not working as hoped, we pivoted to a more traditional plan – evaluating commercial products intended for this use. We contacted manufacturers of thermal imaging devices to see what was in the marketplace that fulfilled our screening needs. After multiple discussions with vendors we landed on the SeeK Scan, developed and manufactured by SeeK thermal.

We selected the SeeK Scan due to its temperature-controlled heating element that is used as an in-frame method — this minimizes any discrepancy in measurements. Using a device with an exact known temperature in the thermal imaging picture, next to the person having their temperature taken, the measurements can be very accurate (±0.3°C or ±0.5°F). This added heating element seemed to allow for a less expensive sensor and device setup to be packaged together for much less than other thermal camera-only systems.

Using the SeeK software on a laptop connected to the thermal/optical camera combo device, the screener is given a pass (green)/fail (red)/measuring (blue) status.

Moving Forward

Our initial plan to create a low-cost thermal imaging device, made from readily available parts, that could be used for screening people going back to work in a COVID-19 world, gave us an opportunity to quickly develop a solution that is now available to clients looking to safely and securely return their people to work, school, stores or wherever people need to gather. Hospitals, retailers, restauranteurs, airlines and many other industries stand to benefit from this solution and Protiviti is ready to partner with them to implement this and other as-yet-undiscovered solutions to the challenges they face in this “new normal.”

Working in a world that has been severely impacted by the coronavirus pandemic demands solutions that are agile, creative, safe, secure and call upon the newest technologies available. We learned that we can, indeed, quickly get people back to successfully living their lives. In the end, this effort showed great minds of different disciplines working together can accomplish great things in short periods of time.

To learn more about our Emerging Technologies capabilities, contact us.