Review on infrared single-pixel imaging

1. Military University of Technology, ul. gen. Sylwestra Kaliskiego 2, 00-908 Warsaw, Poland
2. Fire University, ul. Juliusza Słowackiego 52/54, 01-629 Warsaw, Poland
3. Institute of Electronics, Lodz University of Technology, al. Politechniki 8, 90–590 Lodz, Poland

Abstract

The article discusses modern single-pixel imaging techniques. Different solutions of spatial light modulators (SLMs) used in infrared imaging are presented. The focus is on image reconstruction methods, in particular on the use of a modulator based on orthogonal codes, cyclic matrices, and neural networks for image reconstruction. The potential possibilities and limitations of these new imaging methods are described, emphasizing their usefulness in different ranges of the infrared spectrum. Moreover, the experimental implementation of a single-pixel infrared camera is presented. Possible applications and future development perspectives of this technology are indicated.

Introduction

The aim of the article is to present the latest developments in single-pixel imaging and its potential applications in different ranges of the infrared spectrum. Research in this field is progressing rapidly. Some existing solutions are discussed in [1]. This article focuses on the design of single-pixel camera (SPC) systems, cyclic matrix coding, compressed sensing (CS), and artificial intelligence (AI) applications for oversampled image reconstruction. The article begins with exploring the historical development of imaging technology, tracing its evolution from mechanical scanning methods to modern techniques using various light modulators.

Infrared (IR) imaging is crucial in scientific research, industry, and military applications. It provides information about the structure, temperature, and properties of materials, as well as the characteristics of radiation sources. Although traditional array detectors, such as charge-coupled device (CCD) and complementary metal-oxide-semiconductor (CMOS), dominate visible imaging, their IR counterparts remain expensive and complicated to manufacture. The search for alternative IR imaging technologies leads to a return to SPC solutions, which offer promising oppor-tunities for cost reduction and implementation of other features that array detectors do not possess.

Historically, the first imaging devices, such as the Nipkow disc, using mechanical image scanning, initiated the development of technologies that led to the creation of modern electronic solutions. The Nipkow disc, although primitive, became the inspiration for the construction of systems such as Vidicon electron tubes and later CCD, as well as CMOS matrix detectors. Due to the compatibility of production technology with integrated circuits, the latter gained mass popularity in smartphones and consumer devices. Currently, almost every smartphone produced has at least one built-in CMOS sensor.

An alternative to matrix technology are SPC systems, which use a single detector and spatial light modulators (SLMs). This approach enables image acquisition through sequential signal encoding and reconstruction, effectively bypassing the need for costly array detectors while maintaining high imaging fidelity.

The spectrum of electromagnetic waves

The shortest electromagnetic waves come from space and constitute cosmic radiation. The Earth’s atmosphere partially absorbs this radiation, significantly attenuating its intensity or preventing it from reaching the surface altogether. The lower part of Fig. 1 shows atmospheric windows that are transparent to selected frequency bands. IR radiation was discovered by splitting sunlight with a prism long before X-rays [2]. However, the first successful IR images were not recorded until the second half of the 20th century. The spectral range of IF detectors used for observations in the Earth’s atmosphere is usually limited to a few bands located in the atmospheric windows transparent to IR.

‌

IR detectors

A detailed history of detector development is presented in [3]. Using the wavelength criterion, five types of IF radiation detectors operating in five sub-ranges can be distinguished:

• NIR – near IR. This range, from 0.78 to 0.9 μm, is close to the visible spectrum of light. Properly adapted CMOS sensors can operate in this range. Observation in this range is among the cheapest IR imaging methods.
• SWIR – short-wave IR. Range from 0.9 to 3 μm. The boundaries of this range are determined by the transpar-ency of the atmospheric window for wavelengths emitted by high-temperature objects. The entire SWIR band does not cover the detection capabilities of CMOS matrices.
• MWIR – mid-wave IR spans the 3 to 5 μm range and is particularly suited for detecting high-temperature objects. It is commonly used for monitoring industrial processes involving elevated thermal conditions.
• LWIR – long-wave IR. This range covers from 8 to 14 μm. The maximum energy in this range is emitted by objects at room temperature. In this range, objects effectively emit and can be imaged based on their own thermal radiation, particularly at night.
• FIR – far IR terahertz range, located between the LWIR range and millimeter waves with a wavelength of about 100 μm. This range does not fit completely into the transparent atmospheric window. Detection in this range uses so-called terahertz cameras. Since waves of this length are highly absorbed by the atmosphere, detection necessitates a specialized emission antenna capable of generating radiation, typically stimulated by a laser pulse. The radiation produced penetrates, reflects or is absorbed by the examined object. Imaging can be based on detecting a wave reflected from or passing through an object, in which case the operating principle is similar to X-ray imaging. This type of imaging is a relatively new technology, but it has already been found to be applicable in materials engineering and can be used in the security industry. Since most dielectric materials are transparent in this wavelength range, ethical concerns have been raised about their use in public safety sectors such as airport security, even though the radiation is non-invasive and has no harmful effect on the human body.

Scanning IR cameras

The first IR cameras captured images by sequentially scanning each point line by line. Initially, they used pyroelectric detectors based on triglycine sulphate (TGS) crystals, commonly known as Pevicons. In practice, these were adaptations of the Vidicon tube, originally designed for visible light, in which the antimony trisulfide layer was replaced by a thin slice of TGS [4, 5] and a germanium plate replaced the glass end of the tube. The construction of the Pevicon is shown in Fig. 2.

‌

The device could operate at room temperature with a vertical resolution of 200 lines in the 1–30 μm spectral range. A significant challenge was to remove the charge from the illuminated crystal layer. This type of design required an additional moving shutter synchronized with the vertical deflection of the lamp, which reduced the dark current at the Pevicon output.

Another solution that improved sensitivity and resolution was the development of a camera that scanned the image using a system of moving mirrors. This enabled measurements to be conducted with a single, highly-sensitive detector. The detector could be thermoelectrically cooled using a Peltier stack. Radiation passing through the germanium optics was reflected multiple times by both static and moving mirrors. Fig. 3 shows the mirror system used in the Agema 880 camera.

‌

In this design, IF radiation was reflected eight times before reaching the photodetector. The camera uses a thermo-electrically cooled high-speed photon detector.

In 1994, a pioneering array solution appeared. Honeywell patented a thermal microbolometer detection method using vanadium oxide (VOx) in its detector. In this solution, the detector could operate at room temperature in the LWIR range, and its operation (reading the measured values) was similar to the operation of a CMOS array detector. This is one of the reasons why LWIR is a frequently used spectral range. This spectral range is mainly used in the military, industrial, and automotive sectors. The development of microbolometers is described in [6]. However, despite the undeniable advantages of IF focal plane array (FPA) bolometric detectors, the main challenge is the complex manufacturing process using specialized materials such as HgCdTe. Increasing the production efficiency while maintaining optimal matrix parameters and high sensitivity is still widely studied [7, 8]. Currently, IF detectors require high vacuum during the manufacturing stage and are still expensive compared to visible light detectors. For this reason, alternative solutions for IF imaging are sought.

Single-pixel imaging

SPC for capturing incoherent IR radiation emitted by objects

An alternative to IF array detectors and linear SPC acquisition is image sampling using a single detector combined with transform coding. The principle of this operation is illustrated in Fig. 4 and is closely related to the use of the SLM.

‌

Unlike acquisition with a matrix detector, which records the intensity of light at each point in the image, this method involves the camera making multiple measurements by viewing the scene through different mask patterns and recording the total intensity of the incoming radiation for each mask separately. A series of measurements must be made with different aperture patterns to acquire and reconstruct a single image. Due to multiplexed sampling, such an approach benefits from Fellgett’s advantage and reduces detector noise.

Alternatively, instead of recording linear combinations of radiation reaching the detector by the SLM, the SLM can selectively illuminate the observed object, and the amount of reflected radiation is recorded. The concept of illumination by the SLM and recording the reflected radiation is shown in Fig. 5.

‌

Spatial light modulation (SLM)

The review of SPC approaches with SLM is presented in [9]. Several types of modulators can be found in the literature: liquid crystal (LC) modulators [10–12], digital micromirror device (DMD) mirror modulators [13, 14], complementary DMD modulators with two detectors [15], modulators with caustic patterns [16], LED illumination arrays [17], THz matrix illuminators acting as a modulator [18], cyclic rotating modulators [19], and stationary modulators [20]. In the case of IF, due to its wavelength, transmission-based solutions dominate over reflection-based solutions. Therefore, only some of the presented solutions are used in the IR range due to absorption and diffraction in further IR ranges. The list of parameters of some of the mentioned types of modulators is presented in [21].

Reflective modulators with moving micromirror arrays, like DMD, are used in visible light imaging. This modulator can also obtain hyperspectral images [22, 23]. DMD arrays are readily available and commonly found in digital light processing (DLP) projection systems. However, the DMD design itself limits the range of reflected wavelengths, so this technique cannot be used for longer wavelengths in IF imaging beyond the NIR and SWIR ranges [24] and intermediate wavelengths in the FIR range. For mid- and far-IF ranges, as well as terahertz cameras, modulators are constructed on the transmission principle (Fig. 4). The total radiation reaching the detector is the sum of the radiation from the exposed pixels of the mask and the radiation of the aperture itself in the shadowed areas. The number of measurements required is equal to the number of pixels in the reconstructed image to accurately reconstruct the image. This method originates from techniques used in spectrophotometry (Michelson spectrophotometer) [25], where the original mask pattern was determined using Golay codes.

The number of measurements needed to accurately reconstruct a single image should correspond to the number of pixels in that image. In 2006, Candès, Romberg, and Tao proposed a new method for solving underdetermined linear equations [26–28]. Their work has enabled the construction of SPC cameras that require significantly fewer measurements to reconstruct an image than the number of pixels in the image. This method, known as compressed sensing, allows for the accurate approximation of scattered or compressible signals using significantly fewer samples than traditional methods. Cameras with SLM modulators, which mathe-matically reconstruct images, have certain advantages that are not available in traditional raster imaging. These cameras have a favourable signal-to-noise ratio (SNR) and provide better imaging results for several reasons:

• The same detector performs the measurement for all pixels, which greatly simplifies calibration.
• The measurement error is inherently smaller because the detection of the total intensity is less susceptible to local noise fluctuations compared to FPA.
• The sensitivity of the SPC detector can be lower because it records the sum of radiation from many pixels simultaneously.

The improved dynamic range of the measurements is crucial for very weak signals or high background noise conditions. As such, these cameras can be used for remote observation or medical applications. Cameras with SLM may or may not have an optical lens. An SPC system without a lens minimizes optical aberrations. Changing the focal length involves adjusting the distance between the modulator and the detector.

SLM apertures can be encoded in two ways: block transform encoding or random sequence encoding. In block transform encoded modulators, the aperture patterns perform a linear combination of optical transformation of the observed scene. For this purpose, Golay codes using rows of the Hadamard mask, the Golomb transforms [29], or the Fourier domain [30] can be used. In modulators based on orthogonal codes, the complete measurement basis is derived from the orthogonal basis matrix. Each measure-ment corresponds to a single row of the orthogonal basis matrix. Image sampling using SLM involves performing a linear transformation A on the sample vector X representing the image to obtain a new vector Y (a spectrum) of the same size, as shown in (1):

\( \mathbf{Y}=\mathrm{AX} \) (1)

By selecting the appropriate aperture patterns, encoding is performed during the modulator sampling process (Fig. 7). At the output of the transducer, measuring the total amount of radiation as a function of time, the vector Y can be obtained. During the transformation, information contained in the input samples is mixed. To reconstruct the image, the inverse transformation must be performed by multiplying the vector Y by the inverse matrix A−1, as shown in (2):

\( \text { image }=X=A^{-1} Y \) (2)

If the transformation matrix is orthonormal (e.g., the Hadamard matrix), which in addition is non-singular and square, the reconstruction process becomes simple and linear as A−1 = AT.

‌ ‌

A significant technical challenge is constructing an SLM to generate as many aperture patterns as possible. In some studies, the apertures were manually changed for each measurement, as shown in Fig. 4. In 2008, Chan et al. developed a camera equipped with a terahertz imaging illuminator using this method [20]. Another solution that eliminates the hassle of changing masks each time is to use a rotating disc with patterns corresponding to the cyclic Hadamard mask. The Hadamard mask can be obtained from the Hadamard matrix by removing the first row and column with all elements equal to one, achieving an odd size. In addition, using the twin-prime algorithm, it is possible to create the cyclic transformation matrix (the Hadamard mask) [25]. Cyclic codes can be applied to reduce the physical size of the aperture encoding disc. Once a cyclically coded aperture is prepared, instead of changing the entire aperture pattern for each measurement, it is enough to move the aperture by one row. The operational concept of this solution is illustrated in Fig. 6.

Cyclic code generation involves reducing the size of the entire pattern of openings distributed in the form of a ring in the SLM. This can be achieved using, for example, the twin-primes algorithm, which is based on pairs of adjacent primes p and q, where q = p + 2 [25]. This method of aperture coding has been used for visible light [19] and NIR [31] imaging. Cyclic matrices provide efficient aperture coding because the disc diameter is reduced. Assuming that the number of samples corresponds to the number of pixels in the image, this method provides lossless image reconstruction.

The modulator aperture pattern is selected from a uniform random distribution in random modulators. Such modulators are used in CS when the number of samples is significantly smaller than the number of pixels. The CS technique allows a radical reduction in the number of samples for image reconstruction. Using CS, images can be reconstructed from samples taken with both block transform-coded and random apertures. It has been shown that uniformly distributed random codes give good results [32]. Another interesting solution, which does not involve moving parts, was described by Don et al. [33]. In this solution, a fixed circular aperture is used instead of a rotating disc. Instead of rotating the disc, the entire camera rotates, which can be placed in a rotating munition. The authors also use CS calculations for image reconstruction.

Compressed sensing in single-pixel imaging

It is well known that in order to accurately reconstruct a signal or an image after sampling, the Shannon-Nyquist sampling theorem must be fulfilled. On the other hand, natural signals and images are usually redundant and can therefore be compressed. Compressibility of data denotes that a signal can be represented as a so-called sparse dataset in a certain basis with some elements equal to or smaller than zero. The sparsity allows data compression and, in many cases, the almost exact reconstruction [26,27,34-36,]. It leads to incoherent sampling, which allows the sampling rate to be below the rate specified by the Nyquist-Shannon theorem. The block diagram of such processing, called compressive sensing, is presented in Fig. 7.

Transferring image data to another domain (e.g., using the Hadamard transformation) allows for the sparse data to be easily compressed, e.g., by omitting spectral elements with small values. Various transformations can be applied, as shown in Fig. 7. Compression considers only part of the spectrum, which is then used for decompression. The non-linear image reconstruction is the numerical inverse problem which generally suffers from ill-posing, poor convergence, and in some cases, it leads to either wrong or inaccurate solutions [36]. In practice, compressive sensing implemented in SPCs using incoherent sampling of an object IR emission can be presented schematically as in Fig. 8. Radiation passing through diaphragms with different hole patterns generates the spectrum of an image which is the sparse dataset ready for the decompression and reconstruction.

‌

Typically, the masks with randomly distributed openings fulfil the requirement of incoherent sampling and the bucket detector with or without the lens implements the linear transformation by collecting the radiation emitted by a certain part of an object. Number of masks K < N, where N is the number of pixels of the captured image and defines the compression rate.

Reconstruction uses the convex optimisation algorithms, which frequently use the 11-norm as a criterion (imple-mented in the available l1-magic software package). Among others, various algorithms are used in CS, including greedy methods, total variation minimization, and iterative thresholding [37]. Other reconstruction algorithms are reviewed in [21, 38, 39]. All mentioned optimisation methods fulfil the so-called restricted isometry property (RIP) necessary for reconstructing a compressed signal with several samples below the Nyquist limit [36, 40, 41]. The RIP introduced by Candes [27, 28] ensures a stable measurement matrix that is a sparse but well-conditioned measurement matrix, which is necessary to reconstruct the signal from the compressed data. In other words, the RIP preserves the Euclidean length of a sparse signal and makes the matrix nearly orthogonal [28].

The Hadamard-pattern SLM combined with the deep-learning neural network seems to be a good alternative for solving the SPC inverse problem and for the non-linear image reconstruction [19, 42, 43].

High-speed NIR SPC

A novel method of NIR imaging was proposed by Liu et al. [44] (Fig. 9). The imaging concept is based on illuminating the observed object with pico-second laser pulses. After reflection from the object, the radiation is directed through a lens to a multimode optical fibre. Since the reflected light returns at different angles, the path lengths of individual rays are different. Using a long multimode optical fibre causes the return pulse to be stretched in time. Finally, the reflected and stretched pulse reaches a fast photodetector. The received signal contains spatial information encoded in amplitudes. A trained neural network performs the final reconstruction. This formula allows for fast sampling at 60 MHz for fast image, but such a method has a very limited resolution determined by the number of modes in the fibre (21 × 21 pixels).

‌

THz imaging

SPC imaging in the terahertz range is based on scene encoding and image reconstruction principles as in the VIS or IR ranges. However, significant differences arise due to the physical properties of these wavebands and detection technology. Natural FIR radiation sources are not commonly used because the atmosphere strongly attenuates this range. Furthermore, according to Wien’s law, the temperature of a perfect black body for the FIR range falls within the range of 2.9–116 K. As a result, artificial radiation sources are required. The most commonly used sources include non-linear effect-based sources using crystals, cooled black bodies, quantum cascade lasers (QCL), as well as sources based on Gunn diodes or high-electron-mobility transistors (HEMT). The high cost and complexity of radiation source systems hinder commercialization. Additionally, the wave-length limits the spatial resolution of imaging.

The application of SPC in the FIR range offers a cost-effective approach to FIR imaging by simplifying detector design. SLM in THz cameras can operate based on tuned slit waveguides [45] or Schottky diode-based structures [46]. One of the first SPC projects was presented in [20]. Currently, DMDs are also used for modulating the FIR illuminator beam and creating the pattern displayed on the FIR-generating crystal [47]. A detailed review of hardware solutions and methods is provided in [48] and [49].

Ghost imaging

The origins and inspiration for developing SPC come from ghost imaging, theoretically predicted long ago and currently developed in several configurations [50–52]. Initially, ghost imaging was considered as a purely quantum phenomenon based on the correlation between photons obtained in a spontaneous parametric down-conversion (SPDC). Nonlinear optical crystals (NLCs) are used to perform SPDC. The radiation passing through the NLC is split into two beams of so-called entangled photons. The entangled photons with half the energy and 2 times longer wavelength compared to the excitation photons have orthogonal polarization with respect to each other. Consequently, they can be easily separated by a polarizing beam splitter (PBS) [53, 54]. Photons in the split beams are strongly correlated in space (position) and the momentum domain.

One possible configuration of the quantum ghost imaging setup is shown in Fig. 10.

‌

The quantum ghost imaging system consists of NLC and PBS producing two separate beams of entangled photons, each illuminating the object and semi-transparent binary masks with randomly distributed patterns (SLM). These two light arms are then focused onto the single-point bucket detectors B1 and B2, both connected to a photon coincidence counter (CC). This counter enables signal correlation and final image reconstruction through a simple operation of linear combination of mask patterns scaled by the CC output signal [55].

In addition to quantum ghost imaging using NLC, there is a simpler version of imaging using the natural correlation of photons in laser coherent light (Fig. 11). This is called classical ghost imaging or ghost imaging using classical photon entanglement [52, 56, 57].

‌

Research is still ongoing to compare quantum and classical ghost imaging in terms of spatial resolution, SNR, and computational requirements. Although one might think the quantum approach is superior, experimental results show that the quality of the reconstructed images obtained by both methods is very similar [56].

It is important to emphasize that ghost imaging can be based on recording reflected or transmitted light. In the IR domain, the authors are mainly interested in the incoherent radiation emitted by the object. This configuration is known as single-pixel imaging, initially developed in the visible light environment and now increasingly adapted in different radiation bands.

Imaging with SLM vs. matrix detectors

As an alternative to line scanning, SPCs are increasingly used in combination with various types of SLMs. There are strong reasons to develop this imaging technique separately from matrix-based solutions, as the latter have limitations that SPC overcomes. Fig. 12 shows a diagram of the image quality degradation that occurs in matrix cameras along the path from the objective to the detector and then to the observer.

‌

In a typical camera, quality degradation begins when the radiation passes through the objective lens due to absorption by the optics, optical aberrations, and diffraction caused by refraction of radiation in different media. In SPC, these problems can be reduced because, in some cases, the entire optical system can be reduced to the SLM and the detector. Another problem with matrix detectors is the dynamic range of the measurement and insufficient sensitivity.

Compared to high dynamic range (HDR) matrix cameras, SPC imaging offers a better dynamic range [58]. The SNR is better because statistically ½ of the total incoming radiation reaches the SPC detector directly, whereas in matrix detectors, the average amount of radiation incident on a pixel is 1/n of the total radiation, where n is the number of all pixels. In space applications, the random loss of individual pixels due to cosmic radiation is a significant problem [59]. By using a single detector, the probability of damage is much lower. There are also other reasons, independent of quality aspects, such as imaging partially obscured objects [60–62] and methods to accelerate image reconstruction using magnetic resonance imaging, tomography, or terahertz imaging [63, 64]. There is lossy compression at the end of the path that the image travels from the observed to the observer. Most communication channels impose this type of compression due to limited bandwidth. In case of using CS, the data stream is already compressed by the CS method itself and does not require additional procedures such as the H.265 compression and the like. There are no computational costs on the detector side. The computational costs are transferred to the receiver.

A significant disadvantage of SPC with SLM is its operation on the time-space principle. For accurate image reconstruction, the number of measurements required using this method corresponds to the number of pixels in the image. SPC, unlike matrix detectors (operating in parallel), must take samples sequentially, which is why the measurement time is much longer. This time mainly depends on the detector response time. Another problem is the design of the SLM itself because only some types can work in specific IR bands. In addition, SLM usually has moving elements, which increases its potential failure rate. If SPC has an SLM built based on DMD, then the resolution of such a camera is determined by the DMD parameters, which is currently a certain technical limitation.

The SLM built with DMD can only work in the NIR or SWIR range because the barrier is protective glass and the size of the micromirrors borders on the reflected wavelength for the LWIR range. Although it is possible to use a germanium or ZnSe cover, the main barrier is diffraction caused by the size of the mirrors, whose dimensions approach the wavelength of the reflected IF radiation [65, 66]. When using a rotary modulator with a cyclic code, the resolution is mainly determined by the aperture diameter, which radically reduces the resolution of such a camera. This problem is related to the scalability of the resolution. Although reference [67] indicates a method of increasing the resolution by multiplying small observation blocks, this is an intermediate concept between SPC and conventional imaging. Another difficulty is the measurement time, which is influenced by the number of pixels in the image and the detector response time. Some of the disadvantages mentioned can be reduced by using CS. However, using CS radically increases the computational cost during recon-struction on the receiver side. Table 1 lists and compares the SPC and SLM solutions in the IR range.

Machine learning in single-pixel imaging

Machine learning is more commonly used to reconstruct visual images from SPC. In [71], a deep-learning network was presented in signal processing from a hyperspectral SPC operating in the visible light range using electromechanical optical modulators (DMD). Machine learning methods were also applied to reconstruct images from a camera operating with a high-rate frame generation [44]. Super-resolution methods presented in the literature [55, 72] are a promising tool in reconstructing images from single-pixel systems operating in different radiation spectrum ranges [74]. A new original approach to SPC image reconstruction based on machine learning is presented in Fig. 13.

‌

IR radiation emitted by the object passes through an SLM with a cyclic Hadamard pattern and is then focused on a single NIR/IR detector. A similar SLM was used in [19]. The cyclic Hadamard mask in the authors’ case is 899 × 899. The 31 × 29 matrix contains elements of each row of matrix H in the rectangular form. As a result, for each set of holes arranged in a 31 × 29 pattern, the average IR radiation is measured on a single-pixel detector. The measurements are saved as 1 × 899 vectors. There are 899 lines that provide a full reconstruction of a 31 × 29 image. Then the resolution was increased to 80 × 80. This can be done in various ways. In the given solution for an image reconstruction up to 80 × 80, a convolutional neural network (CNN) is used.

The original 80 × 80 higher resolution IR images are then reconstructed using a CNN with a decoder structure.

The above concept was simulated in TensorFlow AI environment [74]. One of the possible CNN models tested in the current study is shown in Fig. 14. The application of CNN in this case is an original approach to reconstruction and resolution increase in IR SPC.

‌

The proposed CNN decoder architecture consists of several feature map layers of increasing size. The CNN per-forms two important tasks in this processing – reconstructing a low-resolution 31 × 29 NIR image and up-sampling to a higher resolution image with 80 × 80 pixels [70].

Fig. 15 shows examples of the reconstructed IR images using the proposed CNN network. The images were generated by a self-developed 80 × 80 IR bolometric camera. The SLM was modelled and spectra were generated using a CNN-based reconstruction method. Finally, peak signal-to-noise ratio (PSNR) measures were calculated.

‌

Transfer learning was used in CNN training. The training dataset contained 10 000 visual images with a resolution of 80 × 80. The validation dataset consists of 2000 images. The system was tested using 50 IR images. Additionally, learning was supported by augmentation. A stochastic gradient descent optimizer typical for regression networks was used.

A preliminary single-pixel thermal imaging setup was developed after successful SPC simulations using a machine learning approach with promising results. The system core is a dual-channel acquisition module equipped with a high-speed 14-bit ADC. One channel is connected to a PIN diode transimpedance amplifier. The other channel is used for triggering and synchronization, which is performed by a slot optocoupler and a marker on the SLM. A dedicated application was written in C# language to acquire, analyse, and reconstruct signals and display images in real time. A working prototype of the single-pixel NIR camera is shown in Fig. 16.

‌

The SLM in the form of a rotating ring operates at a frequency close to f = 30 Hz. Currently, the modulator is made of a photographic film. In the next stage, modulators will be made of more durable materials, such as aluminium or FR-4 laminate. A slot optocoupler placed on a base printed in 3D technology synchronizes the modulator rotation by acquiring compressed vectors correlated with the analysed scene.

At the current research stage, the simulations carried out and the prototype of a low-resolution SPC (31 × 29) developed were used to verify scientific hypotheses and confirm the design assumptions regarding the possibility of using AI in single-pixel imaging. The proposed solution is fully scalable, but requires the use of a better quality SLM modulator, e.g., in the form of moving mirrors. An important design element of a higher-resolution SPC is the precise synchronization of the disc rotation with acquiring signals from the detector. Two lenses were used in the developed SPC solution presented in Fig. 16. The first lens projects a sharp image of the observed object onto the SLM with cyclic Hadamard patterns. The cyclic Hadamard mask used in the experiment is a non-singular matrix of size 899 × 899. Each row of this matrix is a single aperture of the SLM modulator. Furthermore, the used Hadamard mask is a cyclic matrix that allows for the placement of 899 apertures on the rotating modulator disc. The cyclic nature of the Hadamard mask will cause each successive SLM modulator aperture to shift the previous one by one position. Examples of three successive cyclic Hadamard apertures are shown in Fig. 17.

‌

The second lens placed behind the SLM focuses on the radiation of the observed object precisely onto the NIR detector. The oscillograms shown in Fig. 18 present the recorded oversampled 1D signals up to about 10 000 samples, corresponding to the Hadamard transform of the observed scene.

‌

Blue signals represent waveforms from a slot optocoupler, and yellow ones are from an NIR photodiode. A standard NIR-PIN photodiode in a plastic housing with a diameter of d = 5 mm was used in the experiments. It contained a sensor with an area A = 0.78 mm2 and a field of view ϕ = 20°. The sensor sensitivity was S = 0.55 A/W for a wavelength of λ = 940 nm, and the dark current at a reverse voltage of VR = 20 V was equal to IR = 1 nA.

The spectrum of the SPC detector signal, depending on the radiation of the recorded scene, contains components with both low- and high-frequency values. The charac-teristic envelope comes from large, strongly radiating objects. It results from the correlation of light and dark element positions in the cyclic Hadamard mask and the distribution of the radiation intensity of the observed scene.

The C# application, which allows real-time viewing of the moving image, made it much easier to properly set and calibrate the entire system. Fig. 19 shows sample images taken with a single-pixel NIR camera.

‌

Reconstructions of images with a resolution of 31 × 29 pixels were performed using the transpose Hadamard mask. Then, bicubic interpolation increased their resolution to size 80 × 80.

Assessing the accuracy of the observed scene mapping is difficult at the current research stage due to the lack of reference images. Performing such reference measure-ments would require a camera operating in the same spectral range and field of view, located in an appropriate place, which is rather difficult to carry out. The presented prototype of the SPC system operates at a recording rate of up to 30 frames/s at a sensor signal sampling rate of f = 25 kHz. It is possible to easily increase image recording speed by increasing the modulator rotational speed to 3000 rpm. Further increase leads to the use of high-speed motors.

Using a rotating disc is not a practical solution in general and may lead to reconstruction errors due to the changes in rotation speed over time. For this reason, prac-tical implementations of thermal SPC should use electronic radiation modulators, similar to DMD-type solutions.

It is planned to produce higher resolution images, e.g., 59 × 61 pixels, using another cyclic Hadamard SLM matrix generated by the consecutive twin prime numbers algorithm. Ultimately, the research is focused on thermal imaging in the MWIR or LWIR. The advantage of the SPC is that it allows for quick changes in the detector, thanks to which the system can be used for imaging in different ranges (from NIR to LWIR).

Conclusions

A thermal imaging camera is a device worth investing in due to the flexibility of imaging in different spectral ranges. The SPC imaging systems are based on a different principle of operation. Using a single detector with an SLM in the form of a rotating disc is a simple and low-cost solution in relation to matrix detector systems. In order to record an image using such a simple system, it is necessary to perform many measurements, each of which will provide partial information about the image. The architecture of an SPC consists of just two elements – SLM and a single-pixel detector. Despite its numerous advantages, SPC technology requires further improvement, especially in terms of increasing the resolution and speed of data acquisition. The key challenge remains the optimization of light modulators for different spectral ranges, especially in the MWIR and LWIR. There is still much to be done in this area. There is a significant gap in SLM designs for longer IR wavelengths. Most existing solutions are mechanical. Undoubtedly, progress in this area will come with the development of electronically controlled designs. Great hopes are associated with carbon nanotube technology which exhibits variable absorption and refractive properties under the influence of an applied electric field. However, current SLM solutions already enable the construction of cheap spatial heat sensors. An important research direction is also the development of image reconstruction algorithms, including methods based on machine learning, especially CNN. This can increase computational efficiency and improve the accuracy of detailed reconstruction. The SPC allows for quick and easy detector replacement, so it can be used to observe, e.g., different types of gases, such as carbon dioxide or methane.

Authors’ statement

Research concept and design, B.W. and K.S.; collection and/or assembly of data, K.S., A.S., S.U., and B.W.; data analysis and interpretation, S.U.; writing the article, K.S., A.S., S.U., and B.W.; critical revision of the article, B.W.; final approval of article, A.S.

Acknowledgements

This work was financed/co-financed by Military University of Technology under research project UGB 4720600000-531-0024-W500-22-W523000. This article has been completed while the third author was the Doctoral Candidate in the Interdisciplinary Doctoral School at the Lodz University of Technology, Poland.

References

1. ‌Osorio Quero, C. A., Durini, D., Rangel-Magdaleno, J. & Martinez- Carranza, J. Single-pixel imaging: An overview of different methods to be used for 3D space reconstruction in harsh environments. Rev.Sci. Instrum. 92, 111501 (2021). https://doi.org/10.1063/5.0050358

2. ‌Beckman, J. E. Infrared Astronomy. in Multimessenger Astronomy 81–115 (Springer International Publishing, 2021). https://doi.org/10.1007/978-3-030-68372-6_3

3. ‌Rogalski, A. History of infrared detectors. Opto-Electron. Rev. 20, 279–308 (2012). https://doi.org/10.2478/s11772-012-0037-7

4. ‌Stupp, E. H., Crowell, M. H. & Singer, B. Pyroelectric Vidicons. in 1972 International Electron Devices Meeting 156 (Springer, 1972) https://doi.org/10.1109/IEDM.1972.249374

5. ‌Talmi, Y. Pyroelectric vidicon: A new multichannel spectrometric infrared (1.0–30-μm) detector. Appl. Opt. 17, 2489–2501 (1978). https://doi.org/10.1364/AO.17.002489

6. ‌Fièque, B., Tissot, J. L., Trouilleau, C., Crastes, A. & Legras, O. Uncooled microbolometer detector. Recent developments at ULIS. Infrared Phys. Technol. 49, 187–191 (2007). https://doi.org/10.1016/j.infrared.2006.06.030

7. ‌Xia, H., Nguyen, H., Roy, A., Ohlckers, P. & Aasmundtveit, K. E. Robustness of large-size vacuum sealed packages for micro- bolometer array. IEEE Trans. Compon. Packag. Manuf. Technol. 14, 1731–1736 (2024). https://doi.org/10.1109/TCPMT.2024.3462818

8. ‌Zoschke, K. et al. Wafer Level Capping Technology for Vacuum Packaging of Microbolometers. in 2023 IEEE 73rd Electronic Components and Technology Conference (ECTC) 1571–1578 (IEEE, 2023). https://doi.org/10.1109/ECTC51909.2023.00267

9. ‌Gibson, G. M., Johnson, S. D. & Padgett, M. J. Single-pixel imaging 12 years on: A review. Opt. Express 28, 28190–28208 (2020). https://doi.org/10.1364/OE.403195

10. ‌Brodie, C. H. & Collier, C. M. Single-point detection architecture via liquid crystal modulation for hyperspectral imaging systems. IEEE Access. 8, 185012–185020 (2020). https://doi.org/10.1109/ACCESS.2020.3029550

11. Huang, G., Jiang, H., Matthews, K. & Wilford, P. Lensless Imaging by Compressive Sensing. in 2013 IEEE International Conference on Image Processing 2101–2105 (IEEE, 2013). https://doi.org/10.1109/ICIP.2013.6738433

12. ‌Toyoda, H., Inoue ,T. & Hara, T. Application of Liquid Crystal on Silicon Spatial Light Modulator (LCOS-SLM) for Manipulation and Sensing. in 2015 14th Workshop on Information Optics (WIO) 1–3 (IEEE, 2015). https://doi.org/10.1109/WIO.2015.7206900

13. ‌Duarte, M. F. et al. Single-pixel imaging via compressive sampling. IEEE Signal Process. Mag. 25, 83–91 (2008). https://doi.org/10.1109/MSP.2007.914730

14. ‌Radwell, N. et al. Single-pixel infrared and visible microscope. Optica 1, 285–289 (2014). https://doi.org/10.1364/optica.1.000285

15. ‌Yu, W. et al. Complementary compressive imaging for the telescopic system. Sci. Rep. 4, 5834 (2014). https://doi.org/10.1038/srep05834

16. ‌Toninelli, E., Stellinga, D., Sephton, B. Forbes, A. & Padgett, M. J. Single-pixel imaging using caustic patterns. Sci. Rep. 10, 2281 (2020). https://doi.org/10.1038/s41598-020-59224-8

17. ‌Griffiths, A. D. et al. Hyperspectral Imaging under Low Illumination with a Single Photon Camera. in 2018 IEEE British and Irish Conference on Optics and Photonics (BICOP) 1–4 (IEEE, 2018). https://doi.org/10.1109/BICOP.2018.8658323

18. ‌Jain, R., Hillger P., Grzyb, J. & Pfeiffer, U. R. Silicon-Integrated Single Pixel Terahertz Camera. in 45th International Conference on Infrared, Millimeter, and Terahertz Waves (IRMMW-THz) 1–2 (IEEE, 2020). https://doi.org/10.1109/IRMMW-THz46771.2020.9370707

19. ‌Hahamovich, E., Monin, S., Hazan, Y. & Rosenthal, A. Single pixel imaging at megahertz switching rates via cyclic Hadamard masks. Nat. Commun. 12, 4516 (2021). https://doi.org/10.1038/s41467-021-24850-x

20. ‌Chan, W. L. et al. A Single-Pixel Terahertz Camera. in Conference on Lasers and Electro-Optics and 2008 Conference on Quantum Electronics and Laser Science 1–2 (IEEE, 2008). https://doi.org/10.1109/CLEO.2008.4551487

21. ‌Zhang, Z. et al. A decade review of video compressive sensing: A roadmap to practical applications. Engineering 46, 172–185 (2025). https://doi.org/10.1016/j.eng.2024.08.013

22. ‌Han, S., Sato, I., Okabe, T. & Sato, Y. Fast spectral reflectance recovery using DLP projector. Int. J. Comput. Vis. 110, 172–184 (2014). https://doi.org/10.1007/s11263-013-0687-z

23. ‌Jin, S. et al. Hyperspectral imaging using the single-pixel Fourier transform technique. Sci Rep. 7, 45209 (2017). https://doi.org/10.1038/srep45209

24. ‌Edgar, M. P. et al. Simultaneous real-time visible and infrared video with single-pixel detectors. Sci. Rep. 5, 10669 (2015). https://doi.org/10.1038/srep10669

25. ‌Harwit, M. & Sloane, N. Hadamard Transform Optics. (Academic Press New York, 1979).

26. ‌Candès, E. J. & Tao, T. Near optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans. Inf. Theory 52, 5406–5425 (2006). https://doi.org/10.1109/TIT.2006.885507

27. ‌Candes, E. J., Romberg, J. & Tao, T. Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl.Math. 59, 1207–1223 (2006). https://doi.org/10.1002/cpa.20124

28. ‌Candes, E. J. & Wakin, M. B. An introduction to compressive sampling. IEEE Signal Process. Mag. 25, 21 (2008). https://doi.org/10.1109/MSP.2007.914731

29. ‌Balpande, A., Arora, J., Dhote K. & Khandelwal, R. Spatial Light Modulation Pattern Generation by Golomb Code for Single Pixel Imaging. in 2022 IEEE Region 10 Symposium (TENSYMP) 1–5 (IEEE, 2022). https://doi.org/10.1109/TENSYMP54529.2022.9864522

30. ‌Pastuszczak, A., Stojek, R., Wróbel, P. & Kotynski, R. Differential real-time single-pixel imaging with Fourier domain regularization - applications to VIS-IR imaging and polarization imaging. Opt. Express 29, 2685–26700 (2021). https://doi.org/10.1364/OE.433199

31. ‌Szajewski, K., Szajewska, A., Urbaś, S., Olbrycht, R. & Więcek, B. Imaging in NIR Spectral Range with A Single Detector Using – Pattern-Based SLM. in 17th Quantitative Infrared Thermography Conference (QIRT) (2024). https://qirt2024.org/assets/sazetak/QIRT-2024-046.pdf

32. ‌Bahmani, S. & Romberg J. compressive deconvolution in random mask imaging. IEEE Trans. Comput. Imaging 1, 236–246 (2015). https://doi.org/10.1109/TCI.2015.2485941

33. ‌Don, M. L., Fu, C. & Arce, G. R. Compressive imaging via a rotating coded aperture. Appl. Opt. 56, B142–B153 (2017). https://doi.org/10.1364/AO.56.00B142

34. ‌Candès, E. J., Romberg, J. & Tao, T. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52 489–509 (2006). https://doi.org/10.1109/TIT.2005.862083

35. Hema, M., Gurunadha, R., Suman, J. V. & Mallam, M. Effective Image Reconstruction Using Various Compressed Sensing Techniques. in 2024 International Conference on Advances in Modern Age Technologies for Health and Engineering Science (AMATHE) 1–6 (IEEE, 2024). https://doi.org/10.1109/AMATHE61652.2024.10582191

36. ‌Baraniuk, R. G. Compressive Sensing [Lecture Notes]. IEEE Signal Process. Mag. 24, 118–121 (2007). https://doi.org/10.1109/MSP.2007.4286571

37. ‌Huo, L., Chen, W., Ge, H. & Ng, M. K. Stable image reconstruction using transformed total variation minimization. SIAM J. Imaging Sci. 15, 1104–1139 (2022). https://doi.org/10.1137/21M1438566

38. ‌Gao, L., Zhai, A. & Wang, D. Comparison of common algorithms for single-pixel imaging via compressed sensing. Sensors 23, 4678 (2023). https://doi.org/10.3390/s23104678

39. ‌Bian, L., Suo J., Dai, Q. & Chen, F. Experimental comparison of single-pixel imaging algorithms. J. Opt. Soc. Am. A 35, 78–87 (2018). https://doi.org/10.1364/JOSAA.35.000078

40. ‌Satake, S. & Gu, Y. On Compressed Sensing Matrices Breaking The Square-Root Bottleneck. in 2020 IEEE Information Theory Workshop (ITW) 1–5 (IEEE, 2021). https://doi.org/10.1109/ITW46852.2021.9457623

41. ‌Lampe, B., Chang, C.-I., Bekit, A. & Porta, C. D. Restricted entropy and spectrum properties for the compressively sensed domain in hyperspectral imaging. IEEE Trans. Geosci. Remote Sens. 58, 5642– 5652 (2020). https://doi.org/10.1109/TGRS.2020.2968077

42. ‌Gan, L., Li, K. & Ling, C. Golay meets Hadamard: Golay-paired Hadamard matrices for fast compressed sensing. in IEEE Information Theory Workshop 637–641 (IEEE, 2012). https://doi.org/10.1109/ITW.2012.6404755

43. ‌Lim, J. Y. et al. A comparison between fourier and Hadamard single-pixel imaging in deep learning-enhanced image reconstru- ction. IEEE Sens. Lett. 7, 1–4 (2023). https://doi.org/10.1109/LSENS.2023.3303046

44. ‌Liu, Z., Li, D., Yan, P., Gong, M. & Xiao, Q. Single-Pixel High- Speed Imaging Through A Multimode Fiber. in 2022 Conference on Lasers and Electro-Optics (CLEO) JW3B.56 (2022). https://doi.org/10.1364/CLEO_AT.2022.JW3B.56

45. ‌Li, X., Li, J., Li, Y. & Xiao, Q. High-throughput terahertz imaging: progress and challenges. Light: Sci. Appl. 12-, 233 (2023). https://doi.org/10.1038/s41377-023-01278-0

46. ‌Zanotto, L., Piccoli, R., Dong, J., Morandotti, R. & Razzari, L. Single-pixel terahertz imaging: A review. Opto-Electron. Adv. 3, 200012 (2020). https://doi.org/10.29026/oea.2020.200012

47. ‌Singh, P. & Sonkusale, S. high speed terahertz modulator on the chip based on tunable terahertz slot waveguide. Sci. Rep. 7, 40933 (2017). https://doi.org/10.1038/srep40933

48. ‌Ma, W., Lin, Z., Sun, Y. & Liu, X. Multi-Color Terahertz Spatial Light Modulator for Single-pixel Imaging. in 2024 49th International Conference on Infrared, Millimeter, and Terahertz Waves (IRMMW-THz) 1–2 (IEEE, 2024). https://doi.org/10.1109/IRMMW-THz60956.2024.10697881

49. ‌Shrekenhamer, D., Watts, C. M. & Padilla, W. J. Terahertz single pixel imaging with an optically controlled dynamic spatial light modulator. Opt. Express 21, 12507–12518 (2013). https://doi.org/10.1364/OE.21.012507

50. ‌Einstein, A., Podolsky, B. & Rosen, N. Can quantum-mechanical description of physical reality be considered complete? Phys. Rev. 47, 777–780 (1935). https://doi.org/10.1103/PhysRev.47.777

51. Pittman, T. B., Shih, Y. H., Strekalov, D. V. & Sergienko, A. V. Optical imaging by means of two photon quantum entanglement. Phys. Rev. A 52, 3429–3432 (1995). https://doi.org/10.1103/PhysRevA.52.R3429

52. ‌Shapiro, J. H. & Boyd, R. W. The physics of ghost imaging. Quantum Inf. Process. 11, 949–993 (2012). https://doi.org/10.1007/s11128-011-0356-5

53. ‌Bennink, R. S., Bentley, S. J. & Boyd, R. W. ‘Two-photon’ coincidence imaging with a classical source. Phys. Rev. Lett. 89, 113601 (2002). https://doi.org/10.1103/PhysRevLett.89.113601

54. ‌Bennink, R. S., Bentley, S. J., Boyd, R. W. & Howell, J. C. Quantum and classical coincidence imaging. Phys. Rev. Lett. 92, 33601 (2004). https://doi.org/10.1103/PhysRevLett.92.033601

55. ‌Moodley, C. & Forbes, A. Super-resolved quantum ghost imaging. Sci. Rep. 12, 10346 (2022). https://doi.org/10.1038/s41598-022-14648-2

56. ‌Gatti, A., Brambilla, E., Bache, M. & Lugiato, L. A. Ghost imaging with thermal light: comparing entanglement and classical correlation. Phys. Rev. Lett. 93, 93602 (2004). https://doi.org/10.1103/PhysRevLett.93.093602

57. ‌Padgett, M. J. & Boyd, R. W. An introduction to ghost imaging: Quantum and classical. Philos. Trans. R. Soc. A 375, 20160233 (2017). https://doi.org/10.1098/rsta.2016.0233

58. ‌Liu, Y., Gutierrez-Barragan, F., Ingle, A. & Velten, A. Single- Photon Camera Guided Extreme Dynamic Range Imaging. in 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 41–51 (IEEE, 2022). https://doi.org/10.1109/WACV51458.2022.00012

59. ‌Chapman, G. H., Coelho Silva Menes, K. J., Wu, L. Y., Koren I. & Koren, Z. Image Degradation in Time Due to Interacting Hot Pixels. in 2023 IEEE International Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems (DFT) 1–6 (IEEE, 2023). https://doi.org/10.1109/DFT59622.2023.10313555

60. ‌Wu, J., Hu, L. & Wang, J. Single-pixel imaging for partially occluded objects. IEEE Photonics J. 15, 1–7 (2023). https://doi.org/10.1109/JPHOT.2023.3281820

61. Vaz, P. G., Guerra, B. & Cardoso, J. Single-pixel imaging: Concepts and application to imaging through scattering media. in 2023 23rd International Conference on Transparent Optical Networks (ICTON) 1–9 (IEEE, 2023). https://doi.org/10.1109/ICTON59386.2023.10207213

62. ‌Deng, Z. et al. Seeing through fire with one pixel. Opt. Lasers Eng.183, 108540 (2024). https://doi.org/10.1016/j.optlaseng.2024.108540

63. ‌Nan, Y., Yi, Z. & Bingxia, C. Review of Compressed Sensing for Biomedical Imaging. in 2015 7th International Conference on Information Technology in Medicine and Education (ITME) 225– 228 (IEEE, 2015). https://doi.org/10.1109/ITME.2015.119

64. ‌Kim, K. et al. Adaptive compressed sensing for the fast terahertz reflection tomography. IEEE Trans. Terahertz Sci. Technol. 3, 395– 401 (2013). https://doi.org/10.1109/TTHZ.2013.2267417

65. ‌Folks, W. R. et al. Characterization of digital-micromirror device- based infrared scene projector. Opt. Eng. 44, 086402 (2005). https://doi.org/10.1117/1.2013249

66. ‌Gattinger, P., Zorin, I., Ebner, A., Rankl, Ch. & Brandstetter, M. Mid-infrared DMD-based spectral-coding spectroscopy with a supercontinuum laser source. Opt. Express. 30, 6440–6449 (2022). https://doi.org/10.1364/OE.452221

67. ‌Mahalanobis, A., Shilling, R., Murphy, R. & Muise, R. Recent results of medium wave infrared compressive sensing. Appl. Opt. 53, 8060–8070 (2014). https://doi.org/10.1364/ao.53.008060

68. ‌Gibson, G. M. et al. Real-timeimaging of methane gas leaks using a single-pixel camera. Opt. Express 25, 2998–3005 (2017). https://doi.org/10.1364/OE.25.002998

69. ‌Denk, O., Musiienko, A. & Žídek, K. Differential single-pixel camera enabling low-cost microscopy in near-infrared spectral region. Opt. Express 27, 4562 (2019). https://doi.org/10.1364/OE.27.004562

70. ‌Stantchev, R. I., Yu, X., Blu, T. & Pickwell-MacPherson, E. Real- time terahertz imaging with a single-pixel detector. Nat. Commun. 11, 2535 (2020). https://doi.org/10.1038/s41467-020-16370-x

71. ‌Jin, S. et al. Hyperspectral imaging using the single-pixel Fourier transform technique. Sci. Rep. 7, 45209 (2017). https://doi.org/10.1038/srep45209

72. ‌Więcek, P. & Sankowski, D. A New Deep-Learning Neural Network for Super-Resolution Up-Scaling of Thermal Images. in 15th Quantitative Infrared Thermography Conference 1–7 QIRT Conference Proc. 1–7 (QIRT, 2020). http://www.qirt.org/archives/qirt2020/papers/134.pdf

73. Urbaś, S. & Więcek, B. Application of a deep-learning neural network for image reconstruction from a single-pixel infrared camera. Opto-Electron. Rev. 32, e148877 (2024). https://doi.org/10.24425/opelre.2024.148877

74. ‌https://www.tensorflow.org/ (2024).