Bio-inspired mid-infrared neuromorphic transistors for dynamic trajectory perception using PdSe₂/pentacene heterostructure

Mid-infrared intelligent sensing technology is crucial for detecting and identifying objects in challenging environments, as it can penetrate obstacles like smoke, fog, and dust, providing clear imaging and accurate target detection. In addition, its sensitivity to thermal emissions ensures reliable tracking and identification, even in the presence of visible light, maintaining excellent performance in diverse conditions. Consequently, MIR sensing is extensively used in military surveillance, environmental monitoring, industrial inspection, autonomous driving, and fire warning^1,2,3,4,5,6. However, current MIR vision systems face limitations due to the reliance on separate hardware for image sensors, memories, and processing units, which increases the need for extensive data exchange between image sensors and processing units during motion capture^7,8,9, leading to data transmission delays and inefficiencies^4,10,11. Emerging neuromorphic motion-capture devices offer superior motion processing capabilities, but are currently restricted to the visible and near-infrared (NIR) spectra^{12,13,14,15,16,17,18,19}. This restriction reduces their effectiveness in capturing and analyzing motion information in complex environments. To date, the studies on neuromorphic devices extending to the MIR range, especially over 4 μm, have been rarely reported. Therefore, extending neuromorphic motion-capture devices into the MIR spectrum is essential for efficient motion information capture under challenging conditions.

Biological systems excel in perceiving motion in complex environments with minimal computational overhead, which has inspired new insights in the design of optoelectronic devices^15,17. The Melanophila acuminata, a fire beetle, possesses a unique ability to detect MIR light through specialized protein receptors. By detecting and responding to distant flames even during flight, these fire beetles can accurately locate newly burned wood²⁰, providing an ideal environment for reproduction and improving the survival rates of their offspring^21,22,23. This efficient MIR motion perception, accomplished with minimal computational resources, serves as inspiration for the optoelectronic synapse designed for in-sensor MIR motion detection.

To extend the spectral range of neuromorphic motion-capture devices from NIR to MIR, an MIR neuromorphic transistor inspired by the biological systems of the fire beetle is proposed here. This transistor emulated the efficient MIR motion perception capabilities of the fire beetle, integrating sensing, memory, and processing functions. This was achieved by leveraging the unique properties of PdSe₂ and pentacene materials. PdSe₂, a two-dimensional noble metal dichalcogenide, served as the primary sensing layer due to its exceptional optical properties, narrow bandgap, excellent stability, and good material compatibility^{24,25,26,27,28,29,30,31,32}. Pentacene, selected as the organic semiconductor material with desired HOMO and LUMO levels, could be easily processed to form a heterostructure with PdSe₂. The heterostructure of PdSe₂ and pentacene offered efficient charge separation and achieved PPC in the MIR region^33,34,35,36. The developed optoelectronic neuromorphic transistor could simultaneously perceive MIR optical stimuli and process motion information, closely mimicking biological synaptic characteristics. In addition, real-time hardware implementation further validated the device array’s ability to memorize MIR motion trajectories, underscoring its practical potential for motion detection applications. Operating within the MIR region from 3 μm to 4.25 μm, it possessed detection ability under weak illumination of 0.5 mW/cm² MIR light, which was comparable to the detection threshold of fire beetle³⁷ and lower than the minimum detected light intensity previously reported for PdSe₂ in the MIR range^27,38. Furthermore, an RC system was trained to classify MIR flame movement directions, achieving a recognition rate of 94.79%. This work demonstrates the significant potential of the system to enhance MIR machine vision applications.

Design of the MIR synaptic device inspired by fire beetle for motion perception

The fire beetle possesses a remarkable ability to detect forest fires through its specialized thoracic pit organs. These organs enable the beetle to locate fires and find recently burned trees to lay its eggs, which is a crucial survival trait^{21,22,23,39,40}. As shown in Fig. 1a, the pit organ contains 50 to 100 sensilla²⁰, which function like highly sensitive infrared detectors, similar to the compound eyes of other insects. When exposed to MIR light, the membranes inside these sensilla deform in response to the MIR radiation, generating electrical signals⁴⁰. By integrating data from multiple sensilla, the beetle’s nervous system processes the movement and ___location of flames, guiding the beetle toward the fire⁴¹, as illustrated in Fig. 1a. This ability to perceive and analyze dynamic flame movement in real-time is essential for its navigation.

a Schematic illustrating the process that the fire beetle recognizes forest fires during flight, utilizing specialized MIR receptors located in its thoracic pit organs and transmits signals to the brain to analyze. b Schematic of the PdSe₂/pentacene heterostructure MIR neuromorphic transistor. c The MIR synaptic transistor array memorizes motion trajectories. d The RC system processes output signals from the MIR neuromorphic transistors, analogous to the beetle’s neural processing.

Full size image

The remarkable sensory and processing capabilities of the fire beetle served as a primary inspiration for our design. Specifically, our design was driven by the beetle’s unique ability to detect moving flames during flight via MIR light and navigate based on this information. We aimed to develop a system capable of processing and tracking the motion of MIR targets in a manner analogous to the beetle’s ability. The beetle achieves this through a series of complex biological signal transformations, where MIR radiation from the flames is first converted into mechanical vibrations within its pit organs and then further transduced into electrical signals that are processed by the beetle’s nervous system. In contrast, our design employs functional bio-inspiration to replicate the beetle’s sensing and processing capabilities using modern optoelectronic components. Rather than relying on mechanical vibrations, as observed in the beetle’s pit organs, the system utilizes direct optoelectronic conversion of MIR light into electrical signals. This approach simplifies integration into electronic systems, significantly enhancing the practicality and scalability of the concept.

Inspired by the fire beetle’s MIR perception ability, a PdSe₂/pentacene heterostructure MIR neuromorphic transistor was developed, as illustrated in Fig. 1b. PdSe₂, a narrow bandgap noble metal dichalcogenides^{24,25,26,27,28,29,30,31,32}, exhibited excellent MIR detection capability and optoelectronic properties. Pentacene was elected to form a type-II heterostructure with PdSe₂, achieving efficient charge separation under the MIR illumination and simulating the MIR synaptic characteristics. Just as the beetle processes continuous infrared radiation to track flames, our device tracks and memorizes the motion trajectories of MIR targets, as shown in Fig. 1c. When the devices are integrated with the RC system, the movement direction of flames can be analyzed and classified, effectively simulating the beetle’s ability to detect and respond to moving flame, as shown in Fig. 1d. This functional bio-inspiration approach enables efficient motion tracking, real-time processing, and scalable integration.

The neuromorphic performance of the MIR synaptic transistor

Figure 2 illustrates the characterization of the PdSe₂/pentacene heterostructure and synaptic characteristics of the MIR neuromorphic transistors. The Raman spectra, shown in Fig. 2a, revealed the characteristic peaks of both materials. For PdSe₂, the characteristic Raman peaks were observed at 148, 208, 224, and 260 cm⁻¹, corresponding to the vibrational modes A¹_g-B¹_1g, A²_g, B²_1g, and A³_g, respectively. The characteristic Raman peaks of pentacene were located at 1151, 1172, 1348, and 1367 cm⁻¹, consistent with previous reports^42,43,44. Atomic force microscopy (AFM) was utilized to characterize the thickness of the PdSe₂/pentacene film, as shown in Fig. 2b. The thickness of the heterostructure was about 39 nm, with the PdSe₂ layer being around 2.1 nm (Supplementary Fig. 1), as highlighted in the inset. The microstructure of the heterostructure was further examined using high-resolution microscopy. Figure 2c shows the cross-sectional transmission electron microscopy (TEM) image of the PdSe₂/pentacene heterostructure on Si/SiO₂ substrate, where the interfaces between SiO₂, PdSe_2, and pentacene film were revealed. The corresponding energy-dispersive X-ray spectroscopy (EDS) element mapping images displayed the uniform distribution of C, Pd, Se, and O from a cross-section, confirming the integrity and well-defined configuration of the PdSe₂/pentacene heterostructure. Supplementary Fig. 2 displays a high-angle annular dark-field (HAADF) image of the PdSe₂ film, revealing approximately 5-6 layers of PdSe₂.

a Raman spectra of the PdSe₂ and pentacene film. b AFM image of the PdSe₂/pentacene heterostructure film on a Si/SiO₂ substrate. c HAADF image and EDS mapping of the device. d EPSC responses to MIR pulses at 3100, 3400, 3600, 4000, and 4250 nm, with an intensity of 30 mW/cm² for 1 s. e PPF index as a function of Δt, triggered by 3400 nm and 4250 nm pulses at 30 mW/cm² for 1 s. f Comparison of the maximum PPF index under various wavelengths with prior works. g Transition from STP to LTP with increasing pulse duration at 3400 nm and 4250 nm. h Histogram of the response currents to three-bit light pulse sequences. “0” represents 1 s of darkness, while “1” represents 1 s of 3400 nm MIR light at 30 mW/cm². Each bit interval is 1 s. i Learning-experience behavior simulated in the MIR neuromorphic device using optical spike sequences at 4250 nm, 1 s duration and 1 s interval. Source data are provided as a Source Data file.

Full size image

The bio-inspired optoelectronic neuromorphic transistor demonstrated an MIR response capability, and the basic synaptic features were investigated. Supplementary Fig. 3 illustrates the transfer characteristics of the MIR optoelectronic transistor at wavelengths of 4250 nm and 3400 nm, and Supplementary Fig. 4 exhibits the output characteristics under different light intensities of 4250 nm and 3400 nm, respectively. The transistor demonstrated p-type transport behavior and a clear photoresponse to the MIR light of 4250 nm and 3400 nm. Figure 2d demonstrates the broad response of the device across the MIR band at wavelengths of 3100, 3400, 3600, 4000, and 4250 nm, with the excitatory postsynaptic current (EPSC) characteristics. The MIR synaptic device exhibited an increasing EPSC as the wavelength extended from 3100 nm to 4250 nm, with longer wavelengths (e.g., 4250 nm with 60 s retention time) also showing greater retention capability compared to shorter wavelengths (e.g., 3100 nm with 25 s retention time), under a 1 s photonic pulse. This behavior highlights the device’s capability to emulate synaptic plasticity in the MIR region. The paired-pulse facilitation (PPF) behavior, a critical aspect of neuromorphic systems, was simulated using two consecutive light pulses. Supplementary Fig. 5 demonstrates the PPF behavior at wavelengths of 3400 nm and 4250 nm, respectively. Figure 2e shows that the PPF index, induced by 3400 nm and 4250 nm MIR light pulses, decayed exponentially as the time interval (Δt) between the two light pulses increased. Significantly, with a time interval of 0.3 s, the device exhibited a high PPF index of up to 193% under 4250 nm MIR light. This high maximum PPF index underscores the device’s efficiency in controlling EPSC modulation through consecutive pulse signals. Figure 2f presents a comparison of the maximum PPF index achieved by the devices across various wavelengths, with references to previous studies^{45,46,47,48,49,50,51,52}. Notably, our device achieved synaptic characteristics in the MIR range with a high maximum PPF index, underscoring its excellent ability to operate in the MIR band.

In biological synapses, short-term plasticity (STP) can convert into long-term plasticity (LTP) through sustained external stimuli. Similarly, the device in this study showed a transition from STP to LTP as the duration of light pulses increased, indicating enhanced memory functions under repeated MIR light stimulation. This transition was clearly demonstrated in Fig. 2g, where changes in EPSC were shown for different pulse duration (0.5 s, 2 s, and 5 s) using 3400 nm and 4250 nm MIR light. Initially, STP induced by a short pulse decayed rapidly with a smaller EPSC. However, as the pulse duration increased, the EPSC also increased and its retention time extended, indicating a transition from STP to LTP. This synaptic plasticity transition could also be regulated by varying the number, frequency, and intensity of MIR light pulses, as shown in Supplementary Figs. 6 and 7. To further explore the factors influencing synaptic plasticity, we investigated the effects of channel area, pentacene thickness, and device temperature on STP and LTP in Supplementary Figs. 8–12. As shown in Supplementary Figs. 8 and 9, variations in channel length and width showed no significant impact on either STP or LTP. This can be attributed to the fact that the synaptic behavior primarily depends on the intrinsic properties of the PdSe₂/pentacene heterostructure and the charge dynamics at the interface. Increasing the pentacene thickness had a minimal effect on STP, but extended LTP duration. In Supplementary Figs. 10 and 11, this can be attributed to the slower recombination rate of deeper charge carriers in a thicker pentacene layer, leading to prolonged LTP. Furthermore, as shown in Supplementary Fig. 12, both STP and LTP durations were notably reduced at higher temperatures, suggesting that elevated temperatures accelerate carrier recombination, thereby diminishing synaptic retention. These findings provide critical insights into the design and optimization of MIR synaptic devices for neuromorphic applications. Moreover, to further investigate the growth of EPSC with the number of optical pulses, we calculated the A_n/A₁ index, representing the ratio of the first and last spike currents. As shown in Supplementary Fig. 13, there was a gradual deceleration in EPSC growth, indicating the saturation of excitons, where the device’s response plateaus despite continued stimulation, reflecting the physical limits of the system’s excitatory response.

The memory and decay effect of the device played a critical role in achieving nonlinear responses, allowing the device to exhibit distinguishable responses to optical pulse sequences. This behavior closely mimics the synaptic plasticity observed in biological systems, where signals decay over time to store spatio-temporal information. In our system, this memory and decay effect allowed the 3-bit sequence (ranging from 000 to 111) of each pixel to be accurately represented by different current states of the device. These states effectively reflected the temporal evolution of the input optical pulse sequence, as shown in Fig. 2h, resulting in an easily distinguishable sequence of current states. The experimental results show that the MIR synaptic device holds great promise for in-sensor RC processing, providing a solid foundation for further spatio-temporal data analysis and dynamic signal processing. The relationship between memory retention and time corresponds to the Ebbinghaus forgetting curves, where the multiple learning processes are accompanied by episodes of forgetting. Figure 2i demonstrates this learning-experience behavior. During the learning process stimulated by 4250 nm MIR light, initially, 20 pulses were required to achieve a certain memory level. After the first instance of partial forgetting, only 9 pulses were needed to relearn the same level, significantly fewer than the initial requirement. Following a second period of forgetting, the process became even more efficient, with only 7 pulses necessary to restore the same memory level. The gradual reduction in the number of pulses necessary for learning indicated an increase in learning efficiency. Meanwhile, the time taken to forget to the same level extended progressively with each period, reflecting memory consolidation. To emulate the fire beetle’s ability to detect distant fire through faint MIR light²³, our device was tested for its response to weak MIR signals. As demonstrated in Supplementary Fig. 14, the device could respond to MIR light with a minimum intensity of 0.5 mW/cm², equivalent to the detection threshold of fire beetle³⁷, confirming its capability to detect weak MIR signals.

The mechanism of the MIR synaptic device

Figure 3 provides a comprehensive analysis of the mechanism and performance of the bio-inspired MIR neuromorphic transistors, integrating results from blackbody radiation tests and Sentaurus-Technology Computer Aided Design (TCAD) simulations. Figure 3a illustrates the EPSC characteristics of the device under exposure to IR light at wavelengths of 4250, 3400, and 1550 nm, with each pulse characterized by an intensity of 30 mW/cm² and a duration of 1 s. The results indicated that the EPSC response was strongest at the wavelength of 1550 nm, followed by 4250 nm and 3400 nm. To explain this phenomenon, the n and k coefficients of PdSe₂ were investigated_. Considering the wavelengths, the W-n-k relationship was derived for PdSe₂, which could be found in Supplementary Fig. 15. Subsequently, the optical generation of PdSe₂ was calculated at the three wavelengths using TCAD, with results also available in Supplementary Fig. 16. The calculated optical response order was 1550 nm > 4250 nm > 3400 nm. By calculating based on the intrinsic properties of the material, we successfully explained the relative magnitudes of the responses, which were consistent with the experimental results. Subsequently, Fig. 3b presents the electron current density distribution within the PdSe₂/pentacene heterostructure under illumination at 1550, 3400, and 4250 nm, as obtained from TCAD simulations. This alignment further corroborated the experimentally observed order of the EPSC responses.

a EPSC characteristics under 10 pulses of MIR (4250 nm and 3400 nm) and NIR (1550 nm) illumination, each at 30 mW/cm², 1 s duration, and 1 s interval. b TCAD-stimulated electron current density of the PdSe₂/pentacene heterostructure under NIR and MIR stimulation. c TCAD-simulated energy band diagram under NIR and MIR illumination. d TCAD-stimulated scenarios for the MIR synaptic transistor under the 4250 nm, 1 s pulse. e Schematic diagram illustrating the setup for blackbody radiation measurements. f Responses of the MIR neuromorphic transistor to blackbody radiation at 627–927 °C, with a fixed 15 cm distance and a frequency of 0.5 Hz. Source data are provided as a Source Data file.

Full size image

To explore the underlying mechanisms, the energy band alignment of the PdSe₂/pentacene heterostructure was carefully designed and simulated, as shown in Fig. 3c. Pentacene, an organic semiconductor, has a bandgap energy (E_g) of 1.8 eV, whereas PdSe₂, a TMD, has a much smaller bandgap energy of 0.27 eV^53,54,55, which corresponds to MIR wavelength of 4595 nm. Upon illumination with MIR at 4250 nm and 3400 nm, as well as NIR at 1550 nm, photogenerated electron-hole pairs exclusively generated and separated within PdSe₂, with no optical response observed in pentacene (Fig. 3b). This was further supported by TCAD simulations (Supplementary Fig. 17) and experimental evidence, showing that pristine pentacene exhibited no detectable response to 1550 nm NIR and negligible optical absorption in the MIR range (Supplementary Fig. 18). Due to the alignment of energy bands, the photogenerated holes could transfer from PdSe₂ to pentacene, rapidly elevating the I_d. After the light pulse ended, some accumulated electrons in PdSe₂ quickly recombined with holes in pentacene, causing a rapid decay in I_d. However, other holes in pentacene persisted for a longer time, leading to the observed EPSC. In addition, traps at the interface between PdSe₂ and SiO₂ captured electrons in PdSe₂, hindering electron-hole recombination and prolonging hole retention time, which also extended the retention time of the EPSC. Overall, both heterostructure and interface traps contributed to the PPC effect. For comparison, the energy band diagram of pristine PdSe₂ is presented in Supplementary Fig. 19a. In pristine PdSe₂, electron-hole pairs generated under MIR illumination, with the holes acting as carriers, and a portion of the electrons being trapped at the interface. Upon the removal of light, the untrapped electrons and holes recombined rapidly, while a small fraction of trapped electrons recombined slowly with the holes. The comparison between the two systems revealed that the inclusion of the heterostructure not only enhanced carrier separation under MIR illumination but also slowed the subsequent recombination, thereby strengthening the photoresponse and extending the device’s retention time.

To further validate the role of the heterostructure and traps in device performance, Fig. 3d shows the impact of four scenarios on the synaptic characteristics under a 4250 nm light pulse, as simulated by TCAD. These scenarios included: PdSe₂/pentacene heterostructure with traps between PdSe₂ and SiO₂, PdSe₂/pentacene heterostructure without traps between PdSe₂ and SiO₂, pristine PdSe₂ with interface traps, and pristine PdSe₂ without interface traps. When both factors were absent, PdSe₂ functioned as a photodetector, displaying no retention time or gradual rise in current. When either factor was present alone, the synaptic properties exhibited short retention time. When both factors were present, the simulation closely matched the experimental results shown in Supplementary Figs. 19b and 19c, exhibiting synaptic behavior with longer retention time. The results indicated that the combined effect of the heterostructure and traps significantly enhanced the device’s photoresponse and retention time, aligning with experimental results and TCAD simulation data.

To assess the practical application potential of our device, blackbody radiation testing was employed to provide a standardized and controlled environment for measuring the device’s sensitivity to thermal radiation^56,57,58. Blackbody radiation, which simulates the emission of an ideal thermal source, allows for simulating real-world thermal sources and evaluating the device’s photoresponse across a range of temperatures and emission spectra. The experimental setup, as shown in Fig. 3e, utilized a high-temperature blackbody source placed 15 cm away from the device. The blackbody temperature varied between 900 K and 1200 K, covering a range of typical thermal radiation sources. A chopper modulated the incident radiation at 0.5 Hz to generate pulse measurements of the device’s response to thermal stimuli. The blackbody test results shown in Fig. 3f revealed spectral radiance peaks at wavelengths corresponding to 3220 nm (627 °C), 2898 nm (727 °C), 2635 nm (827 °C), and 2415 nm (927 °C), demonstrating the device’s broad responsiveness to blackbody thermal radiation. Repeated exposure to blackbody thermal radiation at these temperatures resulted in the accumulation of photoresponse over multiple pulses, demonstrating the device’s ability to effectively process and memorize information from weak thermal signals, confirming its potential as an optoelectronic neuromorphic device for applications in low-light environments. This behavior highlights the device’s suitability for real-world applications such as fire detection, environmental monitoring, and intelligent night vision, where accurate detection of subtle thermal signals is crucial in challenging lighting conditions.

Memorization of MIR motion trajectory through MIR synaptic transistor array

Figure 4 illustrates the perception, memory, and preprocessing functions of the MIR synaptic transistor array. Unlike conventional image sensors, which rely on discrete frame readouts and external processing for object recognition^7,8,9, our bio-inspired MIR neuromorphic devices can preprocess the MIR motion information and memorize motion trajectories in a frame. This feature significantly reduces the need for complex post-processing, making our devices highly effective in scenarios that demand quick and reliable motion detection, thereby enhancing motion detection capabilities^{12,13,14,15,17,18}. To demonstrate the performance of the MIR synaptic transistor array, a 4 × 4 device array consists of 16 individual synaptic transistors, designed for efficient motion detection and processing under MIR light. Figure 4a presents the optical image of the completed 4 × 4 MIR synaptic transistor array, which is fabricated on a 1 cm × 1 cm Si/SiO₂ substrate with precise alignment and consistent device spacing. Each transistor functions as a synaptic unit, enabling the processing of MIR signals for motion detection. The layout of the array ensures efficient signal processing while maintaining spatial uniformity across the devices, which is crucial for reliable motion trajectory recognition. The fabrication process, detailed in Supplementary Fig. 20, began with the deposition and patterning of a PdSe₂ layer using photolithography to precisely control the device dimensions and positioning. A pentacene layer was then thermally evaporated to form the required heterostructure, followed by the evaporation of electrodes to establish the electrical connection for each device. Supplementary Figs. 21–23 provide optical images of key fabrication stages, demonstrating the precision of the techniques and the consistent fabrication of each device within the array. Furthermore, to evaluate the device performance in hardware implementation, the 4 × 4 MIR synaptic device array was integrated with a custom-designed real-time readout circuit, which could simultaneously process 16 channels, as shown in Fig. 4b. In addition, Supplementary Fig. 24 presents the actual testing conditions and environment of the MIR synaptic array within the readout circuit. For better understanding, Supplementary Figs. 25 and 26 provide detailed descriptions of the circuit’s functionality and the signal conversion process. In Fig. 4c, the movement of the MIR laser beam across the device array was stimulated. As the laser traced a flipped “L”-shape on the array, the devices perceived and memorized the trajectory. Figure 4d illustrates the synchronization between light input timing and the corresponding pixel outputs, revealing how the device integrated signals from individual pixels into a cohesive motion response. Figure 4e further validates these results by presenting the readout signals from the real-time circuit as the 3400 nm MIR laser followed the “L”-shaped path. The dynamic response of each pixel, represented by V_out over time, illustrated the real-time motion processing capability of the device. In addition, the transfer curves of all 16 devices (Supplementary Fig. 27) confirm the uniformity of the array, ensuring consistent performance. As the laser moved, the relevant pixel units generated significant current responses that gradually decayed after the beam moved away. In Fig. 4f, the slow movement of a 3400 nm MIR laser beam across the array resulted in current responses from the activated pixel units. The laser illuminated each device for 1 s with a 0.5 s interval. Initially, the array was in the dark state. As the laser began its trajectory, irradiating the first device, it moved sequentially across the array. After 10 s, the laser had completed the path, and the motion trajectory became clearly visible through the normalized current mappings, demonstrating the device’s excellent ability to capture and retain motion trajectories under MIR light. During the subsequent 15 s and 30 s stages, the slow decay of EPSC led to a gradual decrease in the normalized current values along the reversed “L” path, indicating the device’s capacity to memorize the motion trajectory even after the illumination had ceased. Furthermore, as the memory effect dissipated, the device showed the capability to return to its initial state, ready to capture new motion events. By directly sensing, memorizing, and preprocessing motion information, our MIR neuromorphic devices effectively simplify the complexity of the MIR vision system and offer a more efficient and intelligent solution for MIR motion detection.

a Optical image of a 4 × 4 MIR synaptic transistor array. b Synaptic transistor array integrated with a 16-channel real-time readout circuit. c Schematic of the MIR laser light moving upon the device array following a flipped “L” trajectory. d Relationship between image pixels and optical input through array preprocessing. e V_out measured from the readout circuit at pixels involved in the trajectory in the device array. f Normalized current mappings during MIR laser movement (first 10 s) and memory fading (after 10 s). Source data are provided as a Source Data file.

Full size image

Classification for flame motion directions based on RC systems

The perception of motion direction is crucial in intelligent vision systems, especially in challenging environments. MIR light, with its superior penetration capabilities, enables clearer detection of motion over long distances or in low-visibility conditions, making it essential for applications such as fire detection, autonomous driving, and intelligent night vision. The fire beetles, for example, can detect distant flames during flight by sensing faint MIR light in forests to locate the flame, at distances where visible light cannot penetrate^20,23,37, showcasing their remarkable ability to sense and analyze moving flames, as illustrated in Fig. 5a. Inspired by this, we developed an in-sensor RC system for MIR motion perception to classify the motion directions of the flames. The RC system is a class of recurrent neural networks that provides an efficient framework for processing spatio-temporal information. The RC system consists of three main components: an input layer, a reservoir layer and an output layer. The input layer is responsible for receiving input signals, while the reservoir layer consists of a group of reservoir nodes that encode the inputs into reservoir states (feature map). The output layer is typically a linear layer that is trained for classification using the reservoir states generated by the reservoir layer. A key advantage of the RC system is its minimal training requirement. While the input and reservoir layers remain fixed, only the weights of the output layer need to be trained, making them highly suitable for edge computing applications, where rapid and efficient learning is essential. In Fig. 5b, our RC system is clearly illustrated with an input layer that received raw MIR optical signals, a reservoir that processed these signals to extract spatio-temporal features of flame motion, and an output layer consisting of a four-layer artificial neural network (ANN) for accurate recognition of flame motion direction. In addition, to further illustrate the advantages of the RC system in processing multiple frames of images, we have included a comparison of the number of weights required for training in networks with and without a reservoir in Supplementary Fig. 28. This further emphasizes that the reservoir network significantly reduces the number of weights required when processing multiple frames of images.

a Schematic of a flying fire beetle detecting fire in motion through MIR light in the forest. b Schematic of the RC system for recognizing and classifying the flame movement directions. c, d Demonstration of flame movement in rightward and upward directions, with final integrated images. Each flame motion was divided into three moments (t₀, t₁, and t₂), each representing one frame. e Confusion matrix for classifying nine flame motion directions. f Classification accuracy over epochs for the flame motion directions. g Loss over epochs for the classification of flame motion directions. Source data are provided as a Source Data file.

Full size image

To train and evaluate the RC system, a comprehensive dataset of MIR images of real-world moving flames was captured, encompassing nine representative movement directions: up, down, left, right, stationary, upper-left, lower-left, upper-right, and lower-right. This dataset, which covered all fundamental motion directions in two-dimensional space, ensured the system’s robust capability to recognize and classify all potential flame motion patterns. To achieve comprehensive coverage of flame dynamics, representative frames were selected through time sampling during the flame’s movement. For example, during motion in the “right” direction, three frames were randomly chosen as a set. The selection process adhered to the following principles: the time interval between frames should capture the dynamic characteristics of the flame’s movement, avoiding both excessively dense and sparse intervals to ensure sufficient motion information; the selected frames should represent key spatial features, such as tail oscillation and overall contour changes; and the three chosen frames should reflect the primary movement trend of the flame, providing adequate spatio-temporal information for classification. In addition, our dataset selection carefully considers changes in motion speed, which is clearly demonstrated by the four data sets in the rightward direction presented in Supplementary Fig. 29.

Figures 5c, d illustrate examples of the “right” and “up” moving through three consecutive MIR images of the flames at time points t₀, t₁, and t₂, and their final images after compressing the spatio-temporal information from the three frames into one. In our experiment to validate the hardware implementation of the RC system, each pixel’s MIR synapse received a light pulse sequence, with the light intensity determined by the pixel value of the input image. The currents of all devices were recorded after three frames, leveraging the memory and decay characteristics of the MIR synaptic devices to compress the three-frame information into a single frame, significantly reducing the data size, as shown in Supplementary Fig. 30. In the RC system, the MIR synapses were employed to simulate reservoir nodes, where their memory and decay characteristics mimic the recursive functionality of traditional reservoir nodes. Then, the compressed image was transmitted to the output layer of the RC system for further processing and analysis. The output layer was structured as a four-layer ANN. By utilizing the final current states of each pixel, which reflect the temporal evolution of the flame’s motion, the ANN was trained to classify the direction of the flame’s movement. Ultimately, the RC system could successfully classify the nine distinctive directions of flame motion in MIR images.

The confusion matrix displayed the model’s prediction performance across different classes in Fig. 5e, showing that the RC system exhibited high classification accuracy across all nine motion directions, demonstrating its strong capability in handling the motion of different directions. Figure 5f shows an overall classification accuracy of 94.79%, further confirming the RC system’s accuracy in recognizing different flame motion directions. In addition, Fig. 5g illustrates the RC system’s loss over the training epochs. The loss decreased progressively as the training epochs increased, with a particularly rapid decline during the first 50 epochs. This indicated that the RC system quickly learned and improved its classification ability for flame motion directions in the early stages. Subsequently, the rate of loss reduction slowed down and stabilized around 150 epochs, suggesting that the RC system had reached a stable state with optimal classification capability. The experimental results above showed that this RC system had satisfactory performance for the classification of nine distinctive directions of flame motion. This aligned with the ability of fire beetles to quickly analyze the dynamic information and respond accordingly. The successfully trained RC system with MIR neuromorphic motion devices also highlights the potential to enhance the intelligent MIR vision system. This capability is highly significant for practical applications, particularly in flame monitoring and fire early-warning systems, as it enables real-time tracking of flame changes and motion patterns, providing timely responses. For example, in forest fire monitoring, the system can identify the direction of flame spread, predicting the fire’s trajectory and facilitating proactive preventive measures. Dynamic flame target recognition can also be applied in industrial safety, drone-based fire detection, post-disaster assessments, and other fields, achieving efficient early warning and response in the initial stages of a fire. Furthermore, the integration of the RC system, which achieves a 94.79% classification accuracy for flame motion directions, enhances the system’s ability to process and analyze dynamic MIR signals. Beyond fire detection, the system’s broad MIR response range (3 μm to 4.25 μm) and ability to process spatio-temporal information make it suitable for applications in environmental monitoring, military surveillance, autonomous driving, and night vision. This versatility underscores the system’s potential to advance MIR machine vision across various fields.

This work has demonstrated an MIR neuromorphic transistor inspired by the biological system of the fire beetle, addressing key challenges in current MIR motion tracking systems. By integrating PdSe₂ and pentacene materials, the developed heterostructure achieved efficient charge separation and PPC effect, and realized the synaptic characteristics in the MIR region. The devices possessed detection ability under weak illumination of 0.5 mW/cm² MIR light, which was comparable to the detection threshold of the fire beetle. Unlike conventional MIR sensing technologies, this optoelectronic neuromorphic transistor could not only perceive optical stimuli within the 3 μm to 4.25 μm MIR range but also preprocess motion information with biological synaptic characteristics. By integrating sensing, memory, and processing functions, the proposed system significantly reduced data transmission delays and enhanced motion processing efficiency. Furthermore, by training an RC system to classify MIR flame movement directions, the system achieved a recognition rate of 94.79%, demonstrating its substantial potential to enhance MIR machine vision capabilities. Overall, this research offers a forward-looking solution for advancing MIR machine vision systems, particularly in scenarios requiring dynamic MIR target tracking. The findings lay a solid foundation for future MIR applications in military surveillance, environmental monitoring, industrial inspection, autonomous driving, and fire warning.

Materials

PdSe₂ films were grown directly on Si/SiO₂ substrates by chemical vapor deposition (purchased from SixCarbon Technology Shenzhen). Pentacene was purchased from TCI and Sigma-Aldrich. Acetone and isopropyl alcohol were purchased from Sinopharm Chemical Reagent Co., Ltd. The gold particles (99.999%) were obtained from ZhongNuo Advanced Material (Beijing) Technology Co., Ltd. All chemicals were used as received without any further purification.

Fabrication of MIR synaptic devices

The MIR neuromorphic transistors were fabricated with a bottom-gate top-contact structure on a Si/SiO₂ substrate with a 300 nm thickness gate dielectric. A six-layer PdSe₂ film with a feature size of 10 mm × 10 mm was synthesized via chemical vapor deposition (CVD) on a Si/SiO₂ substrate. To pattern the PdSe₂ layer, AZ5214E photoresist was spin-coated onto the surface, annealed, and exposed to 365 nm UV light through a shadow mask for 10 s, followed by development in AZ 300 MIF developer. Then, the sample was treated with reactive ion etching (RIE) with CF₄ gas for 60 s to define the device structure. Residual photoresist was removed for cleaning the sample with acetone and isopropyl alcohol. Next, with a shadow mask covered, pentacene was thermally evaporated on the top of the PdSe₂ layer under 5 × 10^-4Pa, forming the PdSe₂/pentacene heterostructure essential for efficient charge separation and MIR detection. Finally, gold (Au) source and drain electrodes were thermally evaporated on the pentacene layer via a shadow mask with the channel length and width of 30 μm and 1000 μm, respectively, and they can be adjusted according to the requirements. For further details, refer to Supplementary Fig. 20, which outlines the fabrication sequence.

Characterization

The Raman Spectroscopy of PdSe₂ was measured by LABRAM ARAMIS (Horiba HR 800). The surface morphology and electrical images of PdSe₂/pentacene film and PdSe₂ film were characterized by Bruker AFM systems in ambient conditions at room temperature under different scanning modes. The HAADF image and EDS mapping were characterized by FEI Themis Z. An MIR laser with tunable wavelengths was used to test the MIR responses of the optoelectronic device. The laser with multiple wavelengths made by CNI was used to test the photoelectric performance of the device in the NIR region. Blackbody testing was measured by a calibrated commercial blackbody furnace (HFY-206A). PDA Fs-Pro semiconductor parameter instrument was used to measure the electrical properties of the devices. All of the electrical properties were measured at room temperature.

Simulation of RC systems

To construct the MIR flame dataset, we captured 700 recordings of moving flame by MIR camera. The dataset consisted of 500 sets for the directions up, down, left, right, and stationary, with 100 for each direction and 200 for the directions upper-left, upper-right, lower-left, and lower-right, with 50 for each direction. To construct the suitable dataset for the RC system, each set of flame moving was divided into three frames of MIR flame, with each frame containing 512 × 640 pixels. Each set was represented as a 512 × 640 × 3 matrix. The data was input into the RC system and passed through the reservoir layer, which consisted of 327,680 reservoir nodes, each corresponding to one pixel. Each node was configured according to the memory and decay characteristics of the MIR synaptic device. With the illumination of each pixel processed according to its respective light intensity, the spatio-temporal information of the three frames was compressed into current signals of one frame after passing through the reservoir nodes, which were then sent to the output layer of the RC system. Then, the output layer of the RC system was constructed for training, consisting of a fully connected layer with 327,680 inputs, 3 hidden layers, and 9 output neurons to classify the nine flame movement directions (“up”, “down”, “left”, “right”, “stationary”, “upper-left”, “upper-right”, “lower-left”, and “lower-right”).