Introduction

Dual-modal imagers that enable the capture of static and process dynamic vision information within a single sensor represent profound implications for industrial surveillance systems and autonomous driving1,2,3,4. Traditional machine imaging, which relies on frame capture, encounters significant challenges in simultaneously processing both static and dynamic information5,6. Particularly when processing dynamic information, the primary challenge arises from the extensive transformations, transmissions, storage, and processing of static data (e.g., frame images), leading to significant data redundancy3,5.

Biological visual systems offer an efficient approach to processing dynamic visual information, with the retina playing a key role in capturing visuals and preprocessing of dynamic information (Fig. 1a). Inspired by the biological paradigm, incorporating image preprocessing tasks into sensors could serve as an effective strategy to reduce complexity and enhance the efficiency of image processing2,7. Biologically inspired neuromorphic devices, such as optoelectronic synaptic devices8,9 and resistive random-access memories10,11 show promise for their time-dependent plasticity. In particular, optoelectronic synaptic devices are notable for their exceptional light responsivity and photonic non-volatility, making them particularly advantageous for neuromorphic imaging12,13. Recent advancements in neuromorphic devices made from two-dimensional materials14,15,16,17, perovskite materials18,19,20, and organic materials21,22,23,24,25 demonstrate significant potential. In particular, organic neuromorphic devices feature a simple device architecture and solution processability, facilitating seamless integration with readout integrated circuits (ROICs)26,27. By incorporating interfacial traps28, energy barriers21,29,30 or ferroelectric materials22 into organic optoelectronic devices31, significant photonic non-volatility characteristics can be achieved. However, the complex device architecture and difficult integration processes hinder their application in high-resolution and dual-modal imaging, presenting challenges for advanced visual systems.

Fig. 1: Organic focal plane array neuromorphic imager for dual-modal imaging.
figure 1

a Schematic of human visual system. Light stimuli trigger electrical responses in retinal photoreceptors, processed by bipolar and amacrine cells. The refined signals are encoded by retinal ganglion cells and sent via the optic nerve to the visual cortex for perception. b The schematic depiction of the organic focal plane array neuromorphic imager. c Photographs of wafer-scale neuromorphic chips, neuromorphic CMOS chip, and neuromorphic imaging module.

Full size image

Here, we introduce a CMOS-integrated organic neuromorphic imager with a resolution of 640 × 512 pixels, capable of dual-modal functionality, including the imaging of both static and dynamic scenes (Fig. 1b). We developed the single-element neuromorphic device, integrated patterning method, and neuromorphic imagers, ultimately achieving dual-modal imaging of real-world scenes. We designed lateral two-terminal architecture organic neuromorphic device with adjustable memory behavior. Though the modulation of charge recombination-trapping dynamics, the device exhibits an adjustable delay time from 101 to 104 ms. We further developed a CMOS-compatible photolithography method for organic neuromorphic devices by leveraging photocrosslinking between polymers. This patterning method offers non-destructive characteristics with resolution approaching the limit of the photomasks (~1 μm in this study), supporting dual-modal neuromorphic device patterning of organic bulk heterojunctions (BHJs). We developed an integrated organic neuromorphic imager (Fig. 1c), enabling static imaging and in-pixel dynamic perception in real-world scenarios. The neuromorphic imager supports in-pixel memorization with a memory retention time exceeding 18 min. Neuromorphic devices’ intrinsic memory allows imagers to operate at lower frame rates for longer periods without losing critical information from brief targets, significantly reducing data storage and processing needs. Our neuromorphic imager can capture brief light events and generate event distribution maps in a single-frame image, demonstrating the imager’s potential for long-duration industrial monitoring. Furthermore, this imager can accurately extract motion trajectory maps with temporal information from single-frame static images, further demonstrating their potential for advanced imaging applications.

Results

Design of a neuromorphic device with adjustable memory properties

Figure 2a depicts the structure of the organic neuromorphic device. This device features a lateral architecture with donor-acceptor BHJs positioned between the Au electrodes. (3-Mercaptopropyl)trimethoxysilane (MPTS) serves as the interfacial trap layer. Figure 2b illustrates the energy band and processes mechanisms of the device. Upon light stimulation, excitons are generated and separated by the built-in and external electric fields. Electrons are trapped by interfacial traps in the BHJs, allowing holes to move towards the electrode. Figure 2c shows the chemical structure of [[2,6′-4,8-di(5-ethylhexylthienyl)benzo[1,2-b;3,3-b]dithiophene][3-fluoro-2[(2-ethylhexyl)carbonyl]thieno[3,4-b]thiophenediyl] (PTB7-Th), 2-((Z)-2-((7-(((Z)-3-(dicyanomethylene)-5-(5-methylthiophen-2-yl)-1-oxo-1,3-dihydro-2H-inden-2-ylidene)methyl)-4,4,9,9-tetrahexadecyl-4,9-dihydro-s-indaceno[1,2-b:5,6-b’]dithiophen-2-yl)methylene)-6-methyl-3-oxo-2,3-dihydro-1H-inden-1-ylidene)malononitrile (PZ1) and MPTS. The absorption of the BHJs is illustrated in Supplementary Fig. 1. The PTB7-Th and PZ1 shows primarily absorbs light between 550 and 750 nm. The photonic response of the device was assessed by applying a light pulse, as depicted in Fig. 2d. Upon light stimulation (power density of 100 mW cm−2, duration of 100 s), typical discernible increase in excitatory postsynaptic current (EPSC) was observed, followed by a gradual decay after cessation of illumination. The photogenerated carriers confined in the interfacial traps lead to a sublinear rise/down in EPSC over time. In our case, following 100 s of optical stimulations, the memory retention time exceeded 2500 s.

Fig. 2: Photonic non-volatility of the single-element neuromorphic devices.
figure 2

a Schematic of the organic device. b Schematic diagrams of the mechanism processes. c Chemical structure of the organic semiconductors utilized in the device. d Photonic responses demonstrating memory characteristics of the device. Applied stimulation light: 455 nm, 100 mW cm−2, and 100 s. e Photon-triggered STP and LTP of the device. Stimulation used 455 nm, 100 mW cm−2 light pulses, 1 s in duration, at 0.1 Hz for STP and 0.5 Hz for LTP. f PPF index of the device relative to spike time interval (ΔT) with the same light condition (power density of 100 mW cm−2 and duration of 1 s). g Photon-triggered LTP of the device with and without MPTS coating. h Characteristic photonic responses of the neuromorphic device. The device was initially stimulated with 10 light pulses (455 nm, 100 mW cm−2, 1 s duration) at a frequency of 0.5 Hz. Subsequently, the power supply was interrupted for 40 to 80 s, as indicated by the pink-shaded region. i Memory retention time traces of devices with varying acceptor (PZ1) weight ratios. Each device was stimulated with 100 light pulses (power density of 100 mW cm−2, 1 s duration) at a frequency of 0.5 Hz. j EPSC (blue line) and delay times (red line) of devices with different PZ1 weight ratios after stimulating with 100 light pulses (100 mW cm−2, duration of 1 s and frequency of 0.5 Hz). k I-T response curves of the devices at acceptor (PZ1) weight of 50%.

Full size image

The neuromorphic device exhibits light-dependent and time-dependent characteristics. Applying optical pulses to the devices, specifically, 0.1 Hz (10 pulses) and 0.5 Hz (50 pulses)—elucidated the transition phenomena from short-term plasticity (STP) to long-term potentiation (LTP) (Fig. 2e). Paired-pulse facilitation (PPF) experiments demonstrate synaptic plasticity similar to biological mechanisms, reflecting dynamic adaptation in visual responses. The PPF index usually decays with the increase in Δt, which may comply with: PPF decay = c1 × exp(−Δt/τ1) + c2 × exp(−Δt/τ2). Here, τ1 and τ2 represent the characteristic relaxation times associated with the rapid and slow phases of PPF delay, respectively. c1 and c2 denote the initial magnitudes of facilitation for the rapid and slow phases. Here, τ1 and τ2 are 0.44 and 32.85 s, respectively, which are compatible with the synaptic time scale32,33. The PPF index in our devices was approximately 1.24 at an illumination intensity of 100 mW cm−2 (Fig. 2f). The neuromorphic device can be modulated by changing the number of optical spikes, intensity, duration, frequency, bias voltage, and operating wavelength. (Supplementary Section 1 and Supplementary Figs. 2, 3). It is noteworthy that the MPTS layer in our neuromorphic device is essential, which can serve as chargeable dielectrics or trap layers due to its nonpolar and hydrophobic characterization34,35. Without the MPTS layer, the device exhibits negligible memory effects, as illustrated in Fig. 2g. Further discussion is in Supplementary Section 2 and Supplementary Figs. 4, 5. In addition, the optoelectronic neuromorphic device showed robust non-volatility, preserving its memory profile even when supply power was interrupted, thereby mitigating data loss during power interruptions (Fig. 2h). The application of reverse gate electrical pulses (indicated by the gray regions, Supplementary Fig. 6) enables the erasure of the non-volatile current, thereby illustrating the neuromorphic device’s ability to achieve photonic writing and electrical erasing. As shown in Supplementary Fig. 6, by applying a pulsed gate voltage of −5 V, we successfully achieved electrical erasing of the stored information. The device demonstrates good durability, after repeated 7 cycles of writing and electrical erasing, the performance remains well-preserved. These results confirm that the resetting process is both effective and non-detrimental to the operational efficiency and longevity of the device.

Energy efficiency is a critical factor for the artificial vision nervous system. The energy consumption (E) per light event can be calculated as E = VDS × IEPSC × t, where IEPSC represents the peak value in the EPSC and t is the light pulse width. As illustrated in Supplementary Fig. 7, the device demonstrates an exceptionally low power consumption of 4 fJ per event when operated at a voltage of 0.2 V with a 5 ms light pulse. When the light pulse duration is extended to 500 ms, the energy consumption increases to 1.2 pJ. These results highlight the device’s energy efficiency across different operational conditions, underscoring its potential for low-power neuromorphic applications.

We emphasize that these neuromorphic devices exhibit adjustable memory properties through the modulation of charge recombination-trapping dynamics. The decay time of the EPSC depends on the dynamics of charge recombination and trapping in the devices. Here, τ0 is defined as the delay time constant, indicating the time it takes for the EPSC to decay to approximately 36.8% (1/e) of its initial peak value following light stimulation. Modifying the weight percentage of PZ1 affects charge recombination center density, enabling control of charge recombination and trapping dynamics, thereby changing the decay time constant (τ0) and the memory retention time. The device shows a τ0 range from 101 ms to 104 ms after 100 light pulses (1 s pulse width with an intensity of 100 mW cm−2 at 455 nm). At 1 wt% PZ1, a significant portion of the photogenerated electrons is trapped by interfacial traps, leading to a longer τ0 of 27.82 s. Increasing PZ1 concentration in the device raises recombination center density, reducing τ0 of EPSC and increasing its amplitude (Fig. 2j). Adding 5 and 10 wt% of PZ1 yields τ0 of 4.04 and 0.030 s. When adding 30 wt% of PZ1, the device shows a τ0 of 0.028 s and retention time of ~191 s, and a maximum EPSC of 3.03 μA. When donor and acceptor material concentrations are balanced (50 wt% PZ1), the device shows negligible memory effects, with minimum τ0 of 0.028 s and retention time of 0.030 s (Fig. 2k). This tunability of delay time and retention time in neuromorphic devices facilitates the processing of dynamic vision tasks with varying temporal resolutions. The mechanisms behind the adjustable memory properties of these devices are depicted in Supplementary Fig. 8.

The device demonstrates good storage stability and operational stability in both the fast-response mode and the low-response mode (or standard mode and synapse mode). Specifically, the synaptic device is able to maintain its high and low conductance states at ~90% of their initial values after a period of 30 days. Similarly, the standard device also shows good stability, retaining over 90% of its initial performance after being stored in ambient conditions for 30 days (Supplementary Fig. 9). Operational stability of the devices is tested in air under a bias voltage of 2 V over 1000 s. Supplementary Fig. 10 demonstrates that the devices operate with good stability in synaptic mode, where variations in dark current and maximum EPSC remain within 3%. Similarly, when operating in standard mode, as shown in Supplementary Fig. 11, the device also exhibits robust operational stability, confirming their reliability under practical working conditions. Moreover, the device shows good batch-to-batch variation. As shown in Supplementary Fig. 12a, the I-V curves of different devices under dark conditions exhibit nearly identical low resistance states, demonstrating the controllability of defects. Additionally, we measured the excitatory postsynaptic current (EPSC) behaviors of the synaptic devices and analyzed the peak EPSC after light pulses for each device Supplementary Fig. 12b. For devices in standard mode, we also evaluated the dark current and detectivity across eight devices. The statistical analysis of their detection rates further corroborates the consistent batch-to-batch stability of these devices (Supplementary Fig. 12c, d).

CMOS-compatible, non-destructive photolithography of organic neuromorphic devices

Transitioning from single-element devices to dual-modal CMOS neuromorphic imagers requires effective patterning methods. This transition is enabled by the direct optical patterning method that utilizes the photochemistry of bisazide crosslinkers with polymer alkyl chains (Supplementary Section 3)24. We selected (1E,4E)-1,5-bis(4-azido-2,3,5,6-tetrafluorophenyl) penta-1,4-dien-3-one as photocrosslinker additives (Fig. 3a). Upon exposure to 365 nm UV radiation, bisazides decompose to release nitrogen and generate highly reactive singlet nitrene radicals at both termini. These nitrene radicals form covalent C-N bonds with the alkyl chains via C-H insertion reactions, leading to the cross-linking of adjacent polymer alkyl chains into insoluble networks within the original solution (Supplementary Fig. 13)36,37. The method consists of three steps (Fig. 3b): (i) coating a substrate with a blend of BHJs and photocrosslinker solution. (ii) exposing specific regions to UV light through a photomask. (iii) developing the coated substrate with chlorobenzene to remove unexposed BHJ films. Through repeated photolithography steps, we can integrate 50% PZ1 BHJ and 1% PZ1 BHJ onto the ROIC to fabricate a dual-mode imager (Supplementary Fig. 14). Supplementary Fig. 15a–c presents microscopic images of patterned BHJ films, including complex logos and microdot arrays.

Fig. 3: Non-destructive photolithography of organic neuromorphic devices.
figure 3

a Chemical structure of the photocrosslinker (left). Schematic diagrams of the mechanism processes (right). Photogenerated nitrene- crosslinkers bridge alkyl-chain on adjacent polymers via C-H insertion. Polymer BHJs lose colloidal stability in the original solution after UV exposure. b Overview of the patterning procedures for BHJs, including ink coating, UV exposure, and development of BHJ patterns using nonpolar solvents. c CMOS-compatible photolithography. The top photograph of the CMOS ROICs with a resolution of 640 × 512 pixels. Optical microscope images of BHJ patterns tailored to the pixel dimensions of CMOS (bottom). d Images of various linewidths of BHJ patterns. e Images of a large-area periodic table of elements pattern on a 4-inch wafer. The insert shows a zoomed-in view.

Full size image

It is noteworthy that all-polymer BHJs demonstrate compatibility with photocrosslinking compared to other types of blends, such as polymer-small molecular acceptor (SMA) or all-SMA BHJ configurations. This enhanced compatibility can be attributed to the higher degree of polymerization of both the donor and acceptor materials, which results in the formation of higher-quality patterns. Conversely, polymer-SMA blends tend to yield lower-quality patterns when subjected to direct photolithography. For example, PTB7-Th and ITIC BHJs form no patterns (Supplementary Fig. 16). Using direct photolithography, we fabricated BHJ patterns featuring a pixel array of 640 × 512 with a pixel pitch of 15 μm, demonstrating the compatibility of all-polymer BHJs with ROIC chip architectures (Fig. 3c). Rectangular patterns with fine shapes ranging in width from 1 to 5 μm, corresponding to the dimensions of the pre-designed photomasks (Fig. 3d). The 1 μm patterning resolution approaches the wavelength of visible light, meeting the requirements of most imaging applications. Furthermore, this photolithographic method is suitable for wafer-scale production, as evidenced by the fabrication of a large-area pattern of the periodic table on a 4-inch silicon wafer (Fig. 3e), suggesting its potential for high-volume production of CMOS-compatible neuromorphic imagers.

To investigate the film retention of BHJ films at different UV doses, we employed 5 wt% of photocrosslinker additives mixed with BHJs. At a low dose of 40 mJ cm−2, the film retention exceeds 60%, and this value saturates at ~90% when the dose is increased to 80 mJ cm−2 (Supplementary Fig. 17). This photolithographic method demonstrates a non-destructive impact on the morphology of the devices. As illustrated in Supplementary Fig. 18, atomic force microscopy (AFM) images of both the pristine and patterned films reveal a uniform morphology, with root-mean-square (RMS) roughness values of 0.58 and 0.73 nm, respectively.

To illustrate the generality of this patterning method in building organic neuromorphic devices, we compared the characteristics of pristine and patterned organic devices. The patterned neuromorphic devices (1 wt% PZ1 within BHJs) show performance comparable to their pristine counterparts. Under identical intensity stimuli (100 mW cm−2), the devices exhibit LTP EPSC values of 3.7 μA (pristine) and 3.8 μA (patterned), with delay times of 15.39 and 19.80 s, respectively (Supplementary Fig. 19a). STP EPSC values are 2.2 μA (pristine) and 3.3 μA (patterned), with delay times of 1.50 and 3.00 s, respectively (Supplementary Fig. 19b). Similarly, the patterned conventional devices (50 wt% PZ1 within BHJs) maintain equivalent performance, with photocurrents of 2.3 vs. 3 μA (Supplementary Fig. 19c, d). The patterned device shows a responsivity of 0.05 A W−1 and a detectivity of 2.85 × 1012 Jones at a bis-voltage of 2 V (Supplementary Fig. 20). This shows the potential of our patterning method in realizing neuromorphic imaging.

Design of a neuromorphic imager

To examine the imaging performance of BHJ films with different memory effects, we expand the neuromorphic device to an array-format FPA imager. Figure 4a shows the architecture of the neuromorphic imager. A pixel electrode is placed in the center, and a common ground gold (Au) electrode grid is placed around it. The layout of ROICs, the function and typical setting of pins and the schematic of the focal plane array (FPA) circuit architectures are provided in Supplementary Section 4. The details of performance experimental measurements are provided in Supplementary Section 5. Neuromorphic array-format FPA imagers with a spatial resolution of 640 × 512 pixels and a pixel pitch of 15 µm was fabricated. The photograph of the organic neuromorphic imager module (top) and chip (bottom) are presented in Fig. 1c.

Fig. 4: Photonic non-volatility of the neuromorphic imager.
figure 4

a Schematic illustrating the architecture of the organic neuromorphic imager and associated electric circuit of chip pixels. Standard imaging mode. Images of a car model (b) and desk tidy (c). d Images of two cars (blue and green), a red traffic cone model and a color card captured by the organic neuromorphic imager using red, green and blue optical filters (left), and the reconstructed full visible color images (right). e Standard mode. The histogram distribution of each pixel’s response in the organic neuromorphic imager under dark conditions, after 5 s of light exposure, and 5 s post-light removal. f Synapse mode. The histogram distribution of each pixel’s response in the organic neuromorphic imager under dark conditions, after 5 s of light exposure and at 5, 25, and 100 s post-light removal. g Imaging pixel numbers of neuromorphic imagers over the years.

Full size image

We initially developed an organic imager with rapid response capabilities and real-time imaging functionality, utilizing a BHJ composed of a PTB7-Th-to-PZ1 ratio of 1:1. As illustrated in Fig. 4b, c, the fabricated FPA imager can capture high-resolution images. Using optical filters, red, green and blue single-visible-color images are achieved, and a reconstructed full-color image is shown in Fig. 4d.

In standard mode, the dark current of each pixel in the organic neuromorphic imager is ~4.6 × 10−13 A (Supplementary Fig. 25). The characteristics of dark current observed in FPAs differ from those measured in single-element devices. This difference arises from fundamental disparities in electrode geometry and defect density, which are caused by different process conditions between FPAs and single-element devices. Supplementary Figs. 26 shows the r.m.s. noise distribution and noise mapping of the FPA imager, with an average value of 1 × 10−3 V. Upon illumination (455 nm, 50 mW cm−2), the imager exhibits a rapid response, as shown in Fig. 4e and Supplementary Fig. 27. The non-effective pixel mapping of the imager reveals a notably low rate of non-effective pixels, with only 15 dead pixels and 7 overheated pixels (Supplementary Fig. 28). The operability of the FPA imager is high at 99.99%. The photoresponse non-uniformity (PRNU) of the FPA imager is as low as 10%. The performance calculation process is detailed in Supplementary Section 5. Upon light removal, the imager’s response rapidly diminishes, returning to a histogram distribution akin to that observed under initial dark conditions, indicating the absence of memory effects in such devices.

In synapse mode, we can modulate the memory behavior of the device by adjusting the component ratios within the BHJs. For instance, devices incorporating 1 wt% PZ1 in the BHJs exhibit pronounced memory effects, as shown in Fig. 4f. By recording the histogram distributions of each pixel’s response under dark conditions, during light exposure, and after light removal, it is evident that the device exhibits significant non-volatility. Even 100 s after the removal of light, the response of each pixel remains distinct from the dark state. Following another series of 50 light pulses, the histogram distribution of the pixel responses returns to a state similar to that observed immediately after the light was removed, as illustrated in Supplementary Fig. 29. This suggests the capability of the device to re-extract image information. Notably, our neuromorphic imager features an array of 105 pixels, significantly larger than those in previous studies, as illustrated in Fig. 4g and Supplementary Table 2. This enhanced pixel density affords a significant advantage in the high-resolution extraction of imaging trajectories, thereby ensuring the spatial resolution and fidelity of the device. Compared to state-of-the-art neuromorphic devices, our device demonstrates competitive performance. Notably, in terms of imaging resolution, our imager exhibits a significant advantage over existing devices. This high resolution demonstrates the advanced capabilities and potential of our device in neuromorphic computing applications.

Dual-modal imaging of the neuromorphic imager

The neuromorphic imager is capable of dual-modal neuromorphic imaging in real-world scenes. The experimental setup is presented in Supplementary Fig. 23. In standard mode, BHJs layer with 50 wt% PZ1 is used as the photosensitive layer, capturing typical images. As illustrated in Fig. 5a and Supplementary Video 1 (left), the moving vehicle is captured by the neuromorphic imager, enabling the precise acquisition of detailed features of both the vehicle and the surrounding environment.

Fig. 5: Dual-modal imaging of the neuromorphic imager.
figure 5

a Illustration of imaging setup and images of car moving captured in standard mode at various time points. b Images of a car captured in synapse mode at various times after the light source has been removed. c Illustration of “FPA” pattern LED arrays pulsed in sequence (time duration of 0.5 s) and images of illuminated LED captured at different times. d Illustration of imaging setup using neuromorphic FPA imagers. A tungsten lamp is used as the light source and placed about 40 cm in front of the imagers. The insert is the photograph of the bump used in the study. e Images of the light filament captured at different times. f The lamp luminance changes of the ROI (blue region in Fig. 5e) over time.

Full size image

In synapse mode, BHJs layer with 1 wt% PZ1 serves as the photosensitive layer. The time-dependent characteristics of the device enable it to process sequential temporal information, thereby achieving in-pixel image memorization. As shown in Fig. 5b and Supplementary Video 1 (right), after removing all light sources, the acquired scene images can be stored on the sensor for over 18 min, demonstrating long-term in-pixel image memorization.

Neuromorphic imagers’ in-pixel image memorization enables operation at lower frame rates without missing critical brief targets (like electrical sparks from momentary industrial leaks). In contrast, conventional imagers require high frame rates for brief target capture, leading to increased energy consumption, redundant data, and complex algorithms. We employed light-emitting diodes (LEDs) to simulate the instantaneous spark generation in industrial production. As shown in Fig. 5c, the LEDs pulsed in sequence (power of 6 mW, duration of 500 ms), followed by the presentation of the ‘FPA’ image. Due to the in-pixel memorization capability of neuromorphic imagers, they can retain the complete spatiotemporal information of the LED light events (Fig. 5c and Supplementary Video 2). By selecting four target LEDs from the last frame and examining their luminance variations, we observe that the retained brightness decreases in the order of D41, D40, D26, D25, D13, and D12. This gradient of brightness directly corresponds to the sequence in which the light events occurred. Thus, this method allows us to infer the temporal sequence of the light events based on the intensity of the captured light (Supplementary Fig. 30). While the conventional imagers can only capture real-time LED light events. Further lowering the frame rate, neuromorphic imagers can significantly reduce storage data volumes.

For motion detection, the organic neuromorphic imager is utilized to capture a moving light source, specifically a light bulb in this study. The bulb is positioned in front of the imager (Fig. 5d). As the motion persists, the contrast of the filament image gradually diminishes following light stimulation. Subsequent frames exhibit higher contrast compared to the preceding ones. The superposition of frames acquired by the neuromorphic imager integrates spatiotemporal motion information through distinct contrast levels, delineating the contour of the entire motion trajectory. (Fig. 5e and Supplementary Video 3). Figure 5f shows the filament luminance variation within the ROI (dotted line in Fig. 5e), which exhibits time-dependent characteristics similar to the device’s EPSC delay. Hence, by integrating the contrast of the trajectory map with the device’s memory decay curve, motion parameters such as velocity can be precisely calculated, thereby substantially reducing abundant data transfer and conversion between imagers, computing, and memory units.

By integrating real-time imaging and motion perception functional units into a single ROIC through direct optical patterning methods, dual-modal imaging can be achieved, enabling the simultaneous acquisition and processing of static and dynamic visual information. This integration allows for efficient and comprehensive analysis of complex scenes, enhancing the overall performance and versatility of the imaging system. As a result, the combined capabilities provide a more robust and adaptive approach to capturing and interpreting visual data, making it particularly useful in applications that require real-time monitoring and motion tracking.

Discussion

In this study, we developed a CMOS-integrated neuromorphic imager capable of dual-modal imaging, which enables the processing of both static and dynamic information in real-world scenes. This neuromorphic device features adjustable memory, allowing for multi-level memory time that can be tuned from 101 to 104 ms. We also devised a non-destructive, CMOS-compatible lithography process suitable for wafer-scale production. We fabricated a CMOS-integrated organic neuromorphic imager with a resolution of 640 × 512 pixels. This imager can accurately extract light maps and motion trajectory maps with temporal information from single-frame static images, demonstrating its potential for advanced imaging applications. The in-sensor motion perception of real-world scenes using neuromorphic devices surpasses the limitations of previous single-point and small-scale array devices, which were restricted to rudimentary imaging or simulation of input image data.

We emphasize the capability to use a direct optical patterning method for seamlessly integrating fast-response devices with synaptic devices on a single chip. This integration enables the chip to perform both real-time imaging and synaptic-like imaging functionalities. Nevertheless, a major limitation in this study that warrant attention in future research. Although the feasibility of electrical erasure has been demonstrated at the single-pixel level, the two-terminal architecture employed in the chip poses challenges for implementing electrical erasing within large-scale systems. To address this limitation, future work will concentrate on designing reconfigurable two-terminal architectures or incorporating gate-control mechanisms into the chip design, thereby enabling more versatile functionality and system-level integration. Further innovations in chip and device architecture are expected to facilitate significant advancements in the practical industrial application of organic neuromorphic imagers.

Methods

Materials

All reagents were purchased from commercial vendors and used without purification. The PTB7-Th and PZ1 were purchased from Solarmer Materials Inc. (1E,4E)-1,5-bis(4-azido-2,3,5,6-tetrafluorophenyl)penta-1,4-dien-3-one was purchased from Dreamchem Inc. 2-Propanol (IPA, 70% in water), MPTS (95%), and chlorobenzene (98%) were purchased from Sigma-Aldrich and used without further purification.

Device fabrication and characterization of single-element photodetectors and neuromorphic devices

Single-element neuromorphic devices were fabricated on the interdigitated electrodes. Prior to spin coating, the silicon substrate was cleaned with IPA, treated with MPTS for 40 s, rinsed with IPA, and blow-dried. The photosensitive materials in the devices are PTB7-Th and PZ1 BHJ blends. For the standard devices (typical photodetector), the materials were mixed in a 1:1 ratio with a total concentration of 10 mg ml−1. For the neuromorphic devices, the materials were mixed in a 100:1 ratio with a total concentration of 10 mg ml−1. Chlorobenzene serves as the solvent, and the solutions are heated and stirred at 50 °C for 12 h until fully dissolved. The prepared solutions are then spin-coated onto the silicon substrates in a glove box at 2000 rpm for 30 s. After spin coating, the films are annealed at 80 °C for 5 min. Finally, the photodetector and neuromorphic device are completed by annealing at 100 °C for 10 min.

For patterned devices, a BHJs solution containing 5 wt% photocrosslinker additives was spin-coated at 2000 rpm for 30 s and exposed to a dose of 80 mJ cm−2 (365 nm, 2 mW cm−2, 40 s). Following this exposure, the films were treated with chlorobenzene via spin coating to remove unreacted photocrosslinker and unexposed polymers. This process concludes with a post-bake at 100 °C for 5 min. The devices feature 25 pairs of interdigitated evaporated gold fingers, with a width of 10 μm, a 10- μm gap, and a length of 1000 μm, covering an area of 0.5 mm2 (0.5 mm × 1 mm).

Photolithography

First, the substrates were cleaned with IPA in an ultrasonic bath for 10 min. Next, a 10 mg ml−1 BHJs solution was spin-coated at 2000 rpm for 30 s. The resulting films were exposed to a UV light source (365 nm, 2 mW cm−2, 40 s) through a quartz mask to crosslink the exposed regions of the BHJs layer. Following irradiation, the films were developed by removing the uncrosslinked regions using the same solvent used for casting the BHJ films.

Device fabrication and characterization of FPA imagers

The FPA imager is constructed using a 640 × 512 pixels ROIC with a 15-μm pixel pitch. Details regarding the layout, pin functions, typical configurations, and a schematic diagram of the FPA circuit architecture are provided in Supplementary section 4. The ROIC is achieved through photolithography and metal deposition. The BHJs layer is deposited onto the ROIC in ambient condition using a spin-coating process similar to that employed for single-element devices.

Characterization techniques for materials

UV–visible absorption spectra of BHJ films were measured with an Agilent Cary 5000 UV-Vis-NIR spectrophotometer in the transmission mode. Atomic force microscopy (AFM) images were taken in the ScanAsyst mode on Oxford Cypher 5. Current versus voltage curves were measured using a source meter (Keithley 2602B). Illumination was provided by a 455 nm LED (M455D3, ThorLabs) with a collimator lens (SM2F32-B, ThorLabs) controlled by an LED driver (LEDD1B, ThorLabs). Light intensity was verified with a Thorlabs SM1PD1B Si p–i–n diode. Water contact angles were measured on a Dataphysics OCA25 instrument.