Extended Data Fig. 6: MNP signal retention, pattern decodability, and information recognition.

(a) ‘needlenet_bit_error_rates’ code for automatic image processing system is programmed to analyze signal retention for 96-bit MNPs. It quantifyies the number of NIR bits that are preserved and detected for 96-bit MNPs and output the number as ‘signal retention %’. Raw fluorescence RGB image is converted to gray-scale and then to BnW, of which a minimum area rectangle tool rotates, crops, and resizes to a target size for an efficient recognition. Each bit is then recognized as either ON or OFF bits to find the percentage of ON-bits out of the 96 bits that were originally transferred on day 0. It is output as bit error rate, which is convertible to the signal retention %. (b) ‘needlenet_error_correction’ code is programmed to decode patterned MNPs and translate the NIR images of patterned MNPs back to the medical information that was originally encoded. Raw RGB image is converted to greyscale using DL-based binarization networks and cropped and rotated using the minimum area rectangle function. Each bit is recognized as either ON or OFF bit using DL-based recognition networks. Error bits are corrected by the RM ECC, and the corrected pattern is translated to information data. If this retrieved information accurately matches the original information data that was encoded on the MNP, then the pattern is processed as ‘successfully decoded’. (c) Initial challenging cases that were used as a guidance to improve simulations for training a robust image recognition network. Main causes of initial image recognition failure are rectification errors due to severe boundary noise, image distortion, and too large or too small global histogram threshold. Simulated spatial variations (e.g., under- and over- exposure, high background noise, high brightness variances) improved the image recognition network. (d) Four different 10×10 patterns were encoded on MNPs for the signal retention and information preservation evaluations in swine. Random medical information was assigned to each pattern.