Fig. 2: Benchmarking NiCo annotation and interaction inference.

a Benchmarking of NiCo annotations with uniPort, Tangram, TACCO, and cell2location for published datasets (mouse intestine and primary motor cortex MERFISH data, and mouse embryo seqFISH data; see text for details) with available ground truths annotations (authors’ annotation). Consistency of annotations between two methods was evaluated by Jaccard similarity (JAC) and adjusted R and index (ARI). b Memory requirement (left) and run time (right) for the cell type annotation and niche prediction task on intestinal MERFISH data (7,417 cells, top) and liver MERSCOPE data (391,679 cells, bottom). SpaGCN was aborted on the liver datasets due to exhausted memory. Runtime is shown on a log₂ scale. c, d Simulation experiments for testing NiCo’s interaction module. The locations of cells were simulated for six cell types in two dimensions (2D) using Lennard-Jones (LJ) pairwise interaction potentials (first panel). Simulated cell type locations were used as input to NiCo’s interaction module (second panel). The confusion matrix (third panel) highlights the predictive capacity of the neighborhood composition. Classifier coefficients (fourth panel) are ordered by magnitude, reflecting co-localization preference, or interaction strength. The confidence score is indicated in parentheses. A more uniform (c) and a more biased (d) scenario were simulated and assessed based on direct neighbors. See Methods for explanations of the parameters. e Same as (d) but using a larger neighborhood radius (R = 5). (c-e) error bars on β coefficients indicate standard deviation derived from five-fold cross-validation (Methods). f Niche cell type interactions inferred with MISTy for direct neighborhoods (left: juxtacrine view, right: paraview radius 5) on the more complex simulation scenario (d). The two-dimensional map indicates the importance of neighboring cell types (Predictor) for predicting central cell types (Target). g, h Comparison of cell type interactions predicted by NiCo with cell type enrichment in topological tissue domains predicted by tissue ___domain detection methods (CellCharter, SpaGCN, Stagate, Banksy, SpatialPCA, Seurat) on (g) Allen brain MERFISH atlas data and (h) STARmap visual cortex data. The Pearson correlation between predicted cell type enrichment Z-score and cell type enrichment in best matching ground truths (GT) tissue domains is compared (Methods). For NiCo, the cell type with the highest correlation of its regression coefficients to the Z-score was selected. Source data are provided as Source Data file (Data Availability).