Extended Data Fig. 1: Illustration of training pipeline of Ark⁺.

Ark⁺ is built on a teacher-student framework augmented with multi-task heads, each corresponding to a specific task, and employs cyclic pretraining to iteratively accrue and reuse knowledge. At each iteration, the student model sequentially scans datasets (tasks) one by one for one epoch, learning from expert annotations through the task-specific head. The knowledge accrued by the student is accumulated into the teacher via exponential moving averages (EMA), enabling the teacher to guide the student in subsequent tasks. To reinforce the feedback loop between the student and teacher, after their encoders, a projector is introduced to map the representations to the same feature space via the consistency loss. The projected representation also serves as the embedding for linear-probing in our evaluation. After pretraining, the accumulated knowledge in the teacher can be reused and transferred to target tasks. Differing from the previous design in Ark¹⁴, Ark⁺ feeds the teacher with the resized original image instead of random cropping. This update in data augmentation ensures the teacher provides a consistent and steady supervisory signal for computing the consistency loss, thereby accelerating training and enhancing performance. Images adapted with permission from ref. ¹⁹, IEEE.