Fig. 1: Our proposed framework of fusion based PD classifier using deep embeddings from WavLM and ImageBind. | npj Parkinson's Disease

Fig. 1: Our proposed framework of fusion based PD classifier using deep embeddings from WavLM and ImageBind.

From: A novel fusion architecture for detecting Parkinson’s Disease using semi-supervised speech embeddings

Fig. 1

First, the speech is separated from video datasets. Then the segment of the audio file where the participants utter the pangram is separated. Vector embeddings from the last layers of WavLM and ImagBind are extracted for the speech data. Then WavLM feautures are projected into the space of ImagBind features set. Finally the projected features are fused and passed through a classification layer that can determine the participant as PD or control. Note that the image of the person is AI generated.

Back to article page