Abstract
With the development of brain-computer interface industry, large amounts of related applications have entered people’s vision. BCI applications based on steady-state visual evoked potentials (SSVEP) are widely used because they do not require pre-training and have high information transmission rates. However, in the actual use of SSVEP stimulus paradigm, the subjects will produce visual fatigue with the use, and fatigue will affect the transmission efficiency. In this experiment, an experimental environment consisting of two paradigm stimulus frequencies (7.5 Hz, 15 Hz), three resolutions (800 × 600, 1280 × 720, 1920 × 1080) and three refresh rates (120 Hz, 240 Hz, 360 Hz) is set up. The Likert scale is used to collect subjective fatigue and preference scores, and the EEG acquisition system and eye tracker are used to collect objective data. Using the proposed information entropy-CRITIC algorithm to combine subjective and objective indicators, a fatigue assessment system (display screen fitness-DSF) is innovated to score different experimental environments. The higher the DSF score, the better the visual experience. The results show that when using the 7.5 Hz SSVEP paradigm, the combination of 360 Hz and 1920 × 1080 can bring the best visual experience. When using the 15 Hz SSVEP paradigm, the combination of 240 Hz and 1280 × 720 is the best. DSF provides powerful help for hardware and software selection guidance and vision protection when using SSVEP-based BCI applications.
Similar content being viewed by others
Introduction
In recent years, brain-computer interface (BCI) has developed rapidly. Among them, steady-state visual evoked potential (SSVEP) is widely used because of its advantages of no pre-training and high information transmission rate (ITR)1,2,3. SSVEP relies on the visual stimulus paradigm to make the subjects produce the corresponding electroencephalogram (EEG) response, and then carry out specific analysis. With the large-scale application of the SSVEP paradigm, the visual fatigue it brings to the subjects has attracted more and more attention. The fatigue caused by long-term use of SSVEP will seriously reduce the ITR and affect the use effect4. Some studies have shown that compared with the low-frequency (5 Hz) stimulus paradigm, the higher frequency (12 Hz) stimulus paradigm has a higher signal-to-noise ratio (SNR)5, and one speller that uses high frequency (30 Hz) can be used for longer time (about 12 min) although it still has a slight decrease in the performance6,7. At the same time, studies have shown that the ITR of the BCI is closely related to the parameters of the electronic screen8. Different screen parameters, such as resolution, refresh rate, brightness, etc., also bring uncertainty to the application effect of BCI based on SSVEP in which the uncertainty is mainly reflected in the comfort of the eyes9. If the screen refresh rate is too low, the stickiness and jam of the computer will reduce the user experience. If the resolution is too low, the blurring of the image quality is more likely to cause visual fatigue. Therefore, understanding the subjective and objective experience of users under different combinations of paradigm stimulus frequency, screen resolution and screen refresh rate has a positive effect on reducing visual fatigue, improving the application experience of BCI, and designing SSVEP paradigm which is more suitable for human physiological habits.
The SSVEP paradigm stimulus frequency is the flicker frequency of the paradigm when inducing the EEG response of the subjects. In the past, fatigue experiments related to SSVEP used low-frequency stimulus (7.5 Hz)9,10. In recent years, more and more SSVEP-based applications, such as brain-controlled spelling systems, have used a larger range of paradigm stimulus frequencies (8–15.8 Hz)11. The effect of high frequency on fatigue is also worth exploring.
There are three main types of screen resolution, resolution (digital image resolution), ppi (digital printing resolution/screen resolution) and dpi (printer resolution)12. This study only focuses on digital image resolution. Resolution (digital image resolution), represents the number of pixels in this photo (not the area), for instance, the resolution of 1920 × 1080 indicates a total of 2,073,600 pixels. Similarly, the higher the resolution, the clearer the picture quality, the better the visual experience13, and it is increasingly becoming an important indicator to measure the performance of electronic equipment. Researchers found that in EEG experiments with different sizes and resolutions of photos, it is not the higher the screen resolution, the better the recognition rate14. Tao Ni et al. confirm that factors such as apparent brightness, contrast, and resolution heavily affect the user’s understanding of information on a display15, thus the upgrade of resolution can bring users a better visual experience.
The screen refresh rate specifically refers to the number of times the display device updates the screen content per second. This number is usually expressed in Hertz (Hz). The higher refresh rate can bring smoother animation and faster response time, so it becomes an important indicator in modern electronic devices. PW Green found that a higher refresh rate can give users a better visual experience16. A team found that 200 Hz produced surprisingly good results compared to 60 Hz and 120 Hz, and the user questionnaire shows that the system with the fastest refresh rate is the least fatigued17. Scholars pointed out that in the field of brain-computer interface, a higher refresh rate can bring a better EEG response, which also means a better use effect8.
In our study, we simulated the combination mode of two paradigm stimulus frequencies (7.5 Hz, 15 Hz), three screen resolutions (800 × 600, 1280 × 720, 1920 × 1080) and three screen refresh rates (120 Hz, 240 Hz, 360 Hz). The screen brightness contrast was adjusted at 287.6 cd/m2 (50% mode). The experimental environment is a darkroom to ensure that there is no influence of natural light. The purpose of this study is to understand how the combination of paradigm stimulus frequency, screen resolution, and screen refresh rate affects the visual perception (visual fatigue) of SSVEP-based BCI applications.
Materials and methods
Experimental chamber
This experiment is carried out in a closed custom darkroom (2.5 m × 2.5 m × 2.5 m) built with an opaque black cloth, and there is no external light source interference for control variables. The computer screen is 55 cm away from the subject, and the perspective of the paradigm stimulus is 4 ° up and down in accordance with the recommended visual angle parameter of previous studies10,18,19. The schematic diagram of the setting of the experimental environment is shown in Fig. 1.
Design of experiment.
Subjects
Twenty-two subjects (fifteen males and seven females) participated in the experiment. All subjects are aged between 25 and 31 years old. All subjects have no eye disease, and the visual acuity or corrected visual acuity is above 1.0. Subjects are asked not to drink alcohol within 24 h before the start of the experiment, and to ensure adequate sleep the night before the experiment. All subjects are given informed consent according to the Helsinki Declaration and approved by the Ethics Committee of Xi’an Jiaotong University, China (No. 2023 − 1552).
EEG recordings
Because this experiment is aimed at visual fatigue, the electrodes are arranged in the occipital region20, left and right temporal lobe21 and parietal lobe22 according to the relevant literature. As Fig. 2 shows, we arrange the electrodes in these areas in turn (O1, O2, Oz, PO3, PO4, POz, T7, TP7, T8, TP8, Cz, FC1, FC2, F3, Fz and F4) according to the international 10–20 systems. The grounding electrode FPz is located at the forehead, and the reference electrode A1 is located at the left earlobe23. The experiment uses an EEG acquisition system (g.tec, Schiedlberg, Austria) with a sampling rate of 1200 Hz and an eye tracker (Tobii, Stockholm, Sweden) with a sampling rate of 120 Hz. In order to ensure the quality and stability of EEG signals, our EEG system uses wet electrodes. In addition, a band-pass filter with a bandwidth of 2–100 Hz and a notch filter with a bandwidth of 48–52 Hz is added to the data preprocessing to remove artifacts and power line interference.
Electrode layout of 16-channel EEG.
Stimulus designs
In this experiment, a display with adjustable screen resolution and screen refresh rate is arranged. The screen brightness is calibrated using a HOPOO spectral brightness analyzer (OHSP-350 L, Hangzhou HOPOO Optical Color Technology Co., Ltd., Hangzhou, China). During the experiment, the monitor is 55 cm away from the subject, and the eyes of the subject look directly at the center of the screen. The viewing angle of the paradigm flicker is 4 °, and the diameter is 148 pixels, which is set according to the relevant literature18,19. The eye tracker is pasted at the bottom of the screen. The stimulus paradigm is proposed by MATLAB 2016b and psychophysical toolbox24. This experiment adopts the international SSVEP stimulus paradigm-light flicker paradigm, which is convenient for others to reproduce the experiment. The stimulus pattern of this paradigm is a brightness contrast from black to white. The mode is modulated by a sine wave, with stimulus frequencies of 7.5 Hz and 15 Hz, respectively. As shown in Table 1, 18 conditions combine two paradigm stimulus frequencies, three screen resolutions and three screen refresh rates.
Experimental procedure
The experiment was conducted in the order of Latin squares (Appendix – Table 1). There were a total of 18 experimental tasks and 22 test subjects. Each participant needed to experience all 18 tasks. Each group of experiments included 23 trials. The prompt time before each paradigm stimulus was 0.5 s, the formal stimulus time was 5 s, and the rest time after the stimulus was 0.5 s, which meant that the complete test time was 6 s.
As shown in Fig. 3, there were a total of 18 tasks. The following was a specific process example of a task. First, the visual perception of the subjects was recorded with the Likert scale (Appendix - Fig. 1)25 before each round of the experiment. Second, adjusted the paradigm stimulus frequency, screen resolution and screen refresh rate. Third, in the 23 trials of each group, the first three trials were designed as a white static ‘x’ marker. The experimental content was that the subject stared at the white static ‘x’ marker on the dark background26. The data of the first three trial records are helpful for the normalization of the subsequent data. A white marker that helps to fix the line of sight appears at the center of the paradigm18. Multiple sensors recorded all data synchronously. Fourth, at the end of the experiment, the Likert scale was used to ask the subjective feelings and preferences of subjects for the whole task.
The process of experiment.
Every time one task was completed, the subjects could rest well until the next task began, and the rest time depended on themselves. The total duration of the experiment for each participant was approximately 90–120 min, which was mainly depending on the time of rest.
Visual fatigue accumulated with the experimental process. Therefore, referring to the existing literature, the formal trials (4–23 trials) of each task were divided into four levels on average27. Level 1 (4–8 trials) represented the state with the lowest degree of fatigue at the beginning, and level 4 (19–23 trials) represented the state with the highest degree of fatigue at the end10,28.
Experimental indicators
The setting of experimental indicators is considered from the perspective of the system. The settings of visual fatigue of subjects (VFS, summation of Likert scale scores) and subjective preference (SP, directly obtained by the Likert scale) are to reflect the subjective quantification of visual fatigue9. EEG activity data (θ + α band energy (EB), EEG response amplitude (EA), response signal-to-noise ratio (ES), all obtained from EEG data processing) and pupil diameter (PD, direct recording of eye tracker) data are utilized to reflect the objective quantification of participants’ visual fatigue from the perspective of the optic nerve and cerebral cortex29,30. In the subsequent analysis, there will be a unified combination of subjective and objective indicators to quantify the overall visual fatigue of the participants in the entire experimental environment.
Likert scale of visual fatigue of subjects (VFS) and subjective preference (SP)
The Likert scale is a subjective question set about the degree of fatigue of the subjects. The idea of our scale comes from the article of Professor Xie25. On this basis, we have made some modifications. The first six questions are composed of 1 to 10, 1 means very disagree, 10 means very agree. By adding the scores of several questions, it is intuitive to see the visual fatigue feeling of the subjects. The seventh question is composed of 1 to 5, 1 means very dislike, 5 means very like. The first six questions in the scale are used to reflect visual fatigue, and the seventh question is used to reflect preference.
EEG spectral powers of θ + α frequency band (EB), EEG amplitude (EA) and EEG signal-to-noise ratio (ES)
In this paper, the canonical correlation analysis (CCA) algorithm is used to analyze EEG data (Appendix-Procedure 1). CCA uses channel covariance information with a high signal-to-noise ratio to process data31,32. Because of its high efficiency, robustness and simple implementation, CCA is widely used to explore the relationship between EEG and visual fatigue33,34. After filtering and screening, the sum of the energy of the θ band (4–7 Hz) and the α band (8–13 Hz) is used as a band index35, called EEG band energy (EB). At the same time, according to the stimulus frequency of the paradigm, the CCA response amplitude and signal-to-noise ratio at the stimulus frequency are also used as quantitative indicators, called EEG amplitude (EA) and EEG SNR (ES), respectively. All data will be normalized for subsequent analysis. The specific normalization procedure is as follows: in 23 trials, the first three stimuli trials are not flashes of light, but a fixed white marker. The first three stimuli here are used as reference values, and then the values of the subsequent 20 real flashes of light are divided by the reference values9,10,28.
Pupil diameter (PD)
The change in pupil diameter can be used to reflect the degree of visual fatigue, which is closely related to visual diseases36. Generally speaking, the average pupil diameter is 4 mm (generally 2–5 mm, maximum up to 8 mm, minimum up to 1.5 mm), and the pupil size difference of both eyes is less than 0.25 mm. The pupil diameter (PD) index directly reflects the eye condition of the subjects and can be used as the corresponding objective quantitative index9.
Information entropy-CRITIC weight method
The traditional CRITIC weight method (Appendix-Procedure 2) includes data volatility and correlation between data and is widely used in the fusion of evaluation indexes in the assessment system37,38. However, the traditional algorithm lacks data size information. This manuscript combines the amount of information of the data with the CRITIC algorithm, which is called the information entropy-CRITIC weight method.
The specific algorithm is as follows: Firstly, the average values of the six parameters (VFS, SP, EB, EA, ES, PD) of the 22 subjects are calculated, and the singular values are removed using the 3-σ principle. Secondly, the traditional CRITIC weight method is used to calculate the weights of the six parameters (see Appendix). Thirdly, the information entropy algorithm is used to calculate the amount of information contained in each parameter. Fourthly, the information entropy is multiplied by the CRITIC weight. Fifthly, the sum is calculated and the final weight value corresponding to each parameter is calculated proportionally, which is called the information entropy-CRITIC weight.
Statistical analysis
Since this experiment is based on six indicators (VFS, SP, EB, EA, ES, PD) to explore the quantitative assessment of visual fatigue under the combination of three different environmental parameters (refresh rate, screen resolution, and paradigm stimulation frequency), the first step is to divide the indicators into subjective quantification (VFS, SP) and objective quantification (EB, EA, ES, PD). The second step, for every single indicator, One-way, two-way and three-way repeated-measures analysis of variance (ANOVA) with significance < 0.05 was used to analyze the above six indicators of 18 conditions at level 1 and level 4. In the third step, Post-hoc Bonferroni adjustment was applied for multiple comparisons when there was a significant main effect (p < 0.05). In the fourth step, the information entropy-CRITIC weight method is proposed to fuse the subjective and objective indicators, and the data is binned and the consistency test is used to verify the effect of the new algorithm. In the fifth step, the new algorithm is used to fuse six indicators. A new indicator named Display Screen Fitness is proposed, and then the combination of three different environmental parameters is quantitatively evaluated for visual fatigue, and the advantages and disadvantages are judged. All the statistical procedures were processed using SPSS 22.0 (IBM, Armonk, NY, USA).
Results
Subjective analysis
Three-way repeated-measures ANOVA (F (3.757, 78.904) = 0.241, P = 0.905) shows that there is no three-factor interaction in the effects of stimulus frequency, resolution and refresh rate on VFS. Two-way repeated-measures ANOVA shows that the interaction of stimulus frequency and resolution (F (1.792, 37.637) = 3.771, P = 0.036*) has a statistically significant effect on VFS. There is no two-factor interaction between stimulus frequency and refresh rate (F (1.839, 38.624) = 0.266, P = 0.750), resolution and refresh rate (F (3.002, 63.049) = 0.558, P = 0.645) on VFS. One-way repeated-measures ANOVA shows that the effect of stimulus frequency (F (1, 21) = 21.027, P < 0.001***) on VFS is statistically significant. The effect of resolution (F (1.688, 35.441) = 0.391, P = 0.645) and refresh rate (F (1.755, 36.861) = 1.092, P = 0.339) on VFS is not statistically significant. It can be seen that different stimulus frequencies cause significant differences in the subjective fatigue degree of the subjects. From Fig. 4, it can also be seen that the visual fatigue of subjects caused by the 15 Hz stimulus frequency is significantly greater than the 7.5 Hz stimulus frequency.
Continue in-depth analysis of resolution and refresh rate. From the Bonferroni post-hoc analysis (Appendix - Table 2) of 800 × 600 resolution (F (3.172, 66.605) = 12.554, P < 0.001***), it can be seen that at low resolution, low stimulus frequency can bring better visual experience to the subjects. At the same time, 120 Hz and 360 Hz refresh rates bring a better visual experience to the subjects than 240 Hz refresh rate.
Comparison of the mean values and SD of the Likert Scale (VFS) for 18 conditions over 22 subjects after each run of the experiment. Statistics were assessed by one-way repeated ANOVA. The difference between Level 1 and Level 4 represents the fatigue value generated by the subjects during the experiment.
From the Bonferroni post-hoc analysis (Appendix - Table 3) of 1280 × 720 resolution (F (2.225, 46.734) = 3.230, P = 0.044*), at medium resolution, the average subjective anti-fatigue performance at low stimulation frequency is better. At the same time, when using BCI devices with low stimulus frequency, the average performance of the three conditions with the highest refresh rate is better. On the contrary, the average performance of the three conditions with the lowest refresh rate is better at high stimulus frequency.
The effect of parameter adjustment on VFS is not statistically significant at 1920 × 1080 resolution (F (2.663, 55.931) = 1.251, P = 0.299).
According to the Bonferroni post-hoc analysis (Appendix - Table 4) of 120 Hz refresh rate (F (3.261, 68.486) = 4.302, P = 0.006**), it can be seen that the average visual experience of the three conditions of low stimulus frequency is better at low refresh rate. At the same time, when using BCI devices at low stimulus frequencies, the three conditions with the lowest resolution perform best on average. At high stimulation frequencies, the three conditions with the highest resolution have the best average anti-fatigue effect.
From the Bonferroni post-hoc analysis (Appendix - Tables 5 and 6) of 240 Hz refresh rate (F (3.520, 73.918) = 4.444, P = 0.004**) and 360 Hz refresh rate (F (3.589, 75.362) = 5.915, P = 0.001**), the influence of different parameters on VFS at medium and high refresh rate is consistent with that at low refresh rate.
Three-way repeated-measures ANOVA (F (3.168, 66.537) = 0.938, P = 0.431) shows that there is no three-factor interaction in the effects of stimulus frequency, resolution and refresh rate on SP. Two-way repeated-measures ANOVA shows that the interaction of stimulus frequency and resolution (F (1.718, 36.077) = 6.776, P = 0.005**) has a statistically significant effect on SP. There is no two-factor interaction between stimulus frequency and refresh rate (F (1.995, 41.898) = 0.003, P = 0.997), resolution and refresh rate (F (2.508, 52.662) = 0.864, P = 0.449) on SP. One-way repeated-measures ANOVA shows that the effect of stimulus frequency (F (1, 21) = 16.196, P = 0.001**) on SP was statistically significant. The effect of resolution (F (1.995, 41.904) = 0.749, P = 0.479) and refresh rate (F (1.596, 33.517) = 2.174, P = 0.138) on SP was not statistically significant. It can be seen that the difference in stimulus frequency will cause significant differences in the subjective preferences of the subjects. It can also be seen from Fig. 5 that the 7.5 Hz stimulus frequency is more liked by the participants than the 15 Hz stimulus frequency.
Continue in-depth analysis of resolution and refresh rate, according to the Bonferroni post-hoc analysis (Appendix - Table 7) of 800 × 600 resolution (F (3.199, 67.186) = 8.242, P < 0.001***), it can be seen that under low resolution, low stimulus frequency is more favored by participants, and at the same stimulus frequency, 360 Hz refresh rate is always more favored by subjects.
The effects of parameters adjustment on SP are not statistically significant at 1280 × 720 resolution (F (3.792, 79.634) = 1.282, P = 0.285) and 1920 × 1080 resolution (F (3.405, 71.508) = 0.728, P = 0.555).
Comparison of the mean values and SD of the Likert Scale (SP) for 18 conditions over 22 subjects after each run of the experiment. Statistics were assessed by one-way repeated ANOVA.
From the Bonferroni post-hoc analysis (Appendix - Table 8) of 120 Hz refresh rate (F (3.717, 78.067) = 3.949, P = 0.007**), it can be seen that at low refresh rate, the average performance of the three conditions of low resolution is better at low stimulation frequency, and vice versa. The average anti-fatigue effect of the three conditions of high resolution at high stimulation frequency is more popular with the participants.
The effect of parameter adjustment on SP is not statistically significant at 240 Hz refresh rate (F (4.220, 88.618) = 2.076, P = 0.087).
According to the Bonferroni post-hoc analysis (Appendix - Table 9) of 360 Hz refresh rate (F (4.384, 92.067) = 4.118, P = 0.003**), the influence of different parameters on SP at high refresh rate is consistent with that at low refresh rate.
Objective analysis
As shown in Fig. 6, there are no statistical significances of EB on three-way, Two-way and one-way repeated-measures ANOVA, respectively (Appendix – Table 10).
Continue in-depth analysis of stimulus frequency, resolution and refresh rate, the effects of parameters adjustment on EB are not statistically significant at 7.5 Hz paradigm stimulus frequency (F (4.701, 84.623) = 0.894, P = 0.484), 15 Hz paradigm stimulus frequency (F (4.676, 70.135) = 0.915, P = 0.471), 800 × 600 resolution (F (4.159, 74.864) = 0.298, P = 0.885), 1280 × 720 resolution (F (3.657, 65.820) = 0.966, P = 0.427), 1920 × 1080 resolution (F (3.040, 54.723) = 1.427, P = 0.244), 120 Hz refresh rate (F (3.864, 77.283) = 0.224, P = 0.920), 240 Hz refresh rate (F (4.045, 68.761) = 1.161, P = 0.336) and 360 Hz refresh rate (F (3.419, 58.123) = 0.518, P = 0.695), respectively.
Comparison of the mean values and SD of EEG θ + α band (EB) for 18 conditions over 22 subjects after each run of the experiment. Statistics were assessed by one-way repeated ANOVA. The difference between Level 1 and Level 4 represents the fatigue value generated by the subjects during the experiment.
As shown in Fig. 7, there are no statistical significances of EA on three-way, Two-way and one-way repeated-measures ANOVA, respectively (Appendix – Table 11).
Comparison of the mean values and SD of EEG response amplitude (EA) for 18 conditions over 22 subjects after each run of the experiment. Statistics were assessed by one-way repeated ANOVA. The difference between Level 1 and Level 4 represents the fatigue value generated by the subjects during the experiment.
Continue in-depth analysis of stimulus frequency, resolution and refresh rate, the effects of parameters adjustment on EA are not statistically significant at 7.5 Hz paradigm stimulus frequency (F (4.952, 94.089) = 0.851, P = 0.516), 15 Hz paradigm stimulus frequency (F (4.594, 82.690) = 0.563, P = 0.714), 800 × 600 resolution (F (3.334, 63.341) = 1.515, P = 0.216), 1280 × 720 resolution (F (3.482, 62.680) = 0.108, P = 0.969), 1920 × 1080 resolution (F (2.641, 50.176) = 0.691, P = 0.544), 120 Hz refresh rate (F (2.791, 53.038) = 0.099, P = 0.953), 240 Hz refresh rate (F (2.386, 42.946) = 0.482, P = 0.654) and 360 Hz refresh rate (F (3.340, 63.459) = 1.541, P = 0.209), respectively.
The above analysis is aimed at the difference between Level 1 and Level 4 because the difference can indicate fatigue9,10, it is found that the difference is not statistically significant, but it is interesting to analyze the data from Level 1 alone (F (1, 18) = 25.951, P < 0.001***), the difference of EA caused by different stimulus frequencies is significant. Similarly, from Level 4 data analysis (F (1, 16) = 25.771, P < 0.001***), the difference in EA caused by different stimulus frequencies is also significant. In summary, the overall induction intensity of the subjects at low stimulus frequency is significantly lower than that at high stimulus frequency.
As shown in Fig. 8, there are no statistical significances of ES on three-way, Two-way and one-way repeated-measures ANOVA, respectively (Appendix – Table 12).
Comparison of the mean values and SD of EEG signal-to-noise ratio (ES) for 18 conditions over 22 subjects after each run of the experiment. Statistics were assessed by one-way repeated ANOVA. The difference between Level 1 and Level 4 represents the fatigue value generated by the subjects during the experiment.
Continue in-depth analysis of stimulus frequency, resolution and refresh rate, the effects of parameters adjustment on ES are not statistically significant at 7.5 Hz paradigm stimulus frequency (F (5.054, 101.081) = 1.145, P = 0.342), 15 Hz paradigm stimulus frequency (F (5.579, 106.009) = 0.698, P = 0.642), 800 × 600 resolution (F (3.099, 65.076) = 1.084, P = 0.363), 1280 × 720 resolution (F (3.963, 75.302) = 0.407, P = 0.801), 1920 × 1080 resolution (F (3.493, 69.860) = 0.494, P = 0.715), 120 Hz refresh rate (F (3.267, 68.605) = 0.278, P = 0.857), 240 Hz refresh rate (F (3.130, 56.342) = 0.951, P = 0.425) and 360 Hz refresh rate (F (3.292, 65.842) = 0.973, P = 0.417), respectively.
Similar to the analysis of EA, the data from Level 1 alone (F (1, 18) = 15.786, P = 0.001**) and Level 4 alone (F (1, 18) = 25.917, P < 0.001***) indicates that the differences in ES caused by different stimulus frequencies are also significant. In summary, the signal-to-noise ratio of the overall evoked signal at low stimulus frequency is significantly lower than that at high stimulus frequency.
As shown in Fig. 9, three-way repeated-measures ANOVA (F (2.915, 46.644) = 0.673, P = 0.569) shows that there is no three-factor interaction in the effects of stimulus frequency, resolution and refresh rate on pupil diameter (PD). Two-way repeated-measures ANOVA shows that there are no two-factor interactions between stimulus frequency and resolution (F (1.684, 26.941) = 0.272, P = 0.727) and stimulus frequency and refresh rate (F (2.561, 40.980) = 0.969, P = 0.406). While the interaction of resolution and refresh rate (F (1.894, 30.305) = 3.998, P = 0.031*) has a statistically significant effect on PD. One-way repeated-measures ANOVA shows that the effects of stimulus frequency (F (1, 16) = 1.263, P = 0.278), resolution (F (1.387, 22.195) = 1.051, P = 0.342) and refresh rate (F (1.889, 30.219) = 0.890, P = 0.416) on PD are not statistically significant.
Comparison of the mean values and SD of Pupil diameter (PD) for 18 conditions over 22 subjects after each run of the experiment. Statistics were assessed by one-way repeated ANOVA. The difference between Level 1 and Level 4 represents the fatigue value generated by the subjects during the experiment.
Continue in-depth analysis of stimulus frequency, resolution and refresh rate, the effects of parameters adjustment on PD are not statistically significant at 7.5 Hz paradigm stimulus frequency (F (4.450, 84.552) = 1.043, P = 0.394), 15 Hz paradigm stimulus frequency (F (3.842, 65.311) = 0.955, P = 0.436), 800 × 600 resolution (F (3.898, 74.056) = 2.292, P = 0.069), 1280 × 720 resolution (F (2.355, 42.394) = 0.627, P = 0.564), 1920 × 1080 resolution (F (3.642, 69.195) = 0.120, P = 0.967), 120 Hz refresh rate (F (2.984, 56.704) = 1.077, P = 0.366), 240 Hz refresh rate (F (3.375, 64.127) = 1.043, P = 0.385) and 360 Hz refresh rate (F (3.431, 61.762) = 0.562, P = 0.665), respectively.
Consistency check
To verify the difference between the evaluation results of the CRITIC weight method and the information entropy-CRITIC weight method, the subjective evaluation data (VFS, SP), the objective evaluation data (EB, EA, ES, PD) and the overall subjective and objective data (all six indicators) are fused by CRITIC and information entropy-CRITIC, respectively, and the evaluation results of 18 sets of conditions are obtained. Then, the sub-box processing is carried out, and then the Kappa consistency test is carried out by SPSS. It can be seen from Fig. 10 that the information entropy-CRITIC algorithm improves the consistency with the subjective (+ 45.90%) and objective (+ 202.34%) evaluation results compared with the CRITIC algorithm, which makes the evaluation results more convincing.
Consistency check.
Anti-fatigue assessment system based on Information entropy-CRITIC weight method
Weight of each index
According to the algorithm mentioned in Sect. 2.7, firstly, the information entropy of VFS, SP, EB, EA, ES and PD is calculated in order, which is 2.3300, 0.3095, 3.0169, 3.9477, 3.4194 and 1.6337, respectively. Then the CRITIC algorithm is used to obtain corresponding weights: 0.1774, 0.1547, 0.1821, 0.1303, 0.1498 and 0.2057, respectively. After the corresponding multiplication, the final weight value of each index is obtained by proportional distribution: 0.1741, 0.0202, 0.2315, 0.2168, 0.2159, 0.1416.
Display screen fitness
Figure 11 shows the scoring results of the anti-fatigue evaluation system under the 7.5 Hz paradigm stimulus frequency. It can be seen from the figure that during the low refresh rate, the low resolution score is the highest, the anti-fatigue effect is the best, and the anti-fatigue effect deteriorates with the increase in resolution. On the contrary, during the high refresh rate, the anti-fatigue effect of high resolution is the best, and the anti-fatigue effect becomes better with the increase in resolution. During the medium refresh rate, the anti-fatigue performance of low resolution is the best, followed by high resolution.
The anti-fatigue scoring of the display under different resolutions and refresh rates at 7.5 Hz paradigm stimulus frequency based on the information-CRITIC weighting method.
Figure 12 is the scoring result of the anti-fatigue assessment system under the 15 Hz paradigm stimulus frequency. It can be seen from the figure that when the refresh rate is low, the high-resolution score is the highest, the anti-fatigue effect is the best, and the anti-fatigue effect becomes better as the resolution increases. During the medium refresh rate and high refresh rate, the anti-fatigue effect of medium resolution is the best, followed by high resolution.
The anti-fatigue scoring of the display under different resolutions and refresh rates at 15 Hz paradigm stimulus frequency based on the information-CRITIC weighting method.
In summary, when using the related applications of BCI system based on SSVEP, the anti-fatigue performance of the system is not only related to the stimulus frequency of paradigm, but also to the resolution and refresh rate. When the paradigm stimulus frequency is low, in order to obtain a higher anti-fatigue effect, the resolution and the refresh rate should be kept consistently low (800 × 600, 120 Hz) or consistently high (1920 × 1080, 360 Hz). When the paradigm stimulus frequency is high, in order to obtain a higher anti-fatigue effect, the high resolution is matched when the refresh rate is low, and the medium resolution is matched when the refresh rate is both medium and high.
Discussion
Significance analysis
As shown in Table 2, most of the quantitative research on visual fatigue is based on a single index. Only the last two articles try to use the method of index fusion. Because the academic community still lacks the gold standard for quantitative evaluation of visual fatigue, a fusion improvement scheme for the CRITIC weight method is proposed, which also shows the novelty and advancement.
The results from the DSF index show that the visual fatigue of SSVEP-based BCI applications is not only related to the stimulus frequency, but also to the resolution and refresh rate of the display.
From the results of VFS and SP, it can be seen that the participants subjectively prefer low stimulus frequency, because the human eye can adapt quickly under low stimulus frequency, while high stimulus frequency is more likely to bring discomfort to the subjects.
From an objective point of view, most of the differences in stimulus frequency are not statistically different, meanwhile, higher stimulus frequency will bring higher CCA response amplitude and signal-to-noise ratio. The subjective and objective results form a trade-off. The low stimulus frequency is more subjectively liked by the participants, but the high stimulus frequency can obtain a better response.
When the BCI application based on SSVEP is used in practice, on the one hand, if it is to be used as long as possible, low paradigm stimulation frequency can be used. At the same time, according to the actual situation of the display, the low refresh rate is matched with the low screen resolution, or the high refresh rate is matched with the high screen resolution to achieve the best anti-fatigue effect. On the other hand, if the BCI device is used only for a short time, in order to achieve a higher information transmission rate, the high paradigm stimulation frequency can be used. At the same time, according to the objective conditions, the highest screen resolution can be used at the low refresh rate, and medium screen resolution is preferred at 240 Hz and above refresh rate to obtain the best anti-fatigue effect.
Limitations of this research
From the results of the overall evaluation system, it can be seen that whether the refresh rate and resolution are improved individually or in combination, can improve the matching degree with the higher paradigm stimulus frequency, and the specific relationship between them remains to be studied.
The duration of use of SSVEP-based BCI applications is generally not too long9,10. The time of this experiment is about two hours, which can cover most of the application scenarios. However, in the future, the development of BCI is bound to be close to human life, and long-term use is inevitable. In this case, if the experimental time can be prolonged as much as possible while ensuring the safety of the subject’s eyes, it may expose more correlation between paradigm stimulus frequency, display screen parameters and anti-fatigue effect.
In reality, BCI application scenarios will also cover factors such as natural light, and some other indoor light environment parameters, such as flash frequency, interior decoration, and spectral power40,41,42. These factors should also be considered in the future.
In the selection of indicators, CCA analysis and pupil data analysis from an objective perspective may lead to the omission of some other EEG and eye movement details. Therefore, in future research, more diversified tools for analyzing EEG data and deeper mining of eye movement data are also meaningful.
In the choice of stimulation frequency, if the experiment time is too long, the subjects will be required to stop the experiment due to fatigue, so the parameters cannot be selected too much. At present, there are only two stimulation frequencies. In the future, the sweep frequency paradigm will be considered to explore the changes in visual fatigue in a larger stimulation frequency range.
At present, most of the related experiments are LCD screens due to site constraints9,43,44,45,46,47,48,49,50, but LED screens are also widely used in real life. In future research, a variety of display screens will be considered, so as to make the experimental data more perfect and more in line with the use needs of BCI applications in real life.
Conclusion
In this manuscript, an anti-visual fatigue assessment system integrating subjective scoring and objective brain-eye information was established to explore the effects of different paradigm stimulus frequencies, display screen resolution and refresh rate on the visual experience of subjects when using SSVEP-based BCI applications. The best anti-fatigue effect can be obtained when the display is set to high resolution (1920 × 1080) and high refresh rate (360 Hz) at 7.5 Hz paradigm stimulus frequency. At the 15 Hz paradigm stimulus frequency, the display is set to medium resolution (1080 × 720) and medium refresh rate (240 Hz) to obtain the best anti-fatigue effect.
Subjective scoring shows that the subjects prefer low-frequency stimulus. Objective data show that high frequency stimulus can obtain higher EEG response amplitude and signal-to-noise ratio, which is more conducive to the use of BCI applications. In practice, if the BCI experiment based on SSVEP is short, it is more inclined to use a high-frequency paradigm stimulus, and if the experiment time is relatively long, it is more inclined to use a low-frequency paradigm stimulus. In the BCI experiment under different paradigm stimulus frequencies, the hardware settings recommended in this paper are matched at the same time, so that the subjects can obtain a better visual experience as much as possible. The research of this manuscript has important reference value for the hardware design direction of subsequent BCI display manufacturers and the matching of hardware and software parameters to minimize visual fatigue when SSVEP-based BCI applications are used.
Data availability
The datasets used and/or analyzed during the current study available from the corresponding author on reasonable request.
References
Chen, X., Chen, Z., Gao, S. & Gao, X. A high-itr ssvep-based bci speller. Brain-Computer Interfaces 1, 3–4 (2014).
Yin, E., Zhou, Z., Jiang, J., Yu, Y. & Hu, D. A dynamically optimized SSVEP brain–computer interface (BCI) speller. IEEE Trans. Biomed. Eng. 62(6), 1447–1456 (2014).
Pan, Y., Chen, J. & Zhang, Y. A survey of deep learning-based classification methods for steady-state visual evoked potentials. Brain-Apparatus Communication: J. Bacomics 2(1), 2181102 (2023).
Cao, T., Wan, F., Wong, C. M., da Cruz, J. N. & Hu, Y. Objective evaluation of fatigue by EEG spectral analysis in steady-state visual evoked potential-based brain-computer interfaces. Biomed. Eng. Online 13, 1–13 (2014).
Duart, X. et al. Evaluating the effect of stimuli color and frequency on SSVEP, Sensors, vol. 21, no. 1, p. 117, (2020).
Saboor, A. et al. Mesh of SSVEP-based BCI and eye-tracker for use of higher frequency stimuli and lower number of EEG channels, In: International Conference on Frontiers of Information Technology (FIT), 2018, pp. 99–104: IEEE. (2018).
Volosyak, I., Valbuena, D., Luth, T., Malechka, T. & Graser, A. BCI demographics II: how many (and what kinds of) people can use a high-frequency SSVEP BCI? IEEE Trans. Neural Syst. Rehabil. Eng. 19(3), 232–239 (2011).
Han, C. et al. Assessing the effect of the refresh rate of a device on various motion stimulation frequencies based on steady-state motion visual evoked potentials. Front. NeuroSci. 15, 757679 (2022).
Tian, P. et al. Effects of paradigm color and screen brightness on visual fatigue in light environment of night based on eye tracker and EEG acquisition equipment, Sensors, vol. 22, no. 11, p. 4082, (2022).
Zheng, X. W. et al. Anti-fatigue performance in SSVEP-Based Visual Acuity Assessment: a comparison of six stimulus paradigms, (in English). Front. Hum. Neurosci. 14, 301 (Jul 31 2020).
Liu, B., Huang, X., Wang, Y., Chen, X. & Gao, X. BETA: A large benchmark database toward SSVEP-BCI application, Frontiers in neuroscience, vol. 14, p. 544547, (2020).
Lemieux, A. & Knoll, E. Digital image resolution: what it means and how it can work for you, in IPCC 99. Communication Jazz: Improvising the New International Communication Culture. Proceedings 1999 IEEE International Professional Communication Conference (Cat. No. 99CH37023), pp. 231–236: IEEE. (1999).
Liu, B., Sun, J., Zhu, B., Li, T. & Sun, F. MadFormer: multi-attention-driven image super-resolution method based on Transformer. Multimedia Syst. 30(2), 78 (2024).
Wang, Q., Li, Y. & Liu, X. The influence of photo elements on EEG signal recognition, Eurasip Journal on image and video processing, vol. no. 1, pp. 1–9, 2018. (2018).
Ni, T. et al. A survey of large high-resolution display technologies, techniques, and applications, in IEEE Virtual Reality Conference (VR 2006, pp. 223–236: IEEE. (2006).
Green, P. W. The effect of visual display terminal flicker on task performance, (1995).
Gembler, F. et al. Effects of monitor refresh rates on c-VEP BCIs, in Symbiotic Interaction: 6th International Workshop, Symbiotic 2017, Eindhoven, The Netherlands, December 18–19, Revised Selected Papers 6, 2018, pp. 53–62: Springer. (2017).
Almoqbel, F. M., Yadav, N. K., Leat, S. J., Head, L. M. & Irving, E. L. Effects of sweep VEP parameters on visual acuity and contrast thresholds in children and adults, Graefes Arch Clin Exp Ophthalmol vol. 249, no. 4, pp. 613 – 23, Apr (2011).
Ng, K. B., Bradley, A. P. & Cunnington, R. Stimulus specificity of a steady-state visual-evoked potential-based brain–computer interface. J. Neural Eng. 9(3), 036008 (2012).
Tian, P. Y. et al. Effects of Paradigm Color and Screen Brightness on Visual Fatigue in Light Environment of Night Based on Eye Tracker and EEG Acquisition Equipment, (in English), Sensors, vol. 22, no. 11, p. 4082, Jun (2022).
Israsena, P. & Pan-Ngum, S. A CNN-based deep learning approach for SSVEP detection targeting binaural ear-EEG. Front. Comput. Neurosci. 16, 868642 (2022).
Guo, M. et al. Neural research on depth perception and stereoscopic visual fatigue in virtual reality. Brain Sci. 12(9), 1231 (2022).
Society, A. C. N. Guideline 5: guidelines for standard electrode position nomenclature. Am. J. Electroneurodiagn. Technol. 46(3), 222–225 (2006).
Chen, Y. C. & Yeh, S. L. Catch the moment: multisensory enhancement of rapid visual events by sound, Exp Brain Res., vol. 198, no. 2–3, pp. 209 – 19, Sep (2009).
Xie, X. J., Song, F. H., Liu, Y., Wang, S. R. & Yu, D. Study on the effects of Display Color Mode and Luminance contrast on visual fatigue, (in English). Ieee Access. 9, 35915–35923 (2021).
Owens, M., Koster, E. H. W. & Derakshan, N. Improving attention control in dysphoria through cognitive training: Transfer effects on working memory capacity and filtering efficiency, (in English), Psychophysiology, vol. 50, no. 3, pp. 297–307, Mar (2013).
Xie, J. et al. Effects of Mental Load and Fatigue on Steady-State Evoked Potential Based Brain Computer Interface Tasks: A Comparison of Periodic Flickering and Motion-Reversal Based Visual Attention, (in English), Plos One, vol. 11, no. 9, p. e0163426, Sep 22 (2016).
Xie, J. et al. Effects of Mental load and fatigue on steady-state evoked potential based Brain Computer Interface tasks: a comparison of periodic flickering and motion-reversal based visual attention. PLoS One 11(9), e0163426 (2016).
Lee, H. J. & Kim, S. J. Factors Associated with Visual Fatigue from Curved Monitor Use: A Prospective Study of Healthy Subjects, (in English), Plos One vol. 11, no. 10, p. e0164022, Oct 4 2016.
Kim, Y. J. & Lee, E. C. EEG based comparative measurement of visual fatigue caused by 2D and 3D displays, in International Conference on Human-Computer Interaction, pp. 289–292: Springer. (2011).
Lin, Z. L., Zhang, C. S., Wu, W. & Gao, X. R. Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs, (in English). IEEE Trans. Biomed. Eng. 53(12), 2610–2614 (2006).
Kalunga, E., Djouani, K., Hamam, Y., Chevallier, S. & Monacelli, E. SSVEP enhancement based on Canonical Correlation Analysis to improve BCI performances, in 2013 Africon, 1–5 : IEEE. (2013).
Bin, G., Gao, X., Yan, Z., Hong, B. & Gao, S. An online multi-channel SSVEP-based brain–computer interface using a canonical correlation analysis method. J. Neural Eng. 6(4), 046002 (2009).
Nakanishi, M., Wang, Y., Wang, Y. T. & Jung, T. P. A Comparison Study of Canonical Correlation Analysis Based Methods for detecting steady-state visual evoked potentials. PLoS One 10 (10), e0140703 (2015).
Cao, T., Wan, F., Wong, C. M., da Cruz, J. N. & Hu, Y. Objective evaluation of fatigue by EEG spectral analysis in steady-state visual evoked potential-based brain-computer interfaces, Biomed Eng Online vol. 13, no. 1, p. 28, Mar 12 (2014).
Kim, T. & Lee, E. C. Experimental Verification of Objective Visual Fatigue Measurement Based on Accurate Pupil Detection of Infrared Eye Image and Multi-Feature Analysis, Sensors (Basel), vol. 20, no. 17, p. 4814, Aug 26 (2020).
Zhao, Y., Guo, M., Chen, X., Sun, J. & Qiu, J. Attention-based CNN Fusion Model for emotion recognition during walking using Discrete Wavelet transform on EEG and Inertial signals. Big Data Min. Analytics 7(1), 188–204 (2023).
Zhi, Y. et al. Quantitative evaluation of anisometropic amblyopia treatment efficacy by coupling multiple visual functions via CRITIC algorithm. BMC Ophthalmol. 23(1), 162 (2023).
Azadi Moghadam, M. & Maleki, A. Fatigue factors and fatigue indices in SSVEP-based brain-computer interfaces: a systematic review and meta-analysis. Front. Hum. Neurosci. 17, 1248474 (2023).
Hartstein, L. E., Tuzikas, A. & Karlicek, R. F. Jr The impact of dynamic changes in light spectral power distribution on cognitive performance and wellbeing, LEUKOS. vol. 16, no. 4, pp. 289–301, (2020).
Pei, Z. F., Lin, B. R., Liu, Y. C. & Zhu, Y. X. Comparative study on the indoor environment quality of green office buildings in China with a long-term field measurement and investigation, (in English), Building and Environment, vol. 84, pp. 80–88, Jan (2015).
Cao, C. Research on innovative application of artistic characteristics of building decoration materials in interior design based on big data, in Journal of Physics: Conference Series. vol. 1744, no. 2, p. 022020: IOP Publishing. (2021).
Makri, D., Farmaki, C. & Sakkalis, V. Visual fatigue effects on steady state visual evoked potential-based brain computer interfaces, in 7th international IEEE/EMBS conference on neural engineering (NER). 2015, pp. 70–73: IEEE. (2015).
Seo, S. P., Lee, M. H., Williamson, J. & Lee, S. W. Changes in fatigue and EEG amplitude during a longtime use of brain-computer interface, in 2019 7th International Winter Conference on Brain-Computer Interface (BCI), pp. 1–3: IEEE. (2019).
Peng, Y. et al. Fatigue evaluation using multi-scale entropy of EEG in SSVEP-based BCI. IEEE Access. 7, 108200–108210 (2019).
Chai, X. et al. Effects of fatigue on steady state motion visual evoked potentials: optimised stimulus parameters for a zoom motion-based brain-computer interface. Comput. Methods Programs Biomed. 196, 105650 (2020).
Peng, Y. et al. Fatigue detection in SSVEP-BCIs based on wavelet entropy of EEG. Ieee Access. 9, 114905–114913 (2021).
Benedetto, S., Drai-Zerbib, V., Pedrotti, M., Tissier, G. & Baccino, T. E-readers and visual fatigue. PloS One 8(12), e83676 (2013).
Guo, F., Liu, L., Li, M. & Greene, R. L. The effect of arm support and control modes on muscle fatigue, eye fatigue, and posture changes in mobile game use among young adults. Hum. Factors Ergon. Manuf. Serv. Ind. 31(5), 496–505 (2021).
Mu, J., Grayden, D. B., Tan, Y. & Oetomo, D. Comparison of steady-state visual evoked potential (SSVEP) with LCD vs. LED stimulation, in 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 2946–2949: IEEE. (2020).
Acknowledgements
We would like to express our sincere thanks to the researchers who put forward the research ideas together, the research predecessors for their relevant research experience, and all the subjects who participated in these experiments.
Funding
This work is supported by the National Key Research and Development projects (No.2021ZD0204300), the Natural Science Basic Research Program of Shaanxi (Program No. 2023-JC-QN-0501), the project of The First Affiliated Hospital of Xi’an Jiaotong University (No.20231055) and the project of The First Affiliated Hospital of Xi’an Jiaotong University (No.20231056).
Author information
Authors and Affiliations
Contributions
P.T. and G.X. wrote the main manuscript text. C.H., C.D. and H.L. prepared all figures. R.C. and J.X. prepared all tables. J.W. and H.J. checked the format of all the figures and tables. X.G, S.Z. and Q.W. wrote the supplementary materials. All authors reviewed the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Institutional review board statement
The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of Xi’an Jiaotong University, China (No. 2023 − 1552). Informed consent was obtained from all subjects involved in the study.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Tian, P., Xu, G., Han, C. et al. A subjective and objective fusion visual fatigue assessment system for different hardware and software parameters in SSVEP-based BCI applications. Sci Rep 14, 27872 (2024). https://doi.org/10.1038/s41598-024-79401-3
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-024-79401-3
