In accordance with the Declaration of Helsinki, informed consent was obtained in writing from all participants prior to their participation. The experimental protocols, which were approved by the Institutional Review Board of the National Institutes for Quantum and Radiological Science and Technology, complied with safety guidelines for MRI research.

A total of 35 healthy female volunteers (mean age 26.9 ± 6.7 years) with no history of neurological disease were selected as candidates for this study. Five subjects’ data were excluded for the following reasons: the image data was damaged due to a technical error (1 subject), the candidate was visually impaired and unable to perform the task correctly (1 subject), there were severe motion artifacts (1 subject), and the candidate failed to perform the task satisfactorily for undetermined reasons (2 subjects).

MRI data acquisition

All subjects underwent 3T MRI with a MAGNETOM Verio scanner (Siemens AG; Munich, Germany). fMRI was performed using a gradient-echo planar imaging (GE-EPI) sequence (echo time: 25 ms, repeat time: 500 ms, tilt angle: 44°, field of view: 1440 mm × 1440 mm, acquisition array: 64 × 64, slice thickness: 4 mm, slices: 30, total scans: 900) during a finger tap task. Additionally, T2*WI were acquired using a two-dimensional (2D) fast gradient echo sequence (echo time: 25 ms, repetition time: 2000 ms, flip angle: 90 °, field of view: 240 mm × 240 mm, acquisition matrix: 128 × 128 and 64 × 64, slice thickness: 4 mm, number of slices: 30). Additionally, T1-weighted MR images were acquired using a three-dimensional (3D) fast gradient echo sequence prepared by magnetization (echo time: 1.98 ms, repetition time: 2,300 ms, flip angle: 9°, field of view: 250mm × 250mm, acquisition matrix: 256 × 256, slice thickness: 1mm). Table 1 shows the parameters of fMRI, T2-weighted MRI* and T1-weighted MRI.

Table 1 Magnetic resonance imaging analysis parameters.

Finger tapping procedure

A finger tapping task was performed during the fMRI scan. Supplemental Figure 1 depicts the task protocol, which included tapping phases of the thumb or little finger of one hand and rest phases between each task. Before starting the experiment, participants had enough time to familiarize themselves with the tasks and choose which hand they would use for typing. Instructions on which finger to tap or rest were provided on a screen behind the participant’s head and were viewed through a mirror mounted on the head coil. The screening was presented using E-prime 1.0 (Psychology Software Tools, PA, USA). Each subject was instructed to tap the cued finger, but not adjacent fingers, at their own pace.

Functional analysis

Prior to functional analysis, the first 60 scans were excluded from analysis to ensure magnetization has reached equilibrium15. After co-registration of T1WI structured data to the Automated Anatomical Labeling Atlas (AAL)16, functional data were co-registered with T1WI data. The transformations were then combined to identify the motor area in the functional datasets. Additionally, linear trends in the time series have been removed and the noise level has been reduced by applying a low pass filter to each pixel. Spatial filtering was also applied using a Gaussian filter with (sigma =1.5).

After this pre-processing, functional activation maps were obtained from the image time series by correlating the time course of each pixel’s signal intensity with an on/off task design convolved with a function of canonical hemodynamic response. SPM12 (revision 7219)17 was used for analysis. The cross-correlation coefficient (CC) was calculated for each pixel using

$$CC=frac{overrightarrow{{R}_{x}}cdot overrightarrow{{R}_{y}}}{left|overrightarrow{{R}_{x}}right| left|overrightarrow{{R}_{y}}right|},$$


where (overrightarrow{{R}_{x}}) is the reference task design and (overrightarrow{{R}_{y}}) is the time evolution of the signal intensity of the pixel15. All image preprocessing and functional analysis was performed in MATLAB R2018b (Mathworks, Natick, MA, USA).

Super-resolution based on Deep Learning

Figure 1 presents an overview of the proposed method. The STSS-SRfMRI scheme includes two unique ideas: first, it uses high spatial resolution static T2*WI as training data; second, it applies subject-specific learning. As described in the introduction, static T2*WIs were used to introduce high spatial resolution information into the training process. Additionally, since functional signal changes are usually quite small, subject-specific learning was used to eliminate any anatomical variation that might be artificially introduced by including T2*WI data from other subjects.

Figure 1

Overview of the subject-specific super-resolution fMRI (STSS-SRfMRI) scheme based on Static T2*WI proposed in this study. The upper and lower parts correspond respectively to the training and testing phases. In the training phase, the generator (G) was optimized to form a relationship between the low-resolution and high-resolution T2*WI. The discriminator (D) decided whether the input was “real” (i.e. the reference high resolution T2*WI) or “false” (i.e. the high resolution T2*WI generated). G learned to generate a more realistic output via feedback from D. In the test phase, a high-resolution functional MRI (fMRI) time series was reconstructed from the low-resolution fMRI data at the using the optimized generator, then a high-resolution functional map. was calculated based on high-resolution fMRI.

Before training, the pixel intensity of the T2*WI training data was adjusted and scaled to match the intensity of the fMRI data. The 30 slices of T2*WI data from each subject were used for training and validation to build a subject-specific model. The trained model was then applied to fMRI data from the same subject.

The SRGAN used in this work has been customized in several ways. Rather than using an oversampling block in the G generator, the low resolution images were scaled to a 128×128 array size using lanczos 3 interpolation18.19 before being seized. All batch normalization layers have also been removed20. A discriminator (D) was applied with the number of convolutional layers set to 10 to accommodate the size of the input. We implemented the modified SRGAN network using an adaptive moment estimation optimizer (Adam) with an initial decay rate of 0.9, scale factor of 2, patch size of 64, a batch size of 2, an initial learning rate of 0.0001, and 100,000 iterations. The training images were the 30 slices of the corresponding T2*WI data. Experiments have been implemented in PyTorch 1.1.0 on Ubuntu 16.04 LTS.

Identification of the region related to neuronal activation

The activation maps generated from the low-resolution fMRI data (the raw map) and the processed output of the STSS-SRfMRI scheme (STSS-SR fMRI map) were compared based on their effectiveness in localizing the region of activation . To this end, regions corresponding to thumb and little finger activation tasks were identified separately for each subject’s fMRI and STSS-SRfMRI raw maps. First, a CC map was calculated for each set of input images (i.e. the raw data or STSS-SR) for each subject and each activated finger. Second, the activation-related region in each CC map was defined as the region consisting of pixels having values ​​equal to or greater than a threshold value, see Fig. 2. The threshold value was defined as

Figure 2
Figure 2

Overview of how the activation-related region was defined for each tapping task. First, activation maps were obtained from the raw image series and the subject-specific super-resolution fMRI (STSS-SRfMRI) image series based on Static T2*WI (bottom row high). Second, the upper 25% between the maximum and minimum CC values ​​was defined as the threshold (middle row). Finally, the region consisting of pixels having values ​​equal to or greater than the threshold value was defined as the activation-related region (bottom).



The number of pixels included in the activation-related region of the raw fMRI map was compared to that of the STSS-SR fMRI map for each finger of each subject. As the STSS-SR fMRI maps had pixels four times smaller than those of the raw fMRI maps for the same size area, the number of pixels in the STSS-SR fMRI maps was reduced by 4 before the comparison.

Independence of activated regions extracted for different tasks

The raw fMRI and STSS-SR maps obtained in the previous subsection were compared to determine which of them has higher functional resolution for thumb and little finger tasks. For this purpose, a Dice coefficient21.22 was calculated for the extracted regions related to thumb and little finger activation for each subject (Fig. 3). This assessment was based on the well-known fact that the motor areas of the thumb and little finger are not the same.23.24.

picture 3
picture 3

Definition of the Dice coefficient used in this study to evaluate the sharpness of the activated regions corresponding to the tasks of the thumb and the little finger. The dice coefficient was calculated for the extracted regions related to thumb (green) and little finger (blue) activation for each subject. The light blue area corresponds to the overlap between the activation-related regions for the thumb and the little finger.

statistical analyzes

The number of pixels included in each activation-related region and the dice coefficient calculated from raw fMRI and STSS-SR fMRI maps were statistically compared using the Wilcoxon signed rank test (p 25was used to make these statistical comparisons.


Stevie Nicks closes in timeless form, a nod to Tom Petty


The Wildlife Center of Virginia treats a record 90,000 wild animals

Check Also