Impact statement
There are various measurement tools that can measure different concepts of social support. The purpose of this study is to test the factor structure of the Multidimensional Scale of Perceived Social Support (MSPSS), one of the most frequently used instruments for measuring perceived social support, using meta-analytic confirmatory factor analysis. In this way, we sought to answer the debates about the internal structure of this measurement tool and obtained some evidence about its reliability and dimensionality. According to this evidence, the MSPSS has a three-factor structure. It has high internal consistency and measurement invariance according to language and sample type (clinical and nonclinical).
Introduction
Since anything that people can exchange can act as a source of social support (Waite, Reference Waite, Majmundar and Hayward2018), social support is considered a multifaceted concept. Therefore, there are different views in the literature on how to conceptualize this concept. The psychological perspective divides social support into two types: perceived and received social support (Vangelisti, Reference Vangelisti2009). The subjective assessment that one’s social networks will provide effective help when needed is known as perceived support (Lakey & Scoboria, Reference Lakey and Scoboria2005). Demonstrated support refers to actual and spontaneous helping behaviors (Norris & Kaniasty, Reference Norris and Kaniasty1996). Studies in the literature reveal that there is only a modest correlation between measures of these two types of support (Eagle et al., Reference Eagle, Hybels and Proeschold-Bell2019; Haber et al., Reference Haber, Cohen, Lucas and Baltes2007; Lakey et al., Reference Lakey, Orehek, Hain and Vanvleet2010). However, compared to received support, perceived support is reported to be more strongly associated with positive health outcomes (Barrera, Reference Barrera, Rappaport and Seidman2000; Eagle et al., Reference Eagle, Hybels and Proeschold-Bell2019; Uchino, Reference Uchino2009). In a meta-analysis study by Prati and Pietrantoni (Reference Prati and Pietrantoni2010), it was found that not all forms of support have the same effect on mental health and perceived support has a greater effect than received support. In addition, Moreira et al. (Reference Moreira, de Fátima Silva, Moleiro, Aguiar, Andrez, Bernardes and Afonso2003) stated that perceived social support can best be seen as an individual difference variable because it remains relatively constant over long periods. Therefore, in the present study, the measurements of perceived social support, which is a more prominent substructure in addressing the concept of social support, were examined. Although there are different instruments for the measurement of perceived social support in the literature (e.g., Perceived Social Support, Friends and Perceived Social Support, Family (Procidano & Heller, Reference Procidano and Heller1983); Child and Adolescent Social Support Scale (Malecki & Demaray, Reference Malecki and Demaray2002); Student Social Support Scale (Malecki & Elliott, Reference Malecki and Elliott1999), this study focussed on the Multidimensional Scale of Perceived Social Support (MSPSS), which is considered to represent the construct of perceived social support more strongly and is widely used in international studies (Zimet et al., Reference Zimet, Dahlem, Zimet and Farley1988). It has been determined that this measurement tool has received nearly 17,000 citations so far.
The multidimensional scale of perceived social support
The MSPSS was originally designed as a 24-item and 5-point Likert scale. It was created and piloted by American university students. Following several pilot studies, it was determined that the scale needed to be modified. The items were updated, and the scale’s final form was produced (Zimet et al., Reference Zimet, Dahlem, Zimet and Farley1988). Following their research, the researchers concluded that the MSPSS only included 12 items and was broken down into three separate categories of social support: significant others (SO), friends (FRI) and family (FAM). This 12-item scale has a 7-point response format; with responses ranging from 1 (very strongly disagree) to 7 (very strongly agree) and is comprised of three 4-item subscales. A higher score on the subscale indicates a higher perception of support. It was determined that the original version of the MSPSS had a three-factor structure, and the internal consistency coefficient and stability coefficient were high. In addition, since MSPSS scores were negatively correlated with anxiety and depression scores, it was stated that it showed moderate construct validity (Zimet et al., Reference Zimet, Dahlem, Zimet and Farley1988).
Due to its many advantages, the MSPSS has been used in many social support studies worldwide. The most important advantage of the scale is that while other scales measuring the perceived social support structure address the FAM, FRI and SO dimensions separately, MSPSS addresses these three dimensions together. While the FAM dimension includes parents, spouses, children and siblings, the SO dimension includes people other than family and friends (e.g., date, fiancee, relative, neighbor and doctor) (Eker et al., Reference Eker, Arkar and Yaldız2001). The short and comprehensible items of the scale and the fact that it is a self-report scale that is not time-consuming are also among its advantages. In addition, MSPSS is psychometrically sound (Zimet et al., Reference Zimet, Dahlem, Zimet and Farley1988). However, alternative factor structures for the MSPSS, which have been adapted to other cultures and groups, have also been discussed in the literature.
Although the unidimensional model is not suitable for the theoretical infrastructure of the scale, Akhtar et al. (Reference Akhtar, Rahman, Husain, Chaudhry, Duddu and Husain2010) in their study with Pakistani women determined that the Urdu version of the MSPSS showed a unidimensional structure. Two different models are proposed for the 2-factorial model. Chou (Reference Chou2000) expressed a separate FAM factor as a second factor in addition to a factor consisting of a combination of FRI and SO items. They claimed that the most comprehensible answer was obtained using this two-factor structure. In a similar vein, Cheng and Chan (Reference Cheng and Chan2004) concluded that the original 3-factor model could be countered by the 2-factor structure created by combining the FRI and SO subscales. Wang et al. (Reference Wang, Wan, Huang, Huang and Kong2017) also determined the factor structure of the MSPSS with exploratory factor analysis (EFA) and determined the FAM factor as a separate factor in addition to a factor consisting of the items of the SO and FRI subscales. Other researchers suggested a 2-factor structure considering FAM and SO items in a single factor. Tonsing, Zimet and Tse (Reference Tonsing, Zimet and Tse2012) determined that a 2-factor structure, FAM (FAM and SO items) and FRI, was appropriate for the Urdu version of MSPSS. Mohammad et al. (Reference Mohammad, Al Sadat, Loh and Chinna2015) found a very high correlation value between FAM and SO, so they combined these two factors under the FAM factor and determined that the 2-factor solution was confirmed. Other studies reached acceptable results for the 2-factor solution, with the first factor mainly consisting of the items in the FAM and SO subscales and the second factor mainly consisting of the items in the FRI subscale (Cobb & Xie, Reference Cobb and Xie2015; Stanley et al., Reference Stanley, Beck and Zebb1998; Zhou et al., Reference Zhou, Li, Wei, Yin, Liang, Zhang and Zhuang2015). In 2-factor structures, it is seen that the SO subscale is included in other subscales. It is suggested that this might be because the subjects were unable to differentiate between SO and other supportive individuals like FRI and FAM (Wongpakaran et al., Reference Wongpakaran, Wongpakaran and Ruktrakul2011).
The original 3-factor structure of the scale was tested and validated in many studies (Bruwer et al., Reference Bruwer, Emsley, Kidd, Lochner and Seedat2008; Calderón Garrido et al., Reference Calderón Garrido, Ferrando Piera, Lorenzo Seva, Gómez Sánchez, Fernández Montes, Palacín Lois and Jiménez Fonseca2021; Canty-Mitchell, & Zimet, Reference Canty-Mitchell and Zimet2000; Cartwright et al., Reference Cartwright, Pione, Stoner and Spector2022; Duru, Reference Duru2007; Ebrahim & Alothman, Reference Ebrahim and Alothman2022; Edwards, Reference Edwards2004; Eker et al., Reference Eker, Arkar and Yaldız2001; Ermis-Demirtas et al., Reference Ermis-Demirtas, Watson, Karaman, Freeman, Kumaran, Haktanir and Streeter2018; Laksmita et al., Reference Laksmita, Chung, Liao and Chang2020; Pedersen et al., Reference Pedersen, Spinder, Erdman and Denollet2009; Pérez-Villalobos et al., Reference Pérez-Villalobos, Briede-Westermeyer, Schilling-Norman and Contreras-Espinoza2021; Robu et al., Reference Robu, Sandovici and Ciudin2020; Tonsing, Reference Tonsing2022; Tonsing et al., Reference Tonsing, Zimet and Tse2012; Wilson et al., Reference Wilson, Yendork and Somhlaba2017). However, in some studies, the original 3-factor structure was confirmed by modifications (Başol, Reference Başol2008; Martins et al., Reference Martins, Peterson, Almeida and Costa2012; Sharif et al., Reference Sharif, Zaidi, Waqas, Malik, Hagaman, Maselko and Rahman2021; Trejos-Herrera et al., Reference Trejos-Herrera, Bahamón, Alarcón-Vásquez, Vélez and Vinaccia2018; Wongpakaran et al., Reference Wongpakaran, Wongpakaran and Ruktrakul2011). It can be said that all these studies conducted in different languages (e.g., Arabic, Chinese, Portuguese, Spanish, Swedish and Turkish) offer strong proof for the generalizability of the scale’s three-factor structure. However, the fact that some studies encountered different situations in the SO subscale (e.g., low factor saturation) (Cheng & Chan, Reference Cheng and Chan2004; Edwards, Reference Edwards2004) may indicate that the 3-factor solution is not a good-enough representation of the MSPSS. In addition, the correlation values of .80 and above between FAM, FRI and SO factors in some studies (e.g., Iwanaga et al., Reference Iwanaga, Wu, Armstrong, Kaya, Dutta, Kundu and Chan2021; Merino-Soto et al., Reference Merino-Soto, Boluarte Carbajal, Toledano-Toledano, Nabors and Núñez-Benítez2022; Pérez-Villalobos et al., Reference Pérez-Villalobos, Briede-Westermeyer, Schilling-Norman and Contreras-Espinoza2021) suggest that there may be a general factor in addition to these three factors.
Later, the factor structure of the scale is also discussed through more recent models such as the bifactor and/or exploratory structural equation model (ESEM). For example, Osman et al. (Reference Osman, Lamis, Freedenthal, Gutierrez and McNaughton-Cassill2014) used the item response theory (IRT) bifactor method. The researchers concluded that there was insufficient empirical basis to calculate the subscale scores of any of the MSPSS-specific factors because the discrimination parameter of each MSPSS item was greater on the general factor than on a group-specific factor. They concluded that the MSPSS offers solid evidence in favor of its application as a unidimensional instrument based on these findings. On the other hand, Merino-Soto et al. (Reference Merino-Soto, Boluarte Carbajal, Toledano-Toledano, Nabors and Núñez-Benítez2022), and Yılmaz Koğar and Koğar (Reference Yılmaz Koğar and Koğar2023) investigated the MSPSS factor structure using bifactor and ESEM techniques in addition to the models previously presented. They concluded from the study that, except the unidimensional model, all other models demonstrated a high degree of model-data fit; nonetheless, the multidimensionality indicators corroborated the superiority of the bifactor-ESEM. Conversely, it was found that the inter-factorial correlations were significantly low, and the general factor was insufficiently strong. It was determined that separate but moderately correlated components might explain the MSPSS and that the presence of potential systematic variations might make it impossible to identify a general factor.
These studies collectively demonstrate that various conclusions are drawn regarding the scale’s factor structure. All models discussed for the MSPSS in the literature are presented in Figure 1.
Figure 1. Conceptual representation of all models tested for MSPSS in the literature.
Note. G = general factor; FAM = family support; FRI_SO = friends and significant others support; FAM_SO = family and significant others support; SO = significant others support.
The current study
Although the MSPSS was created in 1988, the psychometric properties of the scale are still the focus of research in various cultures and groups. Therefore, in this study, we aimed to investigate the factor structure of the scale on a larger sample and to quantitatively synthesize the factor analysis results of previous studies with the help of meta-analytic confirmatory factor (MA-CFA) analysis. To our knowledge, this meta-analytic factor approach has not yet been evaluated for the MSPSS. The compatibility of the scale structure with the bifactor model has not been examined in the majority of studies evaluating the original factor structure. Examining the bifactor model, which has been widely used to address constructs in psychology, through the findings of these other studies may help to better determine the factor structure of the MSPSS. We think that it would be important for the literature to present a more general result of the structure emerging from this scale, which has been adapted in many cultures and used very frequently. We also examined whether the MSPSS is interpreted in a conceptually similar way using a test of measurement invariance (MI) across language and sample type because it is necessary to provide a prerequisite for meaningful comparisons between groups. The construct validity of the MSPSS may be affected by translations of the scale from the original language into other languages, as cultural differences are likely to affect how social support is perceived (Dambi et al., Reference Dambi, Corten, Chiwaridzo, Jack, Mlambo and Jelsma2018). In this respect, we examined the invariance of the factor structure of the scale for the groups we formed according to the language. In addition, the fact that we determined that the MSPSS is frequently used in clinical and nonclinical samples in the literature led us to question whether these groups are comparable. For this reason, we also examined the invariance of the scale over the groups we formed according to the sample type variable. We designed the study to combine the results obtained from more recent studies with those obtained from previous studies. We aim that the findings of the study will provide some data that can help explain the theoretical foundations of the perceived social support structure and contribute to the accumulation of knowledge.
Method
Database, coding and screening
To answer the research questions of this study, a literature search was conducted on many major scientific databases (PsycINFO, PubMed, Web of Science and Google Scholar). In September–October 2022, the literature was searched using the Boolean Operator (MSPSS OR “the Multidimensional Scale of Perceived Social Support”) AND (“valid*” OR “factor analysis” OR “factor structure” OR “psychometric” OR “adaptation”) to reach potential articles and 13,794 scientific studies were reached. The titles, abstracts and tables of these scientific publications were scanned and examined whether they had the following 5 criteria: Each study (a) should be written in English, (b) should be an original study (thesis and conference papers could not be found so all studies are articles), (c) should use the original MSPSS with 12 items and (d) should include loading patterns or covariance/correlation matrix (inter-item correlation matrices, etc.). (e) In the studies, the rotation method should be reported; if oblique rotation was used, correlations between factors should be included. If orthogonal rotation was used, the correlation between factors was accepted as zero.
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA; Page et al., Reference Page, Moher, Bossuyt, Boutron, Hoffmann, Mulrow and McKenzie2021) guidelines were adhered to during this procedure, and reporting was done under these guidelines. The PRISMA checklist is shown in supplementary files. A total of 61 different articles that met the relevant criteria were reached. However, studies that addressed the scale in a way that would change its structure addressing the general population were removed from the database. This situation was experienced in the “significant others” factor. It was observed that this factor was given different names according to the type of sample studied. While studies on the women sample gave this dimension names such as husband and spouse, studies conducted on students gave names such as school staff and professor. We decided to exclude these studies because we thought that the reduction of this factor to a person or specific individuals could affect the structure of the scale. In addition, studies that reported incomplete statistical analyses, that provided almost no information about how the factor analysis was applied, that did not report any reliability coefficients and whose model-data fit indices did not meet the criteria recommended in CFA were also excluded. In this case, the number of articles used for the study decreased to 54. We used the factor loadings obtained from the EFA findings of 32 studies, the standardized regression coefficients obtained from the CFA findings of 21 studies and the inter-item correlation values of 6 studies in 59 samples. More details can be found in Figure 2.
Figure 2. Literature search summary.
However, some studies that altered the structure of the scale (e.g., different factor naming) or were found to have inadequate and/or incomplete reporting in statistical analyses were also excluded, reducing the number of articles used for the study to 54. Studies that used different factor naming also changed the item writing significantly. Since we think that this situation affects the factor structure of the scale, we decided to exclude these studies as well. Studies with a very small sample size (<50) and almost no information on how factor analysis was applied were also excluded from the database. Studies with reliability coefficients below 0.70 or model-data fit indices that did not meet the recommended criteria in CFA were also removed from the database. A database was created with a total of 59 samples from these 54 studies. The factor loadings obtained from the EFA findings from 32 studies, the standardized regression coefficients obtained from the CFA findings from 21 studies and the inter-item correlation values from 6 studies were used from 59 samples. More details are in Figure 2.
Inter-coder reliability for all variables was examined with Cohen’s kappa and inter-coding correlation. The median value of the inter-coding correlation was .98, and the median value of Cohen’s kappa was .99. In the event of a disagreement between the two coders, this disagreement was settled through conversation. If the inter-item correlation values were present in the article, they were undoubtedly used during the data coding process. Six studies have reported inter-item correlations. In other articles, the results of the EFA were prioritized over those of the CFA when they were applied to the same sample. Since there were no limitations in parameter estimation and factor structure testing in EFA, EFA findings were prioritized over CFA. If a rotation technique was used in studies presenting EFA findings, factor loadings of the factor structure obtained after rotation were taken into consideration. The results of the factor structure with the best model-data fit were included in the database if more than one CFA finding was applied to the same sample in an article (to test several factor structures). The multigroup CFA results that were used to test MI were not included in the analysis. Moreover, articles that lacked necessary details (such as model-data fit) were not included in the study. Also, descriptive data regarding the sample (such as the participants’ countries of origin, sample size and average age) and reported factor analysis were gathered (i.e., number of response options and rotation method). Access to the Excel file containing this data is available in the supplementary materials.
MA-CFA procedure
MA-CFA was developed by combining two different disciplines. These two discipline areas are meta-analysis and structural equation modeling (SEM). To apply the traditional meta-analysis, it is necessary to reach all published and unpublished studies. However, MA-CFA can also be used with a small number of samples and a sufficient sample size. According to Strunk and Lane (Reference Strunk and Lane2017), at least 10 samples and a sample size of 1,500 people are needed to reproduce the population parameters. In our study, 59 different samples were reached and the total sample size was 27,905. For this reason, it was decided that MA-CFA could be applied to our database.
Based on the supplied factor pattern matrices, we computed the item-level correlations that the model implies. Since Monte Carlo simulations showed that this method produces unbiased estimates of meta-analytic factor patterns, missing values were handled via imputed factor loadings, which were given a value of zero in the case of missing values (Gnambs & Staufenbiel, Reference Gnambs and Staufenbiel2016). However, our database does not include any study with missing factor loadings or not reported because the factor loading is too small. In the first stage of MA-CFA, using random-effects meta-analysis, a pooled correlation matrix was produced. Because random-effects meta-analysis yields more accurate parameter estimates than fixed-effects techniques, it was selected as the method of choice (Field, Reference Field2005; Hunter & Schmidt, Reference Hunter and Schmidt2004). The generated pooled correlation matrix was put through factor analytic models in the second MA-CFA stage. EFA was used for this, and Horn’s (Reference Horn1965) parallel analysis tests and the minimal average partial (MAP) test were used to find the ideal factor number. In the third stage, CFA was applied to 5 different competitive models, taking into account the dimensionality analysis findings and factor structures tested for the MSPSS in the literature (see Figure 1, Models 1–5). According to Schermelleh-Engel et al.’s (Reference Schermelleh-Engel, Moosbrugger and Müller2003) suggested cutoff points, the overall model fit for each tested model was evaluated using the following metrics: Comparative Fit Index (CFI) and Tucker–Lewis Index (TLI) (≥ .97, desirable, and ≥ .95, acceptable), root mean square error of approximation (RMSEA; ≤ .05, good fit; ≤ .08, acceptable fit) and standardized root mean square residual (SRMR; ≤ .05, good fit; ≤ .10, acceptable fit). Model fit was determined by taking into consideration the values of CFI, TLI, RMSEA and SRMR, as the chi-square test is highly sensitive to sample size. Furthermore, the factor loadings derived from every model were scrutinized to ascertain the degree of well-definedness of the pertinent component. According to Gaskin and Happell (Reference Gaskin and Happell2014), factor loadings of at least ± .40 were recognized as a sign of a well-defined factor. As long as the factor loadings are .40 and above, the model with the best model-data fit will be accepted as the “actual” factor structure of MSPSS. All CFAs were conducted with the weighted least squares (WLS) estimator. After determining the factor structure that best describes the factor structure of MSPSS from these 5 competitive models, factor loadings, omega coefficients and the explained common variance (ECV) coefficient were calculated for the relevant factor structure. The scale is effectively unidimensional because the subscales’ omega coefficients are small (<.50) and the general factor’s omega coefficient is bigger than .75 (Reise, Bonifay et al., Reference Reise, Bonifay and Haviland2013). The scale can be regarded as almost unidimensional when the ECV, or the percentage of common variance in a bifactor model attributed to the general factor, is high (>.60) (Reise, Scheines et al., Reference Reise, Scheines, Widaman and Haviland2013). The Bifactor Indices Calculator Excel file was utilized to compute the omega coefficients and ECV values (Dueber, Reference Dueber2017).
Follow-up analyses were initiated when the best factor structure for the scale was chosen. MI was examined for this aim using language samples as well as clinical and nonclinical samples. MI was tested between English (k = 9, n = 2,710), and the 5 other most used languages (Chinese, Spanish, Malay, Turkish and Urdu [k = 20, n = 10,849]). This approach aims to identify structural deterioration brought on by adaptations and the data set gathered from the clinical setting. The testing of a group of hierarchically nested models, each with a distinct set of constraints on the measurement parameters examined, is the basis of the MI process. The only invariance models that have been tested for MI are the configural model (Model 1), which has the same number of factors and item-factor loading patterns, and the metric model (Model 2), which has identical factor loadings. This is because strong MI is based on comparing factor variance, covariance and mean. With strong factorial invariance, the means of the factors can be meaningfully compared across the groups. However, in this MI study conducted on summary data (correlation matrix), only configural and metric MI was tested since the mean information was not included. Next, the CFI differences between the more constrained and less constrained models (ΔCFI = CFIconstrained model – CFIunconstrained model) and RMSEA differences (ΔRMSEA = RMSEAconstrained model – RMSEAunconstrained model) were analyzed. MI would be indicated by an RMSEA difference of less than .015 and an absolute difference in CFI of less than or equal to .01 (Cheung & Rensvold, Reference Cheung and Rensvold2002).
All the R codes of this study were written by reconsidering the R codes in Schroeders et al.’s (Reference Schroeders, Kubera and Gnambs2022) study. Analysis of dimensionality, pooled correlation matrix and all MASEM procedures were performed with psych package 2.2.9 (Revelle, Reference Revelle2022), metaSEM package 1.2.5.1 (Cheung, Reference Cheung2022) and lavaan package 0.6 (Rosseel, Reference Rosseel2012). MI was tested on Mplus 8.3 (Muthén & Muthén, Reference Muthén and Muthén2017).
Findings
Study descriptions
As stated in the method section of our study, the findings of a total of 59 samples included in 54 unique articles formed the database of this research. These studies were published between 1988, when MSPSS was developed, and 2022. The total sample size of the database is 27,905 (min = 50; max = 2,105; median = 310). Women constitute 59.2% of the whole group. The average age is 32.7. Of the 59 samples, 16 were clinical sample data. This finding shows that cross-cultural studies of MSPSS are mostly carried out on nonclinical samples. While 53 of the 59 samples used 7 response options, which is the number of response options belonging to the original MSPSS, 3 of the studies preferred to use the 4-point response option, 2 of the studies preferred to use the 5-point response option, and one study preferred to use the 6-point response option.
While 7 of the samples were in the USA, 5 of the samples were in Malaysia and 3 of the samples were in China, Singapore, Romania and Türkiye. The most used languages are English (9), Chinese (5), Spanish (5), Malay (4), Turkish (3), Urdu (3) and Romanian (3). The list of included articles and more information about study characteristics can be found in the supplementary files.
Dimensionality analysis
A pooled correlation matrix was produced utilizing factor loadings, normalized regression coefficients and inter-item correlation values derived from various database samples prior to the analyses performed to ascertain the number of dimensions of the MSPSS. In the pooled correlation matrix, the correlation coefficients vary from .196 to .841. The correlation coefficients had an average value of .380.
The factor structure of MSPSS was determined as 3 factors in 47 samples according to EFA and CFA findings, which were applied to 59 samples in the database (since only inter-item correlations were used, EFA and CFA findings of 6 samples were not taken into account). The original factor structure of MSPSS also includes 3 factors. A 2-factor structure was obtained in 5 samples, and a unidimensional structure was obtained in only 1 sample. In this case, it was necessary to perform dimensionality analyses first with the obtained pooled correlation matrix. Auerswald and Moshagen’s (Reference Auerswald and Moshagen2019) simulation study suggests traditional parallel analysis. For this purpose, the MAP test and Horn’s (Reference Horn1965) parallel analysis tests were applied. Both dimensionality analysis tests showed that a 3-factor structure had the best fit with the data. Although this finding is preliminary evidence that the MSPSS has a 3-factor structure, some recent studies (see Merino-Soto et al., Reference Merino-Soto, Boluarte Carbajal, Toledano-Toledano, Nabors and Núñez-Benítez2022; Yılmaz Koğar & Koğar, Reference Yılmaz Koğar and Koğar2023) have shown that there may be different alternatives in the 3-factor structure (i.e., bifactor-CFA, ESEM and bifactor-ESEM). However, since ESEM could not be tested on the summary data (in our study: inter-item correlation matrix), ESEM models were excluded from this study. For this reason, CFA was applied to 5 competitive models to test the factor structure of MSPSS.
MA-CFA analysis
To compare the 5 different competitive models, we applied CFA. The obtained results are given in Table 1. The CFA showed that the model-data fit values of Model 1, Model 2 and Model 3 show sufficient model fit according to CFI and TLI values (> .90). However, in all these models, the RMSEA value is above .08. In this case, it can be said that the model-data fit of these models is not sufficient. It has been mentioned in previous sections that these models have been validated by very few studies. When the model-data fit values obtained for these first three models are examined, it is seen that the factor structure of MSPSS in this study is far from being unidimensional and two-factorial. The original 3-factor model (Model 4), proposed by Zimet et al. (Reference Zimet, Dahlem, Zimet and Farley1988), fitted the data excellent, as in many studies (CFI = 1.000; TLI = 1.000; RMSEA = .008, 95% CI [.007, .010]; SRMR = .007). Besides, Model 5 (proposed by Merino-Soto et al., Reference Merino-Soto, Boluarte Carbajal, Toledano-Toledano, Nabors and Núñez-Benítez2022) was found to have excellent model-data fit values (CFI = 1.000; TLI = 1.000; RMSEA = .006, 90% CI [.004, .007]; SRMR = .003). It can be said that these two competitive models are successful in explaining the factor structure of MSPSS.
Table 1. Fit indices of all the competitive models
Note. CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; CI = confidence interval; df = degrees of freedom; SRMR = standardized root mean square residual; Model 1 = unidimensional model; Model 2 = 2-factor correlated model (FR/SO and FA); Model 3 = 2-factor correlated model (FA/SO and FR); Model 4 = 3-factor correlated model; Model 5 = bifactor CFA model; AIC = Akaike information criterion; BIC = Bayesian information criterion.
For Model 4 and Model 5, the model fit indices indicated good fit. Therefore, factor loadings, omega coefficient and ECV results for both models were examined and a more valid decision was tried to be made (see Table 2).
Table 2. Standardized factor loadings (λ), omega and ECV coefficients for Model 4 and Model 5
Note. FAM = family; FRI = friends; SO = significant others; G = general factor; ω = hierarchical omega coefficient for general factor and hierarchical subscale omega coefficient for specific factors; ECV = explained common variance for general factor and explained common variance subscales for specific factors.
Factor loading values of Model 4 range from .789 to .886. The findings in Table 2 show that each item has a high factor loading on the relevant factor. In addition, it can be said that hierarchical omega coefficients are also quite high as an indicator of composite reliability (ω = .890 to .914, M = .902). However, statistically significant inter-factorial correlations in this model (rfam-fri = .362, rfam-so = .468 and rfri-so = .452; M = .427; p < .001) may indicate a general factor. These correlation coefficients also indicate moderate and positive relationships.
It was determined that the factor loadings of the general factor of Model 5 (λ = .489 to .641, M = .549) were lower than the factor loadings of the specific factor (λ = .489 to .701, M = .625). This finding shows that the variance explained by specific factors is higher. Omega and ECV values were also evaluated to decide which of the general or specific factors were more dominant. In our study, the omega coefficient for the general factor is below .75; also the omega coefficient for subscales is above .50. This shows that MSPSS is not essentially unidimensional. Besides, all ECV values are below .60.
Factor loading values for both models are .30 and above. In this case, model-data fit values were compared to determine the most appropriate factor structure. These findings show that the scale does not have a strong general factor. In addition, the fact that the model-data fit between Model 4 and Model 5 is very close to each other shows that the model-data fit has not improved significantly with the bifactor model. At the end of these reviews, it was decided that the original three-factor correlated CFA model was the “actual” factor structure of MSPSS. Besides all other findings show that the MSPSS has three dominant specific factors.
Measurement invariance testing
In the database we created in our study, it was determined that MSPSS was culturally adapted to 28 different languages. There may be language and translation biases in these cross-cultural studies. Therefore, besides English, 5 different languages that are used most in the research were selected and a pooled correlation matrix of each language was obtained. Then, multigroup CFA was applied for language groups over this correlation matrix. The same evaluation was applied to nonclinical and clinical samples, and the findings are given in Table 3.
Table 3. Fit indices of measurement invariance models
Note. CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; ΔCFI = CFI values’ difference; ΔTLI = TLI values’ difference; ΔRMSEA = RMSEA values’ difference.
Table 3 lists the MI results of the correlated three-factor MSPSS model investigated concerning language and clinical status. The language and clinical status variables have been found to give configural and metric MI. Depending on the clinical status, configural and metric MI was also ensured.
Discussion
The objective of this study is to evaluate the latent factor structure of the MSPSS, the most popular tool in the literature for measuring perceived social support. Since the year it was created, the MSPSS has been modified for hundreds of different cultures. The associated 3-factor structure, which is the MSPSS’s original factor structure, was mostly derived in these adaptation investigations. However, in recent years, various MSPSS factor structures (including bifactor-CFA, ESEM and bifactor-ESEM) have also been investigated (Merino-Soto et al., Reference Merino-Soto, Boluarte Carbajal, Toledano-Toledano, Nabors and Núñez-Benítez2022; Osman et al., Reference Osman, Lamis, Freedenthal, Gutierrez and McNaughton-Cassill2014; Yılmaz Koğar & Koğar, Reference Yılmaz Koğar and Koğar2023). The results of these studies suggest that the 3-factor structure of the scale requires new examination. The motivation for this study came from looking at the novel factor structures investigated for MSPSS utilizing the pooled correlation matrix.
By combining individual studies examining competing models for the dimensionality of MSPSS, the factor structure of MSPSS can be better understood from a more general perspective. In our research, 59 samples from 54 unique articles were used for the database. The average sample size of these 59 samples is 473. This sample size average (N < 500) can produce inaccurate factor loadings (Auerswald & Moshagen, Reference Auerswald and Moshagen2019). Furthermore, depending on the sample, different subgroups (age, gender, clinical sample, etc.) may also have an impact on the factor structure. Therefore, our research aimed to use an MA-CFA to remove any potential influence that sample size and sample subgroups may have had on the factor structure of MSPSS to more correctly evaluate the factor structure of MSPSS. Factor loadings, standardized regression coefficients, inter-factorial correlation coefficients and inter-item correlation coefficients were used in 59 samples to conduct a random-effects meta-analysis for this purpose. In this manner, an item-level pooled correlation matrix was produced. Dimensionality tests performed on the pooled correlation matrix showed that the best representation of the structure was achieved through a three-factor model. However, since there are unidimensional and 2-factor MSPSS factor structure solutions in the literature, 5 different competitive models (details are in the introduction) have been tested with MA-CFA.
The MA-CFA determined that all unidimensional and 2-factor structures (Model 1–Model 3) applied to determine the MSPSS factor structure produced very insufficient model-data fit values. Model-data fits show a perfect fit for both correlated 3-factor CFA (Model 4) and bifactor-CFA (Model 5) models, which is the main focus of this research. Factor loading values for both models are .40 and above. In this case, model-data fit values were compared to determine the most appropriate factor structure. It was decided that the three-factor correlated CFA model with the better model fit was the “actual” factor structure of the MSPSS. These findings are similar to most of the MSPSS’s cultural adaptation studies (e.g., Bruwer et al., Reference Bruwer, Emsley, Kidd, Lochner and Seedat2008; Calderón Garrido et al., Reference Calderón Garrido, Ferrando Piera, Lorenzo Seva, Gómez Sánchez, Fernández Montes, Palacín Lois and Jiménez Fonseca2021; Canty-Mitchell, & Zimet, Reference Canty-Mitchell and Zimet2000). Twenty-one of the cross-cultural studies examined the factor structure of MSPSS with CFA.
It can be said that the MSPSS has a nondominant general factor according to hierarchical omega, hierarchical omega for subscales and ECVs for the bifactor models. Besides, all three factors of the MSPSS are quite strong and dominant. Although the correlation values between the factors are statistically significant, this relationship is at a moderate level. All these findings show that the multidimensional structure of MSPSS still exists today.
Lastly, the internal consistency was high based on the omega values, and the metric MI of the structure of the correlated three-factor model of MSPSS was attained based on clinical status and language. Therefore, it may be concluded that the structure of the correlated three-factor model of MSPSS is strong.
Limitations, future directions and conclusion
The inability to test the ESEM and bifactor-ESEM factor structures for MSPSS is the first and most significant limitation of our research. The bifactor-ESEM model was found to be the model that best characterized the factor structure of the MSPSS in the sole cross-cultural study in which these factor structures were examined (Merino-Soto et al., Reference Merino-Soto, Boluarte Carbajal, Toledano-Toledano, Nabors and Núñez-Benítez2022). This circumstance suggests that a factor structure analysis will be done by taking into account new MSPSS models in various cultural contexts. As a result, more study is required on this topic. It is useful to interpret the results of this study according to the information that all other studies, except the study by Merino-Soto et al., (Reference Merino-Soto, Boluarte Carbajal, Toledano-Toledano, Nabors and Núñez-Benítez2022), did not consider cross-factor loadings. Another drawback is that the publications we used to build our database offer insufficient details and do not employ enough samples for factor analysis. The majority of the publications lacked information on the method of factor extraction they employed. Several articles lack or have insufficient information on certain fundamental demographic factors, such as age and gender. Only 30.5% of the reviewed articles reached the recommended sample size to obtain more stable results in factor analysis (N > 500) (Hirschfeld et al., Reference Hirschfeld, von Brachel and Thielsch2014). This raises questions about the validity of the factor analysis’s conclusions. Another disadvantage of this study is that it utilizes the findings of studies that use factor loadings instead of raw data. It was claimed that the research based on MA-CFA, which is scarce in the literature, used a small amount of raw data (i.e., Gnambs & Staufenbiel, Reference Gnambs and Staufenbiel2016). To effectively perform studies like ours, it is crucial to share the raw data or the inter-item correlation matrix while considering the ethical constraints, especially in cross-cultural investigations. For this reason, it is recommended to include values showing the correlations between the items of the scale in studies examining the psychometric properties of the scales.
The structure of the SO factor of the scale is more controversial than the other factors. The reason for this situation was that the expression “special person” in the items in the SO subscale was not fully understood by the participants (Eker et al., Reference Eker, Arkar and Yaldız2001; Wongpakaran & Wongpakaran, Reference Wongpakaran and Wongpakaran2012). For this reason, additional explanations were given for the items of this subscale and the overlap of the SO items with the other subscale items was tried to be minimized. The way the scale is administered may affect the responses of the participants. Verbally collected responses may provide different information than that collected in a written or online questionnaire, because the practitioner may provide clarifying information to the participant or question the participant to clarify their response (Soszynski & Bliss, Reference Soszynski and Bliss2022). For this reason, future studies that will use the MSPSS may proceed to the data collection phase after explaining the scale to the participants or collect the data verbally. Thus, non-substantive responses of the participants regarding the SO factor can be prevented and the three-factor structure of the scale can be clarified.
The biggest contribution of our study to the literature is that it shows that the MSPSS can better adapt to the three-factor correlated CFA factor structure through a sample of 27,905. More studies are needed on the compatibility of MSPSS with the bifactor-CFA and ESEM structures. The importance of performing statistical analyses correctly is another issue that emerged in this study. While deciding on the number of factors in EFA, a dimensionality analysis like parallel analysis is required. It has been determined that the ML estimation method is generally used in CFA studies. However, studies on estimation methods show that using WLS instead of ML provides more accurate estimations as long as there are enough samples for CFA (Li, Reference Li2016). However, as Bonifay, Lane and Reise (Reference Bonifay, Lane and Reise2017) stated, high model-data fit of bifactor models can be “overfitting.” Overfitting is defined as the tendency of bifactor models to have higher model-data fit compared to other CFA models (Bonifay et al., Reference Bonifay, Lane and Reise2017). Making a profile of the person’s perceived social support with scores from only 3 factors of the MSPSS may lead to a suitable assessment. Considering the situations created by the SO factor, the theoretical structure of the scale can be reviewed and an improvement can be made, especially on the SO factor. Or, more studies can be carried out testing the factor structure of MSPSS with current models. However, the fact that the MSPSS has a robust factor structure in different subsample groups is proof of how strong the scale’s structure is. In this way, MSPSS is a multidimensional scale that measures perceived social support with its different factors, as well as a useful measurement tool that can create a perceived social support profile and can do this with only 12 items.
Open peer review
To view the open peer review materials for this article, please visit http://doi.org/10.1017/gmh.2024.118.
Supplementary material
The supplementary material for this article can be found at https://osf.io/6efq3/?view_only=8e25985f3a15465a908dee36fe4b3f07
Data availability statement
The studies, coding and analyses used in the current research can be accessed by following this link: https://osf.io/6efq3/?view_only=8e25985f3a15465a908dee36fe4b3f07
Author contribution
All authors contributed to the study conception and design. Material preparation and data collection were performed by Author-2; also material preparation, data collection and data analysis were performed by Author-1. All authors read and approved the final manuscript.
Financial support
The authors have no relevant financial or nonfinancial interests to disclose.
Competing interest
The authors declare none.
Ethics statement
An ethics statement is not applicable because this study is based exclusively on published literature.
Open access
Comments
Dear Editor,
Please find attached the paper, The Structure of the Multidimensional Scale of Perceived Social Support: A Meta-Analytic Confirmatory Factor Analysis by Dr. Hakan Koğar and Dr. Esin Yılmaz Koğar, which we would like to submit for possible publication to Cambridge Prisms Global Mental Health. We confirm that this work is original and has not been published elsewhere nor is it currently under consideration for publication elsewhere.
The Multidimensional Scale of Perceived Social Support (MSPSS) is an important measurement tool for which the original scale development study has received over 13000 citations, has been adapted to more than 30 languages, and still examines its psychometric properties over different samples. In this comprehensive study, such an important measurement tool was chosen. In this study, we tried to determine the factor structure of MSPSS in 27,905 people using Meta-Analytic Confirmatory Factor Analysis (MACFA). For this purpose, we used a total of 59 samples from 54 studies. In addition, we tried to obtain factor structures belonging to clinical and non-clinical samples and factor structures belonging to different languages. To the best of our knowledge, this is the first study to conduct MACFA to MSPSS. We think that the study produced important findings. For these reasons, we think that this research falls within the scope of your journal.
This manuscript was previously submitted to your journal with the code JCOP-24-080. However, it was rejected due to high similarity rate. We have made the necessary arrangements in our manuscript and reduced the rate to 10%. We also upload the proof of this to the system.
All supplementary materials here: https://osf.io/6efq3/?view_only=8e25985f3a15465a908dee36fe4b3f07
For any information concerning this manuscript, please contact us preferably by e-mail at [email protected] or [email protected]
Thank you for your consideration of this manuscript.
We look forward to hearing from you soon.
Sincerely yours
Dr. Hakan Koğar
Dr. Esin Yılmaz Koğar