UNESCO defines literacy as “the ability to identify, understand, interpret, create, communicate and compute, using printed and written materials associated with varying contexts.” 1 It is also emphasized in the literature that the concepts of general literacy and disaster literacy are different branches of literacy 2 or that the concepts associated with other types of literacy need to be studied on the basis of health literacy.Reference Çalışkan and Üner 3 While health literacy is denoted through many definitions in the literature, the concept of disaster literacy is still in its infancy. It is generally believed that deploying the approaches utilized in health seems appropriate for disaster literacy, just like other types of literacy.Reference Çalışkan and Üner 3 Disaster literacy is also defined in the literature under more specific categories, such as natural disaster literacy,Reference Brown, Haun and Peterson 4 disaster prevention literacy,Reference Türker and Sözcü 5 disaster mitigation literacy,Reference Tsai, Linliu and Chang 6 and disaster health literacy.Reference Rumambi, Tangkudung and Assa 7 Since covering all phases of mitigation, preparedness, response, and recovery, the concept/model of disaster literacy can conveniently be used in studies involving disaster management.Reference Çalışkan and Üner 3 Çalışkan and Üner define disaster literacy as “the capacity of accessing, understanding, evaluating, and implementing disaster information to make informed decisions in daily life and follow instructions regarding the mitigation/prevention, preparedness, response and recovery/rehabilitation of a disaster to be able to protect or improve one’s quality of life throughout their lives.”Reference Seifi, Ghanizadeh and Seyedin 8
The disaster-causing damage that directly or indirectly affects the entire population varies by the characteristics of that population (gender, age, etc.).Reference Çalışkan and Üner 9 The rapidly increasing older adult population is more vulnerable to disasters and, therefore, suffers more during disasters than the general population, suggesting that the research on disasters needs to consider innovative approaches for the vulnerable groups in the general population, particularly older adults.Reference Çalışkan and Üner 3 Although the literature offers a plethora of studies on the social vulnerability of older adults,Reference Atwill, Sandvik and Kirch 10 –Reference Ayeni, Sharples and Hewson 12 they seem not to reach a consensus on the conceptual framework, dimensions, and affecting factors of their social vulnerability.Reference Mah, Rockwood and Stevens 13 In addition to the geographical location, culture, and economic hardship, the moral losses frequently encountered among older adults exacerbate their vulnerability to disasters.Reference Daddoust, Khankeh and Ebadi 14 Thus, clear, direct, and consistent messages should be conveyed to create disaster literacy at all levels in vulnerable populations. Yet, transmitting such messages seems only possible by first determining the need and then deploying appropriate techniques. Efforts to satisfy one’s disaster literacy needs may carry their points thanks to evaluating disaster literacy through various practices (identifying measurement tools and criteria or developing communication materials). The research on measurement, monitoring, and development of disaster preparedness among older adults seems limited in number and content. Nevertheless, it is key to determine and contribute to disaster literacy among older adults to be able to project the current situation and reduce their vulnerability.Reference Çalışkan and Üner 3 Although disaster literacy is considered to be associated with ensuring the safety of older adults in disasters,Reference Çalışkan and Üner 3 the literature does not describe any measurement tool oriented to measuring disaster literacy among older adults. Thus, the present study attempted to adapt the Disaster Literacy Scale (DLS), developed by Çalışkan and Üner in 2022, to those aged 60 years and older and to reveal its psychometric properties.
Methods
Design
This methodological study aimed to adapt the DLS, introduced for individuals aged 18-60 years, to those aged 60 years and older and to determine its psychometric properties.
Content Validity
Çalışkan and Üner developed the 61-item DLS for individuals aged 18-60 years to be utilized in the Turkish context.Reference Seifi, Ghanizadeh and Seyedin 8 In this study, we investigated whether the scale would be deployed for individuals aged 60 years and older as the “Disaster Literacy Scale – Seniors (DLS-S).” Although the scale is oriented to the Turkish context in both cases, different age groups are targeted in the original study and the current research. In the original study, the authors finalized the scale following the relevant stages of developing a measurement tool (deciding on the structure, generating an item pool, determining the measurement method, preparing the instructions, and resorting to expert opinions for the items).Reference Seifi, Ghanizadeh and Seyedin 8 To seek the DLS-S’s content validity, we also resorted to expert opinions to explore the suitability of the items for the targeted age group. Accordingly, we submitted the 61-item instrument to a total of 10 experts in various fields via e-mails. The experts were requested to assess the suitability of the items and tick one of the options on an evaluation form: “the item fits the context and should remain in the item pool,” “the item is useful but not sufficient,” and “the item needs to be removed.” Moreover, we spared a blank input box for each item to allow the experts to provide further remarks on the items. While evaluating the forms, we only considered the items for which all the experts ticked the option “the item fits the context and should remain in the item pool.” Finally, we calculated the Content Validity Ratio (CVR) and Content Validity Index (CVI) values for each of the items considered “necessary.”
Pilot Study
Prior to the research, we went through all the items considering the individuals aged 60 years and older and carried out a pilot study with a group of 20 participants, similar to the designed sample, to test the intelligibility of the items.
Sample
We carried out the study with a sample of 60 years and older living in the city center of Burdur, Turkey. To study the psychometric properties of an instrument, the relevant literature suggests reaching individuals 10 times the number of items in the scale or recruiting a sample size of 300-400 people regardless of the number of indicators per factor.Reference Çokluk, Şekercioğlu and Büyüköztürk 15 In the literature, a sample size of 400 is also attributed as “good” and 500 people as “very good.”Reference Karagöz 16 Accordingly, we targeted to reach 500 people and collected the data from 550 people considering possible inconveniences on the data (e.g., missing or erroneous responses). Because 22 participants gave consistently repetitive responses to the items, we recruited the responses of 528 participants in the analyses.
The sample was selected using snowball sampling, a non-probability sampling technique. The study included only those living in the city center of Burdur, being 60 years and older, and being able to understand the items and express their thoughts. Additionally, to perform test-retest analysis in the research, we administered the same instrument to 135 participants, who provided their contact information and agreed to participate in the second round of data collection under the same conditions after 2 or 3 weeks of the first round of data collection. The inclusion criteria were that the participants’ place of residence was Burdur, they were 60 years of age or older, and they could understand the questions and express their thoughts. The exclusion criteria were that the participants resided outside Burdur province, were under the age of 60, and had difficulty in understanding the questions and expressing their thoughts. In the first sample, 16 people were excluded during data collection. Because the people sampled for the second time were selected from the first sample, all of them were included in the study.
We collected the data between December 15, 2021 and April 18, 2022. After obtaining their verbal informed consent, we distributed a questionnaire booklet to the participants in public places (streets, squares, workplaces, and cafes) in the city center. We helped those having difficulty understanding some parts of the instruments and responded to their questions. Following data collection, those leaving missing items were requested to respond to the missing items.
Data Collection Tools
In this study, we collected the data using a questionnaire booklet covering an 11-item demographic information form and the DLS.
The DLS is a self-report measurement tool designed by Çalışkan and Üner to measure disaster literacy among Turkish individuals aged 18-60 years. The conceptual framework of the instrument covers a 4*4 matrix structure designed as 4 dimensions (mitigation/prevention, preparedness, response, and recovery/rehabilitation) and decision-making and obtaining information about disaster-related practices (access, understand, appraise, and apply the information); therefore, the conceptual framework of this 61-item scale targets 16 domains. Accordingly, the statements in each of these 16 cells of the matrix were designed to measure the capacity of individuals to read, understand, and utilize the information by following the guidelines for the mentioned 4 stages of disasters: mitigation/prevention (items 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, and 17), preparedness (items 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, and 33), response (items 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, and 46), and recovery/rehabilitation (items 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, and 61). The items are scored on a 5-point Likert-type scale ranging from 1 (very difficult) to 5 (very easy), and there is no reverse-coded item in the scale. The higher scores on the tool indicate an increased level of disaster literacy. The total score is standardized to take values between 0-50 for ease of calculation; 0 point indicates the lowest disaster literacy level, while 50 points suggest the highest disaster literacy level. According to this formula, the authors determined the cut-off scores for the 4 subscales based on the Standard Deviations (SD). Accordingly, 0 - < 30 points refer to inadequate disaster literacy, 30 - < 36 points indicate limited disaster literacy, 36 - < 42 points show adequate disaster literacy, and 42-50 imply excellent disaster literacy.Reference Seifi, Ghanizadeh and Seyedin 8
Data Analysis
We utilized reliability (internal consistency reliability and test-retest reliability) and validity [CVR, CVI, and Exploratory Factor Analysis (EFA)] analyses in the data analysis. The normality of distribution was checked using the skewness and kurtosis values. The analyses were performed on SPSS 23.0, and a P value < 0.05 was considered statistically significant.
Ethical Considerations
The Ethics Committee of the relevant university granted ethical approval to our study (No: GO 2021/404 dated December 1, 2021), and we obtained verbal consent from the participants. Moreover, we obtained written permission from the developers via e-mail to study the psychometric properties of the DLS in participants aged 60 years or older.
Results
Sample Characteristics
In this study, while 58.0% (306) of the participants were women, 47.9% (253) had primary school education. The mean age of the participants was found to be 68.29 (SD = 6.36).
Validity
We considered CVR and CVI values and the results of the EFA in the scope of the validity analyses of the DLS. Initially, we calculated CVR and CVI values for each subscale to explore the validity of the scale. Accordingly, the 61-item measurement tool was reduced to 57 items by removing a total of 4 items from the scale, 2 items with a CVR of zero (0) or negative (less than zero) and 2 items with a CVR less than 0.62. We also discovered that the CVI values of these 57 items varied between 0.70 and 0.90 (mean = 0.78) and that the construct was statistically significant. Before the EFA, we also considered the presence of a multicollinearity problem in all subscales. Accordingly, we calculated the inter-item correlations to vary between 0.15 and 0.85 for the mitigation/prevention items, between 0.24 and 0.87 for the preparedness items, between 0.42 and 0.88 for the response items, and between 0.36 and 0.85 for the recovery/rehabilitation items. In general, the items yielded moderate inter-item correlations. The lack of high correlations suggested that the instrument did not raise a multicollinearity issue.
Then, we performed EFA to explore the construct validity of the DLS for older adults. As accepted in the original study, we sought that the factor loadings of the items should not be less than 0.50 and that the difference between the factor loadings of any 2 items should be a maximum of 0.15 to avoid overlapping items. The EFA was performed for each subscale following the order in the original study. The findings for the mitigation/prevention, response, and recovery/rehabilitation yielded a 3-factorial structure. Then, we re-examined the distribution of the items in these factors by increasing the number of factors to 4 in the “Factor Analysis: Extraction” section (in accordance with the original study and within the knowledge and suggestion of the corresponding authors). While pre-determining the factors, we accepted the eigenvalue to be 1 for the preparedness subscale; however, the mentioned value was ignored because the number of factors was entered manually for other subscales. We chose the direct oblimin rotation technique, an oblique rotation technique, to obtain more evident factors and discovered that the items were loaded within their own constructs. Accordingly, we found the total variance explained for the mitigation/prevention subscale to be 51.56, for the response subscale to be 59.94, and for the recovery/rehabilitation subscale to be 58.78 (Table 1). In the EFA for the preparedness subscale, we initially obtained a 4-factorial structure, discovered the items to load into their own constructs, and calculated the total variance explained for the subscale to be 89.90 (4 sub-matrix with an eigenvalue greater than 1: access [52.85], understand [17.78], appraise [10.91], and apply [8.35]).
Table 1. EFA results for the DLS-S
In the EFA, the Kaiser-Meyer-Olkin (KMO) coefficient was calculated to be 0.91 for the mitigation/prevention subscale (P < 0.001), 0.89 for the preparedness subscale (P < 0.001), 0.89 for the response subscale (P < 0.001), and 0.90 for the recovery/rehabilitation subscale (P < 0.001) (Table 1). The communality was accepted as 0.500 for all subscales, and we found no item falling below this value on any of the subscales.
Reliability
We considered item statistics, internal consistency reliability (Cronbach’s α), test-retest reliability, and Tukey summable test results for reliability concerns of the DLS-S.
Considering item statistics, we discovered the lowest item-total correlation to be 0.38 in item 10 and the highest item-total correlation to be 0.77 in items 31 and 48. The item-total correlations ranged between 0.38 and 0.73 for the mitigation/prevention subscale, between 0.55 and 0.77 for the preparedness subscale, between 0.62 and 0.75 for the response subscale, and between 0.55 and 0.77 for the recovery/rehabilitation subscale (Table 2).
Table 2. DLS-S item statistics and reliability values
* Standard deviation.
a Although the relevant item was included in the DLS, it was excluded from the DLS-S considering CVI and CVR values in this study.
We also calculated Cronbach’s α values to be 0.936 for the mitigation/prevention subscale, 0.935 for the preparedness subscale, 0.939 for the response subscale, and 0.945 for the recovery/rehabilitation subscale (Table 3).
Table 3. Reliability values of the DLS-S items
The Tukey summable test results yielded P = 0.721 for the mitigation/prevention subscale, P = 0.004 for the preparedness subscale, P = 0.019 for the response subscale, P < 0.001 for the recovery/rehabilitation subscale, and P < 0.001 for the total score.
Finally, we calculated the test-retest correlations to be 0.789 for the mitigation/prevention subscale (P < 0.001), 0.888 for the preparedness subscale (P < 0.001), 0.685 for the response subscale (P < 0.001), and 0.671 for the recovery/rehabilitation subscale (P < 0.001).
Scoring
Upon calculating the z-scores of the items remaining in the item pool, we determined the cut-off points for the DLS-S as follows: SS - 1 and below, SS - 1 and 0, SS 0 and + 1, and SS + 1 and above. For the concern of the ease of calculation, we standardized the total score using the following formula,Reference Seifi, Ghanizadeh and Seyedin 8 which would take a value between 0 and 50:
Formula = Index = (arithmetic average – 1) x (50 / 4)
Index = Index original to the calculated person
Arithmetic average = Average responses to each item
1 = Lowest possible value of the average (causes the lowest index to be 0)
4 = Average range
50 = Highest selected value for the new criteria
The lowest DLS-S score = 0
The highest DLS-S score = 50 (Table 4).
Table 4. Cut-off points of the DLS-S scores and their equivalents in the 50-point system
Discussion
Literacy is known to be directly affected by one’s other skills (particularly cognitive skills), the group effect in a specific period (e.g., education may not be offered at consistent quality to all generations due to the radical changes to the education system from time to time), and the period effect (e.g., the entire population’s exposure to war or a famine). Independent of time and geographical location, it is not surprising that one’s skills vanish with aging, that older generations have lower or insufficient educational attainment, and that older adults are more affected by adverse situations such as war. Regardless of age, literacy is highly variable depending on biological, behavioral, environmental, and social factors. Therefore, there is a need for measurement tools that cover the same age, group, and period effects on literacy and that enable the same population to be comparable at different times and internationally.Reference Reiter 17 Recent years have witnessed the introduction of many measurement tools to measure older adults’ health literacyReference Huang, Ruan and Yi 18, health literacy oriented to dentistry practicesReference Mialhe, Tenani and De Checchi 19, internet-computer literacyReference Berman 20, and electronic health literacy.Reference Chung and Nahm 21 In this sense, the importance and acceleration of research on the literacy of older adults in recent years suggest that it may also be noteworthy to explore their disaster literacy. However, there is a paucity of research on the disaster literacy of older adults in the relevant literature. Similarly, the literature does not offer a measurement tool designed to measure older adults’ disaster literacy.
Çalışkan and Üner developed the DLS to measure the disaster literacy of the general population in Turkey (individuals aged 18-60 years).Reference Seifi, Ghanizadeh and Seyedin 8 In this study, we explored whether the scale can be utilized in individuals aged 60 years and older and tested its psychometric properties.
Within the validity analyses of the DLS-S, we first submitted the scale to expert opinions. It is often stated that the smallest CVR for a Likert-type measurement tool evaluated by 10 experts should be 0.62 (P < 0.05)Reference Alpar 22 (0.78 in this study). Following CVR and CVI calculations and expert opinions, we removed a total of 4 items from the scale, 1 from each subscale (item 6 on the mitigation/prevention subscale: Accessing information about the training of disaster volunteers), item 20 in the preparedness subscale: (Accessing the training of first responders to disasters [e.g., police, first-aid personnel, and fire brigade personnel]), item 38 on the response subscale (Understanding the importance of institutions’ disaster equipment kits with items such as medicine, medical supplies, food, and clothing), and item 49 on the recovery/rehabilitation subscale (Accessing the information of what units the disaster and emergency management center staff is divided into in a disaster area). We also revised some items upon expert opinions item 17 (Identifying the citizens needing help around before a disaster), item 34 (Accessing evacuation information if relevant institutions raise warnings and alerts about the danger immediately before the start of the disaster), and item 54 (Understanding what the assistance needs of citizens with special needs might be in the aftermath of a disaster). The final version of DLS-S consists of 57 items on a 5-point Likert-type scale.
The findings revealed that the inter-item correlations were all below 0.90, indicating no multicollinearity problem.Reference Alpar 22 When it comes to the factor analysis, we did not find any item with a factor loading below 0.32, accepted as the cut-off value, on any of the subscales.Reference Daddoust, Khankeh and Ebadi 14 The original study accepted the minimum item factor loading as 0.50, and we discovered the factor loadings of all the items in this study were above that value. Because the number of factors was manually passed as “4” for the mitigation/prevention, response, and recovery/rehabilitation subscales in this study, the condition of an eigenvalue of 1 and above in the literature could not be met for these subscales.Reference Yoo, Mayberry and Bae 23 Such a situation applied only to the recovery/rehabilitation subscale, and the number of factors was passed manually in the original study. Because the factorial structure and number of factors for the scale were clearly explained in the original study, the authors suggested manually passing the number of factors for the mentioned 3 subscales and examining the distribution of item factor loading in this way. Our factor analysis yielded that the items were all clustered under the relevant factors.
For reliability concerns, we initially examined Cronbach’s α values and item-total correlations on the DLS-S. The findings revealed that the item-total correlations of all items were above the cut-off value of 0.20Reference Seçer 24 and that Cronbach’s α values of the subscales were above 0.90.Reference Alpar 22 Next, we considered the test-retest reliability coefficients. Accordingly, while we calculated a strong test-retest reliability for the mitigation/prevention and preparation subscales, it remained moderate for the response and recovery/rehabilitation subscales. Thus, our findings overlap with the cut-off points specified in the literature (0.40-0.69 = moderate correlation and 0.70-0.89 = strong correlation).Reference Alpar 22
Similar to the original study, we also attempted to make a standardization for scoring between 0 and 50 points for the ease of comparison of the measurement results. In the standardization, we designed the cut-off points as “inadequate, limited, adequate, and excellent,” similar to the original study.Reference Seifi, Ghanizadeh and Seyedin 8 Yet, the cut-off points of the standardized scores in the original study were higher than those determined in this study, which may be because older adults may have had difficulties in understanding the concepts of disaster, the scale items may be both too long and high in number, the participants may have become distracted from the scale after a certain period of time, or they may not have wanted to spend a long time with the instrument due to their age.
Some of the limitations uttered in the original study, and even more, also apply to the present study. In the DLS-S framework, the subscales and the sub-indices (access, understand, appraise, and apply) were given one after the other, which may have caused a sorting effect among the participants. Secondly, all of the statements on the scale are positive, which may have hindered testing the response credibility.Reference Seifi, Ghanizadeh and Seyedin 8 In addition, disaster literacy is an infant concept that has just started to be discussed in the literature, and there is still no standard/parallel test to measure disaster literacy among older adults. Therefore, we could not test the parallel forms reliability of the DLS-S in this study.
Moreover, the length of the statements on the items, the large number of the items, and the repeated explanation on some items - the prolongation of the filing out the scale - can be considered among other limitations to the study. In addition, the practices reported in some items must be literally performed physically or cognitively. In this regard, the participants thought that they could not perform such practices due to some reasons (e.g., age, physical competence, cognitive competence, or financial competence), which may be another limitation to this study.
Conclusion
While the extent of the effects of disasters varies by country, vulnerable groups suffer disasters the most. In this sense, disaster literacy becomes key among older adults, considered a vulnerable group, in terms of determining their level of preparedness for disasters, drawing the boundaries of the intervention to be initiated by themselves or relevant institutions, and predicting the consequences of disasters on them.
In this study, we explored the use of DLS, designed for the general population, for individuals aged 60 years and older. Accordingly, the 61-item 5-point Likert-type DLS was found appropriate to individuals aged 60 years and older as the 57-item 5-point Likert-type DLS-S. Similar to the original study, we designed a 50-point scoring system and cut-off points (0 - < 21 points = inadequate literacy, 21 - < 29 points = limited literacy, 29 - < 38 points = adequate literacy, and 38 - 50 points = excellent literacy).
As adapted in this study, the DLS-S can be validly and reliably used for individuals aged 60 years and older. Yet, difficulties in understanding the concepts with technical terms on the scale, beliefs about the inability to perform the practices uttered in the items, short attention span and sudden distractions, and unwillingness to spend a long time responding to the scale items are believed to make administering the scale difficult. We, therefore, recommend introducing brief, clear instruments to measure the disaster literacy of older adults by adhering to the original structure of the 4*4 matrix.
Declaration of Conflicting Interest
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.
