A wide variety of types and definitions of scale exist in geoscience disciplines. Participants in the survey identified ten main limitations of current practices in scale development.
With regard to step 1 of the scale development process, most studies used deductive methods for generating items (literature review and interviews). These approaches often result in a significant loss of items during the final scale construction.
Definitions
A scale is a tool used to measure the weight of objects. A scale is made up of two plates, one on top of the other, which can be weighed against each other to determine which object is heavier. A scale can be found in many places, including kitchens and schools.
Scale is a very important concept for researchers in the geosciences, because patterns are often observed at different spatial and temporal scales. However, it has been shown that there is confusion and ambiguity about the definitions of scale.
The survey asked participants about the importance of the various types of scale and about the corresponding definitions. It was found that participants considered “Cartographic scale” (95%), “Modelling scale” (86%) and “Operational scale” (“Op”) to be important for their work, while the remaining seven types of scale were less well known and ambiguous. Participants also evaluated the corresponding scale definitions by choosing their level of acceptance.
Item Development
The first step in developing a scale is identifying the domain and item generation. This involves a combination of deductive and inductive approaches and consideration of content validity. The domain should be clearly defined and include both theoretically related and unrelated constructs.
Item development should also consider the characteristics of the target population. This can be done through interviews or cognitive interviewing. Items should be clear, easy to understand and respond to, and free of biases such as social desirability.
Item reduction analysis is used to ensure that only parsimonious, functional, and internally consistent items are retained. It is advisable that the initial pool of items developed be at least twice as large as the desired final scale. This will allow for the removal of items that are tangential or unrelated to the domain identified. Often, information collected on sociodemographic questionnaires will correlate with the construct of interest and can be used to identify potential items to be dropped from the initial pool.
Pilot Study
A pilot study is an initial and smaller-scale project conducted before a larger, full-scale research investigation. It serves to refine research questions, objectives and data collection techniques and methodologies. This process can be used in qualitative and quantitative research.
Among other things, pilot studies can help to evaluate sample size requirements and determine if the data will be useful in estimating intervention effect sizes and variability for power calculations for a future large-scale study. Other purposes of a pilot study may include testing a new measurement instrument, determining if a particular design will be feasible (e.g. recruitment, randomized allocation procedures, implementation of interventions and maintaining blinded assessments) or to assist in convincing funding bodies that the proposed research is worth their investment.
Often participants who take part in a pilot study will also be included in the main research. This can influence the results because participants who have already experienced the experimental setting will respond differently than those who are experiencing it for the first time. To account for this potential bias, a sensitivity analysis may be undertaken.
Validation
If a scale’s items are to be added up or averaged into total scores that are intended to represent locations on the latent dimension that represents the construct, it is important for those item scores to be able to predict where each respondent is situated along this latent dimension. This is the essence of validity, and one way to measure this is through internal consistency (e.g., Cronbach’s alpha).
A number of psychometric analyses can be performed to validate a scale and assess its quality. Introductory statistics courses often start at this stage by performing an exploratory factor analysis, and then calculating subscales using the preferred factor solution. These are then correlated to test hypothesized relationships.
More advanced diagnostics such as person separation reliability and a person-item map can also be used to examine the extent to which the scale’s items are able to differentiate individuals at different levels of the latent dimension. Moreover, sensitivity analyses can identify which respondents are not well served by the scale and suggest possible ways of improving it.