Researchers have reported measuring several aspects of participation such as community integration (Royce-Davis, 2001; Wong & Solomon, 2002), engagement (Balcazar, Mathews, Francisco, Fawcett, & Seekins, 1994; Balcazar, Seekins, Fawcett, & Hopkins, 1990; Granlund, Ericksson, & Ylven, 2004), and participation (Heinemann, 2005; Perenboom & Chorus, 2003). To date, however, efforts to evaluate participation have typically used static measures that do not capture the dynamic aspect of individual-environment interaction (Crews & Campbell, 2001). Only a few researchers (e.g., Gray and Whiteneck) have systematically explored how situational or dynamic environmental factors impact immediate participation. Researchers need a more dynamic and comprehensive measurement scheme to adequately answer questions about the interference of participation among people with disabilities.
Currently, several research teams use retrospective survey approaches to measure participation. For example, rates of participation have been evaluated using national data collection instruments, including the National Health Institute Survey Second Supplement on Aging, the National Health Interview on Disability, and the National Organization on Disability (N.O.D.)/Harris Survey of Americans with Disabilities (Crews & Campbell, 2001; Harris Interactive, Inc., 2004). While these established instruments collect baseline data about participation rates of people with disabilities in specific life domains (such as voting, going to restaurants, or attending religious services), they provide a narrow view of participation and do not capture the dynamic interplay between the person and environment.
Other more comprehensive instruments tend to focus on either the environment or participation without blending these two constructs. The Craig Hospital Inventory of Environmental Factors (CHIEF) was developed to assess “perceived physical, attitudinal, and policy barriers” that prevent full participation by people with disabilities. The five domains of the 25-item CHIEF include environmental barriers related to policies, physical and structural space, attitudes at work and school, attitudes in the community, and services and assistance within the community (Whiteneck, Meade, Dijkers, et al., 2004). The CHIEF asks respondents to provide information about the prevalence (daily, weekly, monthly, less than monthly, and never) and degree (big problem or little problem) of each stated barrier. The CHIEF was administered to 2,762 people with spinal cord injury to examine the effect of environmental barriers on participation. Results indicated that 4% of the variance in participation rates and 10% of the variance in satisfaction with participation was attributable to environment factors (Whiteneck ,et al., 2004).
The Craig Handicap Assessment and Reporting Technique (CHART, Whiteneck, et al., 1992) originally was designed to assess how individuals with disabilities fulfill societal roles, it includes five subscales to measure independence, mobility, employment, social relationships, and economic self-sufficiency. A more recent version also includes a subscale to measure “cognitive independence” for a total of 32 questions. The CHART asks several questions related to participation in work, education, homemaking, home maintenance, volunteering, recreation, and self-improvement activities. Respondents indicate how many hours in a typical week they spend engaged in each of these activities. The CHART also asks questions about companionship at home and in the community (as measured by numbers of visits, phone-calls, or written notes per month with relatives, business associates, or friends). The CHART collects meaningful participation data, but doesn’t describe how the environment impacts these engagements or how people feel about their overall participation rates in life domains. And, again, these instruments rely on the memory of the individual to report accurately at some period of time after an event.
The Community Integration Questionnaire (CIQ) developed by Willer, Ottenbacher & Coad (1994) includes 15 items to measure independence in the home (such as meal preparation, housework, childcare, social planning, and personal finance), social integration (such as number of times per month the individual shopped outside the home, engaged in leisure and recreational activities, visited friends), and productive activities (such as travel, employment, education, and volunteer activities). The CIQ was developed specifically for individuals with traumatic brain injury and has not been validated. Like the CHART, the CIQ does not examine how environmental factors shape participation rates and it uses a recall survey approach.
More recently, David Gray and colleagues began blending participation and environmental factors with a two-part measurement protocol, including the Participation of People with Impairments and Limitations Survey (PARTS-G) and the Facilitators and Barriers Scale (FABS; Gray et al, 2007). The PARTS-G asks questions related to frequency, duration, limitation, choice, satisfaction, importance, required assistance, and accommodations to participate in five domains that link with the ICF Activities and Participation chapters (Gray, Hollingworth, Stark, & Morgan, 2006). The FABS asks respondents to recall and rate facilitators and barriers in specific environments (such as restaurants) in terms of access features, attitudes, or personal factors (such as financing of special equipment). The combination of these measures provides a comprehensive look at the individual-environment interaction, but it is difficult to parse out which factors influence the choice to participate in given activities. This is particularly true, given the global nature of the question formats in both the PARTS-G and FABS instruments.
One way to address the gaps associated with static measures is to evaluate participation as it occurs, across time and in multiple environments. Ecological Momentary Assessment (EMA) involves recording situations and events as they occur. Usually, EMA involves an individual recording events on an electronic device (e.g., IPod Touch) either as defined events occur (e.g, urge for a cigarette) or in response to scheduled (e.g, random) prompts. Unlike retrospective measures that are susceptible to recall bias, EMA is designed to capture stimuli within time and place – or within the relevant environment (Barrett & Barrett, 2000; Csikszentmihalyi & Larson, 1987; Smyth, Wonderlich, Crosby, Miltenberger, Mitchell, & Rorty, 2001; Stone & Shiffman, 1994, 2002). As such, EMA is an appropriate data collection strategy to examine immediate response to a setting and offers an innovative method for examining participation interference.
With the advent of computerized prompting and electronic diaries (ED), EMA versatility has grown. In particular, EDs offer data collection and prompting features that are easily accessed in the immediate environment. Using EDs, participant data can be collected at set intervals (interval contingent), at random intervals throughout the day (i.e., signal contingent), or in response to specific events (i.e., event contingent; Barrett & Barrett, 2000). Figure 1 includes a screen shot of a typical EMA questions.
Figure 1 EMA electronic diary screen that uses a "slider" for assessing pain level
Gaps in Methods and Measures
A criticism of the ICF framework is that it does not account for the dynamic individual-environment interaction, which can “fluctuate depending on condition, time, and setting” (Tate and Pledger, 2003, p.290). Current measures rely on retrospective recall for both rates of participation and barriers in the environment. This makes it methodologically difficult to capture accurate situational response. Current measures are not sensitive to the dynamic interplay between the activity, the environment, and personal factors. A better understanding of these dynamic factors could contribute to a variety of interventions that promote participation by people with disabilities.
Static measures that ask respondents to recall their status and activities over some period of time are subject to recall bias (Stone & Shiffman, 1994). Methodological studies demonstrate large discrepancies between momentary and recall assessment (Piasecki, Huffors, Solhan, & Trull, 2007). Paper and pencil diaries have been used to reduce recall bias, but they also have problems. For example, systematic recording errors are common including responses that are “back filled” immediately prior to diary collection (Piasecki, et al. 2007). Because many factors affect recall of pain (e.g., emotional state at recording, peak pain and pain experiences immediately prior to recall) momentary data collection methods provide important advances for studying pain (Gendreau, Hufford, & Stone, 2003).
Ecological momentary assessment. EMA overcomes many of these problems for recalling dynamic events like pain episodes and their effect on participation (Gendreau, et al. 2003; Stone, Shiffman, Schwartz, Broderick, & Hufford, 2002). Because diary recording is burdensome for respondents, creating simple, easy-to-use electronic interfaces decreases that burden. Additionally, because the ED can be used to prompt responses, “forgetting” to record experiences and then “back filling” is much less problematic. A recent study of compliance rate for paper diaries found results consistent with other reports; 90% of entries were completed. However, comparisons between reported and actual compliance suggested that 79% of the entries had been “back filled” long after the recording period (Stone, et al., 2002). In contrast, compliance with the EMA procedure is very high with estimates for data entry compliance ranging from a low of 82% in a study of the general population (Tennen, Affleck & Zautra, 2006) to a high of 94% of chronic pain patients (Gendreau, et al., 2003).
The EMA method has participants carry an iPod or similar device that has been pre-programmed to signal at random intervals throughout the day (across the study period). The individual responds to prompts by accessing a questionnaire on the iPod and selecting choices about the current status of several variables using icons and “drop down” menus. These data are stored on the internal memory of the iPod. At the end of an observation period (i.e. 14 days), these data are uploaded remotely to a dedicated computer for analysis.
For the purposes of this study, we defined participation as a series of engagements between an individual and his or her environment, where engagements are instances of activity within an ecological context. For example, one engagement might occur at a restaurant with coworkers, eating dinner, where the individual was not experiencing any barriers to participating in the activity. Another engagement might occur at home, alone, performing a household chore, while the individual is experiencing pain. Participation, then, varies along dimensions of the rate of engagements, the variety of contexts in which the individual is engaged, and the duration and intensity of activities. Engagements can be analyzed independently to explore how particular types of places, activities, or social contacts are associated with environmental barriers such as access issues, or personal factors such as pain. Data can be analyzed for each individual to present a portrait of individual participation that is later aggregated across individuals to detect common patterns. Further, data can be analyzed across individuals for group comparisons (such as individuals with multiple sclerosis versus those with spinal cord injury).