Read Baum_et_al.pdf text version

Reliability, Validity, and Clinical Utility of the Executive Function Performance Test: A Measure of Executive Function in a Sample of People With Stroke

Carolyn M. Baum, Lisa Tabor Connor, Tracy Morrison, Michelle Hahn, Alexander W. Dromerick, Dorothy F. Edwards

KEY WORDS · executive function · instrumental activities of daily living · performance assessment · stroke

This study examined the reliability and validity of the Executive Function Performance Test (EFPT). The EFPT assesses executive function deficits in the performance of real-world tasks. It uses a structured cueing and scoring system to assess higher-level cognitive functions, specifically initiation, organization, sequencing safety and judgment, and task completion. Seventy-three participants with mild to moderate stroke and 22 age- and education-matched controls completed the 4 EFPT tasks (cooking, using the telephone, managing medications, and paying bills). Significant differences were found between participants with mild and moderate stroke and healthy control participants. The EFPT can help occupational therapists determine the level of support needed by people with cognitive impairments to perform complex instrumental tasks. Objective information derived from this assessment is an essential part of the process of determining whether the person can live independently and helping families understand how to support the performance of their family members at home.

Baum, C. M., Connor, L. T., Morrison, T., Hahn, M., Dromerick, A. W., & Edwards, D. F. (2008). Reliability, validity, and clinical utility of the Executive Function Performance Test: A measure of executive function in a sample of people with stroke. American Journal of Occupational Therapy, 62, 446­455.

Carolyn M. Baum, PhD, OTR, is Professor and Elias Michael Director, Program in Occupational Therapy, Washington University School of Medicine, 4444 Forest Park Avenue, Campus Box 8505, St. Louis, MO 63108; [email protected] Lisa Tabor Connor, PhD, is Assistant Professor, Program in Occupational Therapy and Departments of Radiology and Neurology, Washington University School of Medicine, St. Louis, MO. Tracy Morrison, OTD, OTR, is Research Assistant Professor, School of Allied Health, Occupational Therapy Education, University of Kansas Medical Center, Kansas City, KS. Michelle Hahn, MSOT, OTR, was previously at the Program in Occupational Therapy, Washington University School of Medicine, St. Louis, MO. Alexander W. Dromerick, MD, is Associate Professor, Georgetown University School of Medicine and National Rehabilitation Hospital, Washington, DC. Dorothy F. Edwards, PhD, is Associate Professor, Department of Kinesiology, University of Wisconsin, Madison..

E

xecutiveabilities,orhigher-levelcognitivefunctions,enablepeopletosuccessfullyformulategoals,planhowtoachievethem,andcarryouttheplanseffectively(Kaye,Grigsby,Robbins,&Korzun,1990;Lezak,1982,2004;Stuss,1992). Theabilitytomakedecisions,self-correct,andusejudgmentenablestheperformanceofcomplexactivitiesofdailyliving(ADLs)essentialforfunctionalindependence(Burgess,2000;Burgess,Alderman,Evans,Emslie,&Wilson,1998;Burgess etal.,2006;Crawford&Channon,2002;Goel,Grafman,Tajik,Gana,&Danto, 1997;Lezak,1982,2004).Thesehigher-ordercognitiveabilitiesunderlieandsupportdailylifeperformance;theirimpairmentorlosscompromisesaperson'sability tofullyparticipateinsociety(Goeletal.,1997;Katz&Hartman-Maeir,2005; Lezak,1982). Executive functions traditionally have been assessed through standardized psychometric measurements administered in controlled environments (Lezak, 1995).Althoughneuropsychologicalmeasuresprovidegoodindicatorsoffundamentalcognitiveandexecutivecomponents,performanceonneuropsychological testsoftenisnotpredictiveofreal-worldcomplextaskperformanceandfunctional ability(Alderman,Burgess,Knight,&Henman,2003;Burgessetal.,2006;Gioia &Isquith,2004;Keil&Kaszniak,2002;Shallice&Burgess,1991;Wilson,1993). Dailylifeperformanceandtheexecutiveabilitiesthatsupportitoftenrequiremultitaskingandthegenerationandimplementationofadaptivestrategiestoaccommodatetonovelenvironmentsandperformtasksintherealworld(Manchester, Priestley,& Jackson, 2004).Assessmentof thefull array of executive functions

July/August 2008, Volume 62, Number 4

446

necessaryforcomplexlifetasksrequirestestingbeconducted inreal-worldenvironments(Burgessetal.,2006;Goverover, 2004).Althoughneuropsychologistsacknowledgethatrealworldtasksaresensitivetobraindamage,theyhaveindicated thatthetestingprocessis"toounwieldy"tobeapartofa routineneuropsychologicalevaluation(Godbout,Grenier, Braun, & Gagnon, 2005). Occupational therapists may insteadprovidethisinformationiftheyhavevalidtoolsto measurereal-worldexecutiveperformance. Traditionally,occupationaltherapistshavebeenasked to determine a person's capacity to be safe, live independently,begainfullyandpurposefullyemployed,andparticipateinmeaningfulactivities.Toaddressthoseissues,occupational therapists assess everyday task performance to determine strengths, limitations, and challenges that the personwithcognitiveimpairmentwillfaceinperforming ADLs (Baum & Edwards, 1993). As we become able to answerquestionsaboutpeople'soccupationalperformance needs,ourteamhaselectedtostudyeverydaylifeissuesfrom theperspectiveofthePerson­Environment­Occupational PerformanceModel(Baum&Christiansen,2005),which looks at both the intrinsic and the extrinsic factors that enableoccupationalperformanceandparticipation.Eachof thesefactorsmustbestudiedinthecontextofeverydaylife performance.Thecognitiveabilitiesofparticularinterestin evaluatingoccupationalperformanceincludeinitiation,the process that precedes the performance of a task (DePoy, Maley,&Stranraugh,1990;Kayeetal.,1990;Lezak,2004; Weld&Evans,1990);organization,thephysicalarrangementoftheenvironment,tools,andmaterialstofacilitate efficientsequencingoreffectiveperformance(Lezak,2004; Weld&Evans,1990);judgment(Goeletal.,1997;Lezak, 1982);andtask completion(Goeletal.,1997). Ascrucialasexecutiveprocessesarefordailyfunction, fewassessmentshavebeenspecificallydesignedtocapture themineverydaylife(Burgessetal.,2006;Goeletal.,1997; Lezak,1982).Recently,interesthasincreasedinthedevelopment of real-world structured tasks designed to capture executivedeficitsthroughobservationofsingleormultiple complextasks(Baum&Edwards,1993;Bechara,Damasio, Damasio,&Anderson,1994;Gaudette&Anderson,2002; Goel et al., 1997; Shallice & Burgess, 1991). Real-world performance tests are ecologically valid because they use naturalisticenvironments,donotprovideartificialstructure, requiremultitasking,andreflectthepressofeverydaytask performances(Baum&Edwards,1993;Becharaetal.,1994; Burgessetal.,2006;Gaudette&Anderson,2002;Goelet al.,1997;Shallice&Burgess,1991).Thedevelopmentof these practical tests is critical to the field of occupational therapy and to cognitive rehabilitation because clinicians mustassesstheirclients'potentialtoreturnhomeandregain

The American Journal of Occupational Therapy

functionalindependence(Baum&Edwards,1993;Gaudette &Anderson,2002). Occupational therapists have developed several assessmentstomeasureperformanceofinstrumentaltasks,includingtheAllenCognitiveLevelsTestBattery(Allen,Earhart,& Blue,1992)andtheAssessmentofMotorandProcessSkills (Fisher, 1993). Although these validated measures indicate problems a person is experiencing in cognitive and process skills during performance of an everyday task (Baum & Edwards,1993;Fisher,1993;Linden,Boschian,Eker,Schalen, &Nordstrom,2005;Mercier,Audet,Hebert,Rochette,& Dubois,2001;Nygard,Bernspand,Fisher,&Winblad,1994; Park,Fisher,&Velozo,1994;Secrest,Wood,&Tapp,2000; Velligan,True,Lefton,Moore,&Flores,1995),theydonot recordtheperson'scapabilitieswhenprovidedwithprogressive levelsofsupport.Athirdmeasure,theKitchenTaskAssessment (Baum&Edwards,1993),whichassessesthecapacitytoperformasimplecookingtask,evaluatestheexecutivefunctions ofinitiation,organization,sequencing,safetyandjudgment, andcompletion,anditrecordscapacitiesexhibitedwithprogressivesupport.Itsprimarylimitationisthatitassessesperformanceononlyasingletask. The measure described in this article, the Executive FunctionPerformanceTest(EFPT;Baum,Morrison,Hahn, &Edwards,2003;Katz,Tadmore,Felzen,&Hartman-Maeir, 2007),hasseveraladvantagesoverexistingperformance-based assessments.First,itiseasilyadministeredafterbrieftraining (providedinthemanual)inwhattoobserveandhowtocue andscore.Second,itisolatescognitivecomponentsrelated toexecutivefunctionsduringperformanceoffourinstrumentalactivities;thisinformationiscrucialtothegeneration oftreatmentplans.Third,itusesatop-downapproachthat allowsthepractitionertoobjectivelyassesstheclientduring theperformanceofatask,andunlikemanyotherinstruments assessing instrumental activities of daily living (IADLs),itassessesactualperformanceratherthanrelyon proxyorself-report.Moreover,theEFPTmeasuresthelevel ofsupportthepersonneedsfromanotherpersontobesuccessfulinthefourdailylifetaskscentraltocommunityliving:(1)preparingorheatingupalightmeal,(2)managing medications,(3)usingthetelephone,and(4)payingbills (Lysack,Neufeld,Mast,Macneill,&Lichtenberg,2003).

Development of the EFPT Measure

TheEFPTwasdevelopedasameasureofexecutiveperformance.Itrecordswhatthepersoncandoandthelevelof supportnecessarytosuccessfullyperformatask.Itisdesigned toprovidethepractitionerwithinformationthatwillhelp familymembersunderstandandsupporttheirlovedones' performance.TheEFPTalsowasdesignedtobesensitiveto

447

degreesofperformancebreakdownaccompaniedbyvarious levelsofdiseaseprogressionorseverity. The EFPT uses the format of the Kitchen Task Assessment(Baum&Edwards,1993).TheEFPTreplaces thetaskofpreparingcookedpuddingwithpreparingcooked oatmeal(thetwotasksrequirethesamenumberofsteps), anditisfurtherexpandedtoincludetheassessmentofthe supportneededtocompleteatelephonecall,managemedication,andpayabill.Inaddition,theEFPTenhancesthe methodofrecordingcuestosupportperformance.Unlike othertestsofcognitivefunction(withtheexceptionofthe KitchenTaskAssessment),theEFPTdoesnotassesswhat peoplecannotdo;rather,itidentifieswhattheycansuccessfullyaccomplish.TheEFPT'sstandardizedcueingsystem makes it possible to identify a wider range of abilities in peoplewhomaynotbeabletoperformthetaskiftheywere measuredwithoutsupport.Thiscueingsystemisbasedon theprogressiveneedforassistanceassociatedwithincreasing levelsofcognitiveimpairmentandgivesthetesterastraightforwardwaytorecordtheassistancerequiredtosuccessfully performthetask.Thisinformationmayprovidethebasisfor using learning strategies to help family members or care providersgainskillstoenabletheperformanceoftheperson withcognitiveimpairment. TheEFPT'sreliabilityandvalidityhaverecentlybeen established in two clinical populations. Goverover et al. (2005) reported its validity with a sample of adults with multiplesclerosis,andKatz,Tadmore,Felzen,andHartmanMaeir(2007)reporteditsreliabilityandvalidityinpeople withschizophrenia.Inthepresentstudy,wesoughttoestablishtheEFPT'sreliabilityandvalidityinapopulationof peoplewithstrokeandtodemonstrateitssensitivitytostroke severity.Specifically,wesoughtto(1)determinetheEFPT's interraterreliabilityandinternalconsistency,(2)examineits constructvalidityinpeoplewithmildandmoderatestroke andasampleofhealthycontrolparticipants,and(3)evaluate itsconcurrentvalidityusingabatteryofneuropsychological tests. We hypothesized that healthy control participants wouldperformsignificantlybetterontheEFPTthanpeople withstroke.Wefurtherhypothesizedthatparticipantswith mildstrokewouldperformsignificantlybetterthanthose withmoderatestroke.Finally,wehypothesizedthatEFPT scoreswouldshowsignificantlymoderatecorrelationswith neuropsychologicalassessmentsofexecutivefunction.

inSt.Louis.TheCognitiveRehabilitationResearchGroup registersparticipantsfromtheacuteNeurologyStrokeservice of Barnes­Jewish Hospital; all data were obtained in compliancewithandwithapprovalfromtheWashington UniversityInstitutionalReviewBoard.Anursecoordinator prospectively evaluated and recorded clinical and demographicinformationforallparticipants,andaneurologist confirmed the diagnosis of stroke. Permission to contact participantsforadditionalstudieswasobtainedatthetime oftheacutestrokeadmission.Participantsinthisstudywere admittedfromJuly2002toJune2004.Sixmonthsafter strokeonset,participantswerecontactedandaskedtoenroll inthisstudy.Allparticipantswerefullyindependentbefore theirstrokeasindicatedbyprestrokeBarthelIndex(Mahoney &Barthel,1965) scoresof90andModifiedRankinIndex scoresof2.Allstudyparticipantsresidedincommunity settingsatthetimeofthestudyassessment.Healthycontrol participants were recruited through the Volunteers for HealthProgramatWashingtonUniversity.Controlparticipants were screened for cognitive impairment and health conditions known to affect cognitive performance before studyenrollment.Informedconsentwasobtainedfromall studyparticipants. Wetested73participantswithstrokeapproximately6 monthsafterstrokeonset.Noneoftheparticipantswasina rehabilitationprogram.The6-monthtimeframewaschosen to allow for natural recovery to occur. Participants with strokeweredividedintotwogroupsonthebasisoftheir admissionscoreontheNationalInstitutesofHealthStroke Scale(Brottetal.,1989).Scoresof5orlessareconsidered mild,andscoresbetween6and15representamoderatelevel ofstrokeseverity(Brottetal.,1989;Edwards,Hahn,Baum, &Dromerick,2006).Wealsoassessed22healthycontrol participants. The characteristics of study participants are presentedinTable1. Measures Used in the Study Thespecificsofadministeringeachofthefourinstrumental tasksaredescribedinthetestmanualandcanbeobtained fromtheCognitiveRehabilitationResearchGroup'sWeb site, http://crrg.wustl.edu/outcome_assessment.html. The "simplecooking"taskrequiresthepersontopreparequickcookingoatmealfollowingwritteninstructionsonthepackage (or on a sheet prepared with large print). "Using the telephone"includeslookingupagrocerystorenumberin thetelephonebook,callingthestore,andaskingwhetherthe storedeliversgroceries."Managingmedications"requiresthe persontoselecttheprescriptionmedicationfromthreeavailabledistracterdrugsandtakeitwithfoodasrequiredbythe directions(thepillisasugar-freeplacebo).Toevaluate"payingbills,"twobills,checks,acheckregister,andmailtoserve

July/August 2008, Volume 62, Number 4

Method

Participants Study participants were recruited through the Cognitive RehabilitationResearchGroupofWashingtonUniversity

448

Table 1. Demographic Characteristics and Performance on Executive Function Performance Test (EFPT) by Group

Demographic Characteristic or Test Race

Control (N = 22) n % 6 (27) 16 (73) 6 (27) 16 (73) M (SD)

Mild Stroke (N = 59) n % 22 (37) 37 (63) 15 (25) 44 (75) M (SD) 64.57 (14.28) 12.81 (2.90) 2.00 (1.46) 7.87 (8.42) 2.98 (4.90) 1.83 (2.27) 0.92 (1.34) 1.92 (2.19) 0.83 (1.67) 1.77 (2.22) 3.08 (3.20) 1.32 (2.19) 0.88 (1.90)

Moderate Stroke (N = 14) n % 4 (29) 10 (71) 3 (21) 11(79) M (SD) 64.14 (14.78) 12.5 (3.25) 10.64 (2.99) 24.21 (5.39) 5.57 (7.27) 6.57 (9.00) 5.50 (8.99) 4.43 (6.94) 3.43 (7.14) 6.21 (8.39) 6.93 (7.87) 4.07 (6.67) 3.57 (7.08)

c2(2) 0.91 0.75

African American White Gender Male Female

F(2, 93) 1.10 1.23 15.49 9.28 18.00 13.52 11.07 8.47 18.97 18.33 7.76 8.76

Age (years) Education (years) NIHSS EFPT total score*** EFPT tasks Cooking** Using telephone*** Medications*** Paying bills** EFPT EF components Initiation Organization*** Sequencing*** Safety and judgment** Completion**

59.45 (15.78) 14.68 (2.23) -- 1.51 (2.27) 1.23 (2.81) 0.09 (0.29) 0.42 (1.40) 0.23 (0.69) 0.00 (0.00) 0.13 (0.34) 0.58 (1.05) 0.14 (0.47) 0.11 (0.49)

Note. NIHSS = National Institutes of Health Stroke Scale; EF = executive function. p = .63. p < .68. **p < .005. ***p < .0001.

asadistracterareprovidedinanenvelope.Thepersonis requiredtolocatethetwobillsinthestackofmail,paythem accordingtothemoneyavailableintheaccount,andbalance theaccount. Beforebeginningeachtask,thepersonisaskedabout familiaritywiththetaskandwhetherheorsheperformsit independentlyorwithassistance.Allnecessarymaterials areprovidedfortheassessmentinabox.Inallfourtasks, theEFPTassessestheperson'sabilitytousefiveexecutive functions(components)ofatask:(1)initiationofatask (beginningthetask),(2)organization(retrievalandarrangementoftools),(3)sequencing(executionofstepsinacorrect order), (4) safety and judgment (avoids a dangerous situation),and(5)completion(decidingandacknowledging whenataskiscomplete).Fivelevelsofcueingcanbedelivered: 0 (no cue required); 1 (verbal guidance); 2 (gestural guidance); 3(direct verbal assistance);4(physical assistance); and5(do for the participant). Ahigherscorereflectsaneed formorecueingandindicatesmoresevereexecutivefunction deficits. People with motor impairment are scored accordingtothecueleveltheyneedbutarenotpenalized if they ask for help because the impairment necessitates physicalassistance. The highest level of cueing necessary to support task performanceisrecorded;thus,thetestresultsinthreescores:

The American Journal of Occupational Therapy

(1)theexecutivefunction(EF)componentscore,(2)thetask score,and(3)atotalscore.TheEFcomponentscoreiscalculatedbysummingthenumbersrecordedoneachofthe fourtasksforinitiation,organization,sequencing,safetyand judgment,andcompletion.ScoresoneachEFcomponent canrangefrom0to5,andthetotalforallfourtaskscan rangefrom0to20.Thetaskscoreiscalculatedbysumming thefivescoresforeachtask.Therangeforeachtaskis0to 25.Thetotalscoreisthesumoftheperformanceonallfour tasks;thetotalscoreofperformanceonallfourtaskscan rangefrom0to100.TheformsfortheEFPTcanbeviewed athttp://crrg.wustl.edu/outcome_assessment.html. Measures Used for Classification and Exclusion

National Institutes of Health Stroke Scale. The NIHSS

(Brottetal.,1989) assessescognitive,sensory,andmotor impairments resulting from a stroke. The 13-item test is basedonatotalscorerangingfrom0to46;alowerscore indicatesalowerlevelofimpairment.Weusedthetestto definemildandmoderatestroke.

Short Blessed Test (Memory, Orientation, and Concentration).

WeusedtheShortBlessedTest(Katzmanetal.,1983)to measurecognition.Thetest consistsofsixitemsassessing memory,orientation,andconcentrationandisareliableand validscreeningtoolfordetectionofdementiaincommunity

449

andlong-term-carepopulations.Possiblescoresrangefrom 0to28.Thehigherthescoreis,thegreaterthecognitive impairmentis.Wedidnottestpeoplewithscores>12;thus, peoplewithdementiawereexcluded. Neuropsychological Tests

Animal Naming. The Animal Naming test (Barr &

Maximumtotalscoreforthescaleis84,andhigherscores indicategreaterfunctionalindependence. Procedures All participants were tested individually in a laboratory kitchen designed to simulate a home environment. The roomwasquietandfreefromdistractions.TheEFPTtasks were administered in the same order (cooking, using the telephone,managingmedications,payingbills).Weobtained neuropsychologicalandfunctionalmeasuresduringseparate testsessionsat6months.Demographicandstrokeseverity informationforparticipantswithstrokewasobtainedfrom the medical record for the acute hospital admission, and demographicdataforcontrolparticipantswereobtainedat thetimeoftesting. WeconductedalldataanalysesusingSPSSforWindows (Version13:SPSSInc.,Chicago)andcomputeddescriptive statistics for all variables. One-way analysis of variance (ANOVA)andchi-squareanalyseswereusedasappropriate totestfordifferencesindemographicandneuropsychologicalvariablesacrossthegroups.Intraclasscorrelationcoefficients(ICCs)werecomputedtoestablishinterraterreliability. Cronbach's alpha was used to examine internal consistency. Wecomputedone-wayANOVAstotestthehypotheses ofdifferencesinperformanceacrossthethreegroups.Ana prioriadjustmentofthecriterionpvalueforsignificancewas computedbydividing.05by9(thenumberofANOVAs computed).Thiscalculationresultedinanadjustedcriterion ofp<.005. Giventhenumberofmeasuresandgroups,posthoc analysesusedBonferronitestscorrectedformultiplecomparisons.Pearsoncorrelationcoefficientswereusedtodetermine criterion validity. The median scores for all EFPT variableswere0forthecontrolparticipants.Giventhislack ofvariability,weusedonlythescoresofthestrokeparticipantsforthisanalysis.

Brandt,1996)isameasureofverbalfluencythatasksthe participanttogenerateasmanyanimalnamesaspossiblein 60s.Itassessestheefficiencyofverbalretrieval,short-term memory,andcognitiveflexibility. Trailmaking Test. The Trailmaking Test (Reitan & Wolfson,1995) providesinformationregardingattention, visualscanning,andexecutivefunction.PartArequiresthe participanttodrawlinesandconnect25numbersscattered onapage.PartBcallsfortheconnectionofnumbersand letters in order, alternating between letters and numbers. Twoscoresreflectthetotaltimeinsecondsrequiredtocompleteeachtask.Anoralversionisavailableforpeoplewith motordeficits.

Wechsler Memory Scale­Revised

Digit Span Forward and Backward Test.TheWechsler MemoryScale­Revised,DigitSpanForwardandBackward Test (Wechsler, 1987), measures attention and verbal working memory. The examiner is required to verbally presentdigitsatarateofonepersecond.Theforwardtest requirestheparticipanttorepeatthedigitsverbatim.The backwardtestrequirestheparticipanttorepeatthedigits inreverseorder. Logical Memory Total Recall Test.The LogicalMemory TotalRecallTest,alsobyWechsler(1987),examinesthe abilitytorecallideasintwoorallypresentedstories.Thetest assessesattention,concentration,andverbalmemory.Higher scoresareindicativeofbettermemoryperformance. Functional Outcome Measures

FIM. The interview version of the FIM (Chang,

Slaughter,Cartwright,&Chan,1997)uses18itemstograde thelevelofcognitiveandphysicalassistancenecessaryfor function.Itemscoresrangefrom1(completely dependent)to 7(completely independent).Twosummaryscores(mobility/ self-careandcommunication/cognition)arederived.Total possible scores are 91 and 35, respectively. Higher scores representgreaterindependence. Functional Assessment Measure. The Functional Assessment Measure (FAM; Hall, Hamilton, Gordon, & Zasler,1993;Hawley,Taylor,Hellawell&Pentland,1999) wasdesignedtosupplementtheFIM;ithas12itemsthat further assess motor abilities, community mobility, communication,psychosocialadjustment,andcognition.The items are scored on the 7-point scale used in the FIM.

450

Results

ThecharacteristicsofthestudysamplearepresentedinTable 1.Themeanageoftheparticipantswas64.49(SD=14.28, range=30­90years).Althoughthecontrolparticipantswere slightlyyoungerthantheparticipantswithstroke,nosignificantdifferencesinagewerefoundamongthethreegroups. Thecontrolparticipantshadmoreyearsofeducationthan thestrokegroup(14.68yearsvs.12.81years,respectively), althoughthisdifferencewasnotstatisticallysignificant.The participantswerepredominantlyfemaleandWhite.Nosignificantdifferenceswerefoundinrace(c2=0.91,p=.63, df = 5) or gender (c2 = 0.75, p = .68, df = 5) across the

July/August 2008, Volume 62, Number 4

groups.ThemeanNationalInstitutesofHealthStrokeScale scoreswere2.00(SD=1.50)forthemildgroupand10.64 (SD=2.99)forthemoderategroup. Wecomputedaseriesofone-wayANOVAstoexamine differences across the control, mild, and moderate stroke groups on the neuropsychological tests. In each case, the meanscoresofthecontrolgroupwerebetterthanthosefor themildstrokegroup.Themildstrokegroupperformed betterthanthemoderategrouponallvariables.Significant differenceswereobservedforallvariablesexceptTrailsA. Themeansandstandarddeviationsofthesemeasuresare shownbygroupinTable2. Reliability Wedeterminedinterraterreliabilitywiththreetrainedraters whosimultaneouslyrated10participants,5withmildstroke and 5 healthy controls. The total score and each of four subtestscoreswereanalyzed,andICCswerecomputed.For the total EFPT score,theICCwas.91,and subtest ICC scoreswere.94forthecookingtask,.89forpayingbills,.87 for managing medication, and .79 for the using the telephone. These coefficients are indicative of high levels of interraterreliability. TheinternalconsistencyoftheEFPTforthetotalsamplewashigh(a=.94);subtestCronbach'salphacoefficients were.86forthecookingtask,.78forpayingbills,.88for managing medication, and .77 for using the telephone. These coefficients support the internal consistency of the EFPT.Correlationsbetweeneachofthetestdomainsand thetotalscorewereasfollows:initiation,r=.91;organization,r=.93;sequencing,r=.88;safetyandjudgment,r= .78,andcompletionofallsteps,r=.89. Construct Validity ThemeanEFPTtotalscoreandthescoresforeachtaskand EFcomponentsarepresentedbygroupinTable1.ToexaminetheconstructvalidityoftheEFPTandEFcomponents, weexaminedthescoresacrossthethreegroupsofpartici-

pants.Constructvalidityisestablishedifatestcandiscriminatebetweenpeoplewithandwithoutaknowntrait(Portney &Watkins,2000).Aseriesofone-wayANOVAscomparing performance across the three groups was computed separatelyforthetotalscores,tasks,andEFcomponents;those findingsarealsopresentedinTable1.Ashypothesized,with theexceptionofinitiation,wefoundsignificantdifferences amongthegroupsforeachanalysis.Inspectionofthemean EFPT total scores, task scores, and EF component scores indicatedthatthecontrolparticipantshadthelowest(best) scores,followedbythemildstrokegroup.Theparticipants with moderate stroke had higher mean scores on all measures. WethencomputedposthocBonferronipairwisecomparisonsforeachoftheEFPTtasksandcomponents.The resultsofthepairwisecomparisonsarepresentedinTable2. Thehypothesizeddifferencesbetweenthegroupsweresupported. The control EFPT total scores were significantly lowerthanthemildstrokescores(p<.05)andmoderate strokescores(p<.0001).Mildscoreswerealsosignificantly lowerthanmoderatescores(p<.0001). Twoofthesubtests,CookingandPayingBills,significantlydiscriminatedbetweencontrolandmildstrokeparticipants.Themildstrokeparticipantsperformedlesswell thanthehealthycontrolparticipants.Mildandmoderate strokegroupsweresignificantlydifferentonthreeofthefour subtests. Only the cooking subtest failed to significantly discriminatebetweenthemildandmoderategroups. ThetestEFcomponentswerealsoexamined.Wefound significant differences between control and mild stroke p articipants for sequencing (p < .001) and organization (p < .04). Differences between mild and moderate stroke groupsweresignificantfororganization(p<.0001),sequencing(p<.001),safetyandjudgment(p<.004),andcompletion(p<.01).ThesefindingsarepresentedinTable3. Criterion Validity Weexaminedconcurrentvalidity,aformofcriterionvalidity,bycomparingscoresonabatteryofneuropsychological teststoperformanceontheEFPT.Concurrentvalidityis establishedwhenhighcorrelationsarefoundbetweenthe newmeasureandwell-establishedmeasuresofthephenomena(Portney&Watkins,2000).Onlydatafromthestroke participantswereusedinthisanalysis.Theresultsofthese analysesarepresentedinTable4.Thehypothesisregarding thecriterionvalidityoftheEFPTwasconfirmed.Significant moderatecorrelationswerefoundbetweentheEFPTtotal score and neuropsychological measures assessing working memory,verbalfluency,andattention.Thesecorrelations includetherecallscoreoftheWechslerMemoryScale(r= ­.59),AnimalFluency(r=­.47),TrailsB(r=.39),and

451

Table 2. Performance on Neuropsychological Tests by Group

Variable Trails A (seconds) Trails B (seconds)*** Digits Forward** Digits Backward** Story Recall* Letter Fluency** Animal Fluency*** Control (M ± SD) 31.0 ± 10.8 73.8 ± 29.4 9.2 ± 2.6 5.3 ± 1.6 30.6 ± 6.9 38.3 ± 12.7 22.6 ± 4.9 Mild Stroke (M ± SD) 71.9 ± 63.6 184.1 ± 98.5 6.5 ± 1.3 3.5 ± 1.7 24.8 ± 8.1 14.0 ± 7.5 14.8 ± 5.5 Moderate Stroke (M ± SD) 188.3 ± 114.5 279.6 ± 64.7 6.1 ± 2.3 3.1 ± 1.6 18.0 ± 9.1 25.3 ± 12.7 8.8 ± 5.1

Note. Values are one-way analyses of variance comparing scores across groups. *p < .05. **p < .01. ***p < .001.

The American Journal of Occupational Therapy

Table 3. Significant Between-Group Differences for Executive Function Performance Test (EFPT) Task and Component Scores by Group (p Values)

Variable EFPT total Subtest Cooking Using telephone Medications Paying bills Executive component Initiation Organization Sequencing Safety and judgment Completion .34 .04 .001 .77 .47 .001 .0001 .0001 .0001 .0001 .004 .0001 .001 .004 .01 .008 .06 .07 .03 .0001 .0001 .0001 .0001 .13 .0001 .001 .01 Control vs. Mild .05 Control vs. Moderate .0001 Mild vs. Moderate .0001

Table 4. Pearson Correlation Coefficients Between Executive Function Performance Test (EFPT) Total Score and Neuropsychological and Functional Tests

Variable Digits forward Digits backward Trails A Trails B Story Recall (Wechsler Memory Scale) Animal Fluency Short Blessed FIM total FAM total EFPT Total Score (r) ­.26 ­.49 .21 .39 ­.59 ­.47 .39 ­.40 ­.68 p< .08 .0001 .09 .001 .0001 .0001 .001 .001 .0001

Note. FIM = Functional Independence Measure; FAM = Functional Assessment

DigitsBackward(r=­.49).TheEFPTtotalscorewasalso significantlycorrelatedwiththeShortBlessedScale(r=.39) Lower, nonsignificant correlations were observed for tests that are not thought to assess executive function. These includeTrailsA(r=.21)andDigitsForward(r =­.26). DifferencesinthecorrelationsbetweentheEFPTscores andtheFIMandFAMscoresprovideadditionalevidence ofcriterionvalidity.TheFIMisameasureofbasicADLs, whereastheFAMassessesthemorecomplexinstrumental skillsneededforcommunityindependence.Thecorrelations betweentheEFPTscoresandtheFIM(r=­.40)andFAM (r=­.68)scoresreflectthesedifferences.ThehighercorrelationbetweentheEFPTandFAMscoresfurthersupports thecriterionvalidityoftheEFPTasameasureimportant fortheassessmentofIADLs.

Discussion

ThisstudyexaminedtheEFPT'sreliabilityandvalidityin healthycontrolparticipantsandpeoplewithmildtomoderatestroke.Theneedtomorefullyunderstandtheeffectsof executivefunctiondeficitsoncomplextaskperformanceis well-documented(Kounti,Tsolaki,&Kiosseoglou,2006). Deficitsinworkingmemory,attention,organization,and self-control have been linked to impaired performance of IADLs such as money management, taking medications, shopping,anddriving(Grigsby,Kaye,Baxter,Shetterly,& Hamman,1989).However,noconsensusexistsonthemost appropriatemethodforassessingexecutivefunctiondeficits whentheprimaryinterestisthepredictionofperformance ofcomplexlifetasks(Burgessetal.,2006).Agrowingbody ofliteraturesuggeststhat, althoughneuropsychologicalmeasuresaregoodindexesofisolatedcognitiveandexecutive components,theyoftenarelesseffectiveinpredictingreal452

world complex task performance and functional ability (Aldermanetal.,2003;Burgessetal.,1998;Gioia&Isquith, 2004; Keil & Kaszniak, 2002; Shallice & Burgess, 1996; Wilson,1993). Severalrecentarticleshavesupportedtheassertionthat assessmentofexecutivefunctionsnecessaryforcomplexlife tasksisbestconductedinreal-worldenvironments(Burgess etal.,2006;Godbout,Grenier,Braun,&Gagnon,2005; Goverover et al., 2005). Such assessments will provide a betterunderstandingoftheimpactofenvironmentalfactors onperformance(Goveroveretal.,2005).TheEFPTissuch a tool. It uses real rather than simulated activities as the assessment process, and the assessment is conducted in a real-worldenvironment. WefoundtheEFPTtobeareliableandvalidassessment ofexecutivefunctionabilitiesinpeoplewithmildtomoderatestroke. Ourdatasupporttheconstruct,criterion,and discriminant validity of the measure in this population. SimilarfindingshavebeenreportedwiththeEFPTforpeople with multiple sclerosis (Goverover et al., 2005) and schizophrenia(Katzetal.,2007).Inadditiontoconfirming theresultsoftheseinvestigations,ourstudyprovidesadditionalevidenceofcriterionvaliditythroughthecomparison ofEFPTscorestoacceptedneuropsychologicalmeasuresof executive function. Examination ofthecorrelation coefficientsbetweentheEFPTandtheneuropsychologicaltests illustratestheargumentmadebyBurgessetal.(2006)in theirrecentcallfortheuseofmeasuresbasedonreal-world performanceinclinicalandexperimentalstudiesofexecutive function.ThemoderaterelationshipbetweentheneuropsychologicaltestsandtheEFPTsuggeststhat,althoughboth typesofteststapsimilarconstructs,neuropsychologicaltests cannotfullycapturethecomplexityofperformanceincontextanddonotanswerquestionsaboutaperson'sabilityto safelyperformtheinstrumentaltasksrequiredforindependentdailylife.

July/August 2008, Volume 62, Number 4

Chaytor and Schmitter-Edgecombe (2003) described theconstraintslimitingtheecologicalvalidityofneuropsychologicalassessmentsasincludingthedisagreementindefinitionsofexecutivefunctioning;conductingtestsinastatic testing environment; measuring behaviors during a short period of time, therefore evaluating a limited sample of behaviors;notmeasuringcompensatorystrategies;andnot acknowledging other factors, such as physical disabilities, emotions,andpremorbidfunctioning.TheEFPToffersan alternative approach to identifying executive dysfunction, onethatcanofferboththedescriptionofhigher-ordercognitivefunctionsandaclinicallyusefultoolthatindicatesthe levelofsupportapersonneedstoperformatask.Occupational therapistsoftenareaskedtoprovideanopiniononwhether someonecanlivealoneorbeleftaloneduringtheday.The person's performance on the EFPT offers an empirical underpinningtoadecisionthatsomelevelofassistancewill beneeded. ThemoderaterelationshipbetweentheFIMandthe EFPTalsosuggeststhatsimilarconstructsarebeingtapped, buttheFIMcannotfullycapturethecomplexityofperformance.TheFIMdoesnotanswerquestionsabouttheperson'sabilitytoperforminstrumentaltasksorthelevelofhelp neededtoperformthem.Thesubstantialrelationshipwith theFAMdoessupporttheimportanceofusingtheEFPTto determinetheperformanceneedsofpeoplewithstrokeas theyperformIADL.Informationcollectedwillassistwith familyeducation.

therapist with information about the executive function issues that will interfere with daily life performance. The EFPTisatoolthatoccupationaltherapistscanusetodetermine capacity and guide interventions with people with strokeandotherchronicneurologicalconditions. s

Acknowledgments

We recognize the following people for helping with the conceptualization of this assessment: Noomi Katz, PhD, OT; Adina Harman-Maier, PhD, OT; Deirdre Dawson, PhD, OT(C); Heidi Shambra, MD; and Jen Murawski, MSOT. This study was funded by James S. McDonnell FoundationGrantNo.21002032.

References

Alderman, N., Burgess, P. W., Knight, C., & Henman, C. (2003). Ecological validity of a simplified version of the multiple errands shopping test. Journal of the International Neuropsychological Society,9,31­44. Allen, C. K., Earhart, C. A., & Blue, T. (1992). Occupational therapy treatment goals for the physically and cognitively disabled. Rockville, MD: American Occupational Therapy Association. Barr, A., & Brandt, J. (1996). Word list generation deficits in dementia.Journal of Clinical and Experimental Neuropsychology, 18,810­822. Baum,C.M.,&Christiansen,C.H.(2005).Person­environment­ occupation­performance: An occupation-based framework forpractice.InC.H.Christiansen,C.M.Baum,&J.BassHaugen(Eds.),Occupational therapy: Performance, participation, and well-being(3rded.,pp.242­259).Thorofare,NJ: Slack. Baum, C. M., & Edwards, D. (1993). Cognitive performance inseniledementiaoftheAlzheimer'stype:Thekitchentask assessment. American Journal of Occupational Therapy, 47, 431­436. Baum,C.M.,Morrison,T.,Hahn,M.,&Edwards,D.F.(2003). Test manual: Executive Function Performance Test.St.Louis, MO:WashingtonUniversity. Bechara,A.,Damasio,A.R.,Damasio,H.,&Anderson,S.W. (1994).Insensitivitytofutureconsequencesfollowingdamage tohumanprefrontalcortex.Cognition, 50,1­3,7­15. Brott,T.,Adams,H.P.,Jr.,Olinger,C.P.,Marler,J.R.,Barsan, W.G.,Biller,J.,etal.(1989).Measurementsofacutecerebral infarction:Aclinicalexaminationscale.Stroke,20,864­870. Burgess, P. W. (2000). Strategy application disorder: The role of the frontal lobes in human multitasking. Psychological Research,63,279­288. Burgess,P.W.,Alderman,N.,Evans,J.,Emslie,H.,&Wilson,B. A.(1998).Theecologicalvalidityoftestsofexecutivefunction. Journal of the International Neuropsychological Society, 4,547­558. Burgess,P.W.,Alderman,N.,Forbes,C.,Costello,A.,Coates,L. M.,Dawson,D.R.,etal.(2006).Thecaseforthedevelopment and use of ecologically valid measures of executive function

453

Future Areas for Study

Wewillcontinuetoaddparticipantswithmoreseverityto moredirectlyaddresstheissuesofbothmotorandcognitive impairment.Withmoreparticipants,wewillbeabletodo item analysis to determine the types of cognitive support neededbypeoplewithcertainstrokesyndromestoperform theactivitiesthatallowthemtobesafeintheirinstrumental activitiesathome.

Conclusion

OurstudysupportstheuseoftheEFPTasaperformancebasedtestofexecutivefunctionthatcanbeadministeredas atop-downassessmenttobuildtreatmentplansinahome orclinic.Itoffersaclient-centeredapproachthatmakesit possibleforthecliniciantoconsidersafetyanddetermine independencebyobservingthepersonperformingatask. DuringtheadministrationoftheEFPT,itispossibletogain insightintoenvironmentalbarriersthatmaylimitperformanceandalsoidentifystrategiesthatcanbeusedtosupport performance at home. It also provides the occupational

The American Journal of Occupational Therapy

inexperimentalandclinicalneuropsychology.Journal of the International Neuropsychological Society,12,194­209. Chang,W.,Slaughter,S.,Cartwright,D.,&Chan,C.(1997). Evaluating the fone FIM: Construct validity. Journal of Outcome Measurement,1,192­218. Chaytor,N.,&Schmitter-Edgecombe,M.(2003).Theecological validity of neuropsychological tests: A review of the literatureoneverydaycognitiveskills.Neuropsychology Review, 13, 181­197. Crawford,S.,&Channon,S.(2002).Dissociationbetweenperformanceonabstracttestsofexecutivefunctionandproblem solvinginreal-lifetypesituationsinnormalaging.Aging and Mental Health, 6,12­21. DePoy,E., Maley,K.,&Stranraugh,J.(1990).Executivefunctionandcognitiveremediation:Astudyofactivitypreference. Occupational Therapy in Health Care, 7, 101­114. Edwards,D.F.,Hahn,M.G.,Baum,C.M.,&Dromerick,A.W. (2006).Theimpactofmildstrokeonactivityparticipation and life satisfaction. Journal of Stroke and Cerebrovascular Disease, 15,151­157 Fisher,A.G.(1993).TheassessmentofIADLmotorskills:An applicationofmanyfacetedRaschanalyses. American Journal of Occupational Therapy, 47,319­329. Gaudette,M.,&Anderson,A.(2002).Evaluatingmoneymanagementskillsfollowingbraininjuryusingtheassessmentof functionalmonetaryskills.Brain Injury,16,133­148. Gioia,G.A.,&Isquith,P.K.(2004).Ecologicalassessmentoftests ofexecutivefunctionintraumaticbraininjury.Developmental Neuropsychology, 25, 135­158. Godbout,L.,Grenier,M.C.,Braun,C.M.,&Gagnon,S.(2005). Cognitivestructureofexecutivedeficitsinpatientswithfrontallesionsperformingactivitiesofdailyliving.Brain Injury, 19,337­348. Goel,V.,Grafman,J.,Tajik,D.,Gana,S.,&Danto,D.(1997).A studyoftheperformanceofpatientswithfrontallobelesions inafinancialplanningtask.Brain, 120,1805­1822. Goverover,Y.(2004).Categorization,deductivereasoning,and self-awareness:Associationwitheverydaycompetenceinpersonwithacutebraininjury.Journal of Clinical Experimental Neuropsychology, 26,737­749. Goverover,Y.,Kalmar,J.,Gaudino-Goering,E.,Shawaryn,M., Moore,N.B.,Halper,J.,etal.(2005).Therelationbetween subjectiveandobjectivemeasuresofeverydaylifeactivitiesin personswithmultiplesclerosis.Archives of Physical Medicine and Rehabilitation, 86,2303­2308. Grigsby,J.,Kaye,K.,Baxter,J.,Shetterly,S.M.,&Hamman, R. F. (1989). Executive cognitive abilities and functional statusamongcommunity-dwellingolderpersonintheSan LuisValleyHealthandAgingStudy.Journal of the American Geriatric Society, 46,590­596. Hall,K.M.,Hamilton,B.B.,Gordon,W.A,&Zasler,N.D. (1993).Characteristicsandcomparisonsoffunctionalassessmentindices:Disabilityratingscale,FunctionalIndependence MeasureandFunctionalAssessmentMeasure.Journal of Head Trauma Rehabilitation, 8, 60­74. Hawley,C.A.,Taylor,R.,Hellawell,D.J.,&Pentland,B. (1999). UseoftheFunctionalAssessmentMeasure(FIM+FAM)in headinjuryrehabilitation:Apsychometricanalysis.Journal of Neurology, Neurosurgery, and Psychiatry, 6,749­754.

454

Katz, N., & Hartman-Maeir, A. (2005). Higher-level cognitivefunctions:Awarenessandexecutivefunctionsenabling engagementinoccupation.InN.Katz(Ed.),Cognition and occupation across the life span: Models for intervention in occupational therapy(pp.3­25).Bethesda,MD:AOTAPress. Katz,N.,Tadmore,I.,Felzen,B.,&Hartman-Maeir,A.(2007). ValidityoftheExecutiveFunctionPerformanceTestinindividualswithschizophrenia.Occupational Therapy Journal of Research, 27,1­8. Katzman, R., Brown, T., Fuld, P., Peck, A., Schechter, R., & Schimmel,H.(1983).Validationofashortorientation-memory-concentration test of cognitive impairment. American Journal of Psychiatry, 140,734­739. Kaye, K., Grigsby, J., Robbins, L. J., & Korzun, B. (1990). Predictionofindependentfunctioningandbehaviorproblemsingeriatricpatients.Journal of the American Geriatric Society, 38,1304­1310. Keil,K.,&Kaszniak,A.W.(2002).Examiningexecutivefunctioninindividualswithbraininjury:Areview.Aphasiology, 16,305­335. Kounti,F.,Tsolaki,M.,&Kiosseoglou,G.(2006).Functional CognitiveAssessmentScale(FUCAS):Anewscaletoassess executivecognitivefunctionindailylifeactivitiesinpatients with dementia and mild cognitive impairment. Human Psychopharmacology, 21, 305­311. Lezak,M.(1982).Theproblemofassessingexecutivefunctions. InternationalJournal of Psychology, 17,281­297. Lezak, M. (1995). Neuropsychological assessment (3rd ed.). New York:OxfordUniversityPress. Lezak,M.D.(2004).Executivefunctionandmotorperformance. InM.D.Lezak,D.B.Howieson,&D.W.Loring(Eds.), Neuropsychological assessment (4th ed., pp. 611­646). New York:OxfordUniversityPress. Linden,A.,Boschian,K.,Eker,C.,Schalen,W.,&Nordstrom, C.H.(2005).Assessmentofmotorandprocessskillsreflects brain-injured patients' ability to resume independent living better than neuropsychological tests. Acta Neurologica Scandinavica, 111,48­53. Lysack, C. L., Neufeld, S., Mast, B. T., Macneill, S. E., & Lichtenberg,P.A.(2003).Afterrehabilitation:An18-month follow-upofelderlyinner-citywomen.American Journal of Occupational Therapy,57,298­306. Mahoney,F.I.,&Barthel,D.(1965).Functionalevaluation:The BarthelIndex.Maryland State Medical Journal,14, 56­61. Manchester,D.,Priestley,N.,&Jackson,H.(2004).Theassessmentofexecutivefunctions:Comingoutoftheoffice.Brain Injury,18,1067­1081. Mercier,L.,Audet,T.,Hebert,R.,Rochette,A.,&Dubois,M.F. (2001).Impactofmotor,cognitive,andperceptualdisorders on ability to perform activities of daily living after stroke. Stroke, 32,2602­2608. Nygard,L.,Bernspand,B.,Fisher,A.G.,&Winblad,B.(1994). Comparingmotorandprocessabilityofpersonswithsuspecteddementiainhomeandclinicsettings.American Journal of Occupational Therapy, 48,689­696. Park,S.,Fisher,A.G.,&Velozo,C.A.(1994).Usingtheassessment of motor and process skills to compare occupational performance between clinic and home settings. American Journal of Occupational Therapy, 48,697­709.

July/August 2008, Volume 62, Number 4

Portney,L.G.,&Watkins,M.P.(2000).Foundations of clinical research: Applications to practice(2nded.).UpperSaddleRiver, NJ:PrenticeHall. Reitan, R. M., & Wolfson, D. (1995). Category test and trail making test as measures of frontal lobe functions. Clinical Neuropsychologist,9,50­56. Secrest,L.,Wood,A.E.,&Tapp,A.(2000).Acomparisonof the Allen Cognitive Level Test and the Wisconsin Card SortingTestinadultswithschizophrenia.American Journal of Occupational Therapy,54,129­133. Shallice,T.,&Burgess,P.W.(1991).Deficitsinstrategyapplication followingfrontallobedamageinman.Brain,114,727­741. Shallice, T., & Burgess, P. (1996). The domain of supervisory processesandtemporalorganizationofbehavior.Philosophical Transactions of the Royal Society B: Biological Sciences, 351, 1305­1311.

Stuss, D. (1992). Biological and psychological development of executivefunctions.Brain and Cognition, 20,8­23. Velligan, D. I., True, J. E., Lefton, R. S., Moore, T. C., & Flores,C.V.(1995).ValidityoftheAllenCognitiveLevels Assessment:Atri-ethiccomparison.Psychiatry Research, 56, 101­109. Wechsler,D.(1987).Wechsler Memory Scale­Revised.SanAntonio, TX:PsychologicalCorporation. Weld,E.,&Evans,I.(1990).Effectsofpartversuswholeinstructional strategies on skill acquisition and excess behavior. American Journal of Mental Retardation, 94,377­386. Wilson, B. A. (1993). Ecological validity of neuropsychological: Do neuropsychological indexes predict performance in everyday activities? Applied and Preventive Psychology, 2, 209­215.

The American Journal of Occupational Therapy

455

Information

10 pages

Report File (DMCA)

Our content is added by our users. We aim to remove reported files within 1 working day. Please use this link to notify us:

Report this file as copyright or inappropriate

1134160


You might also be interested in

BETA
370-382_ISP_091659.indd