Bersin High_Impact_Learning_Measurement

JoshBersinPrincipalAnalystNovember2006©BERSIN&ASSOCIATESINDUSTRYREPORT|v.1.0High-ImpactLearningMeasurementStateoftheMarketandExecutiveSummaryBersin&Associates©November2006•NotforDistribution•LicensedMaterialHigh-ImpactLearningMeasurement:ExecutiveSummaryTABLEOFCONTENTSExecutiveOverview3OurResearchMethodology4TheStateofTrainingMeasurement6WhatDoOrganizationsWanttoMeasure?9WhatDoTrainingOrganizationsActuallyMeasure?11BudgetsforTrainingMeasurement14TheImportanceofaBusinessPlan16HowDoOrganizationsMeasureAlignment?17HowDoOrganizationsStayClosetoBusinessPriorities?18TheRoleofPerformanceConsulting19SatisfactionwithLMSSystems21AreYouSpendingEnoughonTrainingMeasurement?22SummaryandHowtoGetMoreDetails23AppendixI:TheBersin&AssociatesImpactMeasurementFramework®4AppendixII:TableofFigures6AboutUs7AboutThisResearch8Bersin&Associates©November2006•NotforDistribution•LicensedMaterialHigh-ImpactLearningMeasurement:ExecutiveSummary3ExecutiveOverviewOneoftheformidablechallengesforlearninganddevelopmentmanagersisthedevelopmentofanefficientandeffectiveprocessforthemeasurementoftraining.Theproblemcoversawiderangeofissues:•Howdowemeasuretheeffectivenessandbusinessimpactofagivenprogram?•Howdowemeasuretheeffectivenessandefficiencyofthetrainingoperations?•Howdowedevelopameasurementprogramthatmeetstheneedsofawiderangeofaudiences,includingourtrainingorganization,seniorHRmanagement,line-of-businessexecutivesandlinemanagers?•Howdowedevelopandimplementaprocess,whichiseasytomanageandsupport?Wehavebeenstudyingbestpractices,approaches,technologiesandmethodologiesinthisareaformorethanfiveyears.Thisreportisanexecutiveoverviewofourcomprehensiveindustryreport,entitledHigh-ImpactLearningMeasurement1.Inthisexecutivesummary,wediscussthestateoftrainingmeasurementtodayandhighlightthesummaryfindingsofourresearch.(Throughoutthisexecutivesummary,wewillrefertomoredetaileddiscussionscontainedintheindustryreportoftopicscoveredherein.)Thedetailedindustrystudyincludesthefollowing:•AreviewofthelimitationsoftheKirkpatrickMeasurementModelandanewmodeltohelpmanagersunderstandpreciselywhatfactorsimpacttheeffectivenessofatrainingprogram(Bersin&AssociatesBusinessImpactModel®);•Ameasurementframework,builtonthisBusinessImpactModel,whichhelpstrainingmanagersunderstandpreciselywhattomeasure(Bersin&AssociatesImpactMeasurementFramework®);1Formoreinformation,High-ImpactLearningMeasurement:BestPractices,Models,andBusiness-DrivenSolutionsfortheMeasurementandEvaluationofCorporateTraining,Bersin&Associates/JoshBersin,November2006.ThiscomprehensiveindustryreportisavailabletoBersin&Associatesresearchmembersorforpurchaseat©November2006•NotforDistribution•LicensedMaterialHigh-ImpactLearningMeasurement:ExecutiveSummary4•Aseriesofbestpractices,whichcanbeimplementedbyanytrainingmanager(regardlessofanorganization’ssize);•AclearunderstandingofwhytheKirkpatrickfour-levelModelislimitedandhowtoapplyitinapracticalway;•Acleardescriptionofthesevensteps,whichorganizationsshouldfollowtoimplementapractical,actionableandaffordablemeasurementprogram;•Aseriesofin-depthcasestudiesthatillustratebestpracticesandmeasurementapproaches,whichotherscaneasilyimplement;and,•Areviewofsomeofthenewtoolsandtechnologies,whichcanhelptrainingmanagersmoreeasilyimplementmeasurementsolutions.Theresearchforthiscomprehensivereportincludestwoquantitativestudies(oneissuedin2004andonein2006);aseriesofmorethan50in-depthinterviewswithtrainingandHRmanagers;and,interviewswithmostofthesolutionproviderswhodeliverthetoolsandservicestohelporganizationsmanageandmeasuretrainingprograms.OurResearchMethodologyTheinformationdiscussedinthisreportcomesfromasurveyofNorthAmericantrainingmanagersconductedinthesummerof2006.Thissurveywasnearlyidenticaltoasimilarsurveyconductedinthesummerof2004,sowecouldseetrendsintheimplementationoftrainingmeasurement.Theaverageorganizationsizeofrespondentswas26,000employees–rangingwidelyfromsmalltoverylarge(seeFigure1).Wedidfindthat,formanymeasurementprocesses,sizedoesmatter.Aswediscussinthisoverview,largerorganizationshavetheopportunitytoinvestmoresteadilyinthemeasurementprocess.Evenso,however,thesmallesttrainingorganizationscaneasilyimplementmeasurementprogramsthatwork.2DonaldKirkpatrick’sfour-levelMeasurementModelhasbeenwidelypublishedinmanyarticlesanditsterminologyiswell-knowntomosttrainingprofessionals.TheoriginalmodelwaspublishedinTrainingandDevelopmentHandbook,R.L.Craig,McGraw-Hill,1976.Bersin&Associates©November2006•NotforDistribution•LicensedMaterialHigh-ImpactLearningMeasurement:ExecutiveSummaryFigure1:DemographicsofRespondentsEightypercentoftherespondentswerefromfor-profitcorporations;sevenpercentwerefromfederal,stateandlocalgovernments;sixpercentwerefromnonprofitorganizations;and,theremainderwerefromconsultantsandvendors.(TheindustrydistributionofrespondentsisshowninFigure2.)Wedidnotfindanysignificanttrendsintrainingmeasurementbyindustry.Figure1:DemographicsofRespondentsSource:Bersin&Associates,2006.TotalCountofEMPLOYEESINRANGE1,000-5,00031%10,000