espms presentation aug 28

7/30/2019 Espms Presentation Aug 28

1/25

1

Employment Service

Performance Management System

Presentation to

Employment Ontar io Service Delivery Advisory Group

August 28 2008


2/25

2

Purpose

Status Report on the Development of the Performance

Management Framework/System for Employment

Services


3/25

3

The Performance Management Framework and System for

Employment Services builds on Employment Ontario principles

Principles


4/25

4

Best Practices - International

The Performance Management Framework and System

incorporate characteristics of effective program management

systems practices

Results measured against three dimensions of service delivery success Effectiveness, Efficiency and Customer service

Dimensions of service delivery defined by core measures and weightedaccording to their value to overall service quality

Measures that quantify input, through/output and impact or results

Service Quality Standards that establish the minimum level of acceptableperformance


5/25

5

Best Practices - OPS Directives on

Performance ManagementThe Performance Management

Framework and System align

with OPS Directives

Key Elements:

Accountability

Transparency

Customer-centric

Integrated Service Delivery

business planprocess

servicedelivery

evaluationand planning

data collectionand reporting

service leveldetermination

PerformanceManagementSystem


6/25

6

Stakeholder Engagement

Delivery Stakeholder Reference Group participated in

developing The Performance Management Framework and

System

External Delivery Stakeholder Reference Group for EmploymentServices, a sub-committee of SDAG

The PMF/S has been developed in collaboration with other keyelements of the implementation and delivery such as contract qualityassurance, resource planning, information systems development,

business supports, etc.


7/25

7

Components of the Performance Management

Framework/SystemService Quality Dimensions, Measures, Values

Service Funding Decision Matrix

Funding Model

Performance Incentive (at maturity)

Performance Management Standards

Stage I Interim

Stage II Maturity

Implementation Contract Management

Awards and Recognition


8/25

8

Service Quality Dimensions, Measures, Values


9/25

9

Compliance

with MTCU

Contract

OverallService

QualityStandardAchieved

Evidence ofOrganizatio

nalCapacity

Reached

Improvement

Targets

MTCU Response

Success Funding approval, Performance Incent ive and/or Award

Potent ial

Organization is in compliance with contract and program guidelines and hasachieved or surpassed Overall Service Quality Standard for service delivery

OC review demonstrates ability for sustained and/or improved results Organization has attained improvement targets identified in last contract

Official Review

Funding Ap proval with Immediate Compliance Direct ive Organization is NOT in compliance with contract despite achieving or

surpassing Overall Service Quality Standard for service delivery and mustsubmit an action plan for achieving compliance

OC review demonstrates little ability for sustained or improved results Organization did not meet improvement targets from last contract.

Directed Improvement - Funding Ap proval with Compliance and In Year

Improvements to Overal l Serv ice Quali ty Standard

Organization is NOT in compliance with contract/guidelines and must submit anaction plan for achieving compliance

Organization has not achieved Overall Service Quality Standard for service

delivery nor improvement targets from last contract OC review demonstrates there is capacity and potential for immediateimprovement

Terminat ion Notice of contract terminat ion, procedures to supp ort c l ient

serv ice transfer

Organization did not achieve in year service improvements to standard and/or is not in compliance with contract/guidelines Organization does not demonstrate necessary OC to support improvement

Service Funding Decision Matrix


10/25

10

Funding ModelEmployment Services

Operating funds provides funding to:

Deliver contracted services meet provincial service

quality standards

Funding Unit Ranges

Activity Range

Minimum Unit

Funding

Maximum

Unit

Funding

Operating Budget

(Range)

Employment & Training

Placement Incentive

& Supports (Range)

up to 399 950 2800 95,000-500,000 20,000-80,000

400-899 880 2000 352,000-1,000,000 80,000-180,000

900-1499 795 1100 715,500-1,500,000 180,000-300,000

1500-1999 750 950 1,125,000-1,895,000 300,000-400,000

750 2800 95,000-1,895,000 20,000-400,000

Employment & TrainingPlacement Incentives

Up to $8000 for on-the-jobtraining placement

Flow through funds

Employment & TrainingSupports

Provision of financialsupports (up to $500) to

remove barriers to job search(i.e. certification,transportation, work boots,temporary childcare)

Available for participantsbased on need and LICOlevel

Flow through funds


11/25

11

Funding Performance Incentive at

Maturity

Operating Budget

AllocationRecognizes:

o Results that achieveoverall service quality

standard,

Operating Budget Allocation

+ Performance Incentive

Level B (up to 3%)

Recognizes: Results that surpass overall

service quality standard within

the Level B range of results

(must exceed results from

84% of organizations SQS

results from previous year

eg. for 2008-09: 7.5)

Operating Budget Allocation +

Performance Incentive

Level A (up to 5%)

Recognizes: Results that surpass service

delivery quality standard within

the Level A range of results

(must exceed SQS results from

95% of organizations results

from previous year eg. for

2008-09: 8.0)

(2013/14)


12/25

12

Funding Performance Incentive at Maturity

A Performance Management System tracks outcomes and holds organizationsaccountable to achieving a provincial service quality standard. The process and

evaluation is transparent, accountable and responsive. It allows for a quantitative

recognition of organizations who surpass quality standards and sets the stage for

continuous improvement.

To encourage, recognize and reward stellar performance, it is proposed that aperformance incentive be integrated into the Employment Service Funding model.

The proposed performance incentive will:

be available to organizations based on full-year results. The organization mustachieve required results levels in each year to receive the performance incentivein subsequent years

be calculated against the organizations program operating budget allocation, butwill not be included in the organizations base operating allocation

recognize and reward two levels of results in overall service quality that exceedsprovincial standard

be recalculated every year based on changes from previous years results


13/25

13


Stage I: (09/10 10/11) Interim Overall Service Quality Target8.25

(linked to sustained funding)Dimension Measure Weighting Interim Target Factor

CustomerService(40%)

Customer satisfaction(participant/employer)

40% 85% 3.40

Effectiveness(50%) Employed/Career Pathand/or Training/Education 50% 79% 3.95

Efficiency(10%)

Intake/Activity 10% 90%* 0.90

Interim targets are based on Job Connect 07-08 data with the exception of Customer Satisfaction (CS)which is consistent with OPS standards. Current JC data exceeds proposed CS standard, reflecting a

mature system

Starting in 09/10 service providers will collect and report data related to all of the core measures at themature state, but at the interim stage, will be held accountable to performance on the above 3.

* Cannot exceed 100% in contribution to Overall Service Quality


14/25

14


Stage II: (11/12)

Proposed Overall Service Quality and Core Measure Standards

Dimension Core Measure Weighting Example of Core

Measure Standard*

Factor

CustomerService(40%)

Customer satisfaction(participant/employer)

15% 85% 1.28

Service Coordination 25% 30% 0.75Effectiveness

(50%)Employed/Career Path 25% 69% 1.73

Training/Education 10% 10% 0.10

Participant Suitability 15% 25% 0.38

Efficiency(10%)

Intake/Activity 5% 100% 0.50

Workshop Activities/Information Sessions 5% 100% 0.50

Proposed Overall Service Quality Standard (Link to sustained funding) 5.24

*Actual standards will be established based on full year data collected on EOIS in 10/11


15/25

15

ImplementationContract ManagementStage I: (09/10)

Set Interim Provincial Targets for initial 3 core measures using 07/08 full yearJob Connect data as proxy

Hold organizations accountable to proposed Interim Overall Service QualityTarget, based on commitments in 3 core measures (as specified on theirSchedule B)

Identify 09/10 data reporting requirements and approach to, at a minimum,capture and report aggregate data to support the 3 core measures for 2009 -2011

Stage II: (10/11)Ensure EOIS has data fields to collect all necessary data starting in 2010.

Integrate additional core measures/standards into OSQS and Schedule B asactual data is available and verified


16/25

16

Implementation - Awards and Recognition

Stage 1: (09/10-12/13)

Establish Ministers Award for Service Innovation and Service Results based on

interim targets and data collection and verification methodologies

Stage 2: (13/14)

Develop full Ministers Award to include Award for Service Quality and align

standards with fully evolved PMF. At this point, all awards based on ServiceQuality Standards and validated EOIS data.

Set Performance Incentive Funding levels and identify recipients based on datacollected in 12/13.


17/25

17

Appendix 1 - Measures, Definitions and Data

Indicators for Customer ServiceDimension Measure Definitions Data Indicators

Customer Service

40%

CustomerSatisfaction

15%

The % of individuals andemployers who reportsatisfaction with theemployment services

On a 1-5 scale, respondentswho rate satisfaction withservice at 4 or 5 level.

ServiceCoordination

25%

The provision of supportedand timely referral andaccess to services andsupports within EO and tocommunity services outsideof EO

Referred in from othercommunityorganizations/services

Registered foreducation/training to completehigh school or equivalent

Registered/participating in

trainingRegistered or confirmedreceiving services with othercommunity resources thatsupport employment


18/25

18

Customer Service (40%)

Service Coordination (25%)

Client Service Planning and Coordination indicators recognize, as part of a clients

service plan development, the support an agency provides to help them access andparticipate in education, training, and/or other types of community services either whilethey are engaged in employment services or at exit from employment services.

A formalized referral (a planned, supported and co-ordinated arrangement) is made bythe employment service provider on behalf of the client to the other organization(education/ training/community services). It also recognizes formalized referrals madeto the employment service provider from another community organization on behalf of aclient.

Customer Satisfaction (15%)

The Customer Satisfaction indicators recognize the value attributed to the service by its

customers. Employment service participants both individuals and employers aresampled at exit from service, on a 1-5 scale, on how likely they are to recommendthese employment services to someone looking for similar services. Service providersare assessed on the percentage of participants sampled who return a 4 or 5 (out of 5)rating.


19/25

19

Measures, Definitions and Data Indicators for

EffectivenessDimension Measure Definitions Data Indicators

Effectiveness

50%

ServiceInputEmployed/on CareerPath

25%

Success at enteringthe labour marketand/or on a careerpath based ongoals/qualifications

experience

Employed (with definition)Self-employedBoth employed and intraining/educationEmployed and a registered apprentice

Employed in a more suitable jobconsistent with individuals career pathEmployed in professional/tradeoccupation or profession/trade relatedoccupation consistent witheducation/training qualifications

Service

Input -Education/Training

10%

Registered in

education/recommended toregister in training tosupport careergoals/path

Individual has returned to school full

time or part-time at exit (includes 4breakout definitions)Individual has been referredto/recommended for registration in EOtraining (includes of 8 breakoutdefinitions)


20/25

20

Effectiveness (cont)

Dimension Measure Definitions Data IndicatorsEffectiveness

50%

ParticipantSuitability

15%

Indicators of participantneed (barriers toemployment) foremployment services

-Age - Under age 20/Over age 44-


21/25

21

Effectiveness (50%)Effectiveness is captured by two key measures which are linked together. The first,

Participant Suitability, is an activity indicator that gathers data on the clientsaccessing employment services from the service provider. It examines andquantifies identified barriers to employment facing the clients. These indicators ofParticipant Suitability work in conjunction with the outcome indicators of serviceimpact to acknowledge the outcomes achieved by the service providers accordingto their client group. This weighted calculation ensures that the service providersare appropriately recognized for service delivery to the most disadvantaged clients.

Participant Suitability (15%)

The Participant Suitability indicators have been identified in employment literatureas potential barriers to successful labour market attachment. Service providerscollect this information at intake (entrance) to the full suite employment service.

Service Impact (35%)

The Service Impact performance indicators are collected at exit from the full suiteemployment service. Service Impact Outcome indicators will be tracked @ exit and3, 6 and 12 months after exit from service.


22/25

22

Measures, Definitions and Data Indicators for

EfficiencyDimension Measure Definitions Data Indicators

Efficiency 10% FundedActivity

5%

The # of individuals forwhich the organization willdevelop EmploymentService plans

Start date at which the organizationbegins providing employmentservices to the individual

Informationsessions/workshopactivity

5%

The # ofworkshop/informationsession participants.

Total # of individuals whoparticipate in workshop/informationsessions (see definition forworkshop/information session)


23/25

23

Efficiency (10%)Funded activity captures activity measures relating to economies of scale and

organizations ability to meet contracted service levels. It recognizes the organizationsactivity to provide full suite employment services and resource and information to self-directed individuals.

Intake/Activity (5%)

Intake/Activity compiles the number of individuals for which the organization develops afull suite employment service plan and calculates it as a percentage of their contracted

service level. It is expected all organizations will meet or exceed this level. It does notinclude individuals for whom the organization refers on to another service withoutengaging in development of service plan. Organizations will not receive additionalrecognition for exceeding 100% of funded activity levels.

Information Sessions/Workshop Activity (5%)

Information Sessions and Workshop Activity is a measure that serves as a proxy foractivity relating to individuals who do not require full suite services and wish to access

resources and information. This measure collects the # of workshop participants.Orientation workshops/information sessions are defined as a structured learningsession that must contain comprehensive and relevant information which can beconducted in group sessions (on or off site), e-learning sessions or one-to-one (whenappropriate) these sessions can be up to 2 days in duration.


24/25

24

Appendix 2:

Analysis to Support Performance Incentive

In 2007-08, organizations under contract to delivery JC services (132 sites) were heldaccountable to a minimum Provincial Service Quality Standard of 6.0.

All but one met and surpassed this level.

Assessment was completed on the distribution of the remaining 131 contracts andfoundthe average result was 6.937, with a standard deviation of .493

71 (54%) agencies posted scores between minimum Provincial Service QualityStandard and the arithmetical average; 41 agencies were between average and 1ststandard deviation15 agencies (12%) scored between 1st and 2nd standard deviations and 4 agencies

(3%) met Provincial Quality Standard of 8 or above.


25/25

25

espms presentation aug 28

Documents