Skill Gains (Real Time Indicator)

Measurable Skill Gains in FutureWork's Performance Matters

The Measurable Skill Gains indicator is the percentage of participants who, during a program year, are in an education or training program that leads to a recognized postsecondary credential or employment and who are achieving documented academic, technical, occupational, or other forms of progress, towards such a credential or employment (see 20 CFR sec. 677.155(a)(1)(v)).

The measurable skill gains indicator is used to measure interim progress of participants who are enrolled in education or training services for a specified reporting period. Therefore, it is not an exit-based measure. Instead, it is intended to capture important progressions through pathways that offer different services based on program purposes and participant needs and can help fulfill the vision for a workforce system that serves a diverse set of individuals with a range of services tailored to individual needs and goals.

Depending upon the type of education or training program in which a participant is enrolled, documented progress is defined as one of the following:

1. Documented achievement of at least one educational functioning level of a participant who is receiving instruction below the postsecondary education level;

2. Documented attainment of a secondary school diploma or its recognized equivalent;

3. Secondary or postsecondary transcript or report card for a sufficient number of credit hours that shows a participant is meeting the State unit’s academic standards;

4. Satisfactory or better progress report, towards established milestones, such as completion of OJT or completion of one year of an apprenticeship program or similar milestones, from an employer or training provider who is providing training; or

5. Successful passage of an exam that is required for a particular occupation or progress in attaining technical or occupational skills as evidenced by trade-related benchmarks, such as knowledge-based exams.

Examples:

• A participant is enrolled in a 4-year registered apprenticeship program: the measurable skill gains indicator tracks the skills the participant gains throughout the reporting period, not just at the end of the 4-year training program.
• Low-skilled adult participants of an adult education program: the measurable skill gains indicator provides an opportunity to track and report gains in reading, writing, mathematics, and English proficiency.

Documenting Progress for Types of Measurable Skill Gains

1. Documented achievement of at least one educational functioning level of a participant who is receiving instruction below the postsecondary level – Programs may measure educational functioning level gain in one of three ways:

(a) States may compare the participant’s initial educational functioning level, as measured by a pre-test, with the participant’s educational functioning level, as measured by a post-test;⁴

(b) States that offer adult high school programs that lead to a secondary school diploma or its recognized equivalent may measure and report educational gain through the awarding of credits or Carnegie units; or

(c) States may report an educational functioning level gain for participants who exit a program below the postsecondary level and enroll in postsecondary education and training during the program year. A program below the postsecondary level applies to participants enrolled in a basic education program.

2. Documented attainment of a secondary school diploma⁵ or its recognized equivalent – Programs may document attainment of a secondary school diploma or its recognized equivalent if the participant obtains certification of attaining passing scores on all parts of a State-recognized high school equivalency test, or the participant obtains a diploma or State-recognized equivalent documenting satisfactorycompletion of secondary studies or an alternate diploma⁶, including a high school or adult secondary school diploma.

3. Secondary or postsecondary transcript or report card for a sufficient number of credit hours that shows a participant is meeting the State unit’s academic standards – For secondary education, this gain may be documented through receipt of a secondary transcript⁷ or report card for one semester showing that the participant is achieving the State unit’s policies for academic standards. For postsecondary education, this gain must demonstrate a sufficient number of credit hours—which is at least 12 hours per semester (or equivalent⁸) or, for part-time students, a total of at least 12 hours over the course of two completed semesters (or equivalent) during a 12 month period that shows a participant is achieving the State unit’s academic standards (or the equivalent for other than credit hour programs). For example, if a postsecondary student completed 6 hours in the spring semester and 6 more hours in the fall semester and those semesters crossed two program years, they would not count as a skill gain in the first program year but they would count as a skill gain in the second program year.

4. Satisfactory or better progress report, towards established milestones, such as completion of OJT or completion of one year of an apprenticeship program or similar milestones, from an employer or training provider who is providing training – Documentation for this gain may vary, as programs should identify appropriate methodologies based upon the nature of services being provided, but progress reports must document substantive skill development that the participant has achieved. The gain may be documented by a satisfactory or better progress report from an employer or training provider. Progress reports may include training reports on milestones completed as the individual masters the required job skills, or steps to complete an OJT or apprenticeship program. Increases in pay resulting from newly acquired skills or increased performance also can be used to document progress.

Note: In the description of this type of Measurable Skill Gains, “completion of one year of an apprenticeship” is just one example of a timeframe that may be established for achieving a satisfactory or better progress report toward a specific milestone, and the “one year” timeframe should not be construed as a required timeframe or the only way that a participant in an apprenticeship can achieve a Measurable Skill Gain.

5. Successful passage of an exam that is required for a particular occupation or progress in attaining technical or occupational skills as evidenced by trade-related benchmarks, such as knowledge-based exams – Documentation for this gain may include passage of a component exam in a Registered Apprenticeship program, employer-required knowledge-based exam, satisfactory attainment of an element on an industry or occupational competency-based assessment, or other completion test necessary to obtain a credential.

Methodology:
Calculation includes all participants:
The number of program participants during the reporting period who are in an education or training program that leads to a recognized postsecondary credential or employment and are achieving measurable skill gains based on attainment of at least one type of gain DIVIDED by the number of program participants during the reporting period who are in an education or training program that leads to a recognized postsecondary credential or employment.

Participants who, during any point in the program year, are in an education or training program that leads to a recognized postsecondary credential or employment are included in the denominator. This includes participants who continue to receive services as well as those who have participated during the reporting period and have exited the program. Data for the denominator in this calculation is drawn from PIRL 1811: Date Enrolled During Program Participation in an Education or Training Program Leading to a Recognized Postsecondary Credential or Employment.

The numerator is the number of program participants defined above who achieved at least one type of gain. A participant may have achieved more than one type of gain in a reporting period; however, only one gain per participant in a reporting period may be used to calculate success on the measurable skill gains indicator. These calculations are described in Attachment 10, Figure 7.

Operational Parameters:
All participants who, during a program year, are in an education or training program that leads to a recognized postsecondary credential or employment are counted in the calculation of this indicator. Participants who exit for any of the reasons listed in Attachment 2, Tables A and B are excluded from the measurable skill gains indicator. The following participants in education or training programs are included:

• Title I Adult and Dislocated Worker – All participants who are in a title I Adult- or Dislocated Worker-funded training program are included in the measurable skill gains indicator (which includes funding a training program for a secondary school program equivalent). This includes all participants in work-based training. (Refer to Incumbent Workers Training section on page 40.)
  
  
• Title I Youth – All ISY are included in the measurable skill gains indicator since they are attending secondary or postsecondary school. Only OSY who are in one of the following are included in the indicator:
  - the program element occupational skills training
  - secondary education at or above the 9th grade level during participation in the title I Youth program
  - postsecondary education during participation in the title I Youth program
  - Title II-funded adult education at or above the 9th grade level during participation in the title I Youth program
  - the YouthBuild program during participation in the title I Youth program
  - Job Corps during participation in the title I Youth program
  
  
• Title II AEFLA – All participants in title II AEFLA programs are considered to be in an education program leading to a recognized postsecondary credential or employment and, as such, would be included in the measurable skill gains indicator.
  
  
• Title IV VR – All VR program participants who are in an education or training program that leads to a recognized secondary or postsecondary credential or employment, which is identified on the individual’s IPE, would be included in the measurable skill gains indicator.

Additional Operational Parameters:

• Participants are only included in the denominator one time per reporting period (i.e., program year), regardless of how many skill gains they achieve in a given program year unless the individual has more than one period of participation in a given program year. Likewise, participants are only included in the numerator one time per reporting period (i.e., program year), regardless of how many skill gains they achieve in a given program year unless the individual has more than one period of participation in a given program year (See Section 9, page 36, for discussion on periods of participation.);
  
  
• A participant who exits the program and re-enrolls in the program during the same program year and is in an education or training program will be in the indicator two times for that particular program year, as explained in the discussion of periods of participation in Section 9 of this guidance;
  
  
• The measurable skill gains indicator is different from the other indicators because it is not exit-based, meaning that a participant can achieve a measurable skill gain while still participating in a program; and
  
  
• Programs should not delay enrollment or services to participants until a new program year even if programs believe there is insufficient time for the participant to make any type of measurable skill gain by the end of that program year.

For performance accountability purposes, the measurable skill gains indicator calculates the number of participants who attain at least one type of gain during each period of participation within a given program year. Since this indicator is not exit-based, each unique program entry date (not exit date) triggers inclusion in the calculation. Participants will achieve a successful outcome in the indicator as long as they attain one type of gain applicable to the core programs. States should report all measurable skill gains achieved by participants in a program year, although only one gain is required per participation period per program year to meet performance outcomes. See the example below for how this would apply in a typical scenario.

Example:
Chris enters an American Job Center and becomes a participant on October 2, 2016 (PY16). He exits the program on February 10, 2017. During this time, Chris achieves two types of gain under the measurable skill gains indicator. He re-enters the program as a participant on June 11, 2017 (PY16). By the end of the program year (June 30, 2017), he is still in his second participation period. During this time, Chris obtains an additional type of gain under the measurable skill gains indicator. In this example, Chris has two periods of participation and two positive outcomes on the measurable skill gains indicator. Although two types of gain were achieved in the first period of participation, only one of the two types of gains counts toward the indicator in the first participation period. During the second period of participation, another type of gain was achieved before the end of the program year, which counts as another positive outcome towards the measurable skill gains indicator.

This information is collected, for all core programs (except the title III Employment Service program), as part of the Measurable Skill Gains Report Template. If a participant achieves more than one type of measurable skill gain in a reporting period, the most recent gain is the skill gain type that should be recorded on the Measurable Skill Gains Report Template.

Operational Parameters – Individual Core Programs:
The appropriate types of measurable skill gains for each core program are detailed in the table below. These parameters are intended to focus performance accountability under measurable skill gains on the services that are allowable under the respective statutory provisions.

Operational Parameters - Individual Core Programs
Core Program	Type of Measurable Skill Gains
Title I – Adult and Dislocated Worker	Measured by achievement of any of the 5 types of measurable skill gains No specific measurable skill gain types required for specific Adult or Dislocated Worker participants
Title I – Youth	Measured by achievement of any of the 5 types of measurable skill gains No specific measurable skill gains types required for specific Youth participants Type of skill gain should be based on the youth’s individual service strategy
Title II – AEFLA	Measured by: - Achievement of at least one educational functioning level, OR - Documented attainment of a secondary school diploma or its recognized equivalent.
Title IV – VR	May be measured by achievement of any of the 5 measurable skill gains.

---

⁴ The approved pre- and post-tests must be based on the list of tests the Secretary of Education determines to be suitable for use in the National Reporting System for Adult Education. The list of approved assessments is published annually in the Federal Register. See Tests Determined To Be Suitable for Use in the National Reporting System for Adult Education; Department of Education, 81 FR 89920 (December 13, 2016). https://www.federalregister.gov/documents/2016/12/13/2016-29899/tests-determined-to-be-suitable-for-use-in-the-national-reporting-system-for-adult-education.
⁵ Secondary school diploma refers to a regular high school diploma, as defined in section 8101(43) of the ESEA, as amended by the ESSA. ⁶ Alternate diploma must meet the requirements under the ESEA, as amended by the ESSA.
⁷ Secondary transcript is specific to youth attending high school.
⁸ For gain type three, the Departments recommend that States and local areas develop policies suitable for the applicable academic system in use by the secondary or postsecondary institution in which the participant is enrolled including, but not limited to, semesters, trimesters, quarters, and clock hours for the calculation of credit hours (or their equivalent) when documenting progress towards Measurable Skill Gains.