Graduate employment: its limits in measuring the value of higher education

Response rates for the survey have been relatively low, at 53% full responses from UK domicile graduates compared with the 70% target. Research found that the response rates had no major impact on the accuracy of statistics drawn from the data. However, there is no guarantee that this will be the case for future cohorts.

Statistics from the GOS have been rated positively by the Office for Statistical Regulation (OSR) in an initial review against the Code of Practice for Statistics which looks at the trustworthiness, quality and value of statistics. 

However, the statistics are still experimental until there is a fuller assessment by the OSR in a few years' time. Until then, the highly skilled employment measure shouldn’t be used for large-scale decision-making without wider context. This is both as a measure by itself, or as part of a composite measure (a combination of measures that make up one measure).

Although good outcomes are important for students, students have different ideas of what a ‘good outcome’ is. This is because:

• Around half of students expect to get ‘graduate-level employment’, but it’s unclear if their definition of ‘graduate-level employment’ is the same as the
  SOC’s definition.
• Students have different reasons for going to university, including valuing education and gaining knowledge.
• Good quality work has many dimensions – of which using skills is just one.
• Someone’s ambitions and expectations can change during and after their studies and throughout their career.

Progression into highly skilled employment only captures outcomes during one week at one point in time (15 months after graduation) This has its limits, for example:

• This can often reflect negatively on graduates with non-standard career pathways, including those taking a break after studies.
• The speed at which someone typically enters a highly skilled role can differ a lot by sector. Some in the creative arts, for example, are often expected to take on roles that fall outside of this category before progressing in their career.
• Even benchmarking by subject may not account for differences at course level.
• Career pathways might be influenced by factors such as a student’s socioeconomic background. This might affect their access to networking and work experience opportunities, and mean they aren’t able to enter a highly skilled role immediately after graduation.

The SOC grouping system was not designed to tell the difference between graduate and non-graduate jobs.

The SOC approach uses job titles to determine whether the role is ‘professional’ or not using a pre-existing list, but it’s questionable how much holding a professional occupation tells us about graduate progress.

Job titles alone do not tell us whether the job is suitable for a graduate, especially as employers benefit from the skills graduate workers bring to all roles.

Classification of the skill levels of occupations isn’t updated regularly. It therefore falls behind changes in the labour market. In particular:

• SOC codes and categorisation systems are only revised every 10 years.
• The current version of SOC (SOC 2020) was recently updated. As we move further away from the last cycle, accuracy of occupation categories across cohorts of graduates will decrease.
• For example, 7% of graduates classified as ‘medium-skill workers’ in earlier definitions are classified as high-skilled in updated categorisation.

Students on ‘one-year interim study’ are studying for one year after they graduate and before they complete the survey. These graduates make up around 8% of respondents with known outcomes. Courses with high numbers of graduates on one-year interim study are likely to have inflated unemployment and non-highly skilled
employment figures.

This is because:

• Many in this category will have completed one-year courses, such as master’s programmes, only three or fewer months before responding to the survey.
• As we might expect, first degree graduates with significant interim study report higher rates of unemployment in the survey.
• UK graduate unemployment rates (including those due to start work or further study) are 10% higher when graduates with interim study experience are included compared to when they are left out.

Whether a graduate is in a ‘highly skilled job’ is less relevant than if they are in a job that matches their skill level.

Graduates who enter jobs that are typically held by those with lower-level qualifications suffer from low job satisfaction and high indicators of poor mental health. Even when compared to their colleagues with lower qualifications.

However, the negative effects of ‘underemployment’ are not seen as clearly when this is identified using job titles and SOC codes. We get a better picture by looking at the skills someone says they are using.

Highly skilled employment measures are only based only on historical data and are unlikely to reflect recent changes in the labour market. This means:

• It doesn’t help current decision-makers, as it’s based on data from graduates who started their studies four or more years ago.
• To be useful for prospective students, the measure assumes that there will be no meaningful change to the labour market in the years between the graduate outcome data being collected and any new user group graduating.
• It doesn’t take into account how much courses can change, particularly as many providers are exploring blended learning and responding to student feedback.

Economic recovery and growth are not just dependent on the level of skills in the workforce, but also how they are used to meet current employer demand.

The World Economics Forum estimates that 97 million new jobs will emerge between 2020 and 2025 and that the demand for different skills delivered through higher education, such as critical thinking and problem-solving, will only increase during this period.

Current measures of highly skilled employment cannot tell us how well graduates fill these skills gaps.

High skilled employment does not help us understand how graduates address local and regional skills needs.

A far more useful measure would show which courses have graduates who take skilled jobs that support growth in parts of the country that would benefit from levelling up.

Such a measure might use data about the location of graduates, as well as up-to-date information about vacancies and skills shortages across different parts of the country.

Graduates engaging in start-ups and other entrepreneurship may be valuable, though access to some relevant details will depend on HESA’s upcoming review of its Higher Education Business and Community Interaction survey.

If used inappropriately, this measure tells us nothing about the value added by higher education.

The likelihood of a graduate being employed in a highly skilled role is influenced by multiple factors outside of a university’s control. This includes where they live, ethnicity, and socioeconomic background.

Without accounting for this, either by benchmarking or using contextual information, there is a risk of negatively affecting providers with high numbers of students who are less likely to access highly skilled employment straight after graduating, even with degrees.

It’s crucial that the level at which graduate employment is assessed ensures students are supported in their career aspirations even where students’ individual circumstances mean they are less likely to gain immediate entry to ‘highly skilled employment’.

The OfS has released a series of experimental statistics to explore how highly skilled employment rates differ by graduate location. It is critical for policymakers and everyone using the data to use the indicators the OfS suggests, or any others that account for regional differences. This will support the efforts universities are making to local employment and widening participation.

Employers should be involved in developing skills and employment measures that reflect their needs. These could inform changing needs across different sectors and different parts of the country.

Apply benchmarking or other techniques that consider how opportunities vary across different student groups. The methods should be evidenced and updated to include new knowledge about external factors of graduate employment and new data.

Data is available that can build more nuanced measures of good employment outcomes.

For example, measures can be developed based on survey responses to:

• How much graduates use skills gained during their studies. To help account for differences in abilities and opportunities for applying higher level skills in the role, even in roles that are technically for non-graduates.
• Whether their current work fits in with their future plans. To help identify career trajectories that aren’t standard and might not be seen using the highly skilled employment’ measure. This measure will also capture some of the variation in graduates’ aspirations.
• Whether their current work is meaningful.
• The qualifications which were required or helpful for securing their role. Unlike using job titles, these responses will be sensitive to variation in similar jobs, and the ability for graduates to add value to traditionally non-graduate roles.

HESA have used some of the data to develop an experimental composite measure of ‘fair work’. Early evidence indicates that, while there are regional differences in job quality, these do not favour areas such as London and South East England that typically perform well in earnings and highly skilled employment rates because of their labour markets.

It is still important for these measures to account for contextual factors such as student demographics – something that will require our understanding and practices to be developed.