Guest post: How to choose the ‘right’ LLN testing tool

25 Sep Guest post: How to choose the ‘right’ LLN testing tool

Posted at 16:35h in Assessment, Foundation Skills, News by

This post is written by my colleague, Ann Leske.

Ann and I work together on a joint venture called, LLN and VET Meeting Place.  Lately, we have received loads of questions and cries for help from RTOs who want to ‘do LLN right’.  They are looking for guidance on how to devise, then implement LLN pre-testing processes that work.

Ann’s post answers many of these questions.  It offers a step-by-step process to help your RTO systematically evaluate and choose the best LLN pre-training testing process for you.

Enjoy…

It makes good sense to identify learners’ support needs

If the Standards for RTOs 2015 guide your training and assessment processes,  then you’ll recognise Clause 1.7 – Support learners:

To maximise the chance of learners successfully completing their training, your RTO needs to… identify any support individual learners need prior to their enrolment or commencement (whichever is the earliest)

 

Foundation skills (including LLN skills) underpin the ability to perform all workplace tasks. Regardless of the task, foundation skills are involved. It makes good sense to have a robust process to identify learners’ needs with a focus on foundation skills.

This post presents 3 considerations to guide your RTO when deciding which LLN assessment tool is best for you and your learners:

  1. Consider the foundation skills that matter most
  2. Consider learners’ characteristics
  3. Consider paper-based, or online tool

 

Step 1: Consider the foundation skills that matter most

Begin by checking the range and complexity of the foundation skills required to demonstrate task competency so that when you identify learners’ existing foundation skills you can make an informed decision about cohort and individual support strategies.

All workplace training requires graduates to perform a range of foundation skills. Clarify the foundation skills that matter most in your industry context now.

We suggest you refer to:

  • industry employers and stakeholders
  • accredited training package, and Unit of  Competence
  • the performance descriptions in the Australian Core Skills Framework (ACSF), and the Core Skills for Work (CSfW).

What to think about

  • When will the assessment be administered – prior to, or upon commencement?
  • Does the Pre-training assessment cover the range of foundation skills you need to know about? What additional evidence may need to be gathered?
  • How are the learner’s foundation skills outcomes reported?
  • Does the report provide sufficient information to make informed decisions about the support learners may need?
  • For auditing, what documentary evidence is required?

Your goals:

  1. identify the range of foundation skills that matter most in your industry context.
  2. identify a pre-training assessment tool that assesses and reports on the range of foundation skills that matter most in your industry context.

 

Step 2: Consider learners’ characteristics

Knowing the learners’ characteristics may influence the process and tool you select.

Generally, generic contexts are preferred. Familiarity with the content is important – for example, content related to the everyday life of an adult living in the community. Unfamiliar content can be a barrier to interpreting instructions, tasks,  and resources. Barriers can skew the outcome,  and provide you and the learner with information that does not indicate learners’ competency.Statistically, the ABS reports that 1.3 million Australian households were without internet access at home in 2014–15 (14%). Even though most homes have a device connected to the internet, the download limit will vary. Accessing a pre-training assessment via a device may be a barrier for some learners.

What to think about:

  • Will the learners manage the language used in both the instructions to get started and the assessment tasks? Is plain English demonstrated in all texts learners must read?
  • How long will the process take to complete?
  • Are learners aware of available support and adjustments where meeting individual needs are necessary?
  • How are the assessment outcomes reported to the learners? Where is the information accessed?

Your goal is to, ask:

  • will the pre-training assessment tool context be recognisable to most learners?
  • is the pre-training assessment tool easily accessed by the learner cohort?
  • if necessary, can the learners access the pre-training assessment in an alternative way?

 

Step 3 Consider: paper-based or online tool?

What to think about

  • Given your annual applicant numbers, which assessment process (face-to-face, or online) can be efficiently, and reliably implemented?
  • What is the process to connect learners to the tool – who will be responsible for this?
  • Does the existing structure and processes allow time for team members to design, trial and validate a tool  (if developing your own)?
  • If using a fee for service on-line tool, how will the administration of participants passwords (or similar) be managed? What time is involved?

Design your own paper-based or online

This can be a complex process to get right. It requires sufficient knowledge of the foundation skills range and complexity to ensure the tasks, tools, instructions, stimuli, and assessor marking guides align with the intended foundation skills level, range and complexity. It’s very easy to be off-track.

There’s a lot at stake for both the learners, and you, if the tasks are not valid and the identified, or indicated foundation skills held NOW, are not reliable.

Helpful resources and advice

Existing paper-based tools

Precision Consultancy ACSF tools offer a range of generic and contextualised paper-based tools. Precision Consultancy was commissioned by the government to prepare a range of tasks across the 5 Language Literacy and Numeracy core skills; each is validated by foundation skills specialists, and freely available.

The tools demonstrate instruction, task, text, and stimuli complexity from ACSF levels 1 – 4. Each task provides Assessor performance description criteria.

Your goal: to identify the foundation skills and knowledge the learners demonstrate NOW, especially the foundation skills that matter most in your training context.

If you use Precision Consultancy ACSF tools as the Pre-training assessment tool:

Your goal is

  •  to match the tasks with the range of foundation skills matter most in your training context (ie writing, numeracy, learning).
  • to consider the ACSF levels you will assess

 

 

Your goals are to ensure:

  • that tasks reflect the foundation skills that matter most
  • the tool is trialled, to check for foundation skills consistency
  • tasks are validated by a foundation skills expert
  • assessors have explicit foundation skills performance criteria

 

Validity and reliability advice

The Assessment of LLN testing tools for the VET student loans program. was prepared to guide to prepare a Pre-training LLN tool for approval under the requirements for VET Fee Help (Now VET Student Loans).

This document comprehensively explains the importance of validity and reliability with a Pre-training assessment tool and provides advice about what’s necessary to achieve validity and reliability with an ACSF focus.

 

ACER Foundation Skills Assessment Tool (FSAT) includes a mix of online and paper-based assessments

FSAT is the first of its kind in Australia.  It covers all 15 foundation skills across 5 levels of complexity – LLN (ACSF) and Employability skills. (CSfW) It has been extensively validated by foundation skills experts (this is ongoing as it is in a thorough trial phase) – and it’s freely available.

For more information about the tool including how the different core skills are assessed:

At present, FSAT may not be used for VET Student Loan learners.

Addendum 01 July 2018:  We are sad to say that the government has chosen to discontinue offering FSAT.  I – and many others – are dismayed by this decision, particularly because:  1) This was the only assessment tool that assessed both the ACSF core LLN skills, and the Core Skills for Work, and 2) FSAT was the result of years of work and testing by some of Australia’s top foundation skills practitioners and was, therefore, one of the highest – if not THE highest – quality assessment tool of its type available in Australia; 3) It was available at no cost to users.

DET approved on-line tools

These Language Literacy and Numeracy (LLN) assessment tools have been approved by DET for VET student loan program student entry requirements. Currently there are 6 approved tools. Each can be used as a Pre-training assessment tool for all students.

The tools must identify if learners Reading and Numeracy skills competency is at, or above, or below ACSF Level 3. Some of the tools on the list include assessment of Oral communication, Writing and Learning as well. Each requires  a service fee payment.

The list current as at 9 March 2017, is below and is updated as tools are approved.

Your goal: identify the foundation skills and knowledge the learners demonstrate NOW, especially the foundation skills that matter most in your training context.

Ask:

  • does the tool assess and report the foundation skills information I, or our trainers, need to know?
  • has more than one tool been trialled?
  • what are the implementation considerations for us? (see below)

 

 

Yes, there’s a lot to think about!  We encourage you to share your experiences.

 

Print page