Part 6: Monitoring and assessment tool for activities with young people
Monitoring and assessment tool for activities with young people
Introduction
Ongoing monitoring and assessment, as well as summative evaluation of youth work trainings, workshops and interventions is a must for several reasons. We need monitoring and assessment in order to be able to adjust the process to the participants and their needs, so that we achieve the desired results. We need evaluation at the end in order to measure the results of the initiative against its goals. This chapter is focused on understanding monitoring and assessment as well as evaluation and developing skills to apply them. It also prepares the youth workers to apply monitoring, assessment and evaluation specifically to the FABLE workshops.
In detail, we focus on the following topics:
- What are monitoring, assessment and evaluation and why they are important?
- Evaluation
- Ongoing monitoring and assessment
- Monitoring, assessment and evaluation methods and how to use them
- FABLE evaluation tools
At the end of the module we expect that youth workers will be able to implement ongoing assessment and summative evaluation in their trainings and interventions. Also they will be prepared for assessment and evaluation of the FABLE G-Local Hub workshops.
What are monitoring, assessment and evaluation and why they are important?
Constant monitoring and assessment of trainings and interventions with young people is a prerequisite for their effectiveness. Ongoing assessment, also called formative assessment, increases the chances of achieving the goals of youth work interventions, like trainings, workshops and other. It involves continuously gathering information, which is then used to adjust the instruction and work process in order to achieve the goals set. Evaluation on the other hand is done at one moment in time in order to find out if the goals that have been set at the beginning for the respective unit or for the whole intervention have been met.
To understand better what assessment and evaluation are, see the video: Evaluation and Assessment, by Rebecca Burton (automatic subtitles available).
Evaluation
A very good evaluation model is the Kirkpatrick Model, which has been a standard for training evaluation in the last fifty years. It is also completely applicable to youth work interventions.
The model suggests that the training results should be evaluated at 4 levels. These 4 levels, applied to youth work are:
Reactions – impact on the young person. The extent to which young people find the training or intervention useful, engaging, and relevant to their situation and needs. Reactions are usually evaluated using questionnaires or interviews, conducted at the end of the training or intervention. They measure the subjective perception of the young person about the process and results of the training, but not the impact on behaviour or performance. These surveys can collect rich, qualitative data, but they can also turn the program toward what is enjoyable rather than what will be most effective.
Learning – impact on the competences that are expected to be developed. Have the desired new knowledge, skills and attitudes been acquired as a result of the participation in the training or intervention? The learning process is about acquiring new competences: knowledge and skills, attitudes, motivation. Evaluation at Level 2 is usually done by comparison: before/after and measurement of changes in competences that can be directly attributed to the training, workshop, or other intervention. Formal assessments and examinations are examples of evaluation at Level 2.
Behaviour – impact on the behaviour of young people, for example, for the FABLE programme, are they applying more creativity and better problem-solving strategies, are they behaving more confidently and proactively? Behaviour refers to the successful application of the acquired new competences. Level 3 evaluation can be carried out using formal assessment or through less formal approaches such as observation.
Results – impact on the quality of life of the young people. The extent to which targeted outcomes are achieved as a result of the intervention. Are the young people more successful in finding jobs or starting initiatives that improve their wellbeing? Do they feel more connected, included, and meaningful? This level is the hardest to measure, since it requires some long-term relationship with the people that have taken part in the initiative, but on the other hand, if possible, to be implemented, it is the ultimate measure of the usefulness and effectiveness of the activity.
In order to be able to monitor, assess and evaluate any type of training or intervention we need to first define their goals. These goals are defined mainly at Level 2: Learning and at Level 3: Behaviour, but may also include more long-term goals at Level 4: Results. Based on the goals, the trainer or youth worker sets a number of indicators, which will be used to evaluate the quality and the effectiveness of the training/intervention.
In order to understand better the Kirkpatrick Model, see the video The Kirkpatrick Model of Training Evaluation by Devlin Peck (automatic subtitles available).
Ongoing monitoring and assessment
In order to ensure the achievement of the goals set for a training or intervention with young people we implement continuous monitoring and assessment during the course of the training/intervention. This is also called formative assessment.
Ongoing assessment shows the young person’s progress throughout the training or intervention. This approach requires the youth worker to assess the participant’s engagement, motivation, level of understanding and participation throughout the course of the activities. Ongoing assessments is extremely important for interventions with young people, because it allows the youth worker to identify how to adjust the course of the intervention or training to the particular group of people, they are working with in order to achieve the goals. For example, we ask the group a question in order to see if they understand what we are talking about or the task they have to perform. If they do not understand, we go back and explain in another way.
A key principle of using ongoing assessment is to clearly communicate the learning/intervention objectives to the participants. This communication not only sets expectations for them, but also enables them to self-monitor their learning progress. Another approach to using ongoing assessment is to require learners to record conclusions (reports) on their tasks and exercises in a workbook or electronic file. This self-assessment approach allows the participant to focus
on what they have learned.
Ongoing assessments requires the trainer or youth worker to provide clear feedback to the participants, to clarify and correct errors, to discuss observations and conclusions from participatory interventions. This is an important contribution to acquiring the competences and changing the behaviours.
Monitoring, assessment, and evaluation methods and how to use them
These are more formal and individual methods, which allow the trainer/youth worker to gather some specific information without any social influence of the group. However, they require knowing what kind of information we need and what questions will lead the young people to give it us.
Individual or in group, they are probably the best way to observe the evolution of the young people during the training or intervention. They:
- allow young people to explore and experiment with their new knowledge and competencies;
- give some concrete aspects to the theory learned;
- show to both young people and youth worker the practical competency acquired (or not).
Very close of the previous method, simulations put the young people in a situation which could really happen in their life, for example when looking for a job or starting new work/initiative. They allow the youth worker to see how prepared the participants are for one or another particular situation and observe their behaviour.
Informal and individual, they are probably the best way to gather all the information needed because they offer the possibility to ask questions individually. However, they are more difficult to use without influencing the answers. Moreover, they require a good level of trust between the people involved to assure the honesty of the answers.
The participants are probably the best source or information about the right progress of a training or intervention. Feedback and comments can be intentionally gathered by the youth worker, at important milestones of the training or intervention to solicit information about progress and understanding. It is also important to provoke spontaneous comments and feedback throughout the whole duration of the intervention or training.
A lot of information comes from the simple observation of the young people during the workshop. The youth worker can prepare some checklist/table to help them qualify the participant’s behaviour and competencies. Observation of the young people and the results of their work can provide information on all 4 levels of The Kirkpatrick Model.
- Level 1: Reaction. For example, if the youth worker observes negative reaction to his/her facilitation style they can adjust it. Generally, motivation and engagement are good signs of a youth worker – young person relationship that will lead to positive change.
- Level 2: Learning and Level 3: Behaviour. The youth worker can prepare some scale, based on the training/intervention goals to help them evaluate and analyse the participant’s behaviour and competencies.
- Level 4: Results. Evaluating at Level 4 requires long-term commitment of either the youth worker or young person, ideally of both. The most important part is to set measurable indicators against which results can be monitored and evaluated.
There are a large number of methods that can be used to implement ongoing assessment in face to face and in online activities. Here are some videos where you can find some inspiration.
FABLE evaluation tools
Within the FABLE project we are going to implement summative evaluation (evaluation at the end of the work with young people) through questionnaires for all initiatives undertaken with young people. Here we provide the goals of these initiatives (G-Local Hub activities), indicators for evaluation and a questionnaire (Annex 1) to be used at the end of every session with young people.
For the formative (ongoing) monitoring and assessment youth workers are free to apply what they have learned in this chapter as well as their creativity and choose the most appropriate according to the situation methods.
G-Local Hub Workshop goals
- to promote the acquisition of youth entrepreneurship skills through the sustainable fashion
- acquisition of transversal skills like critical thinking, creativity
- promoting a sense of belonging to a community
- motivation in acquiring transversal competences for the future
- awareness on the topic of waste resources in the local community and sense of initiative to create together something that can make a difference
- to stimulate a change of mind and perspective towards green transition and an ethical approach to the fashion world
- to stimulate the curiosity of young people towards sustainable topics and propose the idea that the new careers on the green transition can be the future for everyone, especially for young people
- to create a safe and inclusive space for young people who are looking to build competences and at the same time explore their potential future
Indicators to gather feedback on, at the end of the workshops
- level of satisfaction of young people who participated in the workshop
- acquired new entrepreneurship skills
- acquired new transversal skills like critical thinking, creativity
- extent to which participants have felt a sense of belonging and community as a result of their participation in the G-Local Hub workshops
- extent to which participants have felt safe and included, while looking to build new competences during the workshops
- increase of motivation to acquire transversal competences in the future
- increase of motivation to shift towards green transition and an ethical approach to the fashion world
- acquired curiosity towards sustainable topics and careers in the green transition
- acquired awareness on the topic of waste resources in the local community and sense of initiative to create together something that can make a difference
Activities
Now, we invite you to some activities that will help you understand and apply monitoring, assessment, and evaluation in your activities with young people.
Objectives
- Understand some additional reasons to implement evaluation in youth work and design an evaluation activity.
Preparation
For this activity you need a computer and internet, access to YouTube.
Total duration
45 min
Description
- Watch the video Transformative Evaluation Process of The Youth Work: https://youtu.be/GJChoKMQCiw
- Research more information about the Transformative Evaluation Process
- Answer the following questions:
– Do you think it will be useful to apply the Transformative Evaluation Process in FABLE?
– Why?
– How would you apply it if you organize a FABLE workshop? - Design an activity to select suitable stories and share it with us .
Materials
Computer, internet access, YouTube, text processing programme, email.
Objectives
- Understand better the Kirkpatrick model of evaluation and develop skills to apply it.
Preparation
For this activity you need a computer and internet access.
Total duration
1 hour
Description
- Figure out 1 assessment or evaluation method to be applied to a FABLE workshop for each level of the Kirkpatrick model.
- Describe shortly the four methods and why you choose them.
- Share the methods with us.
Materials
Computer, internet access, YouTube, text processing programme, email.
Objectives
- Understand better the ongoing monitoring and assessment.
- Learn to apply new assessment methods.
Preparation
For this activity you need a computer and internet access.
Total duration
1 hour
Description
- Research about different ongoing assessment methods.
- Find at least 5 that you have never tried before and note them down. How are they applied? In what circumstances is it most suitable to apply them?
- Choose 2 of them that you would apply during a FABLE workshop and share them with us.
- Why did you choose exactly these 2 methods?
Materials
Computer, internet access, YouTube, text processing programme, email.
Further resources
Kirkpatrick’s Four Levels of Training EvaluationJames D. Kirkpatrick and Wendy Kayser Kirkpatrick, Association for Talent Development; 1st edition (October 7, 2016)
Panadero, E., Andrade, H., & Brookhart, S. M. (2018). Fusing self-regulated learning and formative assessment: A roadmap of where we are, how we got here, and where we are going. The Australian Educational Researcher, 45(1), 13-31. doi:10.1007/s13384-018-0258-y
Formative Assessment, Making It Happen in the Classroom, Margaret Heritage, October 2021, Corwin
Formative Assessment for Teaching & Learning; Bill Boyle & Marie Charles, SAGE Publications Ltd, 2014 DOI: https://dx.doi.org/10.4135/9781526401861
Kirkpatrick’s Four Levels of Training EvaluationJames D. Kirkpatrick and Wendy Kayser Kirkpatrick, Association for Talent Development; 1st edition (October 7, 2016)
Part 5: Integration of previous methodologies
