Parte 6: Herramienta de seguimiento y evaluación para actividades con jóvenes

Monitoring and assessment tool for activities with young people

Introduction

Ongoing monitoring and assessment, as well as summative evaluation of youth work trainings, workshops and interventions is a must for several reasons. We need monitoring and assessment in order to be able to adjust the process to the participants and their needs, so that we achieve the desired results. We need evaluation at the end in order to measure the results of the initiative against its goals. This chapter is focused on understanding monitoring and assessment as well as evaluation and developing skills to apply them. It also prepares the youth workers to apply monitoring, assessment and evaluation specifically to the FABLE workshops.

In detail, we focus on the following topics:

  1. What are monitoring, assessment and evaluation and why they are important?
  2. Evaluation
  3. Ongoing monitoring and assessment
  4. Monitoring, assessment and evaluation methods and how to use them
  5. FABLE evaluation tools

At the end of the module we expect that youth workers will be able to implement ongoing assessment and summative evaluation in their trainings and interventions. Also they will be prepared for assessment and evaluation of the FABLE G-Local Hub workshops.

What are monitoring, assessment and evaluation and why they are important?

Constant monitoring and assessment of trainings and interventions with young people is a prerequisite for their effectiveness. Ongoing assessment, also called formative assessment, increases the chances of achieving the goals of youth work interventions, like trainings, workshops and other. It involves continuously gathering information, which is then used to adjust the instruction and work process in order to achieve the goals set. Evaluation on the other hand is done at one moment in time in order to find out if the goals that have been set at the beginning for the respective unit or for the whole intervention have been met.

To understand better what assessment and evaluation are, see the video: Evaluation and Assessment, by Rebecca Burton (automatic subtitles available).

Evaluation

A very good evaluation model is the Kirkpatrick Model, which has been a standard for training evaluation in the last fifty years. It is also completely applicable to youth work interventions.
The model suggests that the training results should be evaluated at 4 levels. These 4 levels, applied to youth work are:

In order to be able to monitor, assess and evaluate any type of training or intervention we need to first define their goals. These goals are defined mainly at Level 2: Learning and at Level 3: Behaviour, but may also include more long-term goals at Level 4: Results. Based on the goals, the trainer or youth worker sets a number of indicators, which will be used to evaluate the quality and the effectiveness of the training/intervention.

In order to understand better the Kirkpatrick Model, see the video The Kirkpatrick Model of Training Evaluation by Devlin Peck (automatic subtitles available).

Ongoing monitoring and assessment

In order to ensure the achievement of the goals set for a training or intervention with young people we implement continuous monitoring and assessment during the course of the training/intervention. This is also called formative assessment.

Ongoing assessment shows the young person’s progress throughout the training or intervention. This approach requires the youth worker to assess the participant’s engagement, motivation, level of understanding and participation throughout the course of the activities. Ongoing assessments is extremely important for interventions with young people, because it allows the youth worker to identify how to adjust the course of the intervention or training to the particular group of people, they are working with in order to achieve the goals. For example, we ask the group a question in order to see if they understand what we are talking about or the task they have to perform. If they do not understand, we go back and explain in another way.

A key principle of using ongoing assessment is to clearly communicate the learning/intervention objectives to the participants. This communication not only sets expectations for them, but also enables them to self-monitor their learning progress. Another approach to using ongoing assessment is to require learners to record conclusions (reports) on their tasks and exercises in a workbook or electronic file. This self-assessment approach allows the participant to focus
on what they have learned.

Ongoing assessments requires the trainer or youth worker to provide clear feedback to the participants, to clarify and correct errors, to discuss observations and conclusions from participatory interventions. This is an important contribution to acquiring the competences and changing the behaviours.

Monitoring, assessment, and evaluation methods and how to use them

These are more formal and individual methods, which allow the trainer/youth worker to gather some specific information without any social influence of the group. However, they require knowing what kind of information we need and what questions will lead the young people to give it us.

Individual or in group, they are probably the best way to observe the evolution of the young people during the training or intervention. They:

  • allow young people to explore and experiment with their new knowledge and competencies;
  • give some concrete aspects to the theory learned;
  • show to both young people and youth worker the practical competency acquired (or not).

Very close of the previous method, simulations put the young people in a situation which could really happen in their life, for example when looking for a job or starting new work/initiative. They allow the youth worker to see how prepared the participants are for one or another particular situation and observe their behaviour.

Informal and individual, they are probably the best way to gather all the information needed because they offer the possibility to ask questions individually. However, they are more difficult to use without influencing the answers. Moreover, they require a good level of trust between the people involved to assure the honesty of the answers.

The participants are probably the best source or information about the right progress of a training or intervention. Feedback and comments can be intentionally gathered by the youth worker, at important milestones of the training or intervention to solicit information about progress and understanding. It is also important to provoke spontaneous comments and feedback throughout the whole duration of the intervention or training.

A lot of information comes from the simple observation of the young people during the workshop. The youth worker can prepare some checklist/table to help them qualify the participant’s behaviour and competencies. Observation of the young people and the results of their work can provide information on all 4 levels of The Kirkpatrick Model.

  • Level 1: Reaction. For example, if the youth worker observes negative reaction to his/her facilitation style they can adjust it. Generally, motivation and engagement are good signs of a youth worker – young person relationship that will lead to positive change.
  • Level 2: Learning and Level 3: Behaviour. The youth worker can prepare some scale, based on the training/intervention goals to help them evaluate and analyse the participant’s behaviour and competencies.
  • Level 4: Results. Evaluating at Level 4 requires long-term commitment of either the youth worker or young person, ideally of both. The most important part is to set measurable indicators against which results can be monitored and evaluated.

There are a large number of methods that can be used to implement ongoing assessment in face to face and in online activities. Here are some videos where you can find some inspiration.

FABLE evaluation tools

Within the FABLE project we are going to implement summative evaluation (evaluation at the end of the work with young people) through questionnaires for all initiatives undertaken with young people. Here we provide the goals of these initiatives (G-Local Hub activities), indicators for evaluation and a questionnaire (Annex 1) to be used at the end of every session with young people.

For the formative (ongoing) monitoring and assessment youth workers are free to apply what they have learned in this chapter as well as their creativity and choose the most appropriate according to the situation methods.

G-Local Hub Workshop goals

Indicators to gather feedback on, at the end of the workshops

Activities

Now, we invite you to some activities that will help you understand and apply monitoring, assessment, and evaluation in your activities with young people.

Objectives

  • Understand some additional reasons to implement evaluation in youth work and design an evaluation activity.

Preparation
For this activity you need a computer and internet, access to YouTube.

Total duration
45 min

Description

  1. Watch the video Transformative Evaluation Process of The Youth Work: https://youtu.be/GJChoKMQCiw
  2. Research more information about the Transformative Evaluation Process
  3. Answer the following questions:
    – Do you think it will be useful to apply the Transformative Evaluation Process in FABLE?
    – Why?
    – How would you apply it if you organize a FABLE workshop?
  4. Design an activity to select suitable stories and share it with us .

Materials
Computer, internet access, YouTube, text processing programme, email.

Objectives

  • Understand better the Kirkpatrick model of evaluation and develop skills to apply it.

Preparation
For this activity you need a computer and internet access.

Total duration
1 hour

Description

  1. Figure out 1 assessment or evaluation method to be applied to a FABLE workshop for each level of the Kirkpatrick model.
  2. Describe shortly the four methods and why you choose them.
  3. Share the methods with us.

Materials
Computer, internet access, YouTube, text processing programme, email.

Objectives

  • Understand better the ongoing monitoring and assessment.
  • Learn to apply new assessment methods.

Preparation
For this activity you need a computer and internet access.

Total duration
1 hour

Description

  1. Research about different ongoing assessment methods.
  2. Find at least 5 that you have never tried before and note them down. How are they applied? In what circumstances is it most suitable to apply them?
  3. Choose 2 of them that you would apply during a FABLE workshop and share them with us.
  4. Why did you choose exactly these 2 methods?

Materials
Computer, internet access, YouTube, text processing programme, email.

Further resources

Part 5: Integration of previous methodologies