Evaluation

What is evaluation?
Evaluation is the process of making judgments about the worth of something. It provides information about achievement of training objectives and provides data required to increase efficiency and improve training standards

Why evaluate?

• To provide a basis for sound decision making.
• To obtain a measure of trainer achievement.
• To gain professional reinforcement.
• Accountability for effective training.

Every element of a training session builds upon the previous element until the training is hopefully, a well orchestrated, synergistic whole. Skillfully obtaining and interpreting evaluations from the participants of workshops and training is the royal road to continual improvement. (Ref. Dynamic Trainer by Laurie Kagan, Kagan Publishing 2007)

What’s involved in an evaluation?
Evaluation takes into account the participants, the trainer, the program, the environment, administration and resources. The evaluation process could be broken up into the following three steps:

1.  Preparing for the evaluation:
   a. Deciding on who will assist in the evaluation.
   b. Clarifying the objectives of the program/activity/policy to be evaluated.
   c. Deciding on the technique to be used.
   d. Designing the evaluation instrument.
2.  Data collection – conducting the evaluation.
3.  Data analysis and formulation of recommendations.

Elements of Evaluation Forms
The first step in receiving helpful feedback on training is to distribute a well-designed evaluation
form. The following are important elements:

• The details (Presentation date, topic, trainer name)
• What the participant learned (content)
• Comment on the instructor
• Additional comments
• An overall scoring mechanism (1-5, smiles or frowns, etc)

The language on the evaluation form should be positive and the questions open ended. The evaluation process is a step in the wrapping-up of the training, and it needs to avoid drudgery for the participants.

Types of Evaluation
Evaluation is basically summative, formative and process or combinations of all of these.

Summative Evaluation involves a summary of the success or failure of a program after its completion. All participants at workshops and training sessions conducted by Kagan Australia will be requested to complete a summative evaluation form.

Formative Evaluation continues throughout the lifetime of a program. It focuses on program improvement through continual evaluation.

Process Evaluation describes and records the process, or processes that take place throughout the length of the program.

Evaluation of the Program
In evaluating the program the following questions could be used:

1.  Do the program aims and objectives reflect the content of the course?
2.  Is this program appropriate to the interests of the participants?
3.  Does the course cater for participants of all abilities?
4.  Are the appropriate resources available?
5.  Do the trainers have sufficient skill, background knowledge and interest to present this course?
6.  Are the allocations of time to the various components appropriate?
7.  Is there adequate formative evaluation of the program?

Evaluation of the Teaching
This may be achieved by posing such questions as:

1. Are resources being used effectively to achieve the program’s objectives?
2. Do the teaching strategies contribute to the achievement of the
   program’s knowledge and understandings, and skills objectives?
3. Do the activities develop the capacity in participants to apply what has been learnt in the training to their own workplace?
4. Is the possession of specific pre-requisite skills being verified before commencing each new section of the course?
5. Does ongoing assessment throughout the training indicate that the objectives are being achieved?
6. Does the approach by participants to course work indicate that their attitudes are supportive of the course?
7. Have trainers retained their enthusiasm for all aspects of the course?

How will we analyse the information gathered?
Quantitative analysis of straightforward counts and percentages will be applied to scores on evaluation forms. This will provide information about the overall level of satisfaction with the courses we conduct.

Qualitative analysis of open ended questions can be treated in a quantitative or qualitative way.

The written statements may be categorized or coded (ie given a number) and these numbers may then be given over to statistical analysis.

Alternatively the written responses can be treated by logical analysis. Instead of being concerned merely with the numbers of people who responded in certain ways, we can examine the responses to determine the concepts which have emerged and then determine what links exist between concepts.

In this way we can build a complete picture of what the respondents think about any particular aspect of the course.

Whatever method is used we will be able to draw conclusions and make recommendations for program improvement.
Evaluating the Learning and Assessment Regardless of the experience level and backgrounds the participants the trainer must consider the participants instructional needs. The degree of mastery of skills by course participants and the trainers teaching strategies need to be determined.

Modes of assessment will vary according to the purpose for which they are employed, but the continuing progress of participants within the content of the course will always be the central focus.

Assessment can take a variety of forms. Some examples that are utilised within Kagan trainings are:

• Questioning
• Oral presentations
• Observation
• Written tests
• Team discussions
• Completion of worksheets

Kagan Australia will provide participants with an opportunity to follow-up and expand their expertise and support them in the implementation of the course content. The forms “Kagan Cooperative Learning Follow-Up Action Plan” are provided to participants for this purpose and as a means of self assessment.

Confidentiality
In order to obtain honest and accurate information staff of Kagan Australia are to assure the respondents to any surveys and evaluations that the information they provide is confidential.

This assurance is to be taken seriously by all Kagan Australia staff. If the evaluation form calls for individuals to be identified, permission to release any information collected about those individuals should be obtained. Similarly data collected from small groups of people must be protected. The assurance that will be given is that “it will not be possible to identify individuals in the reporting of the evaluation.”

There are advantages in keeping information confidential and in being seen to do so. Apart from increasing the likelihood of obtaining honest answers, it helps the evaluators to gain the cooperation and openness of people, resulting in more complete and better quality data.