Evaluation

There are a wide variety of methods for evaluating programs or events and a vast number of ‘how to’ books – just Google the subject and you will see!

You do not have to be an expert to carry out a useful program evaluation. Many funding bodies require some form of evaluation of the initiative funded to verify that the desired outcomes have been achieved and to inform their decisions about future funding.

• Evaluation is not about proving the success or failure of a program, it’s about receiving feedback and identifying the strengths and weaknesses to enable you to adjust and improve the program accordingly.
• Evaluation helps you understand the impact or outcomes of your program.
• Evaluation is not something that happens at the end of your program. When you think about it from the beginning, it can assist with keeping you on track.
• Evaluation helps you to identify whether you are doing what you think you are doing and, perhaps, whether you should be doing it at all!

You probably already evaluate informally in some way such as talking things over with your team. A formal evaluation involves:

• Identifying what information you need in order to make informed decisions about program issues or needs and putting in place systems to accurately collect that information.

How you do this depends on the resources you have. Aim to integrate the evaluation processes into the project from the outset. When designing a program evaluation you need to identify:

• The purpose – what decisions do you want to take following the evaluation
• e.g. do you wish to know the extent to which a program is meeting predetermined goals or objectives
• Who will receive the information – your board, funding bodies, audiences or participants?
• What type of information is required to enable you to make the decisions you need to make and/or inform the intended audiences? e.g. information to understand the processes you have in place, the strengths and weaknesses of the program from an audience perspective and/or the benefits to the audience
• What are the sources of the information?
• How can that information be easily collected?

Research can be qualitative (e.g. questionnaires, interviews, conducting focus groups, taking photos or videos) or quantitative (e.g. data relating to audience numbers and ticket sales). For more information on research see Cultural planning and policy making with community.

Pitfalls in data collection
In choosing and designing a collection method, you need to be aware of how much can be collected and readily analysed, given the resources available.

Questionnaires can be non-threatening but recipients need to be motivated to respond and the wording may bias responses, just as an interviewer can bias the response of the person interviewed.  Test questionnaires with a sample group.

Focus groups are great for exploring a topic in depth through group discussion, and can be fun but they require an experienced facilitator who can accurately interpret the discussions and report on them.  Focus groups can be drawn from the stakeholder group, artists involved, audience members, local or visiting cultural leaders.

For further detailed information on evaluating an arts project visit:

Evaluating Community Arts and Community Well Being (2002) Arts Victoria available on line www.arts.vic.gov.au > publications
Strengthening local communities Arts in Community Settings - The Evaluation of Two Community Support Funded Arts Programs (2006) Strategic Policy and Research Division Department for Victorian available on line at www.arts.vic.gov.au > publications