This page has been developed by M.Peyre, CIRAD-AGIRs; B. Haesler, RVC and L.Hoinville, RVC

Importance of the evaluation question

An evaluation comes with a cost, the process could be ressources demanding (both in terms of human and financial ressources) according to the type and level of evaluation performed. An evaluation should be therefore done with a clear objective, which comes with one or multiple clearly formulated question. This will ensure framing the evaluation process exactly to the needs and to ensure saving ressources

How to define the evaluation question?

The important elements to consider when defining an evaluation question are :

Which evaluation level : System or component ?
Which element of evaluation to include : process, effectiveness, economic ?
Which economic evaluation question : cost-optimisation, cost effectiveness, cost-benefit ?

As part of the RISKSUR project, a guidance pathway was created to assist with the choice of the evaluation question. This pathway is part of the EVA tool and could be used by clicking on this link.
Click here for more information on this guidance pathway.

The different type of Evaluation questions

(note : the numbering of the question is referring to the numbering used in the EVA tool)
The Evaluation question could be descriptive: what is the current situation, how is my system organised?;
it could be causal, trying to understand the factors that contributes to the results of the surveillance, for example what is the effectiveness of the system?;
It could be a synthesis question, trying to understand what could be done better and which alternatives will be best, for example, is my system/component cost-effective?
It could be an action question related to identifying changes that should be made to improve the system.
It is important to notice that those questions could be related and more than one type of question could be asked during an evaluation process: for example if someone wants to know how is the surveilllance system performing, what are the factors influencing its effectiveness and which corrective action should be implemented to improve it.

Evaluation question	Evaluation criteria	Evaluation method
Q1. Assess whether one or more surveillance component(s) is/are capable of meeting a specified technical effectiveness target	Effectiveness	Effectiveness attribute assessment
Q4. Assess the technical effectiveness of one or more surveillance components	Effectiveness	Effectiveness attribute assessment

Cost-optimisation

Evaluation question	Evaluation criteria	Evaluation method
Q2. Assess the costs of surveillance components (out of two or more) that achieve a defined effectiveness target, where effectiveness is already known	Effectiveness Cost	Least cost assessment
Q3. Assess the costs and effectiveness of surveillance components (out of two or more) to determine which achieves a defined effectiveness target at least cost, the effectiveness needs to be determined	Effectiveness Cost	Least cost assessment

Evaluation of Net Benefit

Evaluation question	Evaluation criteria	Evaluation method
Q5. Assess whether a surveillance component generates a net benefit for society, industry, or animal holder(s):		-
Q5a. Benefit to be measured in monetary terms	Effectiveness, Monetary benefit Cost	Cost benefit assessment
Q5b. Benefit to be measured in non-monetary terms or to be expressed as an effectiveness measure	Effectiveness Non-monetary benefit Cost	Cost effectiveness assessment
Q5c. Benefit to be measured in both monetary and non-monetary terms (or to be expressed as an effectiveness measure)	Monetary benefit Non-monetary benefit/effectiveness Cost	Cost benefit and cost effectiveness assessment

Evaluation of Biggest Benefit

Evaluation question	Evaluation criteria	Evaluation method
Q6. Identify the surveillance component (out of two or more) that generates the biggest net benefit for society, industry, or animal holder(s):		-
Q6a. Benefit to be measured in monetary terms	Effectiveness, Monetary benefit Cost	Cost benefit assessment
Q6b. Benefit to be measured in non-monetary terms or to be expressed as an effectiveness measure	Effectiveness Non-monetary benefit Cost	Cost effectiveness assessment
Q6c. Benefit to be measured in both monetary and non-monetary terms (or to be expressed as an effectiveness measure)	Monetary benefit Non-monetary benefit/effectiveness Cost	cost benefit and cost effectiveness assessment

Evaluation of Biggest Net Benefit

Evaluation question	Evaluation criteria	Evaluation method
Q7. Identify the surveillance component (out of two or more) that generates the biggest net benefit for society, industry, or animal holder(s) under a budget constraint		-
Q7a. Budget constraint and benefit to be measured in monetary terms	Effectiveness, Monetary benefit Cost	Cost benefit assessment
Q7b. Budget constraint and benefit to be measured in non-monetary terms (or to be expressed as an effectiveness measure)	Effectiveness Non-monetary benefit Cost	Cost effectiveness assessment
Q7c. Budget constraint and benefit to be measured in both monetary and non-monetary terms (or to be expressed as an effectiveness measure)	Monetary benefit Non-monetary benefit/effectiveness Cost	Cost benefit and cost effectiveness assessment

Evaluation question	Evaluation criteria	Evaluation method
Q8. Assess the functional aspects of surveillance which may influence effectiveness	Function	Functional attribute assessment
Q9. Assess the technical effectiveness of one or more surveillance components and the functional aspects of surveillance that may influence effectiveness	Effectiveness and function	Effectivness and functional attribute assessment

Evaluation question	Evaluation criteria	Evaluation method
Q10. Assess the technical effectiveness of the surveillance system	Effectiveness	Effectiveness attribute assessment
Q11. Assess the surveillance structure, function and processes	Strength and weakness	Process, structure and function assessment