The evaluation question is the most important aspect of the evaluation process; evaluation is intrinsically linked to action, it makes little sense, and is of limited interest, to perform an evaluation without a specific objective for action.
Indeed an evaluation process with the aim to gather knowledge is qualified as a “quasi-evaluation” as it provides elements on how the system is working but no elements and why and is therefore limited in terms of recommendation for corrective actions and improvement.
This is especially important to consider in resource scarce situation as evaluation has a cost, both in terms of human and financial resources.
A list of 11 evaluation questions were defined within the EVA tool trying to account for all the diversity of evaluation need (Table 1). However this list might not be exhaustive and could be reviewed based on feedback from users of the tool and/or comments made on this EVA wiki.
Table 1. List of evaluation questions relevant to evaluation surveillance systems
| Evaluation question | Evaluation criteria | Evaluation method |
| |
||
| ===== Evaluation questions at the component level ===== | | |
| Q1. Assess whether one or more surveillance component(s) is/are capable of meeting a specified technical effectiveness target | Effectiveness | Effectiveness attribute assessment |
| Q2. Assess the costs of surveillance components (out of two or more) that achieve a defined effectiveness target, where effectiveness is already known | Effectiveness Cost | Least cost assessment |
| Q3. Assess the costs and effectiveness of surveillance components (out of two or more) to determine which achieves a defined effectiveness target at least cost, the effectiveness needs to be determined | Effectiveness Cost | Least cost assessment |
| Q4. Assess the technical effectiveness of one or more surveillance components | Effectiveness | Effectiveness attribute assessment |
| Q5. Assess whether a surveillance component generates a net benefit for society, industry, or animal holder(s): |
||
| Q5a. Benefit to be measured in monetary terms | Effectiveness, Monetary benefit Cost | Cost benefit assessment |
| Q5b. Benefit to be measured in non-monetary terms or to be expressed as an effectiveness measure | Effectiveness Non-monetary benefit Cost | Cost effectiveness assessment |
| Q5c. Benefit to be measured in both monetary and non-monetary terms (or to be expressed as an effectiveness measure) | Monetary benefit Non-monetary benefit/effectiveness Cost | Cost benefit and cost effectiveness assessment |
| Q6. Identify the surveillance component (out of two or more) that generates the biggest net benefit for society, industry, or animal holder(s): |
||
| Q6a. Benefit to be measured in monetary terms | Effectiveness, Monetary benefit Cost | Cost benefit assessment |
| Q6b. Benefit to be measured in non-monetary terms or to be expressed as an effectiveness measure | Effectiveness Non-monetary benefit Cost | Cost effectiveness assessment |
| Q6c. Benefit to be measured in both monetary and non-monetary terms (or to be expressed as an effectiveness measure) | Monetary benefit Non-monetary benefit/effectiveness Cost | Cost effectiveness assessment |
| Q7. Identify the surveillance component (out of two or more) that generates the biggest net benefit for society, industry, or animal holder(s) under a budget constraint |
||
| Q7a. Budget constraint and benefit to be measured in monetary terms | Effectiveness, Monetary benefit Cost | Cost benefit assessment |
| Q7b. Budget constraint and benefit to be measured in non-monetary terms (or to be expressed as an effectiveness measure) | Effectiveness Non-monetary benefit Cost | Cost effectiveness assessment |
| Q7c. Budget constraint and benefit to be measured in both monetary and non-monetary terms (or to be expressed as an effectiveness measure) | Monetary benefit Non-monetary benefit/effectiveness Cost | Cost benefit and cost effectiveness assessment |
| |
||
| ===== Evaluation questions at the system level ===== | | |
| Q8. Assess the functional aspects of surveillance which may influence effectiveness | Effectiveness | Functional attribute assessment |
| |
||
| Q9. Assess the technical effectiveness of one or more surveillance components and the functional aspects of surveillance that may influence effectiveness | Effectiveness | Effectiveness and functional attribute assessment |
| Q10. Assess the technical effectiveness of the surveillance system | Effectiveness | Effectiveness |
| Q11. Assess the surveillance structure, function and processes | Effectiveness | Process |