//This page has been developed by M.Peyre, CIRAD-AGIRs; B. Haesler, RVC and L.Hoinville, RVC//\\ \\ ====== Importance of the evaluation question ====== An evaluation comes with a cost, the process could be ressources demanding (both in terms of human and financial ressources) according to the type and level of evaluation performed. An evaluation should be therefore done with a clear objective, which comes with one or multiple clearly formulated question. This will ensure framing the evaluation process exactly to the needs and to ensure saving ressources\\ \\ ====== How to define the evaluation question? ====== The important elements to consider when defining an evaluation question are :\\ \\ * Which evaluation level : System or component ? * Which element of evaluation to include : process, effectiveness, economic ? * Which economic evaluation question : cost-optimisation, cost effectiveness, cost-benefit ? === === As part of the RISKSUR project, a guidance pathway was created to assist with the choice of the evaluation question. This pathway is part of the EVA tool and could be used by [[https://survtools.org/evaluation/|clicking on this link.]]\\ Click [[/Evaluation-question-guidance-pathway|here for more information on this guidance pathway]]. \\ \\ ====== The different type of Evaluation questions ====== //(note : the numbering of the question is referring to the numbering used in the EVA tool)//\\ The Evaluation question could be descriptive: what is the current situation, how is my system organised?; \\ it could be causal, trying to understand the factors that contributes to the results of the surveillance, for example what is the effectiveness of the system?;\\ It could be a synthesis question, trying to understand what could be done better and which alternatives will be best, for example, is my system/component cost-effective?\\ It could be an action question related to identifying changes that should be made to improve the system.\\ It is important to notice that those questions could be related and more than one type of question could be asked during an evaluation process: for example if someone wants to know how is the surveilllance system performing, what are the factors influencing its effectiveness and which corrective action should be implemented to improve it.\\ \\ ===== Component level / EFFECTIVENESS EVALUATION ===== \\ | **Evaluation question **\\ | **Evaluation criteria**\\ | **Evaluation method **\\ | | Q1. Assess whether one or more surveillance component(s) is/are capable of meeting a specified technical effectiveness target \\ | Effectiveness\\ | Effectiveness attribute assessment\\ | | Q4. Assess the technical effectiveness of one or more surveillance components \\ | Effectiveness\\ | Effectiveness attribute assessment\\ | \\ ===== Component level / ECONOMIC EVALUATION ===== \\ ==== Cost-optimisation ==== \\ | **Evaluation question **\\ | **Evaluation criteria**\\ | **Evaluation method **\\ | | Q2. Assess the costs of surveillance components (out of two or more) that achieve a defined effectiveness target, where effectiveness is already known \\ | Effectiveness \\ Cost\\ | Least cost assessment\\ | | Q3. Assess the costs and effectiveness of surveillance components (out of two or more) to determine which achieves a defined effectiveness target at least cost, the effectiveness needs to be determined\\ | Effectiveness \\ Cost\\ | Least cost assessment\\ | \\ ==== Evaluation of Net Benefit ==== \\ | **Evaluation question **\\ | **Evaluation criteria**\\ | **Evaluation method **\\ | | Q5. Assess whether a surveillance component generates a **net benefit** for society, industry, or animal holder(s):\\ | \\ | -\\ | | Q5a. Benefit to be measured in monetary terms\\ | Effectiveness, Monetary benefit\\ Cost\\ | Cost benefit assessment\\ | | Q5b. Benefit to be measured in non-monetary terms or to be expressed as an effectiveness measure\\ | Effectiveness\\ Non-monetary benefit\\ Cost\\ | Cost effectiveness assessment\\ | | Q5c. Benefit to be measured in both monetary and non-monetary terms (or to be expressed as an effectiveness measure)\\ | Monetary benefit \\ Non-monetary benefit/effectiveness\\ Cost\\ | Cost benefit and cost effectiveness assessment\\ | \\ ==== Evaluation of Biggest Benefit ==== \\ | **Evaluation question **\\ | **Evaluation criteria**\\ | **Evaluation method **\\ | | Q6. Identify the surveillance component (out of two or more) that generates the biggest net benefit for society, industry, or animal holder(s):\\ | \\ | -\\ | | Q6a. Benefit to be measured in monetary terms\\ | Effectiveness, Monetary benefit\\ Cost\\ | Cost benefit assessment\\ | | Q6b. Benefit to be measured in non-monetary terms or to be expressed as an effectiveness measure \\ | Effectiveness\\ Non-monetary benefit\\ Cost\\ | Cost effectiveness assessment\\ | | Q6c. Benefit to be measured in both monetary and non-monetary terms (or to be expressed as an effectiveness measure) \\ | Monetary benefit \\ Non-monetary benefit/effectiveness\\ Cost\\ | cost benefit and cost effectiveness assessment\\ | ==== Evaluation of Biggest Net Benefit ==== | **Evaluation question **\\ | **Evaluation criteria**\\ | **Evaluation method **\\ | | Q7. Identify the surveillance component (out of two or more) that generates the biggest net benefit for society, industry, or animal holder(s) under a budget constraint\\ | \\ | -\\ | | Q7a. Budget constraint and benefit to be measured in monetary terms\\ | Effectiveness, Monetary benefit\\ Cost\\ | Cost benefit assessment\\ | | Q7b. Budget constraint and benefit to be measured in non-monetary terms (or to be expressed as an effectiveness measure)\\ | Effectiveness\\ Non-monetary benefit\\ Cost\\ | Cost effectiveness assessment\\ | | Q7c. Budget constraint and benefit to be measured in both monetary and non-monetary terms (or to be expressed as an effectiveness measure)\\ | Monetary benefit \\ Non-monetary benefit/effectiveness\\ Cost\\ | Cost benefit and cost effectiveness assessment\\ | \\ ===== System or component level / PROCESS EVALUATION ===== \\ | **Evaluation question **\\ | **Evaluation criteria**\\ | **Evaluation method **\\ | | Q8. Assess the functional aspects of surveillance which may influence effectiveness\\ | Function\\ | Functional attribute assessment\\ | | Q9. Assess the technical effectiveness of one or more surveillance components and the functional aspects of surveillance that may influence effectiveness\\ | Effectiveness and function\\ | Effectivness and functional attribute assessment\\ | \\ ===== System level / EFFECTIVENESS EVALUATION ===== \\ | **Evaluation question **\\ | **Evaluation criteria**\\ | **Evaluation method **\\ | | Q10. Assess the technical effectiveness of the surveillance system\\ | Effectiveness\\ | Effectiveness attribute assessment\\ | | Q11. Assess the surveillance structure, function and processes \\ | Strength and weakness\\ | Process, structure and function assessment\\ | \\ \\ ====== ======