acceptability-and-engagement [Surveillance Evaluation Wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
//This page was developed by F. Goutard, CIRAD and C.Calba, CIRAD//\\ \\ **Publications**: [[https://www.ncbi.nlm.nih.gov/pubmed/26489602|Calba et al.]], 2015, PVM ; [[http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0159041|Calba et al., 2016]], PLOSONe; [[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5339697/|Schulz et al., 2017, Nature Scientific Report]]; [[https://www.ncbi.nlm.nih.gov/pubmed/27601050|Schulz et al., 2016, BMC Vet. Res.]]\\ \\ **RISKSUR Case studies**: [[/CSF-Surveillance-in-wild-boars-in-Germany|CSF surveillance in Wild boars]]\\ \\ [[/Acceptability-and-engagement#Accept|AccEPT METHOD]]
===== 1. WHAT IS ACCEPTABILITY? =====
==== 1.1. Definition and importance in the evaluation process ====
**//Acceptability//** refers to the willingness of persons and organizations to participate in the surveillance system, and to the degree to which each of these users is involved in the surveillance [[/#_ftnref1|[1]]]. This evaluation attribute has been listed by the CDC as one of the main quality of surveillance. Indeed, the decision to report a suspected event is a critical function of an emerging infectious disease surveillance system. It reveals important perception factors and sociological aspects due to the fact that reporting an outbreak may be conflicting economic, cultural, and/or political incentives. \\ In order to limit the under-reporting of suspected cases and to identify the best ways to improve the current surveillance system, it is crucial to assess stakeholders’ willingness to participate in this system.\\ 
==== 1.2. Elements to take into consideration ====
Acceptability is relevant to different aspects of surveillance system. It first refers to the acceptance by the actors of the system **objective** and to the way it is **operating**. The acceptance of the way the system is operating refers to //(i)// the role of each actor and the representation of its own utility, //(ii)// the consequences of the flow of information for each actor, and //(iii)// the relations between stakeholders. Another essential element of acceptability is the **trust** in the system as a whole but also in each other stakeholders involved in it.\\ \\ \\ 
===== 2. HOW TO EVALUATE ACCEPTABILITY? =====
==== 2.1 Participatory Tools ====
**//Participation//** empowers people to find solutions to their own development challenges. It is both an attitude and a philosophy that encourages learning, discovery and flexibility.\\ **//Participatory appraisal//** is a family of approaches and methods that enable people to present, share and analyze their knowledge of life and conditions, so as to plan and act. It is participatory, flexible, lightly structured, adaptable, exploratory, empowering and inventive.\\ Different participatory appraisal tools can be used to investigate the same issues. Although the methods are intended primarily to explore different aspects of the issue, there will be some overlap in the information generated. Three main groups of tools will be presented during this training:\\ 

  * **Informal interviewing:**Semi-structured interviews, key informant meetings, focus-group discussions.
  * **Ranking and scoring:**Simple ranking, pair-wise ranking, proportional piling, matrix scoring.
  * **Visualization:**Mapping, timelines, seasonal calendars, Venn diagrams.
  * Qualitative research methods require data checking and analysis. In participatory appraisal, this process begins during fieldwork. Throughout an investigation the practitioner is analyzing information, and as a result the hypothesis guiding the investigation is continually evolving. There are several ways in which data is crosschecked, validated and analyzed.
  * **Probing:** During an interview, information provided by informants is explored for internal consistency. When an interviewee responds to a question, the interviewer usually asks additional questions to verify and deepen his or her understanding of the interviewee’s viewpoint.
  * **Triangulation:**An analytical process where data collected by different methods and from different sources is compared. Triangulation is used to explore patterns and coherence, as well as to understand the bias of different informants. Triangulation occurs during an interview, when comparing secondary information and interview results, and during final analysis.
  * **Conflict of interest:**It is important to understand any potential conflict of interest your informants might have in order to interpret the information you gather.
  * **Weighing of evidence:**The practitioner needs to weigh the evidence gathered from different sources to make a judgment on which to prioritize and investigate further.
  * **Laboratory diagnostics:**Results, particularly regarding a diagnosis, should be confirmed with biological testing.
\\ \\ It is important to be aware of our own attitudes and behavior, and how they might influence our work. Human beings communicate with all five of our senses: sound, sight, hearing, touch and smell. Most of us are used to communicating primarily with the sense of sound, by speaking. However, when we interact with others we are consciously or subconsciously communicating in other ways. Our attitudes are often displayed through these other means of communication; we exhibit behavior that the people become aware of and respond to in unanticipated ways. Our attitude and behavior may bias our study.\\ It is also important to be aware of the attitudes and behaviors of those with whom we are working. The attitudes and behaviors of our interviewees provide us important information for our study, as well as how well we are facilitating the interview!\\ \\ 2.1 Participatory Toolbox for the evaluation of acceptability (AccEPT)\\ \\ The objective of AccEPT is to engage representatives of all types of stakeholders of the surveillance (e.g. farmers, hunters, veterinary services) through individual face-to-face interviews or through focusgroup discussions. Each interview consists of three steps.\\ 

  * Step 1 - Identifying surveillance system stakeholders’ professional network and assessing the satisfaction of the relations among them, through the elaboration of relational diagrams and the use of smileys.
  * Step 2 - Representing the information flow within the system and assessing the trust devoted to fulfil its objective(s), with the use of flow diagrams associated with proportional piling.
  * Step 3 - Assessing the satisfaction of the information flow (i.e. positive and negative impacts following a suspicion), with the elaboration of impact diagrams associated with proportional piling.
\\ \\ 
{{ :files:image1.png?nolink |}}
 \\ Each element was assessed using specific participatory appraisal tools, adapted and developed to collect data related to specific questions as stated in the following table.\\ \\ 
{{ :files:image2.png?nolink |}}
===== 3. AccEPT =====
\\ The assessment of the acceptability using participatory approaches can be implemented with all types of stakeholders in the system, from farmers and hunters to Veterinary Services. Nonetheless, each meeting should join only one type of actor. The preferred way to implement the process is by the organisation of focus group discussions. It is better not to have too many people involved in each group. Five participants appeared to be manageable. It is also feasible to implement the approaches through individual interviews. A focus group will last approximately three hours; an individual interview approximatively two hours.\\ Several steps will be implemented for each meeting, as describe below. You will have to start first with introductions, from your side (e.g. project, objectives) and from participants’. You should specify that the information that will be collected will be anonymous. Also, an important point the participants need to understand is that there is no question of judgement in this process. The objective is to assess their point of view, and to understand about their feeling regarding the surveillance system targeted.\\ Finally you will have to present them the organisation of the day. “Three exercises will be implemented with you. The first one will target your professional network and your relations. The second one will be related to the sanitary information. And the last one will be related to the consequences of some specific event.”\\ \\ 
{{ :files:image3.png?nolink |}}
\\  It is better to implement these tools with a team: one person in charge of facilitating the discussions, and one (or two) persons in charge of taking notes and of observing participants’ behaviour. If you have no choice than implementing it with only one person, it is advisable to use a digital recorder. Indeed, you will not be able to facilitate the meetings and taking notes in the same time.\\ \\ 
==== 3.1 Tools ====
==== 3.1.1 Relational diagrams & smileys ====
Relational diagrams are used to identify respondents’ professional network and interactions between stakeholders. This tool is a good way to introduce the evaluation process with participants as they are talking about their professional relations. The objective here is not to focus on relations related to the surveillance system, but to have an overview of participants’ relations.\\ After drawing the diagram with participants, the objective is to assess their satisfaction level for each relation. Five smileys will be used on the relational diagram, representing five levels of satisfaction: very unsatisfactory, unsatisfactory, medium, satisfactory, very satisfactory. The objective is to have one, and only one smiley per identified stakeholder/organisation.\\ \\ **__Method__**\\ **1.**Draw a box in the middle of a flipchart, with the status of participants (e.g. farmers, hunters).\\ **2.**Ask respondents about stakeholders or organisations they have interactions with in the frame of their activity (e.g. farming, hunting). Draw a box for each of them.\\ **3.**Ask respondents to describe these relations. Are these interactions on one side only or on both sides? Could these interactions be defined as //(i)// not enough, //(ii)// enough, or //(iii)// more than necessary? Draw arrows accordingly.\\ **4.** For each interaction ask //(i)// what information / services do stakeholders exchange, and //(ii)// why did they define them as not enough, enough or more than necessary.\\ **5.**Summarize the discussions by going through the diagram. Be sure no stakeholder / organisation are missing in the diagram.\\ **6.**Once the diagram is drawn, ask participants to devote one and only one smiley per box according to their satisfaction. Be sure participants understood that it is not a judgement but a representation of their own feelings. Ask them to explain their choice.\\ **7.**Summarize the discussions and the results by going through the diagram.\\ \\ 
{{ :files:image11.png?nolink |}}
==== 3.2.1. Flow diagrams & proportional piling ====
Flow diagrams are used to assess participants’ knowledge about the flow of information following a suspicion and to identify the different pathways where this information can circulate. This exercise is based on stakeholders’ experience, knowledge and/or future attitude. This diagram highlights the knowledge of the system by stakeholders.\\ \\ **__Method (for hunters)__**\\ \\ **1.**Ask respondents if they once had a sanitary problem with animals. If yes, ask them what did they do? If no, ask them what they would do? Go through the discussion to identify which actor or organization will have the information related to a suspicion in wildlife.\\ \\ **2.**Once the first(s) stakeholders receiving the information (i.e. suspicion) has been identified, ask participants if they know where the information is going. List the stakeholders who will have this information and draw arrows to show this information flow.\\ \\ **3.**Once the flow up of information has been completed, ask participant if they know if the information is going down, and how? Do they have feedback after reporting a suspicion?\\ \\ **4.**Once the diagram is drawn, ask respondents if they know why this system is in place? What is the objective of implementing surveillance?\\ \\ **5.**Using 100 counters, start implementing the proportional piling. First, ask participant to divide the counters into two piles. One pile representing their trust in the system, and the other one representing their lack of trust. Remember, the more you put counters the more you trust/don’t trust the system. Be sure to explain participants that this is not a question of judgement. The objective is to take into consideration every aspect: human and/or budget constraints, relations between stakeholders, etc.\\ \\ **6.**Using the counters devoted to the trust in the system, ask participants to split them on the stakeholders / organisations represented in the diagram. Once more, the more you put counters the more you trust the stakeholder.\\ \\ **7.**Ask participants to explain about their choices. By going through the diagram, sum up the results to be sure to probe the data.\\ \\ \\ {{ :files:image5.png?nolink |}}
==== 3.3.1. Impact diagrams & proportional piling ====
Impact diagrams are used to assess both positive and negative impacts of a specific event and to document the consequences as experienced directly by respondents. The specific event experienced here is a suspicion.\\ \\ **__Method__**\\ **1.**Ask participants to detail the potential consequences of a suspicion at their own level. If you have the information related to a suspicion, what will you do? Do you think your relations will change?\\ **2.**Ask participants if the consequence is positive, negative or both, and ask them to explain about their choice.\\ **3.**Once all the impacts are identified, sum up the discussions by going through the diagram.\\ **4.**Implement the proportional piling in two steps. First, ask participants to divide the counters on the positive and on the negative part according to influence on their activity. Be sure they understand that the more they put counters the more the influence will be high.\\ **5.**Ask respondents to split the counters of each category (i.e. negative or positive) according to the probability of occurrence of each impact. The more they put counters the higher the probability of the impact is.\\ **6.**Sum up the discussion in order to probe the results.\\ \\ 
{{ :files:image6.png?nolink |}}
==== 3.2 Analysis of results ====
In order to make the assessment of the acceptability, you will have to analyse the diagrams drawn by participants and the discussions they had during the meetings.\\ The first step of the process will be to analyse the result for each individual interview and for each focus group implemented. To obtain the final scores, you will just have to calculate the mean obtained.\\ The following tables present the evaluation criteria developed to provide score for each element of acceptability.\\ \\ 
{{ :files:image8.png?nolink |}}\\ 
{{ :files:image9.png?nolink |}}\\ 
{{ :files:image10.png?nolink |}}\\
----
[[/start|[1]]] RISKSUR Consortium definition