Elle est en ligne ici.

Rural development

• Programme evaluation
• Mid term
• Europe and a French region in partnership

Seul centre technique de France à avoir fait le choix d’une évaluation externe et indépendante en 2009, le Ctifl a décidé de poursuivre sa collaboration avec Euréval au terme d’une année concluante.

Après 3 évaluations l’année dernière, cette année deux prestations sont prévues : l’évaluation des activités de documentation du Ctifl et l’évaluation des programmes phytosanitaire « fruits à pépins » et « légumes en sol ».
Pour en savoir plus, delahais@eureval.fr

Your learning programme will follow a nine-step process covering all successive phases of the management of an evaluation, plus a tenth step on evaluation systems.

• Read the methodological guidance
• Look at one of the real-life case studies
• If your are a beginner, then briefly answer the questions asked and send your answers to Thomas Delahais (delahais@eureval.fr). As far as necessary, discuss your answers with Thomas by email or by phone (+33 4 53 63 83 98)
• Participate in the two half-day training sessions in Brussels.

• Theory : 25%
• Practice : 25%
• Bridging theory and practice : 50%

La Gazette des communes, parue le 8 décembre 2008, fait la part belle à notre métier avec un dossier de 8 pages consacré aux bénéfices de l’évaluation pour les collectivités locales. Annie Fouquet, présidente du conseil d’administration de la SFE (dont Euréval est un membre fondateur), y rappelle les principes de la démarche évaluative, et précise qu’une charte de l’évaluation, dont Euréval est signataire, encadre les pratiques des cabinets spécialisés.
Par ailleurs, nous sommes heureux de voir que deux de nos études, en Alsace et en Rhône-Alpes, sont mentionnées comme exemple de la façon dont l’évaluation peut contribuer à enrichir les débats et changer les cultures au sein des collectivités locales.
Un dossier à lire dans la Gazette de cette semaine.

Have an evaluation mandate signed

Prepare a brief and clear document formalising the decision to evaluate and have it signed at the highest relevant level in the organisation commissioning the evaluation.A mandate legitimises the whole evaluation process, attracts high level support to the evaluation and helps preventing risks of conflicts. Prepare the mandate carefully

Long enough in advance, check the following points :

• regulatory framework applying to the evaluation, if there is any
• purpose of the evaluation
• main intended users, their fears and expectations
• deadline after which the evaluation report will become useless
• main issues to be addressed or not addressed
• whether you need recommendations, options, or nothing
• how will the report be published and disseminated
• how will responsibilities be shared (external or internal evaluation team, reference group or evaluation committee, evaluation manager)

Foresee utilisation

Put yourself at the time when the evaluation will be finalised and develop a utilisation scenario. For instance, your evaluation will :

• make an input into a policy reform process or into a major budgetary discussion
• make an input into a public debate involving politicians and media
• provide fuel for thought before partners re-negotiate their co-operation
• validate and disseminate the lessons learnt from a pilot programme

If there is no foreseeable use, then the evaluation is at risk of being just a bureaucratic process, and you should recommend abandoning or postponing it.

If you have several scenarios, then select the main one

Manage risks

Only evaluate activities which can be legitimately judged by the authority(ies) who signs the mandate. An institution is not legitimate for evaluating the policy of another institution, except in the framework of a joint mandate.

Ensure that the evaluation will not touch issues that the commissioning body considers as absolutely unmanageable. If necessary, specify that the corresponding issue(s) is excluded from the evaluation mandate.

Identify which institutions or interest groups are the most likely to resist the evaluation conclusions. Then consider involving them into the evaluation process or entrusting the evaluation to a committee, a panel, or an expert whose impartiality is fully trustable.

Identify stakeholders

Public authorities, i.e. those who have the power to adjust, to reform, or even to kill the intervention under evaluation :

• policy-maker(s), parliament, ministers
• funding institution(s), donor agencies in the case of development aid
• parent ministry in the case of an agency

Operators, i.e. those who are responsible for the implementation of the intervention :

• implementing authority or agency, programme/project manager
• agents who enforce laws, allocate grants or subsidies, provide public goods or services, whether they belong to a governmental, non-governmental or private organisation, provided that they have to report on their implementation/enforcement activities on an annual basis at least

Targeted publics, i.e. those who are directly targeted by the intervention in the sense that :

• their needs are being satisfied in a direct manner
• their behaviour is to be changed

Ultimate beneficiaries, those who are facing the problem which is to be addressed by the intervention, for instance :

• individuals or groups whose needs are being satisfied in an indirect manner 
• the society at large
• future generations

Winning and loosing tiers, those who benefit or loose from the unintended consequences of the intervention such as :

• economic agents whose activity will increase if the intervention is implemented (winning tiers) 
• economic agents whose activity will decrease if the problem is solved (loosing tiers)

Related authorities that implement connected activities such as :

• public agencies targeting the same public
• civil society organisations pursuing the same goals

Involve stakeholders in the evaluation process 

Because they are resource persons who can help in :

• clarifying the objectives or the intervention logic
• suggesting indicators and benchmarks
• opening access to relevant documents, databases and informants
• interpreting data, identifying biases and checking quality

Because they may strengthen the legitimacy of the evaluation process, especially in

• selecting the questions and setting the judgement criteria
• interpreting information
• ensuring that value judgements are impartial

Because they may improve the utility of the evaluation in :

• focusing on useful questions
• contributing to make recommendations feasible
• advocating the evaluation in their own organisation or network

Because some stakeholders are likely to criticise the evaluation anyway. Involving them in the process may prevent such criticism, at leat in part.

Invite the right participants

Preferably invite people who :

• are respected by decision makers and interest groups
• have as diverse views as possible
• are ready to engage in an open minded reflection
• have programme-wide views instead of specialisms in one or another component
• have a first hand knowledge of the intervention, of the concerned public, of the context
• are likely to attend all meetings over the evaluation process

Preferably avoid inviting :

• decision makers themselves (they need to feel free to adopt or reject conclusions and recommendations)
• lobbyists and those who are likely to speak just the official language of their institution
• people who hold purely private business stakes

Set up an evaluation committee or reference group

Identify six to twelve participants who will speak from the standpoint of various categories of stakeholders, e.g. public authorities, operators, concerned publics. They may also bring a particular expertise

In the invitation letter, specify that participants are invited ‘ad nominem’ and not as representatives of their institution.