Mid-term evaluation of single farm assistance under the Regulation (EC) No. 1257/1999: methodological problems, data shortcomings and possible solutions

Assistance of farm investments and less favoured areas are core elements of agri-structural policy in Germany and make up about 16% and 13% respectively of the rural development plans 2000 to 2006. Due to legal obligations, these measures have to undergo a systematic evaluation process. For that purpose the European Commission (COM) elaborated guidelines, which are very helpful for designing evaluations. They contain most relevant questions and propose predominantly sensible criteria and indicators. However, while performing the mid-term evaluation, important difficulties occured, especially from the application of with/without- and before/after-comparisons. Regarding the EU-level synthesis seeked by the COM, many indicators lack a precise definition which is an essential precondition for homogeneous measurement and comparable assessment. In addition to the methodological problems, many analyses suffer from major data shortcomings. Necessary data is often only available with a large time-lag and/or requires enormous effort to make it usable. Some of these problems can certainly be linked to the specifics of German federalism and the numerous particularities of the respective intervention designs and data gathering on state level. These could lead to considerable additional evaluation costs in comparison to unitarian member states. Finally, some recommendations for further evaluations are made. The authors see a growing necessity to supplement the evaluation process with scientific studies which are not squeezed into the rigid timetable and analysis frame of the COM, but focus on thematic aspects that need clarification. Currently, there is a trend to invest much money in evaluation at the cost of more profound scientific studies. Among other recommendations, this papaer concludes that more efforts should be made to improve the systematic exchange of evaluators' experiences of different member states.