Evaluating Foresight: Fully-Fledged Evaluation of CTFP
The Foresight wave is growing. Interest in using Foresight exercises to inform policy-making in science, technology, and innovation (STI) is continuing to extend around the world. It now seems safe to say that this is no mere fashion.
The Foresight approach combines three elements: prospective (long-term) studies; planning (and priority-setting) inputs; and participative processes (engaging stakeholders and knowledge sources).
This combination of elements is well matched to the challenges currently confronting STI policy. These include increased emphasis on innovation as a tool for competitiveness and sustainability, alongside pressure on government and university budgets, uncertainty about environmental risks and ethical dimensions of new technologies, and a proliferation of opportunities for strategic R&D.
Foresight is liable to be needed more, rather than less, in years to come. If we need Foresight, then we need to learn about Foresight. This means going beyond merely the formal results of Foresight exercises, in terms of what forecasts and analyses of future opportunities and risks have been developed, or what plans have been proposed and priorities targeted. We also need to learn how best to design and deploy Foresight.
Foresight activities are demanding of time and resources, and it is important to ensure that these are well used. We are confronting big challenges, and the quality of Foresight will affect our readiness to address them. One lesson from the last decade or so of Foresight practice is that “one size does not fit all”. Different problems and contexts require different configurations of Foresight approaches. It is necessary to draw lessons not about “the” Foresight method, but about how Foresight approaches and techniques can be tailored to particular countries and circumstances. This means that evaluation of Foresight efforts is not just a matter of examining the efficiency of the activities. Evaluation must also consider their effectiveness in promoting change to meet the challenges confronting us, and it must take into account the creativity exercised in their design. Foresight must be fit for purpose. Simplistic benchmarking, that matches each feature of the programme against similar features in other countries, is not enough. It is important to relate the complex of activities pursued to the specific objectives of the programme. With such an approach to evaluation, the Colombian Technology Foresight Programme (CTFP) provides an excellent opportunity to draw lessons about how Foresight can be introduced and implemented, and what sorts of design challenges need to be tackled if Foresight is to meet the STI challenges.
For this reason, this study proposes the methodological framework and concept of fully-fledged evaluation of a (Foresight) programme as:
…a systematic process aimed at assessing the appropriateness and level of achievement of the programme’s objectives, its performance (using cost-benefit analysis), efficiency of organisational structure (i.e. approaches and methods) and effectiveness of implementation and aftercare. The process should assess the level of capacities and Foresight culture achieved; its national, sub-national and international reach; level of commitment of participants; and novelty and impact of its internal activities (i.e. studies and projects). In addition, with the aim of aligning a Foresight with the implementation environment, the evaluation should try to measure the impact on public and private policies and strategies; agendas of science, technology and innovation (STI) programmes and institutions; consolidation of research groups; consolidation of S&T capacities; and internationalisation of R&D. Finally, a fully-fledged evaluation of Foresight should also identify new products and services; new policy recommendations and research agendas; new processes and skills; new paradigms and visions; and new players.