New framework to assess the success of climate adaptation services
Eva Boon is researching what makes a climate service successful. Together with colleagues from NGI and WUR, she has published an article in the Elsevier journal Climate Services about a framework to assess the success of climate services. The article demonstrates how the framework was tested in two cases and how it was evaluated by researchers and professionals working with climate services.
The success of climate services
More and more, climate services are being developed to support adaptation processes. At the same time, they are almost never evaluated. That is why Eva has developed a robust evaluation framework in three steps:
- Designing the Framework: First, Eva designed the framework by selecting 12 success criteria and making them measurable. She did this through a Delphi study, in which experts assessed which criteria are most relevant for determining the success of climate services.
- Testing the Framework: Then she tested this framework on two climate services that were developed in the REACHOUT project.
- Getting Feedback: Finally, researchers, developers, and users of climate services assessed how usable, credible, and transparent the framework is. Some of these were researchers and implementers involved in the climate services in which the framework was tested.
If needed, new criteria can be added to the framework.
What does the framework look like?
The framework consists of 20 indicators established through a literature study. Each criterion can be evaluated on a scale from 1: not successful to 5: successful. This makes it easier to compare climate services and monitor them more effectively. Furthermore, the framework can be used for different types of evaluations (summative, formative, developmental) and applied to all kinds of climate services.
Moving towards a standard approach
The research shows that the framework can provide a standard approach, with indicators, a measurement system, and guidelines, enabling each criterion to be assessed quantitatively. In general, researchers and users found the framework useful for multiple purposes, such as supporting monitoring and evaluation or guiding the development process of a climate service. The study emphasizes that a more robust evaluation is possible if criteria are considered early in the development of a climate service. In this way, the framework offers direct tools to design more successful climate services.
Future use of the framework
For the success of climate services, it is important that the framework is actually used and that evaluating climate services becomes a common practice. To further develop climate services, it is therefore smart to encourage the use of the success criteria as a useful tool. By increasing awareness of the criteria, a standard approach for evaluating climate services can be fostered. In addition to the framework, a short guide has also been developed as a tool to help incorporate the criteria in the development of climate services, including a checklist and guiding questions.
Want to know more?
Are you curious to learn more? Download the article. Would you like to use the framework yourself? Then you can download the guide below:
Checklist and guiding questions for climate service development and evaluation
The success of climate services
More and more, climate services are being developed to support adaptation processes. At the same time, they are almost never evaluated. That is why Eva has developed a robust evaluation framework in three steps:
- Designing the Framework: First, Eva designed the framework by selecting 12 success criteria and making them measurable. She did this through a Delphi study, in which experts assessed which criteria are most relevant for determining the success of climate services.
- Testing the Framework: Then she tested this framework on two climate services that were developed in the REACHOUT project.
- Getting Feedback: Finally, researchers, developers, and users of climate services assessed how usable, credible, and transparent the framework is. Some of these were researchers and implementers involved in the climate services in which the framework was tested.
If needed, new criteria can be added to the framework.
What does the framework look like?
The framework consists of 20 indicators established through a literature study. Each criterion can be evaluated on a scale from 1: not successful to 5: successful. This makes it easier to compare climate services and monitor them more effectively. Furthermore, the framework can be used for different types of evaluations (summative, formative, developmental) and applied to all kinds of climate services.
Moving towards a standard approach
The research shows that the framework can provide a standard approach, with indicators, a measurement system, and guidelines, enabling each criterion to be assessed quantitatively. In general, researchers and users found the framework useful for multiple purposes, such as supporting monitoring and evaluation or guiding the development process of a climate service. The study emphasizes that a more robust evaluation is possible if criteria are considered early in the development of a climate service. In this way, the framework offers direct tools to design more successful climate services.
Future use of the framework
For the success of climate services, it is important that the framework is actually used and that evaluating climate services becomes a common practice. To further develop climate services, it is therefore smart to encourage the use of the success criteria as a useful tool. By increasing awareness of the criteria, a standard approach for evaluating climate services can be fostered. In addition to the framework, a short guide has also been developed as a tool to help incorporate the criteria in the development of climate services, including a checklist and guiding questions.
Want to know more?
Are you curious to learn more? Download the article. Would you like to use the framework yourself? Then you can download the guide below:
Checklist and guiding questions for climate service development and evaluation