Impact of COnstructing Non-motorised Networks and Evaluating Changes in Travel
register | login
Home | work areas | WP2: evaluation framework and measurement tools | develop overall evaluation framework

Task: develop overall evaluation framework

You need to be logged in to edit this content.

The first main task is to develop an evaluation framework which adequately reflects the nature of interventions of this kind, i.e. that they are typically highly specific to their context and embedded in complex systems from which they cannot readily be disentangled (Kelly et al., 2005; Ogilvie et al., 2006). Our approach will therefore be informed (but not defined) by the realist approach to evaluation (Pawson and Tilley, 1997) which advocates not simply trying to establish whether an intervention has 'worked' but rather to investigate why it is (or is not) effective, in what ways, for whom and in what circumstances. Our evaluation framework will therefore be designed around the need to collect data on context, mechanism and outcomes. Under the heading of 'mechanism', we propose to collect data on putative mediating factors comprising psychosocial characteristics (such as self-efficacy), changes in perceptions of the local environment, and changes in objective measures of accessibility of local destinations. Under the heading of 'outcomes', we intend to examine changes in one core primary outcome of walking and cycling behaviour and a variety of secondary outcomes: (a) trip generation or trip suppression and modal shift in overall travel behaviour, (b) household fuel use, fuel and other transport expenditure and carbon emissions, and (c) overall physical activity. We will also critically examine the use of GIS-based measures of accessibility (such as the Accession accessibility planning tool), a measure of general health and wellbeing such as the Short Form (SF-8), and routinely-collected injury data from hospitals (HES for England, SMR01 for Scotland) or the police (STATS19).

We will also examine economic outcomes in terms of cost-per-QALY (quality-adjusted life years) from a cost-utility perspective, cost-benefit ratios from a societal perspective and costs and consequences from a realistic evaluation perspective. Methodological guidance on the economic appraisal of health effects related to walking and cycling has recently been released under the auspices of WHO Europe (Cavill et al., 2007). Members of this consortium (Rutter and Bull) were involved in the production of this guidance and the development of the WHO Health Economic Assessment Tool (HEAT), a very new tool for including health benefit outcomes in the cost-benefit analysis of infrastructural interventions. At present, HEAT can be applied to estimate the outcomes, benefits and resource savings of interventions that encourage cycling. As part of this work package we intend to test and refine HEAT for cycling and to initiate the development and testing of HEAT for walking.

by anonymous on 13 October 2008, 10 AM :
Added weblink to HEAT Uploaded HEAT user guide
by anonymous on 12 October 2008, 12 AM :
highlighted c, m and o
site by Christian Brand, ECI