Evaluating infrastructure project planning and implementation: A study using qualitative comparative analysis

Many evaluations of infrastructure projects rely on methods that ignore the complexity of the projects. Although case studies are attentive to project complexity, it is difficult to identify general patterns that would apply to a larger sample of projects. Qualitative comparative analysis is a method that preserves the complexity of projects and generates insights across cases. In this contribution, we discuss our experiences with using qualitative comparative analysis for the evaluation of the planning and implementation of complex infrastructure projects. We will provide a short introduction into the main properties of the method (complex causality, systematic comparison) as well as describe some of the main operations (calibration, truth table analysis, interpretation). This will serve to demonstrate why qualitative comparative analysis is a fitting evaluation method in project development and implementation. Next, we will show how we used the method in a research project that aimed to find out under what conditions unplanned events in the implementation of infrastructure projects were dealt with satisfactorily, that is, what it took to respond to these events in an apt manner. Based on our experiences, we will summarize main lessons learned for conducting qualitative comparative analysis proper and provide suggestions for further reading.

Publication | Verweij, S. & Gerrits, L.M. (2019). Evaluating infrastructure project planning and implementation: A study using qualitative comparative analysis. Sage Research Methods Cases, 2, 1-16.

Call for papers: Case-oriented & set-theoretic approaches to comparative policy analysis

Together with Eva Thomann and Valérie Pattyn, I invite you to submit a paper proposal for our panel Case-Oriented and Set-Theoretic Approaches to Comparative Policy Analysis at the International Conference on Public Policy (ICPP4) in Montreal, organized by the International Public Policy Association. The deadline for your paper proposal is January 30th, 2019. More information about the Call for Papers can be found here. If you have any questions about the call, please do not hesitate to contact me.

The policy process is characterized by a considerable degree of complexity regarding institutional settings, actor and preference constellations, policy goals, contents, and tools. Simultaneously, there is a practical demand for better knowledge of “what works” in public policies and under what conditions or in what contexts. In order to better match methods with theories and empirical realities, the analysis of public policies faces several challenging tasks (Brans & Pattyn, 2017). First, it needs to model the complexity that characterizes the policy process and trace the underlying mechanisms. Second, comparative policy analysis detects regularities and achieves a modest degree of generalization. Finally, comparative policy analysis often deals with small or intermediate numbers of cases.

Case-oriented and set-theoretic approaches to comparative policy analysis, such as Qualitative Comparative Analysis (QCA), Coincidence Analysis (CNA), explanatory typologies, and comparative process tracing, are designed to address these challenges. Situated within a “critical realist” paradigm of social research (Gerrits & Verweij, 2013), they model different aspects of causal complexity, such as configurations of different factors leading to policy outputs or outcomes, equifinality (multiple configurations can result in the same outcome), contextual contingencies, and causal asymmetry. Moreover, they can be applied within a variety of small-N or large-N research approaches to evaluate as well as generate theories through a combination of systematic comparison with targeted in-depth case studies (Thomann & Maggetti, 2017). As interactive and iterative methods, they also lend themselves to interpretative comparative analysis (Brans & Pattyn, 2017).

Set-theoretic and case-oriented methods are increasingly common in comparative policy analysis (see e.g., Rihoux et al., 2011; Thomann, 2019), particularly in policy implementation and evaluation research (Gerrits & Verweij, 2018; Pattyn et al., 2017). This panel gathers both theoretical, conceptual, and empirical contributions that deal with the state of the art of case-oriented and set-theoretic approaches and illustrate their potential and limitations to contribute to the theory and practice of policy analysis.

Meerwaarde door PPS: Welke meerwaarde?

Publiek-Private Samenwerking (PPS) in de transportinfrastructuur is populair vanuit het idee dat samenwerking meerwaarde zou opleveren voor de betrokken partijen. Sterker nog, het zou meerwaarde opleveren die anders niet bereikt kan worden. Maar de wens lijkt de vader van de gedachte. Het bewijs voor meerwaarde door samenwerking in PPS is schaars.

Publicatie | Verweij, S. (2018). Meerwaarde door PPS: Welke meerwaarde? Agora, 34 (3), 34-37.

Slim investeren in netwerken: Oude infrastructuur, nieuwe kansen

De Nederlandse infrastructuur is van hoge kwaliteit en betrouwbaar, maar wel verouderd. In zijn recent verschenen rapport Van B naar Anders: Investeren in Mobiliteit voor de Toekomst voorspelt de Raad voor de Leefomgeving en Infrastructuur dat de kosten voor onderhoud zullen en moeten toenemen. Dat kan neerkomen op 350 miljard euro extra de komende decennia. Niet voor niets noemt minister Cora van Nieuwenhuizen (Infrastructuur en Waterstaat) het vervangen van infrastructuur de grootste opgave ooit in Nederland. Want de winkel moet tijdens de verbouwing wel open blijven. Kansen dus voor slimme investeringen.

Publicatie | Neef, M.R., Verweij, S. & Busscher, T. (2018). Slim investeren in netwerken: Oude infrastructuur, nieuwe kansen. ROmagazine, 36 (7-8), 8-11.

The evaluation of complex infrastructure projects: A guide to qualitative comparative analysis

Our book is now available at Edward Elgar! The book can be ordered at the website of Edward Elgar (here) with a promotional discount of 35% using the code “VIP35” (valid in July 2018).

Infrastructure projects are notoriously hard to manage so it is important that society learns from the successes and mistakes made over time. However, most evaluation methods run into a conundrum: either they cover a large number of projects but have little to say about their details, or they focus on detailed single-case studies with little in terms of applicability elsewhere. This book presents Qualitative Comparative Analysis (QCA) as an alternative evaluation method that solves the conundrum to enhance learning.

“Disentangling within-case complexity is a challenging task; even more so if one examines multiple cases. Gerrits and Verweij brilliantly demonstrate, using the latest conceptual and technical innovations, and through the concrete example of infrastructure projects, that QCA can produce qualitative leaps in taking on this challenge. This book is a must-read for researchers, evaluators, and practitioners who take both complexity and comparison seriously.” – Prof. Dr. Benoît Rihoux, Université catholique de Louvain, Belgium

Publication | Gerrits, L.M. & Verweij, S. (2018). The evaluation of complex infrastructure projects: A guide to qualitative comparative analysis. Cheltenham: Edward Elgar.

Voorbij de dichotomie: Op zoek naar een succesvolle combinatie van contractuele aspecten en relationele aspecten in publiek-private samenwerking

Het contract vormt een belangrijke basis in de samenwerking tussen overheid en markt. Een belangrijk kenmerk van Publiek-Private Samenwerking (PPS) is ook dat het gaat om een juridisch gestructureerd samenwerkingsverband. PPS valt echter niet te reduceren tot contractuele aspecten alleen. Succesvolle PPS behelst meer dan een contractuele relatie tussen publieke en private partijen. Naast contractuele aspecten zijn relationele aspecten in PPS evenzeer belangrijk. Het uitgangspunt in dit hoofdstuk is dat succesvolle PPS vooral afhangt van hoe contractuele en relationele aspecten worden gecombineerd. PPS-onderzoekers hebben wat dat betreft reeds gewezen op het belang van het vinden van een ‘balans’ tussen de contractuele en de relationele aspecten van PPS. Bestaand onderzoek richt zich echter vaak op de dichotome vraag of het nu de contractuele aspecten of de relationele aspecten zijn die er echt toe doen. Deze dichotome vraagstelling maakt het lastig om inzicht te krijgen in hoe de aspecten precies zouden moeten worden gecombineerd. Met andere woorden: hoe ziet die balans tussen contractuele en relationele aspecten er precies uit? Daar is dan ook nog veel onderzoekswerk naar te verrichten, naar welke combinaties van contractuele en relationele aspecten resulteren in succesvolle PPS. Het kernargument in dit hoofdstuk is dat succesvolle PPS vraagt om een combinatie van de aspecten en dat dit vraagt om een onderzoeksperspectief dat inzicht kan bieden in welke combinaties succesvol zijn.

Publicatie | Verweij, S. (2018). Voorbij de dichotomie: Op zoek naar een succesvolle combinatie van contractuele aspecten en relationele aspecten in publiek-private samenwerking. In: M. Sanders (red), Publiek-Private Samenwerking: Kunst van het Evenwicht (pp. 91-106). Den Haag: Boom Bestuurskunde.

Kwaliteitsdoelstelling Ruimte voor de Rivier geanalyseerd: De ontwerper als spil in het gebiedsproces

Na tien jaar nadert het programma Ruimte voor de Rivier de eindstreep. De resultaten zijn ernaar: het rivierengebied is zeker veiliger gemaakt voor de gevolgen van de klimaatverandering. Onderzoekers van de Rijksuniversiteit Groningen analyseerden wat er terecht is gekomen van de tweede belangrijke doelstelling van het programma: het versterken van de ruimtelijke kwaliteit in het rivierengebied. En welke strategieën er zijn toegepast om dat te bereiken. Denk hierbij aan de project-gedreven strategie, de programma-als-achtervang-strategie en de all-in strategie.

Publicatie | Verweij, S., Van den Brink, M.A, Bouwman, R. & Busscher, T. (2017). Kwaliteitsdoelstelling Ruimte voor de Rivier geanalyseerd: De ontwerper als spil in het gebiedsproces. ROmagazine, 35 (12), 12-17.

Conditions for addressing environmental determinants of health behavior in intersectoral policy networks: A fuzzy set qualitative comparative analysis

Improving health requires changes in the social, physical, economic, and political determinants of health behavior. For the realization of policies that address these environmental determinants, intersectoral policy networks are considered necessary for the pooling of resources to implement different policy instruments. However, such network diversity may increase network complexity and therefore hamper network performance. Network complexity may be reduced by network management and the provision of financial resources. This study examined whether network diversity – amidst the other conditions – is indeed needed to address environmental determinants of health behavior. We included 25 intersectoral policy networks in Dutch municipalities aimed at reducing overweight, smoking, and alcohol/drugs abuse. For our fuzzy set Qualitative Comparative Analysis we used data from three web-based surveys among (a) project leaders regarding network diversity and size (n = 38); (b) project leaders and project partners regarding management (n = 278); and (c) implementation professionals regarding types of environmental determinants addressed (n = 137). Data on budgets were retrieved from project application forms. Contrary to their intentions, most policy networks typically addressed personal determinants. If the environment was addressed too, it was mostly the social environment. To address environmental determinants of health behavior, network diversity (>50% of the actors are non-public health) was necessary in networks that were either small (<16 actors) or had small budgets (<€183,172), when both were intensively managed. Irrespective of network diversity, environmental determinants also were addressed by small networks with large budgets, and by large networks with small budgets, when both provided network management. We conclude that network diversity is important – although not necessary – for resource pooling to address environmental determinants of health behavior, but only effective in the presence of network management. Our findings may support intersectoral policy networks in improving health behaviors by addressing a variety of environmental determinants.

Publication | Peters, D.T.J.M., Verweij, S., Grêaux, K., Stronks, K. & Harting, J. (2017). Conditions for addressing environmental determinants of health behavior in intersectional policy networks: A fuzzy set qualitative comparative analysis. Social Science & Medicine, 195, 34-41.

Research grant awarded: Responsive infrastructure through responsive institutions (RITRI)

Update: We are Hiring!

The vacancy for a PhD Student at the University of Groningen, to be working in the research project, has now appeared online. The vacancy can be found here.

Research Grant Awarded

Last week, we received the great news that our research proposal Responsive Infrastructure Through Responsive Institutions (RITRI) will be awarded a €500,000 grant from NWO, in cooperation with Next Generation Infrastructures. The project has been granted to, and will be led by, Dr. Andreas Hartmann (University of Twente), Dr. Tim Busscher (University of Groningen) and myself. Two PhD-students will be hired on the project. One will be based at the University of Twente and one will be based at the University of Groningen. Together with Tim Busscher, I will supervise the PhD-student in Groningen.

Summary of the Research Proposal

Infrastructure administrators have been confronted with considerable investment needs in their infrastructure networks that mainly stem from an increased demand encountering an aged and deteriorated infrastructure stock. Since these investment needs can overlap in time and geographically, they can become cross-sectoral investment opportunities for increasing the responsiveness of the infrastructure system as a whole. However, for investment opportunities to be seized, infrastructure administrators have to be able to align their individual decision-making processes: the institutional system has to be responsive as well. The research project “Responsive Infrastructure through Responsive Institutions (RITRI)” addresses this challenge. It is guided by the following research question:

What are investment opportunities to make infrastructure systems responsive now and in the future, and what institutional rules need to be adapted so that these investment opportunities can be seized?

In order to answer the question, the research adopts an engaged scholarship approach bringing together in-depth scientific and practical knowledge on infrastructure planning and management and institutional analysis and design. It deploys multiple methods including scenario building, institutional analysis, stress-testing, and serious gaming to gain insights into the infrastructural opportunities and institutional requirements for increasing infrastructure responsiveness. By introducing responsiveness as the capability of a system to anticipate and shape future societal and technological changes, the research is particularly interested in how infrastructure administrators can play a more constitutive role in change trajectories. With its focus on cross-sectoral decision-making, the research also studies how infrastructure administrators can take up such a role in a joint manner.

The research will be conducted in close cooperation with the following infrastructure administrators: Alliander, the Port of RotterdamRijkswaterstaat, ProRail, and Schiphol.

Public private partnerships in transport: Trends and theory

Public-Private Partnerships (PPPs) continue to attract considerable attention worldwide as a governance model for delivering public services, particularly in the transport sector: in Europe, the transport infrastructure sector represents about 60% of the PPP market. PPPs are popular with policymakers since they allow the public sector to benefit from private sector capacities and resources, leading to increased quality of transport infrastructure development and management, for lower prices, and with faster delivery times. Compared to traditional procurement of transport infrastructure services, however, PPPs are complex governance arrangements involving many stakeholders with often conflicting interest, and are hence challenging to implement. Unsurprisingly then, the increasing popularity of PPPs is accompanied with calls for more research into their functioning and performance. The edited volume Public Private Partnerships in Transport: Trends and Theory (2016) addresses these research gaps. In this book review, I have discussed the strengths and weaknesses of the book.

Publication | Verweij, S. (2017). Public private partnerships in transport: Trends and theory. Transport Reviews, 37 (5), 685-687.

Load more