Insights from the field

Methodological note · Programme design · 9 May 2026

Theory of change isn't a flowchart

Most theories of change we inherit are decorative. Boxes, arrows, every line pointing up and to the right; no branches, no failure modes, no assumptions stated specifically enough to be wrong. A useful TOC commits the programme to a falsifiable mechanism — if no finding could disconfirm the theory, what you have is a programme description, not a theory. Contribution analysis, taken seriously, imposes the discipline that flowcharts evade. Write the theory as prose first; the prose forces commitment to mechanisms that diagrams smooth over.

Theory of Change Programme Design Contribution Analysis

Methodological note · Indicators · 9 May 2026

The composite index that hid the policy choice

Composite indices look technical but bury policy decisions in weight selection. Equal weighting is not neutral — it embeds the substantive claim that a unit of heat-vulnerability matters as much as a unit of livelihood-vulnerability. Run the same data through five reasonable weighting schemes and the top and bottom deciles tend to be stable; the middle of the distribution swings substantially, which is exactly the part of the distribution most allocation policy is operating on. Then Goodhart's law arrives: once the index is a target, sub-component manipulation begins.

Indicators Composite Index Policy

Methodological note · Survey design · 9 May 2026

What translation does to a survey instrument

Forward-back translation is the standard procedure and it catches a small fraction of the failures that matter. Three failure modes survive almost untouched: register mismatch (formal Hindi or Tamil reads as alien to most respondents), examples that don't carry across regions, and policy idioms that exist in only one language. Cognitive interviews are the corrective; bilingual enumerators detect the failures and also smooth them over invisibly in the field. Instrument adaptation across languages is research, not translation, and budget accordingly.

Survey Design Translation Multilingual

Methodological note · Panel data · 9 May 2026

Attrition isn't random: what panel surveys quietly lose

Households drop out of panel surveys for reasons correlated with the outcomes the panel is trying to measure — migration after a livelihood shock, household dissolution, illness, withdrawal as programme contact diminishes. Inverse probability weighting on baseline observables corrects for the easy part. The bias that survives weighting is the bias that matters most. Panels that report low attrition often achieved it by selecting on stability at baseline, which is the selection problem moved earlier in the pipeline. Above 15% attrition, the analysis frame should change, not just the weights.

Panel Data Selection Bias Methodology

Methodological note · Beneficiary feedback · 9 May 2026

Why we stopped trusting satisfaction scores

Eighty-two per cent of beneficiaries are very satisfied. The figure is almost meaningless. Three mechanisms drive systematic inflation: courtesy bias, withdrawal fear, reciprocity. The empirical signature is recognisable — ceiling effects, low variance, satisfaction scores that don't discriminate across visibly different facilities. A measure that doesn't vary isn't a measure. Behavioural data, comparative phrasings ("would you recommend this to your sister?"), and third-party CATI vendors get closer to the truth than face-to-face Likert scales ever will.

Beneficiary Feedback Courtesy Bias Survey Design

Methodological note · IHCRF

Why quantitative surveys in India need Human-Centred Design

While building the IHCRF district learning architecture, we piloted quantitative surveys using Human-Centred Design — not as a stylistic choice, but because standard instrument development kept producing questions that didn't map to how respondents actually think about their own lives. Iterating with frontline workers before enumerator training surfaced structural problems: response categories that collapsed meaningful distinctions, recall windows mismatched to agricultural and health-seeking rhythms, scales respondents politely answered but did not use discriminatingly. HCD applied to instruments narrows the gap between the question we wrote and the question respondents actually answer — which is where most measurement error hides.

HCD Survey Design IHCRF

Field note · IHCRF District Research

What district base papers revealed about climate-health interdependence

Writing district base papers for Chamarajanagar, Dhubri, Khunti, and West Singhbhum for the India Health and Climate Resilience Fund forced us to confront how rarely health and climate data are held together at a district scale. Climate vulnerability layers — heat, flood, air quality, water stress — sit in one set of ministries; health system capacity and morbidity data in another. Even where both exist, the administrative geographies rarely line up. District learning architectures are valuable less for novel findings than for forcing integrated reads on systems the administrative state keeps apart.

Climate-Health District Research IHCRF

Methodological note · DIU / Transform Rural India

Listening for aspiration in rural and peri-urban India

The DIU / Transform Rural India study on women's aspirations is teaching us how much mainstream livelihoods research has flattened the concept of "choice." Questions about preferred occupations routinely return sanitised answers because respondents have been trained by decades of programme intake forms. Getting past the scripted answers required layered tools: open-ended life-history prompts, mobility mapping, and decision-scenarios rather than preference rankings. The pattern that emerges isn't "no aspiration" — it is constrained aspiration: sharply specific, highly contextual, and often foreclosed by factors women can name very precisely.

Gender Aspirations Rural India

Field note · Alliance for Responsible Aquaculture

Evidence standards in a sector without a research tradition

Leading India operations for the Alliance for Responsible Aquaculture — working with fish farmers across Andhra Pradesh on water quality, stocking density, and engagement with the Animal Welfare Board and Bureau of Indian Standards — put us inside a sector where the research-to-policy pipeline is almost non-existent. The same evidentiary questions that are routine in human welfare work (what counts as a standard, how thresholds are justified, how compliance is measured) arrive here without the scaffolding of peer-reviewed literature or established indicators. In emergent sectors, the first contribution isn't a study — it's the measurement vocabulary the sector will argue over for the next decade.

Animal Welfare Policy Aquaculture

Field note · Railway Children India

Reunification research along the Delhi–Bengal corridor

Studying family reunification strategies for runaway platform children for Railway Children India meant working along the Delhi–Bengal corridor across six central Indian states — and quickly dissolving the assumption that "reunification" is a single event with a single outcome. For the children we spoke with, reunification was iterative: a return home, a re-departure, sometimes several cycles, shaped by livelihood shocks, climate stress, and household conflict the originating ecosystems had never resolved. Protection-system metrics that treat reunification as an endpoint create perverse incentives.

Child Protection Qualitative Railway Corridor

Methodological note · India Institute

Advising a critical thinking RCT in government schools

The India Institute's RCT on a critical thinking curriculum — commissioned by the Tamil Nadu government and evaluated across government schools — raises a question we keep returning to: what exactly are we measuring when we measure critical thinking? Standardised instruments for reasoning, argument evaluation, and epistemic humility exist, but each carries assumptions about language, testing culture, and classroom norms. A defensible evaluation has to triangulate — test scores alongside classroom observation and teacher practice measures — or the headline finding will be an artefact of the instrument choice.

RCT Education Tamil Nadu

Methodological note · Bihar & Odisha

Co-designing MEL frameworks with women's collectives

When we began evaluating a multi-state women's empowerment programme, the existing monitoring framework captured outputs but missed the nuanced changes in agency that participants valued most. The co-designed framework revealed that traditional indicators like "number of loans taken" missed critical dimensions. Women identified confidence in public speaking, ability to negotiate with officials, and shifts in family decision-making as the outcomes that mattered. Participatory MEL design requires upfront time investment but creates more accurate measurement — the measurement process becomes part of the intervention itself.

Gender MEL Bihar & Odisha

Methodological note · Bangladesh

What photovoice revealed about climate displacement

In Chittagong's informal settlements, standard survey methods struggled to capture the compound vulnerabilities faced by climate-displaced families. Photovoice — where participants document their lives through photography — surfaced issues no questionnaire would have found: not just physical infrastructure gaps, but the social dynamics of exclusion — being turned away from clinics, children unable to enroll without address proof, the daily negotiations required to access water. Visual methods shift power in the research relationship, giving participants editorial control over their narrative.

Climate Participatory Bangladesh

Methodological note · Bangladesh

When RCT results complicated the narrative

Our mobile money training trial with garment workers produced surprising results. While savings rates increased significantly, the mechanism wasn't what we expected: workers used mobile wallets primarily to hide savings from family members, not to access formal financial services. Financial inclusion isn't just about access to tools — it's about navigating household power dynamics and social expectations around income sharing. Without the accompanying interviews, we'd have published a misleading success story.

RCT Financial Inclusion Bangladesh

Methodological note · Sri Lanka

Facility assessments don't tell the full story

In a primary health care evaluation across three districts, facility-level data showed strong performance on process indicators. But patient experience data told a different story — long wait times, poor communication, and low trust in referral pathways. Health workers themselves identified the disconnect: they were optimising for metrics that got reported upward, not for the dimensions of care that mattered to patients. Multi-perspective evaluation combining supply-side and demand-side data is essential.

Health Evaluation Sri Lanka

Methodological note · Rajasthan

The gap between stated and revealed preferences

In rural Rajasthan, surveyed parents overwhelmingly expressed support for daughters' secondary education. But enrolment data showed persistent dropout at the transition to secondary. Focus groups revealed individual preferences were overridden by collective social norms: families feared sanctions if daughters travelled to distant schools, even when they personally supported education. Individual-level surveys can mask community-level norm effects.

Education Social Norms Rajasthan

Policy note · Odisha

When programmes interact: PDS meets ICDS

Studying child nutrition in tribal districts, we found that household-level analysis missed a key dynamic: families receiving both PDS and ICDS had worse nutrition outcomes than those receiving ICDS alone in some blocks. The mechanism was administrative: dual enrolment created documentation burdens that caused families to miss ICDS supplementary feeding days. Two programmes designed independently were undermining each other. Convergence analysis should be standard practice.

Social Protection Policy Odisha