Insights from the field

Methodological note · Official statistics · 23 September 2026

The consumption survey came back with two headline numbers

India went eleven years without a published consumption expenditure survey. The series that came back publishes every figure twice, with and without the imputed value of items received free through welfare programmes, and the choice between them moves nominal rural growth between the two rounds from 9.3 per cent to 10.0 per cent. Nobody has to declare which series they used. When two reports disagree by three per cent on rural consumption, the first question is not which is wrong but whether they are the same series.

Consumption Official Statistics Imputation

Methodological note · Data quality · 23 September 2026

The total row

Published tables carry their own total, in the same column with the same formatting. Sum the column instead and you get exactly twice the real figure, or three times it where NCRB publishes Total (States), Total (UTs) and Total (All India). The error defeats every check a careful person would apply: the magnitude is plausible, the parts still sum to the total because the total is one of the parts, and every state’s share is wrong by the same uniform factor, so the chart looks right and only the axis is wrong.

Data Quality Official Statistics Validation

Methodological note · Indicators · 23 September 2026

When a scale stops discriminating

A case-study library graded its evidence strong, moderate or emerging on a reference count. Eight of 204 read emerging — a plausible tail, and entirely an artefact: those eight were the ones whose citations the parser could not read, because they used a different heading. Fix the parser and all 204 read strong, which is worse, because a field with zero variance still reads as an assessment having been made. Tabulate the derived variable, not the rule.

Indicators Measurement Evidence

Methodological note · RCT design · 29 May 2026

Power calculations in cluster RCTs with realistic ICCs

Cluster-randomised trials in Indian development contexts produce more underpowered studies than any other design. The reason is almost always the same: the intra-cluster correlation coefficient assumed at the power-calculation stage was wrong, usually low, copied from a paper on a different outcome in a different country. In education outcomes at school level, ICCs cluster between 0.10 and 0.25, not the 0.02 the pre-registration named. Pull ICCs from the actual data source. Report sensitivity across the plausible range. Budget for a real baseline — a five-cluster pilot is the only way to know.

RCT Power Calculation ICC

Methodological note · Survey design · 29 May 2026

Non-response as data

Every survey produces two data sets: the substantive variable, and the response pattern. The first is what the report presents; the second is what most reports discard. Refusal is item-specific, has a subgroup structure, accumulates within an interview, and "don't know" is not the same as "refused." A published table of refusal rates by item is often more informative than the corresponding substantive table — particularly on sensitive topics. Timing data from ODK / SurveyCTO makes pauses at sensitive items available at almost no marginal cost. Most analyses discard it, which is a shame.

Non-Response Survey Design Missing Data

Methodological note · Labour economics · 29 May 2026

The wage-gap decomposition that didn't decompose cleanly

Blinder-Oaxaca decompositions report an "explained" share and an "unexplained" residual usually labelled as discrimination. When the residual is sixty to eighty per cent of the gap — as it usually is in Indian gender and caste wage analyses — that is not a finding about discrimination. It is a finding about occupational segregation, selection into the labour force, the limits of years-of-schooling as a skill proxy, and noisier wage measurement for the populations the residual absorbs. The residual is whatever the model did not have a variable for.

Wage Gap Blinder-Oaxaca Labour Economics

Methodological note · Administrative data · 29 May 2026

When admin data is the only data

NREGA muster rolls, PMAY-G housing records, PDS ledgers, UDISE+ — for low-frequency outcomes in remote districts, administrative data is often the only source large enough to work with. It is collected for administration, not measurement. Recorded attendance is not work done; job-card holders are a selected population; "person-days generated" has shifted definition; absence in the MIS is ambiguous between "did not happen" and "was not recorded." Document the administrative process that produced the record. Triangulate with a small ground-truth subsample. Report the administrative variable as what it is.

Administrative Data NREGA PMAY-G

Field note · Caste & access · 29 May 2026

The interview that ended at the doorway

Caste-based access constraints in Indian fieldwork are invisible in most methods sections, but they shape every sample we collect. Doorway refusals mediated by a non-respondent intermediary. Household heads answering on behalf of the woman who was on the list. Enumerator caste and gender affecting what is said and what is omitted. The replacement protocol absorbs the gap, the response rate looks intact, the bias is structural. The sample frame is who we asked. The achieved sample is who answered. Pretending the gap is random is the methodological choice that protects the headline.

Caste Fieldwork Access

Methodological note · Programme design · 9 May 2026

Theory of change isn't a flowchart

Most theories of change we inherit are decorative. Boxes, arrows, every line pointing up and to the right; no branches, no failure modes, no assumptions stated specifically enough to be wrong. A useful TOC commits the programme to a falsifiable mechanism — if no finding could disconfirm the theory, what you have is a programme description, not a theory. Contribution analysis, taken seriously, imposes the discipline that flowcharts evade. Write the theory as prose first; the prose forces commitment to mechanisms that diagrams smooth over.

Theory of Change Programme Design Contribution Analysis

Methodological note · Indicators · 9 May 2026

The composite index that hid the policy choice

Composite indices look technical but bury policy decisions in weight selection. Equal weighting is not neutral — it embeds the substantive claim that a unit of heat-vulnerability matters as much as a unit of livelihood-vulnerability. Run the same data through five reasonable weighting schemes and the top and bottom deciles tend to be stable; the middle of the distribution swings substantially, which is exactly the part of the distribution most allocation policy is operating on. Then Goodhart's law arrives: once the index is a target, sub-component manipulation begins.

Indicators Composite Index Policy

Methodological note · Survey design · 9 May 2026

What translation does to a survey instrument

Forward-back translation is the standard procedure and it catches a small fraction of the failures that matter. Three failure modes survive almost untouched: register mismatch (formal Hindi or Tamil reads as alien to most respondents), examples that don't carry across regions, and policy idioms that exist in only one language. Cognitive interviews are the corrective; bilingual enumerators detect the failures and also smooth them over invisibly in the field. Instrument adaptation across languages is research, not translation, and budget accordingly.

Survey Design Translation Multilingual

Methodological note · Panel data · 9 May 2026

Attrition isn't random: what panel surveys quietly lose

Households drop out of panel surveys for reasons correlated with the outcomes the panel is trying to measure — migration after a livelihood shock, household dissolution, illness, withdrawal as programme contact diminishes. Inverse probability weighting on baseline observables corrects for the easy part. The bias that survives weighting is the bias that matters most. Panels that report low attrition often achieved it by selecting on stability at baseline, which is the selection problem moved earlier in the pipeline. Above 15% attrition, the analysis frame should change, not just the weights.

Panel Data Selection Bias Methodology

Methodological note · Beneficiary feedback · 9 May 2026

Why we stopped trusting satisfaction scores

Eighty-two per cent of beneficiaries are very satisfied. The figure is almost meaningless. Three mechanisms drive systematic inflation: courtesy bias, withdrawal fear, reciprocity. The empirical signature is recognisable — ceiling effects, low variance, satisfaction scores that don't discriminate across visibly different facilities. A measure that doesn't vary isn't a measure. Behavioural data, comparative phrasings ("would you recommend this to your sister?"), and third-party CATI vendors get closer to the truth than face-to-face Likert scales ever will.

Beneficiary Feedback Courtesy Bias Survey Design

Methodological note · IHCRF

Why quantitative surveys in India need Human-Centred Design

While building the IHCRF district learning architecture, we piloted quantitative surveys using Human-Centred Design — not as a stylistic choice, but because standard instrument development kept producing questions that didn't map to how respondents actually think about their own lives. Iterating with frontline workers before enumerator training surfaced structural problems: response categories that collapsed meaningful distinctions, recall windows mismatched to agricultural and health-seeking rhythms, scales respondents politely answered but did not use discriminatingly. HCD applied to instruments narrows the gap between the question we wrote and the question respondents actually answer — which is where most measurement error hides.

HCD Survey Design IHCRF

Field note · IHCRF District Research

What district base papers revealed about climate-health interdependence

Writing district base papers for Chamarajanagar, Dhubri, Khunti, and West Singhbhum for the India Health and Climate Resilience Fund forced us to confront how rarely health and climate data are held together at a district scale. Climate vulnerability layers — heat, flood, air quality, water stress — sit in one set of ministries; health system capacity and morbidity data in another. Even where both exist, the administrative geographies rarely line up. District learning architectures are valuable less for novel findings than for forcing integrated reads on systems the administrative state keeps apart.

Climate-Health District Research IHCRF

Methodological note · DIU / Transform Rural India

Listening for aspiration in rural and peri-urban India

The DIU / Transform Rural India study on women's aspirations is teaching us how much mainstream livelihoods research has flattened the concept of "choice." Questions about preferred occupations routinely return sanitised answers because respondents have been trained by decades of programme intake forms. Getting past the scripted answers required layered tools: open-ended life-history prompts, mobility mapping, and decision-scenarios rather than preference rankings. The pattern that emerges isn't "no aspiration" — it is constrained aspiration: sharply specific, highly contextual, and often foreclosed by factors women can name very precisely.

Gender Aspirations Rural India

Field note · Alliance for Responsible Aquaculture

Evidence standards in a sector without a research tradition

Leading India operations for the Alliance for Responsible Aquaculture — working with fish farmers across Andhra Pradesh on water quality, stocking density, and engagement with the Animal Welfare Board and Bureau of Indian Standards — put us inside a sector where the research-to-policy pipeline is almost non-existent. The same evidentiary questions that are routine in human welfare work (what counts as a standard, how thresholds are justified, how compliance is measured) arrive here without the scaffolding of peer-reviewed literature or established indicators. In emergent sectors, the first contribution isn't a study — it's the measurement vocabulary the sector will argue over for the next decade.

Animal Welfare Policy Aquaculture

Field note · Railway Children India

Reunification research along the Delhi–Bengal corridor

Studying family reunification strategies for runaway platform children for Railway Children India meant working along the Delhi–Bengal corridor across six central Indian states — and quickly dissolving the assumption that "reunification" is a single event with a single outcome. For the children we spoke with, reunification was iterative: a return home, a re-departure, sometimes several cycles, shaped by livelihood shocks, climate stress, and household conflict the originating ecosystems had never resolved. Protection-system metrics that treat reunification as an endpoint create perverse incentives.

Child Protection Qualitative Railway Corridor

Methodological note · India Institute

Advising a critical thinking RCT in government schools

The India Institute's RCT on a critical thinking curriculum — commissioned by the Tamil Nadu government and evaluated across government schools — raises a question we keep returning to: what exactly are we measuring when we measure critical thinking? Standardised instruments for reasoning, argument evaluation, and epistemic humility exist, but each carries assumptions about language, testing culture, and classroom norms. A defensible evaluation has to triangulate — test scores alongside classroom observation and teacher practice measures — or the headline finding will be an artefact of the instrument choice.

RCT Education Tamil Nadu

Methodological note · Bihar & Odisha

Co-designing MEL frameworks with women's collectives

When we began evaluating a multi-state women's empowerment programme, the existing monitoring framework captured outputs but missed the nuanced changes in agency that participants valued most. The co-designed framework revealed that traditional indicators like "number of loans taken" missed critical dimensions. Women identified confidence in public speaking, ability to negotiate with officials, and shifts in family decision-making as the outcomes that mattered. Participatory MEL design requires upfront time investment but creates more accurate measurement — the measurement process becomes part of the intervention itself.

Gender MEL Bihar & Odisha

Methodological note · Bangladesh

What photovoice revealed about climate displacement

In Chittagong's informal settlements, standard survey methods struggled to capture the compound vulnerabilities faced by climate-displaced families. Photovoice — where participants document their lives through photography — surfaced issues no questionnaire would have found: not just physical infrastructure gaps, but the social dynamics of exclusion — being turned away from clinics, children unable to enroll without address proof, the daily negotiations required to access water. Visual methods shift power in the research relationship, giving participants editorial control over their narrative.

Climate Participatory Bangladesh

Methodological note · Bangladesh

When RCT results complicated the narrative

Our mobile money training trial with garment workers produced surprising results. While savings rates increased significantly, the mechanism wasn't what we expected: workers used mobile wallets primarily to hide savings from family members, not to access formal financial services. Financial inclusion isn't just about access to tools — it's about navigating household power dynamics and social expectations around income sharing. Without the accompanying interviews, we'd have published a misleading success story.

RCT Financial Inclusion Bangladesh

Methodological note · Sri Lanka

Facility assessments don't tell the full story

In a primary health care evaluation across three districts, facility-level data showed strong performance on process indicators. But patient experience data told a different story — long wait times, poor communication, and low trust in referral pathways. Health workers themselves identified the disconnect: they were optimising for metrics that got reported upward, not for the dimensions of care that mattered to patients. Multi-perspective evaluation combining supply-side and demand-side data is essential.

Health Evaluation Sri Lanka

Methodological note · Rajasthan

The gap between stated and revealed preferences

In rural Rajasthan, surveyed parents overwhelmingly expressed support for daughters' secondary education. But enrolment data showed persistent dropout at the transition to secondary. Focus groups revealed individual preferences were overridden by collective social norms: families feared sanctions if daughters travelled to distant schools, even when they personally supported education. Individual-level surveys can mask community-level norm effects.

Education Social Norms Rajasthan

Policy note · Odisha

When programmes interact: PDS meets ICDS

Studying child nutrition in tribal districts, we found that household-level analysis missed a key dynamic: families receiving both PDS and ICDS had worse nutrition outcomes than those receiving ICDS alone in some blocks. The mechanism was administrative: dual enrolment created documentation burdens that caused families to miss ICDS supplementary feeding days. Two programmes designed independently were undermining each other. Convergence analysis should be standard practice.

Social Protection Policy Odisha