When we began designing the baseline survey instruments for the India Health and Climate Resilience Fund's district learning programme, we started where most quantitative studies start, with a literature review, a conceptual model and a set of constructs we wanted to measure. Once we took those instruments into the field, we found that the questions we had written and the questions respondents were answering were different. This happens often, and it is rarely written down.
The four districts in the IHCRF programme (Chamarajanagar in Karnataka, Dhubri in Assam, Khunti in Jharkhand, and West Singhbhum in Jharkhand) differ widely in ecology, health system structure and livelihood calendar. We were measuring how climate and health connect: heat exposure, water access, illness patterns, health-seeking behaviour, and how these related to household economic shocks. The constructs are well established in the global literature. The instruments that measure them are usually not designed around the seasons and routines of these communities.
Three problems came up repeatedly in piloting. HCD methods found each of them, and standard cognitive pre-testing missed them.
The first was response categories that collapsed real distinctions. A question on water source access offered options that made administrative sense (borewell, handpump, piped water, open well, river/stream). Respondents judged their own water security by whether a source was reliable across seasons, and the options hid that. In Dhubri, a handpump and a river can both be primary sources, but one dries to saline in March and the other floods in July, and each carries disease risk at a different point in the year. A single question about water source, asked without seasonal anchoring, produced data that looked clean and misrepresented water access.
The second was recall windows that did not match how people count time. Standard household survey recall periods, such as "in the last 30 days" and "in the last 12 months," follow the calendar. In agricultural communities organised around sowing, harvest and the lean season, people count time by the agricultural cycle, and a calendar month means little. Asking about healthcare expenditure "in the last 30 days" during the harvest season in Khunti gave a very different picture from asking the same question three months later. Frontline workers told us this during the HCD iteration sessions. No enumerator training protocol would have brought it out.
The third was scales that respondents answered politely without discriminating between items. Likert-scale questions on women's decision-making autonomy, a staple of empowerment measurement, returned heavily compressed distributions, with most women marking the highest positive response on nearly every item. The compression comes from the social setting of a survey interview, conducted by a stranger in or near the home, which pushes answers toward what seems socially desirable, and it says little about the actual level of autonomy. We rewrote the same questions as decision scenarios, for example "your child is ill and your husband is at work; who decides whether to take her to the health centre, and how?" The answers became far more variable and far more informative.
Applied to instrument design, HCD narrows the gap between the construct a study intends to measure and the measure it actually obtains. Most measurement error sits in that gap, and development research discusses it far less than it should. One reason is that the gap shows up clearly only between instrument design and fieldwork, a stage most studies treat as logistics, so nobody examines what the instrument is measuring at that point.
We ran two iterations with frontline health workers before the instrument was finalised, and one round of cognitive interviewing with community members in each district. This added roughly three weeks. The final instruments produced distributions that matched the qualitative patterns we already had from the district base papers. That agreement is corroboration, though it does not prove validity.
The IHCRF district base papers for all four districts are available through the Fund. The National Family Health Survey and India's Health Management Information System provide district-level comparison data for health system indicators, and we used both as anchor points while designing the instruments.