The programme we were evaluating worked with women's collectives in Bihar and Odisha: self-help groups federated into producer collectives and supported with a mix of livelihood inputs and social empowerment activities. The monitoring system we inherited at the start of the engagement was organised around outputs, namely groups formed, members trained, loans disbursed, and the value of goods sold through collective marketing channels. These are defensible indicators. They could not track the programme's own theory of change, which put women's agency ahead of income.
The mismatch showed in the first round of data collection. Programme data showed strong performance on outputs. Community-level qualitative work found that women who had been in the collectives for three years or more described changes in their lives that the monitoring system did not capture. The system also did not weight those changes correctly against the income gains the programme prioritised in its reporting.
We ran a series of co-design sessions with women in both states to learn which outcomes they themselves used to judge whether the programme had made a difference to their lives. The method drew on Most Significant Change. Women narrated specific changes they had experienced, and we worked through with them what made each one significant. We started with their stories and brought no proposed indicator list.
Three outcome areas came up consistently that the original indicators had missed or underweighted. The first was voice in dealings with officials. Women across sites described a change in how they handled government officials at the panchayat office, the ration shop and the health centre. Having more information was part of it. The larger part was being taken seriously by officials, and knowing what to do when one dismissed them. Several women described specific incidents, such as a ration entitlement restored or a delayed MGNREGA payment followed up, as markers of a change the income figures could not represent.
The second was negotiation within the household. Women in longer-standing collectives said household financial decisions were now made differently. Husbands consulted them about expenditure above a certain threshold and told them when a large purchase was planned. A loan-disbursement indicator does not capture this, and an external enumerator asking a standard question about financial decision-making would find it hard to measure.
The third concerned their daughters. In several groups in both states, the change women named as most significant was one they saw in their daughters: confidence in school, a willingness to argue, a sense that their own futures were negotiable. Women in their thirties and forties described an intergenerational change and attributed it, at least in part, to what their daughters had seen them do in the collective.
The co-designed indicator set covered all three areas, and its indicators were built from the words the women used for the changes they described. It took about four weeks of iteration up front. It also meant managing a tension between indicators that mean something to participants and indicators that funders can read, and that tension does not resolve neatly.
Participatory MEL design often produces better indicators. In this case the co-design process also produced programme data of its own. How women describe change tells you things about the programme's theory of change that an externally designed indicator set cannot: which changes they weight, which they dismiss as trivial, and which they attribute to the programme and which to other factors.
Co-design at the collective level risks giving more weight to the voices of the more confident, better-connected women in each collective. We held separate discussions with newer members and with women who had dropped out of groups. Their indicator priorities were somewhat different, which gave us a useful check.
Bihar and Odisha NFHS-5 district data on women's autonomy indicators are available through the DHS Programme. NRLM's data portal carries SHG membership and loan disbursement data by state.