The trial tested a mobile money training programme with garment workers in Bangladesh. It was designed to measure financial inclusion: whether workers who received structured training in mobile wallet use saved more, relied less on informal moneylenders and made more use of formal financial products. Assignment was randomised, the endline came after eight months, and attrition was low. The primary outcome, the savings rate, increased significantly in the treatment group.
The difficulty came from the qualitative interviews we had run alongside the trial. We had designed them to explain the mechanism, meaning why the treatment group was saving more, and the answer did not match what the programme theory had predicted.
Workers were using mobile wallets to hide savings from family members. Their primary purpose was to keep a pool of money that husbands, brothers or in-laws could not see. Access to formal financial services and building emergency buffers for their own households were secondary.
This finding sits awkwardly with the programme's account of its own success. The rise in savings was real, and so was the mechanism. The theory of change that justified the intervention, in which expanded access to formal financial services reduces financial vulnerability, did not describe what was happening. The wallets were letting women make financial decisions on their own in households where those decisions were otherwise subject to oversight.
A write-up framed only around the savings impact would have been accurate as far as it went, because savings did increase. It would also have implied that the programme worked for the reasons it was designed to work, and it did not.
For trial design, this raises the question of how much mechanism matters next to outcome. One view holds that if the outcome is positive, the mechanism is a secondary concern that programme designers can work out later. We disagree, for two reasons.
First, if you do not understand the mechanism, you cannot predict where the programme will work and where it will not. Hiding savings depends on a specific social structure: households where other adults monitor women's financial decisions, and where each mobile phone is owned and used by one person. If either condition changes, the mechanism may stop operating, and a trial result that rests on it cannot be generalised to places where it does not hold.
Second, some mechanisms have ethical implications that the outcomes alone do not show. Women hiding savings from their households is not a problem in itself. Financial autonomy is a legitimate goal, and the women we spoke with said that being able to save without telling anyone was important to their sense of security. It also points to a household dynamic that should be understood before anyone designs a scaled programme, particularly where household conflict over resources is already a documented risk factor.
The programme funder accepted the finding, and the publication reported the mechanism alongside the impact estimate. This took some negotiation, because there was early concern that a complicated account of the findings would weaken the case for continued investment. The position everyone settled on was that findings which held up to scrutiny would do more for the programme's credibility over the long term.
For mixed-methods components in RCTs, the methodological point is about design. Qualitative work that runs alongside a trial with no defined role in testing mechanism tends to produce illustrative case material, which goes into a separate section that fewer people read. Qualitative work designed to probe the theory of change, by finding out which link in the causal chain is actually operating, changes how the quantitative findings are interpreted. The two need different protocols and different sampling strategies.
Data on financial inclusion in Bangladesh are available from MicroSave Consulting and the BRAC Institute of Governance and Development. The World Bank's Findex provides regional benchmarks on mobile money adoption and savings behaviour.