Chapter 13: Monitoring and Proving Results¶

September 2023, Hassa, in Hatay province, in southern Turkey. Seven months earlier, the earthquakes of February 6 devastated the south of the country. The municipality set up prefabricated shops in a business park for merchants who lost theirs; Chapter 9 told how. An international NGO, CORE, paid 58 of them a grant in two installments. A Turkish NGO, YSYD, collected the data. The program ends on September 30.

Then comes the final evaluation. In March, the baseline survey was done in person. This time, the merchants are called on the phone. Fifteen out of fifty-eight answer.

The results are encouraging. 86% of respondents say they are satisfied or very satisfied with the aid they received. In their households, the share of unemployed people fell from 67% at enrollment to 14% at the end of the program. The reported spending is all business purchases, stock and equipment, in line with the business plans. 93% say they had access to a way to make a suggestion or a complaint, and none says they were asked for anything in return for the aid.

The report published in April 2024 does not stop there. It states that the final sample represents only 26% of beneficiaries. That the baseline survey covered only 4 of the 12 domains of the self-reliance index used, so that the start and the end "were generally not comparable." That most of the work found again in households is temporary or irregular. That over such a short period, income only stopped falling. And that no local monitoring officer was in post at the start, which delayed the baseline survey. It presents its own results as "a descriptive approximation."

A report that says what it does not know is rarer, and more useful, than a report that shows off percentages.

What the scene teaches¶

Let us go back over the Hassa figures, separating what they establish from what they suggest.

What they establish. The money was spent as planned: invoices and monitoring visits confirm it. The fifteen respondents are satisfied. A complaint channel existed and people knew about it. These are management results, and they count: a donor, a foundation, a municipality first want to know that the money went where it was supposed to go.

What they suggest without proving it. The drop in household unemployment is real for those fifteen. But nothing shows it came from the grant: other factors may have played a part at the same time, and the survey has no comparison group. Nothing shows either that the 43 merchants who did not answer are doing as well; other surveys, as we will see, take a number that no longer answers as a sign of closure. And nothing shows that the shops will hold out another year.

What they teach for next time. Nearly all the weaknesses of the Hassa monitoring were born at the start: an incomplete baseline survey, no monitoring officer at launch, delays piling up. The report draws the lesson itself: bring in a local monitoring specialist from the launch. Monitoring is won or lost in the first month.

Your club will not do better than CORE with fewer resources. It can be just as honest, and plan earlier.

Why measure, and for whom¶

You measure first for the affected people. The Core Humanitarian Standard (CHS), in its 2024 edition, presents itself as a tool that lets people and communities hold those who help them to account. Its commitments are written from their point of view: they must be able to give their opinion, complain without risk, and learn what was drawn from their answers.

You measure next for the Rotary Foundation and for donors. The global grant terms in force in July 2026 require a project to be measurable: sponsors choose standard measures from the Foundation's monitoring and evaluation supplement, and may add others.

Finally, you measure to decide. The Minimum Economic Recovery Standards (MERS) make monitoring a quality condition throughout the program: data collected regularly, reviewed in meetings, and a program adjusted when the market changes. Monitoring that leads to no decision has only served to fill a report.

Measure little, measure well¶

The temptation for a club discovering monitoring is to measure everything. You need to do the opposite.

The MERS recommend not waiting for annual evaluations and collecting monitoring data regularly, and they specify: "Lean data can be used to collect a small amount of information relatively frequently." The CHS 2024 asks for disaggregated data (women and men, ages, circumstances) to be collected "in ways that minimise demands on them": without weighing on people. Chapter 3 already described assessment fatigue; heavy monitoring prolongs it.

A clarification about the MERS, because a confusion is going around. Their "key indicators" are not income or job indicators to reuse in your project. The standard defines them itself: "Key indicators are 'signals' that show whether a minimum standard has been met." They serve to judge the quality of a program (is it market-aware, coordinated, monitored?), not to measure what merchants earn. For outcome indicators, you have to look elsewhere.

The baseline after a disaster¶

The baseline is the snapshot taken before the project starts, against which everything that follows will be compared. The International Federation of Red Cross and Red Crescent Societies (IFRC) livelihoods guidelines of 2010 define it as the set of conditions existing at the start of a program. They add two useful points.

First, the initial assessment is not enough: it does not always provide data precise enough to measure an effect, and the baseline must be measured separately, usually after the assessment and before the project starts. Your assessment form from Chapter 3 serves to choose the project; the baseline serves to judge it.

Second, after a disaster, the baseline has two parts: the original situation and the effects of the disaster. You measure today's situation, and you record what people remember from before. In Hassa, the report thus compares income in 2022 with income in the month following the earthquake.

The IFRC gives, in a table, examples of baseline indicators for livelihoods programs. Among them, the ones that best serve an economic recovery:

the number of people who have been able to resume their pre-disaster activity;
the number of people who have access to credit to replace lost assets;
the number of people selling their main productive assets;
the level of debt;
the number of family members who have work;
the number of people relying on relatives to get by.

The MERS add a warning, in the form of an example with no location: a baseline taken in a year with a very good harvest said nothing about a normal year, and the organization had to look for other data when drought came. After a disaster, the risk runs the other way: a baseline taken at the worst moment makes any return to normal look spectacular.

The Foundation, for its part, leaves no choice. The monitoring and evaluation supplement makes the baseline the fourth of its seven steps, and the community economic development guidelines (April 2024) ask for starting data to be collected before the project so results can be measured. One practical detail follows: a global grant pays for nothing that comes before its approval (Chapter 10). Plan who will do the baseline, and with what money, between approval and the first payment to beneficiaries.

Indicators that have worked elsewhere¶

No international framework provides a single, quantified list of economic recovery indicators. Real evaluations use a few, always the same ones. The table covers five families.

Indicator	Real example	What to know
Businesses reopened at 3, 6, 9, and 12 months	Katrina: 441 cumulative reopenings by late September 2005, 1,190 in October 2007	easy to count on foot; says nothing about income
On-time repayment	Nepal: 99% in July 2018, recovery credit	the lender's figure; cross-check with arrears
Employment at end of training	construction trades training (Chapter 7)	also measure six months later
Reported income, before and after	Haiyan, CARE program	self-reported; ask for a range
Coping strategies	Hassa: savings drawn down, borrowing, asset sales	easier to collect than income

The last row deserves a closer look. Asking a merchant how much they earn is delicate, and the answer is often wrong. Asking whether they had to draw down their savings, borrow outside the family, sell a tool, or take a child out of school is simpler, and more telling. In Hassa, the merchants' households report having dipped into their savings (48%) or borrowed from people outside the family for basic needs (28%). The IFRC lists the sale of productive assets among its baseline indicators, and the World Food Programme and FAO use coping strategy indices of the same kind.

Here, to give an order of magnitude, is what a final program evaluation can produce with these tools. After Haiyan, the evaluation of CARE's program in the Philippines, carried out in 2016 by a Philippine university center, finds that 73% of households receiving transfers earn an income, with an average monthly net income of 2,785.15 pesos against 2,000 before. But it also notes that debt repayment and savings remain "not improved or not yet met" for a significant share of households. The figures come from beneficiaries' own statements, with no comparison group.

Contribution, not attribution¶

When there is no way to compare with people who were not helped, honesty means speaking of contribution. The CARE evaluation gives the wording. Noting that households had reduced their coping strategies, it writes: "As CARE was the major implementer of livelihoods in each Barangay, it is reasonable to assume that CARE's intervention contributed to this reduction." Not "caused," not "thanks to": contributed, and because the organization was the main actor on the ground. Your club can write its reports the same way.

Survivorship bias¶

Business surveys after a disaster share a flaw that their best authors acknowledge: they interview the businesses that still exist. The Federal Reserve Bank of New York says so about its survey of small businesses in Puerto Rico after the 2017 hurricanes: businesses that disappeared are not systematically included, and its results "most likely understate the adverse impact of the hurricanes." In New Orleans and along the Gulf Coast, a 2008 survey uses disconnected phone numbers as a sign of closure, and finds nearly one in four among Black-owned businesses.

For your club, the consequence is practical. Keep the complete list of starting beneficiaries, and try to find out what became of those who no longer answer. In Hassa, that is 43 merchants out of 58.

One last example, already seen in Chapter 6: Fonkoze, in Haiti, interviewed 274 of its affected clients a few months after the 2010 earthquake, with an assessment tool that existed before the disaster. Its figures are the institution's own, and it says so. A tool ready before the shock is what makes monitoring possible afterward.

What the Rotary Foundation requires¶

The rules below are those of the documents in force in July 2026; check them in the Terms and Conditions in force on My Rotary before submitting an application (Appendix E).

The monitoring plan. The Foundation's monitoring and evaluation supplement, dated April 2015 and still in use, describes seven steps: set clear goals, choose the standard measures, add project-specific measures, establish the baseline and collection methods, submit the plan with the application, collect data during the project, then evaluate and report the results. It adds two instructions: avoid double counting, and do not simply report all the residents of a community or region as beneficiaries.

The standard measures for community economic development are six: the total number of direct beneficiaries, the number of entrepreneurs supported, the number of businesses supported, the number of people trained, the number of jobs created (measured through employer surveys), and the number of young people employed in income-generating activities. Since the supplement is old, check in the Grant Center that no more recent version exists.

The budget. Global grant sponsors can devote up to 10% of the budget to measuring results. That is the line that will pay for your baseline, your surveys, and a local partner's time.

The duration. The Foundation's Code lists monitoring among the six dimensions of sustainability, and asks for a measurement plan that confirms significant improvement, "ideally for three or more years." This is a recommendation, not an obligation; but it goes beyond the length of most projects.

The reports. Deadlines differ by grant.

Grant	Reports required in July 2026
Disaster Response Grant	report within six months of payment to the district
District grant	final report within 12 months of payment, or 2 months after full disbursement if earlier
Global grant	first progress report within 12 months of the first payment, then each within 12 months of acceptance of the previous report
Global grant	final report within two months of the end of activities

A global grant report must in particular account for the measures chosen and the data collected, and explain how the results will be sustained over time. Receipts and bank statements are kept for at least five years. Finally, a rule that affects the whole district: the Foundation approves no new application from sponsors who have an overdue report, for any grant whatsoever. A forgotten report on a small project can block the big one (Chapter 11).

Listen, give back, document¶

Listen. The CHS 2024 calls for safe, accessible, and appropriate means so that all groups in a community can give feedback and file a complaint (requirement 5.1). In Hassa, 79% of respondents preferred a combination of channels, such as leaflets and SMS, to a single channel. For a club, this can be a phone number held by a member who did not select the beneficiaries, and a regular visit on site.

Give back. The same standard calls for sharing the analysis and lessons drawn from feedback and monitoring with the people and communities supported (requirement 7.4). The merchants who gave you their figures are entitled to the summary, presented simply, with what the club concluded from it.

Document. The disaster guide already set the rule: no identifiable photo without consent, and for children, permission from the person responsible for them. The CHS 2024 extends it to data: safe, ethical, and effective management of information, to limit risks to people (requirement 4.3). A list of beneficiaries with their debts and incomes is sensitive data; it does not circulate in a club messaging group.

Share failures. The Hassa report is public, with its limits. The MERS ask, at the end of a program, to check whether the initial assessment actually guided the program. Chapter 14 will say that exit strategies are rarely evaluated; failures, even less. A club that publishes what did not work does the next club a service.

What the club does: the monitoring plan¶

The monitoring plan fits on one page. It names five to eight indicators, no more, chosen from the Foundation's standard measures and from the outcome indicators described above. For each one, it states the question asked, who asks it, when, and to whom.

Take a typical example: your club funds, with a local lender, restart grants for forty small businesses on a flooded street, with management training.

Indicator	Question or source	Who collects	When
Businesses supported (standard)	project records	project treasurer	at each payment
Businesses open	walk down the street	club pair	3, 6, 12 months, then yearly
Prior activity resumed	"Have you restarted your business?"	local partner	start, 6 and 12 months
Coping strategies	savings drawn down, borrowing, tool sold	local partner	start, 6 and 12 months
On-time repayment	lender data	lender	every quarter
People trained (standard)	attendance sheets	trainer	end of training
Jobs created (standard)	survey of merchants	local partner	12 months
Complaints received and handled	complaints register	designated member	every month

Three rules go with this table. The baseline comes before the first payment: without it, none of the later figures will prove anything. The local partner collects: a 2005 guide on exit strategies, published by the C-SAFE consortium, recommends handing data collection to local partners as much as possible, to build their capacity and lighten the sponsor's workload. Monitoring outlives the project: if the Foundation's Code talks about three years, someone must still be counting open businesses when the grant is closed. Chapter 14 will say who.

The pitfalls¶

Pitfall	Why it happens	Remedy
Counting all residents as beneficiaries	the number impresses	the Foundation forbids it; count direct beneficiaries
Taking the baseline after the first payments	the emergency presses	budget the baseline before the first payment
Measuring twenty indicators	you want to know everything	five to eight, tied to the goals
Asking for exact income	it is the obvious indicator	coping strategies, ranges
Interviewing only those who answer	it is easier	follow the full list, look for the missing
Writing "thanks to the club"	you want to thank donors	speak of contribution
Citing MERS indicators as income	confusion of vocabulary	the MERS judge program quality
Forgetting a report to the Foundation	change of president	report calendar kept by the district
Keeping results within the club	it does not occur to you	give them back to the people interviewed

What your club now has in hand¶

At the end of this work, your club has a monitoring plan: five to eight indicators, a baseline taken before the first payment, a three-year schedule, a person or organization responsible for each data collection, a complaint channel, and a date for reporting back to beneficiaries. This is the piece this chapter adds to the recovery file; Appendix B gives the template.

The Hassa report did not prove that its grants had saved 58 businesses. It showed what was known and what was not, and it said why. That is all you can ask of honest monitoring. One question remains that this plan raises without resolving: who will still be counting the open shops when the project is over? Preparing for that moment from day one is the subject of the next chapter.