Clear reporting will get us moving in the right direction. Always.
What’s in a number? As we let the latest Costa Rica COVID-19 stats sink in, Jelena Radic puts forward argument that perhaps we’re all looking at the wrong numbers and asks what the plan is here?
For me, the daily dose of cringe sets in when I start reading updates on COVID-19 in Costa Rica.
The “numbers matter, but remember there are people behind the numbers” debate has no place here, as no one is questioning that.
This is a rant about which numbers are being reported and how… and about the consequences of the current reporting methods. It’s about why it’s important to question the reporting method, and adjust some of it. I believe it’s important for anyone who wants to understand how Costa Rica is handling the pandemic.
One caveat, I don’t actually watch the daily conferences. Maybe I miss some of what they say. I just look up different news outlets later in the day and read the reports.
What’s wrong with that, you may ask. Well, the first question any analyst asks themselves is “what is missing in this data set?”
And I can say that what keeps my frustration growing every day is the missing context of the shared numbers.
Costa Rica coronavirus data for June 25, 2020: https://t.co/9FqNoAQ30i
— The Tico Times (@TheTicoTimes) June 25, 2020
It could be that they shared the context early on and still assume we know it.
But that’s a horrible assumption. I want journalists to assume I know nothing. Every time they report on anything. And especially when they report the pandemic numbers.
To clarify my position, I wouldn’t call myself an analytics professional, but I’ve dabbled enough in the field to understand that what they’re sharing arn’t necessarily the main KPIs one would pursue when trying to understand the current underlying shape of the pandemic and its behavior.
So, what is this data set they present to us every day?
Yesterday’s ‘score’ and a tally of all accumulated cases are two ‘lead’ indicators. (Spoiler alert, cumulative ‘anything’ is rarely a leading indicator, as it doesn’t give a clue of what to expect in the future.) It’s kinda ‘nice to have’ (or lagging indicator).
Then they slice the total cumulative number by gender (meh, who cares), and count of nationals vs foreigners (yikes! What happened to tracking the origin, rather than the nation?), which is an easy way to fuel nationalism and xenophobia.
Finally, the reports move into active vs recovered numbers, regions, and age brackets. And then the death toll.
Four months into this thing, is this the best we can muster? And even if the government is parsing the data this way, could anyone try saying something different?
We went from high praises of “almost no cases in Costa Rica” (only for El Salvador to call us out for underreporting cases) to hyping up the scare tactics once the ‘Tico Triumph’ became an unsustainable image.
Such a limited set of data, measured differently across different countries, gives too little to work with in the first place.
Yet we have dead set opinions and all kinds of outrageous conclusions on the matter. I insist that the insight can only get better if we ask what’s missing. What are we not seeing?
Common sense tells me there’s a whole lot we’re not seeing.
Is the underside of this iceberg three, five, or ten times bigger than what we currently see? We don’t know yet. We may never know.
Recent studies of serologic (antibody) testing show that COVID-19 antibodies can disappear after two or three months. And with current testing rates that sample a tiny part of the population, we’ll never be able to go back and test it properly. This means we need to take everything with a grain of salt and question each metric with scrutiny.
— Medscape (@Medscape) June 23, 2020
Monitoring the outbreak, or beautifying the metrics.
Some countries that seem to know what they’re doing focus on the number of current hospitalisations (non-critical and critical), count of active cases, and testing rates FIRST.
Those are not vanity metrics, but actionable numbers that offer a better understanding of the current situation. What does the cumulative number of 26,331 tested people in Costa Rica (as today) mean? Not much other than just over 0.5% of the Costa Rica population got at least one test in the last four months.
How many tests do they execute per day? Same as a week ago or more? Could the change in the testing rate have had an impact on the increased case loads we’re now seeing?
The test rate has dropped to 6.29 whereas the minimum recommended is between 10 to 30 tests per found case, according to the WHO. So perhaps the daily test count should have increased, but earlier and in higher numbers?
Again, they should explain test spikes, not just lump them together into a cumulative number that doesn’t say anything.
What about track, trace, and test strategy?
The Costa Rican government has introduced effective measures to ease the population into accepting the new norms. Removing license plates, closing down businesses reported by friendly neighbors, and now mandatory mask-wearing rules. (To be clear, I welcome and respect them all. Except for maybe the snitching part.)
Face Masks to be Mandatory in Public in Costa Rica from June 27 https://t.co/hzqtFSaxGC
— CentralAmericaLiving (@VidaAmerica) June 22, 2020
Still, where is the same effectiveness in obtaining mass tests, or fully-developed tracing strategies (mobile apps anyone?), that would help keep the virus numbers low?
Because let’s face it, the virus is here to stay. While measures like physical distancing, wearing masks and reducing social interactions skirt the infection rates, COVID-19 won’t disappear any time soon.
The challenge for the government is to ensure that COVID-19 doesn’t overburden the healthcare system, not to completely eradicate it.
Is Costa Rica testing enough? Where are the goals?
Perhaps there are some goals I’m unaware of. But that means they are not clearly communicated. And no, “We’ll take it slow” is not a goal.
Let’s look at the UK as an example. Their goal (yes, a goal!) by the end of May was to have 200,000 tests executed on a daily basis. They are currently testing 1.01 per 1,000 (rolling seven-day average).
In Costa Rica, the current test rate is 0.07. The recent increase of tests from daily average of ~250 to 600+ is encouraging, but Costa Rica trends on the very bottom of the charts like daily test rate, or the adjusted test rate relative to the size of the population.
I know Costa Rica isn’t an advanced industrial economy like the UK, but does it have to be so far behind?
While Costa Rica is in fact, on top of tracing all cases, and is doing a good job of it, testing is only open to symptomatic cases and key groups. They’re leaving behind anyone with light symptoms and testing of the public.
And that’s something that will leave Costa Rica behind.
Some examples to follow…
One of South Korea’s key indicators is that the bed occupancy rate of its hospitals does not exceed 70%.
Where is Costa Rica’s intention on that matter? With a total of 30 hospitalizations counted yesterday, the ‘bed count’ comparison is fuzzy to me. It’s not reported on in the daily briefings. I’d still assume far below 70%. Do I know for sure? No. Is 70% something Costa Rica is pursuing or less? Or more? I don’t know. And therein lies the rub. And I’m not the only one. Yes, you might have mentioned it once ages ago. No, no one remembers that anymore.
Another example of clear goals/guidelines (vs spewing numbers with no reference points) are these key indicators tracked in the San Francisco Bay Area:
- The number of new cases identified per day is flat or decreasing
- The number of hospitalized patients with COVID-19, across all hospitals, is flat or decreasing for 14 consecutive days
- For at least a week (seven-day rolling average), no more than 50% of patients in staffable non-surge hospital beds are COVID-19 positive.
- At least 200 COVID-19 viral detection (PCR) tests are being conducted per 100,000 residents per day.
When reporting against accountable goals, it’s easy to see where one is underperforming or reaching acceptable rates.
To recap. Reporting the numbers for pure fear mongering has passed its solstice.
Nobody wants to see another fiasco like last Friday when they rescinded Phase Three of opening at the last minute. It causes confusion, frustration, and even mass hysteria.
The best way forward is always up for debate and something to improve on. Taking notes from the rest of the world, three key points come to mind:
- Clearly define realistic indicators (e.g. flatline of new cases) and report on it. Report on how they correlate with testing practices. No need for cumulative charts. They don’t help understand the current trends!
- Define hospitalization rate, and report on it. That will help everyone understand how far off the mark we are.
- Create a user-friendly report that shows current and relevant information in an easily digestible way. How Iceland displays its stats is a sight for sore eyes:
Paralyzing the entire country for several weeks and even two months was a necessary evil.
And, I must say, timely executed, before Costa Rica even had a chance to experience any kind of ‘wave’.
I won’t go into what is right or wrong for the local economy as I’m no expert on the matter. I will say something about the travel industry I work in, though.
Having no clear answers on the strategy of opening up the country—the fourth month into the lockdown—doesn’t give anyone confidence that Costa Rica knows how it would deal with even a small outbreak.
“We’re not ready yet”, sure. That’s understandable.
But it needs following with, “our plan is __________ and___________.”
Government officials/erstwhile rulers, please fill in the blanks!
Oh, and stay true to your numbers.
That will help us move forward in the right direction.
“We haven’t heard the government talk about reactivating our beach economies. There’s no rescue plan. We’re abandoned, and if the beaches don’t open we’ll starve.”#DespiertaCostaRica #CostaRicahttps://t.co/WIlSrzw15k
— CentralAmericaLiving (@VidaAmerica) June 24, 2020
Beyond reporting, the promise of travel.
We’re still far from finding a silver bullet for the pandemic. But we needn’t be so far from finding the right path back into semi-normalcy.
I won’t pretend I’m not looking forward to a moment where Iceland is finding themselves right now. As they take control of virus infections, they’re implementing a simple yet very effective strategy to draw in visitors again.
Dependent largely on tourism, Iceland’s tour operators are offering a promotion better than any seasonal discount – free coronavirus tests for visitors. What a great way to reaffirm confidence.
— TravelDailyNews (@TravelDailyNews) June 22, 2020
Costa Rica, start taking notes.
Jelena Radic is a marketing professional who lives in Escazu, Costa Rica.