How to manage a drug-resistant pandemic? (2) Estimating the burden of disease at a national level…

In the previous blog we discussed the ways that the WHO generally use to estimate the proportion of the global TB pandemic that's drug-resistant, concluding that the method seems to be fundamentally flawed and that it's almost certainly dangerously under-estimating the threat.

 In contrast, in this blog we take a look at one particular country – at South Africa – and we examine  how the estimates for MDR-TB and the notified numbers are being incoherently represented and reported for it. We then explore some of the reasons behind these variations. Some of them relate directly to the previous Blog (How to manage a drug-resistant pandemic? [1]);  but mostly they seem to expose fresh issues of concern.

In the most recent WHO Global TB report (published late in 2015) the WHO’s estimate of the burden of MDR-TB for this rainbow nation was 6,200 incident cases (comprising 4,700 new and 1,500 re-treatment cases). This can be seen in the upper ringed text in the graphic below which is reproduced from page 141 of the WHO’s most recent Report. It’s very important to bear in mind that this number is the one that’s used to help build the estimate of MDR-TB for the African region, and also, therefore, for the whole world.

So far so good, but with only a minute’s further review of this same graphic we can spot an embarrassingly large South African elephant in a roomin Geneva because we can see (in the lower ringed text) that the number of actually reported laboratory confirmed cases of MDR-TB for the same year was 18,734 – or pretty much three times the published estimated number. Unless expert opinion considers the country’s laboratory-confirmed bacteriological data to be gobbledygook (which it doesn’t), this makes the WHO’s estimate of MDR burden for South Africa a de facto statistical impossibility.

Of course any proper estimate of incidence for a disease like TB has to always include the number of cases that are believed to be being missed by the national TB program (which globally was a worrying 37% in the last Report, but for South Africa it was a bit less at 32%). In any imaginable scenario, of course, this total must always be higher than the number of reported cases, but in this instance it’s not – it’s a lot less.

Might we have got this wrong? Well we know that we haven’t because we tried to resolve this anomaly with two WHO experts in Geneva and also two experts in South Africa. All of them were helpful, but nothing got resolved and our concerns were actually left considerably more acute at the end of their efforts.  

The two South African experts explained that enormous efforts have been made to improve general epidemiological intelligence regarding to their national epidemic. And they reckoned that about about 18,000 (or between 15,000 and 20,000) incident reported MDR cases is "close to the truth" for the country. Furthermore they didn't think that many MDR-TB cases who were attending for treatment were being missed (largely because of the roll-out of GeneXpert diagnostic device). And as a result we agree that the case detection rate for MDR in this country can be quite reasonably assumed to be similar to that for all-TB (i.e. 68% which is far higher than the 26% which is the optimistic global average).

So we tried running this same estimate using the principles described in Method I as described in the previous blog, using the notified number of cases and then inflating it by using a case detection rate that (in this case) we’re pretty confident with. Our calculation runs as follows: 18,734 divided by 68 multiplied by 100. This gave us 27,750 which is certainly a lot more than the WHO's estimate of just 6,200 - in fact it's four-and-a-half times more.

 What initially alerted us to all of this were these curious percentages of new and re-treatment MDR cases that have year-on-year been being entered for South Africa in the WHO Reports. Thestwo percentages have for the past years always been 1.8% of all cases for new TB cases and 6.7% of retreatment ones (which you can also see in that graphic above in the upper ringed text). We noted that these percentages are a lot less than the most recent global percentages which were 3.3% and 20% respectively and that this would suggest, therefore, that the rate of new MDR cases occurring in South Africa is around half the global average, and the rate of retreatment cases a third of it. Yet South Africa is not only recognised as a high burden MDR-TB country by the WHO itself, it’s also widely recognised to have an MDR-TB problem that is out of control. As such these consistently reported percentages just don’t add upfor us at all.

With the help of the WHO experts we managed to establish that these percentage estimates actually relate to a survey that was carried out in 2002 so are obviously way out of date. This may well be the explanation, but we still don’t understand why they haven’t been adjusted by a single point in the thirteen years since. Not just this, but we are appalled that they could still be used to calculate the numbers which are then fed into both regional and global estimates of MDR-TB, despite their being fundamentally meaningless to the current state of the pandemic and so misleading. To put it bluntly they render any resulting estimate of the regional and global MDR pandemic illegitimate.

A couple of simple sums can prove that these percentages have indeed been used, incidentally. The total estimated number of new all-TB cases in South Africa for 2014 (the year reported on in the 2015 Report) was 261,955, and 1.8% of cases gives us one the numbers that appear in the graphic above (4,700). Meanwhile the total estimated number of relapsed or previously treated was 22,716, and 6.7% of this number also gives us the second number entered above (1,500). So we can confirm that the estimated percentages for 2002 have been used to misleadingly calculate the MDR estimates for this country in 2015 and in every other year since 2002.

As mentioned above, we’ve not been given any acceptable explanation as to why these percentages from the 2002 survey haven’t ever been adjusted by expert opinion. We have learnt, however, that there was supposed to have been another drug-resistant survey conducted in the meantime (though we don’t know why it never happened). But we’ve also learnt that the results of a new and long-overdue survey are nowimminent. From the existing correspondence, however, we understand that the experts in Geneva aren’t expecting the existing percentages to change much in the new survey. It’s not at all clear how this can possibly be the case, but it’s what we’ve been told nevertheless.

But let’s go back to the actual numbers for a minute. If we accept that the South African 18,734 MDR laboratory notifications are correct (and if we assume most of them to be pulmonary which we think is probable), we can compute this number as a percentage of the current Report's 261,955 notified new pulmonary cases, and then we can calculate that the overall percentage of TB in the country is MDR may be 7%. This is higher than the overall global average which is 5.5%, and it’s a lot nearer what we’d expect it to be.  

We appreciate that there’s a lot of detail here and a also lot of tentative playing around with the numbers. Some of it may be difficult to appreciate in a single read, but we want any reader to understand that all we're doing is trying to apply some logic to a quite illogical report. We're sure that submitting illogical and misleading numbers into regional and global estimates isn't be any way of monitoring and responding to a drug-resistant pandemic which also includes XDR-TB – what Mario Raviglione of the WHO himself calls the “worst thing we could have imagined”. The idea that these percentages can be so casually misrepresented actually appals us, not least because of what it implies for other neighbouring countries given known migratory patterns and their comparative lack of resources in comparison to South Africa.

Such concerns are even more worrying when it’s appreciated how acutely aware of this risk the WHO was in earlier years. Back in 2009 several top TB experts in the WHO presented a paper at the Union Conference in Mexico. They reckoned that 75,000 new MDR cases were probably occurring that year in the African region (i.e. more than twice what they reckoned for the region last year!) – and bear in mind this was six years ago. This wasn’t even a one-off: the year before that (in a specialist WHO Report on MDR-TB) the number of new African cases was reckoned at 69,000, so it would appear that six years ago the WHO was comfortable in suggesting that the incidence of African MDR might have increased by 9% in a single year. Since then, though, the picture that’s been presented hasn’t just been flatlining (which is what’s happened for the global picture) - for Africa it appears to have actually collapsed! The current Report estimated that only 32,000 new African MDR cases emerged in the last reported year (albeit pulmonary-only) – in other words it seems to be being belived that there’s been a reduction in African MDR-TB of over 50% in five years. Can this possibly be the case given everything we know about MDR-TB? Of course it can’t.

But here’s one further calculation.

The recent Report also recorded 25,531 MDR cases being notified in the Africa region (18,734 of whom we already know were South African), which leaves less than 7,000 for the whole of the rest of the region. Of course this is nonsense as well, but it’s all in there in the reported numbers and it’s there partly because of those thirteen year-old South African percentages, of course! 

But then we know we’re all only pretending at this anyway because the surveillance is so poor. So let's finally pretend for a minute in this way: if the WHO’s 75,000 estimate of 2009 was right and the regional epidemic has been rising at 9% a year the total number of new cases of TB that are MDR in Africa can’t possibly be 32,000; it could easily be over 100,000...

 This might not be such nonsense as it first sounds because that WHO 2008 specialist MDR Report made a very serious statement indeed. It suggested that the incident rates of MDR-TB in “some southern African countries” might have then been “5-6 times higher than that of China or India”. This seems to us to be a sentence which was deliberately worded to alert the world to a drug-resistant disaster that the organisation realised threatened the region. Currently, however, no country in the African region is being reported to have percentages of MDR-TB that are even equal to the global average let alone above it. Something seems to have changed in the last six years.

But is it in Africa that something might have changed wthout being properly reported, or is it in Geneva?