Trouble Brewing around NBER Paper on Indian Pharma – Part I

We’ve recently been informed of two events, one flowing from the other, both of which I would term as very disturbing in their own way.

First, is this NBER Working Paper that concludes that it is likely that Indian pharmaceutical firms differentiate drug quality based on destination of consumption, to the detriment of African countries.
Secondly, is the follow up development that the Indian government may be taking legal action against the authors of the above paper. Needless to say – both of these developments call for urgent attention.

In this post, I shall be dealing with the first of these developments: the paper itself. In a follow up post, (available here) Gautam Bhatia, a free speech scholar will be examining the second of these developments. Since I won’t be discussing that second issue, I’d like to state up front – regardless of the content and quality of this paper – I think it is an extremely distressing state of affairs if academia needs to start worrying about defamation (or any other) law suits. Exponentially more so if the action comes from the government!! – Not to mention, I’m not sure how the government can claim defamation when products of (unnamed) companies are supposedly being defamed. Anyhow, I trust Gautam will discuss that issue in suitable depth. Moving on to the first issue: I will first give the basic thrust of the paper, followed by my own observations and comments, followed by the reaction from Indian pharmaceutical companies. Relevant links are found at the end of the page. Long post ahead.

Basic info on the paper: 

The paper titled “Poor Quality Drugs and Global Trade: A Pilot Study” was published in September 2014, and authored by 1. Roger Bate (of the thinktank ‘American Enterprise Institute’), 2. Ginger Zhe Jin (of University of Maryland, Dept of Econ), 3. Aparna Mathur (also of ‘American Enterprise Institute’), and 4. Amir Attaran (of University of Ottawa, Faculty of Law).

As the paper states, they examined 1470 samples of antiobiotics that claim to be made in India, distributed across 22 cities spread over 18 low to middle income countries across the world. The samples included two broad-spectrum antibiotics (ciprofloxacin and erythromycin) and two first line antibiotics for tuberculosis (isoniazad and rifampicin).

They found that 3.9% of samples contain no Active Pharmaceutical Ingredient (API), and 7% of samples contain less than 80% API. They also state that these bad samples tend to be unregistered products that are disproportionately found in the African cities they’ve sampled, and that it is likely that Indian manufacturers intentionally send bad samples to African countries as it is cost effective.

Analysis and comments: 

Given the nature of this claim, it is natural that it be taken very seriously. And to be perfectly frank, I wouldn’t find it surprising that not only Indian companies, but that all pharmaceutical companies try to cut costs by exploiting weaker regulatory regimes. For that reason alone, I commend the taking up of this study. That being said, I’ve only had a chance to take a preliminary look at their paper and have noticed some problematic parts. Given that this is a ‘working’ paper, I hope that these gaps are taken note of and addressed.

1. Country selection & grouping bias?

The paper bases much of its conclusions on ‘comparing’ distributions between different groups of countries. It categorises them into three groups: India, African countries (12 of them) and Non-African countries (5 of them). It doesn’t however explain the selection of these countries. As the study itself notes, there is a high correlation between GDP and ‘bad samples’. Therefore its worth noting (see Table 4 of their paper) that the “Non-African group” consists of 5 countries, all with 5-digit city GDP per capita, while all the other countries in the list, with the exception of Egypt, have a 4-digit city GDP per capita (or less). Therefore, first off, this doesn’t seem to be a comparison of equals.  The study notes that outside factors have been controlled for – but given the very small sample size for non-African countries, I don’t know how useful this ‘control’ is. But regardless of that –  simply the selection and grouping of countries, I think, leaves a lot to be desired. For example, I would be very interested in seeing whether Egypt (being the only African country with a GDP comparable to the Non-African countries), sees similar rates of bad samples as the rest of Africa. Or how much DRC (as the only country with a 3-digit city GDP per capita) brings down the average for African countries. Or if non-African poor countries such as Nepal, Bangladesh, Afghanistan, Haiti, etc also get similarly bad samples. Currently, simply stating a discrimination against “African” countries seems a tad bit more sensational grouping, rather than useful information. [It goes without saying that low GDP or not, no manufacturer ought to be intentionally sending low quality drugs anywhere].

2. Sampling bias?

I’ve drawn up a chart of the samples they’ve taken based on their data provided in the tables at the end of the paper: (Fake = 0% API; Substandard = < 80% API)

Samples	India	Africa	Non-Africa	Total
All	956	430	84	1470
Ciprofloxacin	456	151	84	691
(Fake) (SubStandard)	(4.8%) (3.3%)	(3.3%) (8.6%)	(4.8%) (0.0%)	(4.5%)(4.1%)
Erythromycin	167	119	0	286
(Fake) (SubStandard)	(1.8%) (10.8%)	(7.6%) (11.8%)	0	(4.2%)(11.2%)
Isoniazad	166	57	0	223
(Fake) (SubStandard)	(1.8%) (4.8%)	(8.8%)(7.0%)	0	(3.6%)(5.4%)
Rifampicin	167	103	0	270
(Fake) (SubStandard)	(2.4%) (7.8%)	(1.9%) (17.5%)	0	(2.2%)(11.5%)
Total	(3.3%)(5.6%)	(4.9%)(11.4%)	(4.8%) (0.0%)	(3.9%)(7.0%)

The first thing one notices is that there were no samples taken of 3 of the 4 drugs, from Non-African countries! It’s worth noting that this breakdown of samples is not mentioned in the main text of the paper; it is only found in the tables after the end notes. The only reason I found mentioned for differences in quantities of samples from different countries was this line: “Because drug availability varies across purchase countries (in part because the targeted disease varies)”.

A few quick points jump out:

1) that this once again sharply demonstrates the problem with comparing these countries – since clearly more than 50% of the drugs compared aren’t even required/weren’t sampled in one of the two export groups – yet the claim of being discriminated against, is being made for the sole export group.

2) that their given reason doesn’t actually explain why samples of the other broad-spectrum antibiotic Erythromycin weren’t taken in Non-African countries. [Admittedly, I am making an assumption that where one broad-spectrum Indian manufactured antibiotic is available, the other one also should be.

3) In the only comparison where all three groups are covered, African countries actually have the least percentage of fake drugs.

4) In two out of the four drugs, India has a higher percentage of fake drugs than do the African countries. However, in a statement on page 7, they say: “However, drugs purchased from Africa are more likely to fail the TLC test than the same type of drugs in the Indian domestic group.”

This statement would only be true if patients were willing to substitute one drug with another (which obviously doesn’t apply here), or if more types of drugs were likely to be fake in Africa, than in India. However, as their own data shows, 2 types are more likely to be fake in Africa, while the other 2 types are more likely to be fake in India. And clearly a sample of only 4 types of drugs isn’t a big enough one to be making broader generalizations on.

5) With regard to drug quality, they also state, “In comparison, drugs from Non-Africa have a greater passing rate than the Indian domestic ones within the same drug type.”  While technically true – this is misleading, as only one type of drug has actually been compared between Non-Africa and Indian domestic – and even in this one drug – there was an equal percentage (4.8%) of failed drugs in both.

Only in the substandard test, did Indian domestic (3.3%) have a worse rate than Non-Africa (0%). This is also to be kept in context of 456 samples being taken from India, while only 84 samples were taken from across 5 countries for Non-Africa.

6) Of the two drugs that India has more fake drugs than Africa, it is curious to note that one drug has a ridiculously high percentage of sub-standard availability in African countries (Rifampcin: 17.5%). As I’ve often anecdotally heard that TB medicines often get degraded in storage, I decided to do a preliminary check and found that Sanofi’s rifampcin capsules, tablets as well as infusion solutions are required to be kept under 25C, and away from light and moisture. For comparison with another drug – Bayer’s ciprofloxacin tablets don’t require any special storage conditions. Thus – I have to ask – Has this properly been controlled for, in making the determination that the drug was not degraded due to storage and handling? I am presuming most of Africa does not remain at 25 degrees C. Therefore questions of storage and handling must be properly looked into.

[Edit: Madhulika has brought some studies to my attention, on when Rifampcin may degrade in quality. They have been mentioned in the comments below]

There were a few more minor (presumably non-consequential) discrepancies I came across, such as the authors making a counting mistake in stating that 9 Indian pharmaceutical companies exported more than 75% of their net sales, while I could count only 8 in the chart they referred to. (footnote 4). Similarly, they state that they used Ciprofloxacin data from 3 of their previous papers – But when I took a look at the latest one of these papers, I found that there they had found 683 samples from Indian manufacturers, while this latest study has 691 samples. This latest addition of 8 samples isn’t explained.

Indian Industry Reaction

As this post has already gone on for quite a length, I will only briefly mention the main points of issue that the Indian industry have brought up. I will also upload their statements for our readers to view it on their own.

The Indian industry has chosen to respond via the India Brand Equity Foundation (IBEF) “on behalf of the Government of India”. IBEF is a Trust established by the Ministry of Commerce and Industry, Government of India. [Comment: This is curious on its own – that the government is choosing to respond to a working paper that discusses 4 drugs made by private manufacturers based in India.]

Calling it a smear campaign, the IBEF has raised several points of contention, including:

1. The NBER Paper has not been peer-reviewed, nor been subject to a review by the NBER Board – yet it is using NBER’s well established name to gain credibility. 

2. The methodology and treatment of samples make the conclusions suspect. Degradation by transportation and storage conditions, though mentioned, haven’t adequately been discussed. Similarly, counterfeit products (made elsewhere but with a “Made in India” label) have not adequately been accounted for. 

3. The samples were collected over 3 years but no mention of where and when the testing was done, and why it took 2 years to write this report. [Comment: It is true that the report is only 25 pages, of which 16 are text. However, there can be any number of reasons for taking 2 years to write this report – I’m not inclined to see this as ‘necessary’ to explain. However, I believe this 2 year question is brought up more as it makes it difficult/impossible to cross-check the samples after so long.]

4. The authors should have given the Indian manufacturers a chance to comment on the test observations. Pharmexcil’s Director General, Dr P.V. Appaji states that there are at least 8 technical issues with the sampling and data techniques.

5. Due to the above reasons, the report has no value and is written for the purpose of maligning India’s image in the international market.

6. Realising the serious damage that such an irresponsible report is trying to make, the pharma industry has asked the Government of India to initiate legal proceedings against all concerned.

Conclusion

As the one of the largest suppliers of generics, and the oft touted tag “pharmacy of the developing world”, it is certainly terrific that Indian manufacturers are being scrutinized.

And if one were to assume the paper’s conclusion as a valid one, this is naturally a very disturbing thought – that Indian manufacturers are intentionally exploiting poorer countries with weak regulatory oversight. And quite frankly, given anecdotal evidence, I personally would not be surprised by this from any manufacturer, not just Indian. And given the volume and necessity of Indian generics, quality should always be maintained as a top-priority. After all, the majority of the sick – the poor of the world – are not going to be able to afford the large branded drug prices.

However, anecdotes aside, real world studies must be undertaken with appropriate rigour and from my admittedly limited perspective, I believe this present study has certain important gaps it is required to fill, in order to be taken seriously. In the given context of India’s pharma regime being constantly scrutinized and attacked, unfairly or inadequately reporting on Indian manufacturers could be devastating on patient populations everywhere. [Indian Pharmaceutical Association has also listed some comments on the paper. Linked below]

All that being said – I see absolutely no reason for the Government of India to institute legal proceedings against the authors / publishers of the paper! Even if one were to assume that the authors were out to malign India – The only proper response is to appropriately deconstruct and de-credit that paper. It is an absolutely terrible signal to send out – that the Government of India is willing to sue academics! This would certainly have a chilling effect on any critical study done on the pharma industry – something that NO sector should be free from, especially not one as crucial as the pharma industry. And even strategically, this would only serve to further allegations of the Indian Government protecting its domestic manufacturers – for that’s exactly what they would be doing here. And giving any strength to that allegation would be devastating for India’s pharma patent regime.

Readers can look forward to a further examination of this legal threat in Part II – coming up soon. [Edit: Part II is now available here]

Relevant Links: 
1. Poor Quality Drugs and Global Trade: A Pilot Study (The paper in question)
2. IBEF Press release (The “official response”) 
3. IPA Comments on AEI study (Further comments by IPA)