Guest Post: “India’s misguided response to our findings on the quality of drug exports”

We recently wrote a two part piece on an NBER paper that has been garnering quite a bit of attention for two reasons. First, because the authors conclude that India’s pharmaceutical companies may be discriminating against African nations in their exports; and secondly, because the Indian government has indicated they may file a defamation action against the authors for this paper. In covering these developments, first I presented some of my critiques of the paper; and then, Gautam Bhatia wrote about the unlikelihood of any defamation action holding up against the authors. We had also invited the authors to respond if they wished and we’re glad to say that they have sent across a response for us to publish. Without further ado:

India’s misguided response to our findings on the quality of drug exports

Roger Bate, Ginger Zhe Jin, Aparna Mathur, Amir Attaran
October 10, 2014

A recent National Bureau of Economic Research working paper that we co-authored on the quality of Indian medicines exported to Africa has drawn surprising over-reaction from the Indian government and pharmaceutical industry. In a recent article in the Economic Times, officials said that India’s government is planning a defamation suit against our research team for running a “smear campaign” against the industry. The government’s threat of legal action is a shocking development, and if it proceeds, will hurt India more than it will hurt us.

Our research team is composed of four highly trained PhDs in health sciences, economics and law. All of us are professors or scholars at universities or think tanks in Canada and the United States. Everything about this study—the collection of data, the analysis, and the write-up and publication—was done in a rigorously academic fashion. The paper is a working paper, which means it has not yet been through extensive peer review and is disseminated for the purpose of idea sharing, discussion, comments, and suggestions. But the vast majority of the quality data have already appeared previously in some of the top medical, scientific and socio-economic journals after extensive peer review.* What is new in this paper is that we track manufacturer identity as labeled on the package and for the first time compare drug quality of the same manufacturer across different purchase countries.

In order to prevent anyone from making further sweeping generalizations regarding the study, we think it bears repeating the main findings from our study. For the paper, we examined 1,470 samples of antibiotics and tuberculosis medicines that claim to be made in India but were purchased from India, Africa and other mid-income countries. We then tested these drugs for quality using simple physical and chemical methods. From the overall sample, 10.9% failed basic quality tests, which means they had less than 80% of the standard amount of active ingredient (API). Drugs that fail the quality test are classified as either substandard or falsified: substandard drugs contain some positive active ingredient (but less than 80%), while falsified drugs have no active ingredient at all.

We find that a significantly higher fraction of these “made-in-India” drugs are substandard or falsified if they were purchased in Africa than if they were purchased in India or non-African middle income countries such as China, Brazil, Turkey, Thailand and Russia. We find geography best explains this disparity in drug quality. Other variables such as GDP per capita, adult literacy rates, price regulations, criminal penalties for counterfeiting, the rule of law, or intellectual property protections have limited power explaining this finding.

But our most interesting result has to do with medicines that are sold illegally in Africa because they are not registered with the government for sale. Overall, the passing rates of registered drugs were similar across Africa, India, and non-Africa, averaging more than 90%. However, among unregistered products, the passing rate was 49.4% in Africa and 67.8% in India. The presence of Indian drugs in the unregistered market is worrying enough, but our data suggest that these medicines are especially poor quality and could be dangerous.

Based on these data, we consider a few possible explanations in the paper. After articulating our assumptions, we argue that the prevalence of poor quality drugs in Africa is likely driven by Indian firms – whether manufacturers or exporters – that do not bother to register products in the African countries where they are sold or consumed. This is a serious failure that both India’s government and industry should investigate and resolve. Due diligence, obviously, never allows a medicine to be sold where it is not registered—and yet this is happening all too often with “made-in-India” medicines.

Due to limited resources, ours is a pilot study with clear limitations. Our effort in sample collection, going back to as early as 2007, aimed at assessing problems with products in various markets. We never targeted any manufacturing nation or any individual companies in the sampling design. After speaking with Ranbaxy whistleblower Dinesh Thakur about practices at some Indian companies, we decided to use existing data to investigate whether the same products performed worse in different locations – the NBER working paper is the result.

We started working on this paper months before the Indian election and aimed to circulate it as a working paper to coincide with a Capitol Hill event we co-organized in September 2014, featuring the heads of various donor/drug buying agencies (GFATM, PMI, OGAC). The paper release was not intentionally timed with Prime Minister Modi’s visit to the US.

Because our data are limited in geography, drug type and sample size, we aim for statistical regularities rather than criminal investigation. The problem of poor-quality drugs is real, but the international community has done too little to measure this problem. We hope our pilot will drive more detailed and extensive studies on this topic, identify reasons behind the problem, and move towards an effective solution. Litigation (or merely its threat) will undermine that effort.

We welcome any legitimate criticism, as exemplified by the recent blog of Mr. Swaraj Paul Barooah on spicyip.com. Mr. Barooah is correct that our data coverage on all three purchase-country groups – Africa, India-domestic and Non-Africa – is limited to ciprofloxacin only. For the other three drugs, the comparison focuses on Africa versus India domestic.  Our sample size on ciprofloxacin is greater than other products, partly because the initial aim of data collection placed more emphasis on ciprofloxacin, partly because there is more restricted access (to TB meds) and some products were not available in markets where we expected them to be. In the current regression analysis, we control for manufacturer-drug fixed effects to ensure that our comparison of drug quality within a manufacturer is also within the same drug type. With these controls, the difference between Africa and India-domestic  (in the regression results) is identified from all drug types, while the difference between non-Africa and India-domestic is identified from ciprofloxacin only. In a robustness check, we repeated the analysis for the subsample of ciprofloxacin only. Our key results are robust to this sample change. We also repeated the exercise for the subsample of non-ciprofloxacin only, which restricts the comparison to Africa vs. India domestic. Despite this limitation, the message is the same: the overall passing rate of non-ciprofloxacin is significantly worse in Africa than in India domestic, partly driven by unregistered drugs that turned out to be substandard. Both robustness checks will be presented in our next revision of the paper.

Another limitation of our study lies in our sample size. No doubt, there are variations across drugs and different types of failures. Our statement of “drugs purchased from Africa are more likely to fail the TLC test than the same type of drugs in the Indian domestic group” (page 7) refers to the total failure rate, not just the falsification rate. If one wants to focus on a particular drug and a particular type of failure (say falsification of Rifampicin), the statistical comparison should account for the sample size of each cell. With only 167 Rifampicin samples from India Domestic and 103 from Africa, the 2.4% vs. 1.9% difference in falsification rate is not statistically significant. More importantly, because we are after statistical regularities, we believe regression results with a large sample size and explicit control of manufacturer-drug fixed effects are more informative than simple bean counts in a small subsample.

Our critics in India have alleged that our study findings should not be taken seriously because we did not take into account the possible degradation of medicines.  That is a desperate and ironic objection coming from India’s pharmaceutical industry and India’s government, because when they last worked together to survey drug quality in India, they did not test for degradation either (see the report here). Instead, like ourselves, they took the view that a medicine within its expiry date should be stable enough that it would pass quality tests, even in the ambient conditions of a tropical country.

That said, the ambient conditions of India are actually worse for medicines than Africa.  The four drug types in our sample, all tablets, require storage at room temperature below 25°C or 30°C. If we count the number of months in a year that have an average high temperature above 25°C or 30°C, this number is actually higher in the Indian cities than in the African and non-African cities of our sample.  Thus medicines in India should be more degraded, and therefore lower quality, than medicines in Africa.  But we find just the opposite, suggesting that degradation is not the cause of our findings.

Further, our previous paper (Bate, Jin and Mathur 2014) also found that the innovator-brand of ciprofloxacin claiming to be made in the United States and Western Europe never failed quality tests as substandard, whether purchased in Africa or elsewhere.  If degradation were the driver of our results, these medicines should have failed just like the ones claiming to be made in India.

All this strongly suggests that something other than degradation is the cause of poor drug quality in our tests.  Cheating, or more precisely, negligence and fraud, on the part of Indian medicine suppliers is most likely explanation, which deserves further study.

The Indian pharmaceutical industry has also questioned the timing of our quality tests and criticized us for not contacting manufacturers before making the study public.  All tests were conducted within 60 days after purchase in the Africa Fighting Malarial Minilab in Cambridge, UK (the lab was closed in 2013 due to lack of funding). We generally contacted companies about problems with their products. This effort was most significant in 2009/2010. When it comes to Indian companies the vast majority never replied. Those that did said that since we were not regulators or clients they would provide no information. We had only useful information from one Indian company who confirmed to us a fake of its ciprofloxacin product sold in Nigeria had incorrect batch numbering for the manufacturing date and no API. The dilemma we face as independent researchers highlights the importance of involving regulators in the research itself, not a lawsuit against researchers.

Our work is a pilot study. Its conclusions may not repeat in larger, more detailed studies, which we welcome. However, we do believe we have made a reasonable interpretation of our data because none of our results are surprising. Anyone familiar with the pharmaceutical industry in India or Africa knows that a large number of Indian medicines are traded through poorly regulated channels where both compliance with the law and quality standards can be easily evaded. Our point is that if India wants to be regarded as a serious and ethical drug exporting country—does it?—then this has to stop. Yet for saying the obvious, India’s government is now threatening to sue us.

It is quite common for academics to publish studies that governments dislike, but not common for governments to sue them for it. However, as a recent paper in Foreign Policy noted, the Bharatiya Janata Party (BJP) and the Modi government have a nasty habit of suing and criminally prosecuting their opponents. Not even North Korea or Russia have ever taken the illiberal and foolish step of suing scholars for publishing academic research, but India is poised to become the first government to do so. Is that really how India wants to be viewed in the world: as illiberal, hypersensitive, and more concerned about shooting the messenger than responding to our message to improve drug quality and save lives?

Notes: *For example, see Bate R, Jensen P, Hess K, Mooney L & Milligan J. 2013. Substandard and Falsified Anti-Tuberculosis Drugs: A Preliminary Field Analysis. International Journal of Tuberculosis and Lung Disease. Bate R, Mooney L, Hess K, Milligan J & Attaran A. 2012. Anti-Infective Medicine Quality: Analysis of Basic Product Quality by Approval Status and Country of Manufacture. Research and Reports in Tropical Medicine.  Bate R, Jin G.Z. & Mathur A. 2011. Does Price Reveal Poor-Quality Drugs? Evidence from 17 Countries. Journal of Health Economics. Bate R, Jin G.Z. & Mathur A. 2014. Falsified or Substandard? Assessing Price and Non-Price Signals of Drug Quality, forthcoming the Journal of Economics & Management Strategy.

Prof. Ginger Jin is at the University of Maryland, Prof. Amir Attaran is at the University of Ottawa and Dr. Aparna Mathur and Dr. Roger Bate are scholars at the American Enterprise Institute.