One of the worst ever tobacco control papers is ‘corrected’

Papers by tobacco control academics are often truly awful – thinly disguised self-serving polemics masquerading as objective inquiry. But sometimes, the scientific paper and research conduct are so terrible that they become usefully revealing. This blog looks at a case in point.

The original terrible paper

This paper was published in November 2022.

Yassine, A., el Hage, R., El-Hellani, A., Salman, R., Talih, S., Eissenberg, T., Shihadeh, A., & Saliba, N. (2022). Did JUUL alter the content of menthol pods in response to US FDA flavour enforcement policy? Tobacco Control, 31(Suppl 3), s234–s237. [Web archive]

This paper was a formal version of research first presented at the SRNT conference in March 2022, “POD6-1: Alteration of Juul content in response to the US FDA flavor ban”.

The SRNT presentation (March) and then the formal paper (November) essentially accuse Juul, a leading vaping company, of changing the formulation of its products in response to FDA efforts to control the flavours available in the US market. The abstract of the original paper explains:

Results: Menthol concentration was significantly higher in 2020 products than in products from prior years. Moreover, other cooling agents varied across pods. The PG/VG volume ratio was 27/63 in all pods examined.

Conclusion: This study highlights how regulations intended to reduce e-cigarette prevalence among youth may influence changes in tobacco product characteristics in ways that regulators may not have foreseen.

Tobacco Control, 31(Suppl 3), s234–s237. [Web archive]

In February 2020, the US Food and Drug Administration issued an enforcement policy to remove all flavoured cartridge/pod-based e-cigarettes from the market except for tobacco and menthol. The allegation made by the authors is that Juul had manipulated its menthol products as a way to circumvent some of the negative commercial effects of this regulation.

A grave allegation with serious consequences

This amounts to a serious allegation. It is not just about marketing practices but about legal compliance and law-breaking. This is because for a vaping company to change the formulation of its products at the time would have been illegal and opened it up to enforcement action that could have removed some or all of its products from the market. Under the FDA’s enforcement regime, companies were permitted to keep products on the market that had been available on 8 August 2016 while they made a pre-market tobacco application (PMTA) to show their product was appropriate for the protection of public health. But they were not allowed to modify their products unless and until a PMTA for the modified product had been successful.

Not only would this mean the company was acting unlawfully and, therefore, open it up to enforcement action, it could also have invalidated the real-world data Juul was collecting to support its pre-market tobacco applications (PMTAs) for its products. That may have been enough to cause FDA to deny authorisation of its products and thereby destroy the company while destabilising the harm reduction gains made by Juul’s three million customers.

So it is not an accusation to make lightly.

An allegation made by FDA-funded researchers

The threat is further compounded by the funding source disclosed in the report. The regulator with the enforcement powers has paid for the research:

This research is supported by grant number U54DA036105 from the National Institute on Drug Abuse of the National Institutes of Health and the Center for Tobacco Products of the US Food and Drug Administration.

The first problem

In reality, Juul did not alter its formulation. It should always have been obvious that this would have been a reckless act, given the potential implications described above. Major companies with professional management and demanding investors do not just willfully break the law and put themselves at mortal risk as described above. The authors’ claims should have been met with incredulity from the outset.

In fact, the authors made the rookie error of mistaking a change of menthol content over time caused by the evaporation of the volatile menthol flavouring agent with a deliberate product modification, which would have been illegal under FDA’s rules. The authors should have known this or at least given greater attention to it as an explanation consistent with Juul’s compliance with the law. And even if the authors didn’t know, <draws breath> they should have asked the company before making wild and false allegations about its conduct. It is a basic rule of journalism, ethics, and common decency to put allegations of impropriety to those accused before going public.  

The second, much bigger problem

The authors knew. Juul staff had seen the presentation in March 2022 and written to the authors in April 2022 explaining that it had not changed the formulation of its products and that the likely reason for their observations was the loss of the menthol flavour agent over time through a well-established mechanism. Nevertheless, the authors went ahead with the publication of this article in November 2022. Perhaps with some concession to doubt by adding a question mark to the title, but the substantive claim was unchanged.

Response on PubPeer

Juul would be appropriately dismayed to see this article drop out of a clear blue sky in November with no mention of its communication to the authors in April and no sign the authors had addressed the concerns raised.

So Juul published a detailed response on PubPeer in November 2022 – see PubPeer comment. In this comment, Juul reproduces its April communication with the authors and summarises its concerns as follows:

A summary of that April 5 correspondence to authors is as follows:

1. Juul Labs has not altered or modified the e-liquid formulations subject to the authors’ analysis since these formulations were introduced into the market before August 2016 (i.e., FDA’s deeming date).
2. Juul Labs has supporting documentation, including batch records and certificates of analysis to confirm this.
3. Without knowing the study’s methodology, there are a number of possible explanations for the variations authors found. For example, one potential explanation for the differences in tested products is the loss of menthol over time. It is well-documented in scientific literature that menthol may migrate from areas of high concentration to low concentration, and therefore flavor levels may decrease over time.
4. I and others from Juul Labs would appreciate the opportunity to engage with authors to better understand their methodology and findings, and provide additional information relevant to the researched products.

I never received a reply to this correspondence from the publication’s authors. And many of the questions posed to them remain unanswered post-publication of the manuscript.

The authors did not bother responding to the critique on PubPeer. The journal, Tobacco Control, did not rapidly publish the ‘Rapid Response’ to the paper that Juul submitted in November shortly after the paper was published, setting out its position. It all just went quiet.

Correction in Tobacco Control

Nothing much happened until 27 April 2023, more than five months after Juul raised its concerns on PubPeer and via a Rapid Response.

A “correction” was then published in Tobacco Control.

Correction: Did JUUL alter the content of menthol pods in response to US FDA flavour enforcement policy? http://dx.doi.org/10.1136/tobaccocontrol-2022-057506corr1

The correction notifies readers of specific changes in the text made to the original. The original study has now been changed and republished with a new title and with most traces of an alleged scandal scrubbed away.  The original paper is no longer accessible other than via the web archive. The correction simply lists changes to fragments of text before and after correction. It provides no context or explanation for the changes.

The new publication thus emerges.

Yassine, A., el Hage, R., El-Hellani, A., Salman, R., Talih, S., Eissenberg, T., Shihadeh, A., & Saliba, N. (2022). Levels of menthol, nicotine and cooling agents measured in JUUL products purchased across a three-year period. Tobacco Control, 31(Suppl 3), s234–s237. https://doi.org/10.1136/TC-2022-057506

This pretty well voids all the exciting findings of the original and turns allegations of a scandal to be drawn to the attention of the FDA into a series of uninteresting banalities about flavours evaporating.

Other than the change of title, my favourite revision is the change to the conclusion:

Replaced text

Original: In conclusion, our findings suggest the possibility that the menthol content of JUUL pods changed in a manner that may have increased the appeal of these products when other flavoured pod- based products were no longer available. This work highlights the need to consider menthol and other natural and synthetic coolants in e- cigarettes as a possible target for regulation, as well as the need for regular independent testing to assure that products remain compliant with regulation.

Revised: This study shows that concentrations of nicotine and menthol in JUUL products procured over a 3 year period decreased over time, possibly due to product degradation during storage. The findings highlight challenges of reconstructing product characteristics at the time of manufacture and time of consumption, and suggests the utility of an ENDS product bank that could preserve samples for retrospective studies.

The authors now characterise the problem not as Juul’s lawlessness but as the authors’ own difficulty in conducting amateur sleuthing operations to hunt down non-existent problems.

Rapid Responses reveal authors knew and published anyway

Though the publication of the Rapid Response was certainly not rapid, a response was eventually published.

Juul submitted its Rapid Response to Tobacco Control at around the same time it published its rebuttal on PubPeer, on 9 November 2022. It took until 27 April 2023 (> five months) to publish this – essentially an abridged version of Juul’s PubPeer comment and its April 2022 communications with the authors. 

Implications made about JUUL products in this study are contradicted by our own contemporaneous and detailed product and manufacturing records, and are more readily explained by a methodological artifact that the authors have not fully reported

If you thought the authors might be humbled, apologetic or even gracious about their error, you would likely be disappointed. In fact, the authors make a strong case for their own research misconduct in their Rapid Response to Juul, also published on 27 April: 

Time-Dependent Changes in JUUL Pod Composition

The authors’ rapid response contains this extraordinary anti-scientific defence of their conduct: 

We acknowledge receipt of a private e-mail message from JLI regarding our paper (Yassine et al., 2022). Given the industry‘s long history of industry obfuscation, interference, and deception regarding research on tobacco products, we decided that the most transparent approach to the private e-mail that we received from an employee of a tobacco product manufacturer would be for us to report our results independently and respond to any public discussion of our work if and when it arose. Now that public discussion has arisen, we are pleased to respond to it.

So they knew that Juul had offered to provide evidence that it had not changed its products. The authors knew about this obvious flaw in their reasoning, and they were aware of Juul’s detailed rebuttal using a perfectly rational and convincing science-based argument. But they decided to ignore it. They went to their comfort zone of generalised industry scientific misconduct (which, in reality, is largely related to the tobacco industry pre-2000 Master Settlement) to justify their conduct.

Yet, they have NO evidence that Juul has ever engaged in “obfuscation, interference or deception” because there isn’t any.  Under most reality-based definitions, Juul isn’t even a tobacco company and didn’t even exist in Big Tobacco’s most mendacious era. Even if Juul did have a blemished record of some sort (it doesn’t as far as I know), then that does not give an open-minded researcher carte-blanche to ignore everything it ever says, however correct and reasonable.

They did not even investigate Juul’s explanation (and citations) using their own independent resources. They went ahead anyway, promoting their preferred narrative in a soft-touch journal, knowing there was a better explanation for their findings. Then they have the brazen cheek to describe this as “the most transparent approach“.  

Not a secret… I don’t have a high opinion of much of the mainstream of tobacco control science. But even I did not expect tobacco control scientists to sink so low. I assumed that they would argue that Juul’s email had been lost, misinterpreted, or treated as spam. But no; they knew.

It seems to me that they are so deeply immersed in tobacco control mythology that they believe any malpractice can be justified with reference to “the tobacco industry”, even if utterly disconnected from reality. For readers with an interest in this corrosive anti-scientific dynamic, I have written about the way false (and easily falsified) assumptions about the industry drive harm to public health: The irreconcilable conflict principle.

Note: the authors’ Rapid Response is the only reason I have written this blog rather than merely being privately appalled by their conduct – my usual reaction. It is very revealing: it suggests that they think normal rules of research and investigation can be suspended for them on account of what they perceive to be a uniquely evil commercial adversary. They are wrong about that.

Implications

This is not the work of novices; leading tobacco control researcher Tom Eissenberg is an author (grants to date > $50 million). We have to assume purposefulness and calculation, rather than mere incompetence, sit behind the research and publishing conduct here.

This work is, however, laden with questions and lessons for more conscientious researchers. These are discussed by me here on PubPeer and in a few points listed below.

• Juul had warned the authors about the error when they first published it in a presentation at SRNT.  The authors ignored the warning and dismissed the reasoning, and went to publication anyway. Why? What was their motivation? Is this acceptable research conduct?
• Did the authors mislead the journal editors, peer reviewers and, ultimately, the readers by not disclosing the communication and explanation from Juul in the original paper? Would the publication decision have been different if the authors had been actually transparent about Juul’s communication?
• Why haven’t the authors referred to Juul’s communication in either the correction or the corrected version of the paper, given this is the only reason the authors have been forced into making a correction? Why not be candid?
• The authors’ interpretation, peer reviews, and editorial oversight were obviously very poor – they missed the obvious explanation. This is a cautionary tale about confirmation bias and groupthink. What will those involved do to prevent this, or is this a feature of the science in this field?
• It seems no one in the production chain (peer reviewers, editors, and even readers) asked the basic quality assurance question: what is Juul’s reaction to these allegations? Even the lowest gutter journalists have an ethical requirement to put allegations to those accused. Just because it is notionally an academic publication, the journal is not excused basic investigative good practice.
• The journal refused to publish Juul’s Rapid Response until 27 April 2023, so readers were denied easy access to the backstory to the ‘correction’. Why? What sort of science is served by that? Why weren’t readers given ‘rapid’ access to the concerns about this paper?
• Why wasn’t the original paper retracted rather than corrected? The entire premise of the original has been undone by the ‘correction’, and I doubt it would ever have been published if it had been submitted in its corrected nothing-burger form. Why not admit that the whole thing was a mess and do the right thing instead of leaving a published corpse dressed up with blusher and lipstick?
• There is no sign of an apology (or even contrition) from the authors or journal for the error and misrepresentation of Juul. On the contrary, there is a display of aggressive hubris. Even the sleaziest newspapers apologise when they make and then correct errors. Are tobacco control researchers above decent and contrition when appropriate?
• Why have the Tobacco Control journal and BMJ Publishing Group tolerated this purposeful deception of their readers and protected the authors by allowing correction rather than retraction and holding up the publication of the Rapid Response? They should be extremely concerned about this conduct. 
• The funders should take a dim view of this research conduct. What will FDA, NIDA, or NIH do next? Will they examine the standard of research conduct here, or are they waiting for someone to make a formal research misconduct complaint? Who should do that? Where is the accountability?

Accountability

The authors accused Juul of deliberately breaking the law – a claim they were warned was baseless but repeated anyway. What are the accountabilities of researchers and journals in this situation?

Academic accountability? Will these authors suffer any sort of reputational damage among their peers in tobacco control? I’m not hearing much of an outcry. Or will they be lauded for taking a shot that didn’t quite land? Will they face inquiries in their own institutions? Will career progress slow? Will there be a lessons-learned wash-up and new guidance put in place?

What about the journal? The journal appears to have gone to some lengths to protect the authors’ reputations by not retracting the paper or even insisting on a clear articulation of the reasons for correction. Yet the journal itself was abused by the authors’ decision to conceal the Juul communication from editors, peer reviewers and readers – and then allowed them to claim this was done in the name of transparency. Maybe the journal will be less of a pushover when it has a new Editor-in-Chief along the lines I have proposed.

What about the funders? Will FDA/NIDA show their displeasure in some way? Will they withdraw grants or put these researchers on a blacklist or other form of naughty step? Are they, in fact, even displeased by this? Could this sort of work be seen as a helpful contribution to their politically-motivated war against Juul?

Will the media report it? I doubt it. The media seems largely locked into a ‘good versus evil” narrative about vaping, and it could be discordant to start lifting the drains on what passes as science in this field. I mean, what else might they find?

Political oversight? Nope.

Accountability – the nuclear option

Maybe someone harmed by this kind of behaviour will finally draw a line in the sand. In my view, this sort of contemptuous conduct goes routinely unpunished. However, those following Dominion Voting Systems versus Fox News Network will know that even in the United States with a strong presumption in favour of free speech, the law protects corporations from malicious or negligent defamation.

This can become very expensive. 

“The truth matters. Lies have consequences,” Dominion lawyer Justin Nelson told reporters outside a Delaware courthouse, 

…as he trousered the $787.5 million paid by Fox News to settle the case.

As this sorry episode suggests, the academic publishing world appears incapable of exerting discipline on claims made about tobacco or nicotine companies.  I suspect Juul is too busy trying to survive the politically motivated onslaught it faces to defend its reputation in this way.

Still, one day I hope a company will hit back at a complacent journal or negligent authors with a $100m defamation lawsuit. What more will it take to remind this dire activist science and propaganda publishing complex that ‘lies have consequences’?

If you are on the editorial board, it might be an idea to check your litigation liability insurance.