In Praise of Non-Inferiority

Dear All (wonkish but intended for everybody to read and absorb):

27 Jan 2021 update: This topic now has its own YouTube video! As the ideas here are VERY confusing, I suggest you make the time to watch the video, read the newsletter, and then watch the video one more time. I have yet to find a short, simple way to explain this but I’ll keep trying!

There’s been a lot of recent discussion about “lessons learned from COVID-19.” Indeed, there are many valuable points: global cooperation, preparedness, supply chains, and so forth. As an example, I note the 17 Sep 2020 communique from the G20 Health Ministers (link): it was heavily focused on COVID-19 but it did note the need to deliver on “previous G20 commitments to tackle antimicrobial resistance.” All good stuff!

Among these important ideas, I would like to draw out today the lesson learned from COVID-19 about the importance of non-inferiority (NI) trials as tools for developing new antimicrobial agents of all types (antibacterial, antiviral, etc.) in advance of widespread need.

Unless otherwise cited, the text below draws from two papers that I co-authored in 2017 and 2019 (link and link, see full cites below my signature). In outline, there are five key ideas:

  1. There are two types of trials: Superiority and non-inferiority (NI)
  2. Infections can be lethal: Placebo is a bad idea!
  3. Antibiotics are best developed before we need them!
  4. NI trials are the public good that allows us to avoid COVID-19-like situations.
  5. Superiority efficacy is (hopefully) unattainable, but superior utility is possible.

1) There are two kinds of clinical trials: All pharmaceutical products must (a) show which individuals can benefit from the product, (b) demonstrate a way to identify those individuals, and (c) document the benefit received from the product. Clear answers to these questions are required for product approval and acceptance. To achieve this, one of two types of trial designs can be used:

  • In a superiority trial, the goal is to show that NEW is measurably superior to EXISTING or to a placebo.
  • In a non-inferiority (NI) trial (sometimes formerly referred to as an “equivalence trial”), the goal is to show that NEW has efficacy similar (within the bounds of a pre-specified non-inferiority margin) to that of EXISTING.

2) Infections can be lethal: From a public health perspective, new antimicrobial agents with an improved microbiologic spectrum of activity should be developed before widespread bacterial resistance emerges. This desire leads immediately, however, to a paradox: although it is easy to demonstrate that a novel test agent has an improved spectrum both in vitro and in preclinical animal infection models, rigorous demonstrations of the NEW agent’s superior clinical efficacy over EXISTING drugs both are and should be difficult to implement on a routine basis in trials of human infections:

  • Infections move quickly and can be fatal! As inadequately treated acute infections can be rapidly fatal and enrollment into a trial must often be undertaken empirically before culture results are known, it is obviously desirable that the control arm be predicted to be efficacious in all studies, including studies of potentially superior new agents.
  • We always seek to give effective therapy to all patients! Indeed, if resistance is known or suspected to the control arm, then the control arm should always be adapted to offer some form of best available therapy that is predicted to be efficacious. In short, it is important from an ethical viewpoint that the trial make every attempt to use an efficacious control and not anticipate showing superiority relative to an ineffective or substandard control.

3) The time to develop new antimicrobial agents is before we need them: When we lack ANY effective therapies, placebo-controlled trials are possible … at least in theory:

  • COVID-19 highlights the only exception to the prior rule! The only exception to the above-noted ethical imperative would be if there were either no efficacious options whatsoever for the infecting strain (or if all forms of best available therapy were meaningfully suboptimal). It is of course obvious that such a situation would imply a situation with grim public health implications.
  • For (fatal!) infections, study enrollment is made challenging by both (i) the impact of infection control efforts and (ii) the desire to avoid placebo (link): Again COVID-19 has shown us that trials may not run as quickly as you’d expect. If local control measures are able to reduce the spread of the target infection (good for the community!), infected patients can become hard to find (bad for the study!). And the desire to be treated with something (anything!) can overwhelm the ability to enroll even in properly designed and ethical trials.
  • And once you definitively have an effective therapy, the rules change! The window of opportunity to reliably design trials to show superiority because of a complete lack of therapeutic options would close with emergence of a new efficacious therapy. Would you be willing to be randomized to placebo if you were in the ICU with COVID-19?

4) NI trials are a public good: By using NI trials to develop new antibiotics in advance of the (next) pandemic, we can save lives and avoid panic. Think about how different the world would be right now if we (as a global community) had substantially completed the development a few years ago of one or more therapies and vaccines for the coronaviridae (SARS, COVID-19, etc.) based on measures of immune response and perhaps even controlled human infection models (CHIMs, link to NEJM editorial on this approach).

5) Superior utility is possible: Although superior efficacy is thus hopefully something we rarely see, superior utility is entirely possible. Oral rather than IV? Shorter course of therapy? Fewer associated side-effects? Better taste in a pediatric formulation? All of these are entirely possible and could be pursued — the only thing that is off the table is superior efficacy. Indeed, the ARLG (Antibiotic Resistance Leadership Group) is pursuing studies of this type based on the Desirability Of Outcome Ranking (DOOR) suggested by Scott Evans and colleagues (link).


There we have the lesson of COVID-19: Antibiotics are the #FireExtinguishersOfMedicine and need to be developed, purchased, and installed in our pharmacies around the world before the fire breaks out. NI studies give us the opportunity to make this happen – provided we are willing to pay for them in advance (and that’s a topic for another day).

In praise of non-inferiority!

Best wishes, John

Key papers:

  • 26 May 2017 newsletter (link) on Rex JH, Talbot GH, Goldberger MJ, Eisenstein BI, Echols RM, Tomayko JF, Dudley MN, and Dane A. Progress in the fight against multidrug-resistant bacteria 2005-2016: Modern non-inferiority trial designs enable antibiotic development in advance of epidemic bacterial resistance. Clinical Infectious Diseases. 2017;65:141-6.
  • 6 Aug 2019 newsletter (link) on Rex JH, Fernandez Lynch H, Cohen IG, Darrow JJ, Outterson K. Designing development programs for non-traditional antibacterial agents. Nature Communications. 2019;10(1):3416.

John H. Rex, MD | Chief Medical Officer, F2G Ltd. | Operating Partner, Advent Life Sciences. Follow me on Twitter: @JohnRex_NewAbx. See past newsletters and subscribe for the future: https://amr.solutions/blog/. All opinions are my own.

Current funding opportunities (most current list is here):

  • Novo REPAIR Impact Fund closed its most recent round on 31 Jul 2020. Go here for current details.
  • 2020 funding rounds for CARB-X have not been announced.
  • The Global AMR R&D Hub’s dynamic dashboard (link) summarizes funders and projects by geography, stage, and more.
  • It’s not a funder, but AiCuris’ AiCubator offers incubator support to very early stage projects. Read more about it here.
  • You might also be interested in the most current lists of R&D incentives (link) and priority pathogens (link)


Upcoming meetings of interest to the AMR community (most current list is here):

  • (it’s over now, but you can still watch it!) 24-28 Aug 2020 BEAM Alliance-sponsored AMR Conference. Go here to access the recorded sessions. This was a very, very good meeting! In particular, please note these two sessions from the final day … both are free on YouTube:
    • “Spotlight on UK | The Investor Action on AMR” (link). Chaired by Louise Norton-Smith (Head of Global AMR Strategy & Delivery, UK Dept of Health) with a key note by Dame Sally Davies (UK Special Envoy on AMR) in which we learn why we should #SaveTheLobster (see notes in this newsletter).
    • “The future of AMR – How a new post COVID19 policy roadmap could look like” (link). Chaired by Marc Gitzinger (CEO BioVersys and VP of BEAM Alliance, the core sponsor of the AMR Conference), this session has speakers from the Government of India (Renu Swarup), the UK (Dame Sally), FIND (a global non-profit focused on diagnostics), EU Parliament (Tiemo Woelken, MEP), and the US (Evan Loh, CEO Paratek and Chair of the Antimicrobial Working Group).
  • [Registration link now posted] 24 Sep (online, 9-10.30a EST): webinar chaired by Dame Sally entitled “The global movement of microorganisms:  Tracking the spread of difficult-to-treat infections”, from a 4-part series sponsored by Wellcome Trust entitled “AMR in the Light of COVID-19 Webinar Series; From hypothetical to reality: How COVID-19 foretells a world without antibiotics.” Go here to register. Really unfortunate overlap with the next meeting, but both are being recorded … you’ll have to pick one to attend live and the other for replay!
  • 24 Sep 2020 (online, 15.30-17.30 CEST; 9.30-11.30a EST; 7-9p IST): Antibiotic Bootcamp Series webinar entitled “Moving from preclinical to clinical-stage: Challenges and opportunities.” Moderated by Erin Duffy (CARB-X), this webinar is jointly sponsored by GARDP, CARB-X, Novo REPAIR, JPIAMR, Wellcome Trust, ASM, and ESCMID. Since we can’t the ASM-ESCMID meeting, we’re still going to have the bootcamps! Go here to register.
  • 24 Sep 2020 (online, 17.15-18.15 CEST): ECCVID (ESCMID Conference on Coronavirus Disease) includes an AMR-focused symposium entitled “Antimicrobial resistance and COVID-19: learning policy lessons from one another” featuring Christine Ardal (Norway), Chantal Morel (Switzerland), and Kevin Outterson (US). You have to dig a bit in the online program to find this one, but it’s there! Go here to register.
  • 25 Sep 2020 (online, 9a-5p EST): FDA Workshop entitled “Addressing Challenges in Inhaled Antifungal Drug Development.” Go here to register.
  • 30 Sep 2020 (online, 2pm BST): Longitude Prize Sprint Workshop 2 entitled “Building Medtech Companies: Learn how to attract investment to fund product development and scale your company.” Details and registration are here.
  • 2 Oct 2020 (online meeting): 7th annual Boston Area Antimicrobial Research Network (BAARN) meeting. Go here for details.
  • 21-25 Oct 2020 (online meeting), IDWeek 2020. Go here for details.
  • 26-29 Oct 2020 (online meeting), Annual ESPID meeting (European Society for Pediatric ID, #38)
  • [NEW] 27 Oct 2020 (online, 9a-5p EST): FDA Workshop entitled “Development Considerations of Antimicrobial Drugs for the Treatment of Gonorrhea.” Go here to register.
  • 27 Oct 2020 (online meeting), BARDA Industry Day, a discussion of U.S. Government medical countermeasure priorities. Mark your calendar now and watch this website for details.
  • 9-12 Jul 2021 (Vienna): Annual ECCMID meeting (#31)
  • 18-21 May 2021 (Albuquerque, New Mexico): Biannual meeting of the MSGERC (Mycoses Study Group Education and Research Consortium). Save-the-date announcement is here, details to follow.
  • 20-24 June 2021 (Toronto): International Symposium on Pneumococci and Pneumococcal Diseases (ISPPD-12). Go here for details.
  • 3-7 Jun 2021 (Anaheim), ASM Microbe 2021. Go here for details.
  • 27 Jun-2 Jul 2021 (Ventura, CA): Gordon Research Conference entitled “Antimicrobial Peptides”. Go here for details, go here for the linked 26-27 Jun Gordon Research Seminar that precedes it.
  • 5-21 Aug 2021 (Marine Biology Laboratory, Woods Hole, MA): Residential course entitled “Molecular Mycology: Current Approaches to Fungal Pathogenesis.” This 2-week intensive training program has run annually for many years and gets outstanding reviews. Go here for details.
  • 8-11 Oct 2021 (Aberdeen, Scotland): 10th Trends in Medical Mycology. Go here for details.
  • 16-24 Oct 2021 (Annecy, France): Interdisciplinary Course on Antibiotics and Resistance (ICARe). This is a soup-to-nuts residential course on antibiotics, antibiotic resistance, and antibiotic R&D. The course is very intense, very detailed, and gets rave reviews. Registration is here and is limited to 40 students.
  • 6-11 Mar 2022 (Il Ciocco, Tuscany): Gordon Research Conference entitled “New Antibacterial Discovery and Development”. Go here for details, go here for the linked 5-6 Mar Gordon Research Seminar that precedes it.
Scroll to Top