Delusional Biomedical Researchers Seek Repeal Of Arithmetic

There is a letter being sent around via e-mail that expresses the views of biomedical researchers who are pushing a petition to repeal the new NIH rule that only a single resubmission of an unfunded competing grant application is permitted. Here is the text of the letter that I received via an e-mail forward:

Dear Colleagues,

I am writing to solicit your help in changing a new NIH policy that I
believe will have an enormous negative impact on our field. As most of
you know, a recently adopted rule states that if a grant proposal is not
funded on the first submission, only one revision can be submitted with
the same specific aims. If that revision is not funded, the proposal
must be “substantially” changed. As far as I understand, the rule was
adopted to discourage “serial resubmitters”. While such a policy could
make sense in an era of reasonable paylines, with the projected budgets
rumored to be funding at the 7th percentile in some sections, this could
have a dramatic and I would argue devastating effect on the research
efforts in this country. Consider the following:

The rule will have a disproportionately negative impact on young
investigators with early stage and therefore less diverse programs, or
more senior investigators who also have more narrowly focused programs.
How can a young investigator, for example, who is just starting
“substantially” change their aims when they have to focus their efforts
on a very limited number of projects undertaken with limited funds and
staff. These people are often hired by senior faculty on the basis of
their first projects and to be told they must change on the basis of
applications that might fail despite being ranked better than 90% of
grants submitted, seems patently absurd. And worse, it is likely to be
profoundly discouraging and destructive.

All of us who have sat on study section know that we cannot distinguish
a 15th percentile grant from a 5th percentile grant. It is simply beyond
the resolution of the process. Therefore, this new rule will have the
consequence of redirecting the science of many of our very best
scientists on the basis of what will essentially be an arbitrary
criterion.

The meaning of “substantially changed” has not been clearly defined.
Program Officers themselves are not sure what this term means and are
not being given adequate guidance. I have heard things from “51%
different”, change the tissue or cell type you are working on, any aim
included in either the first application or revision cannot be included,
etc. We need clear and unequivocal guidance on this point, and I would
argue we need it immediately as “new” applications are being prepared by
a large number of investigators at this time.

The alternative that I advocate would be to go back to a system where at
least 2 revisions of the same application would be allowed. While we
will still obviously lose some superb applications if the pay line stays
where it is, I think this would provide a much fairer assessment of the
research proposals received by the NIH.

My intention is to let the feelings of a large number of scientists on
this subject be known. If you are willing to sign an email that will be
sent to both Francis Collins and Tony Scarpa (Director of Center for
Scientific Review) that raises these points, please let me know by
simply responding to this email and (if possible) forwarding it to 10
people who you know (not on the current recipient list) that might also
want to sign. If I can accumulate a large enough number of signatures
(100-500, say) I will draft a letter and send it first to all who have
expressed interest in signing to get feedback.

I must say, I am not generally prone to such activism but I think things
have just gotten to the tipping point.

I look forward to your responses.

There is a lot of fundamental misunderstanding going on in this letter. We can start with this:

As far as I understand, the rule was adopted to discourage “serial resubmitters”.

No, the rule was adopted to try to stop the common study section behavior of putting meritorious applications in a “holding pattern” for one or two resubmissions, so they can fund the applications that have already been in a holding pattern. The other change made by many ICs to prevent the “holding pattern” is the adoption of different paylines for A0 and A1 applications, with the former substantially more generous.

More importantly, however, there is a serious delusion that underlies this letter. There is only so much money available to fund competing applications, and the only effect changes in peer review in terms of actual funding of such applications could possible have is a change in which applications get funded. So the notion of “meritorious applications going unfunded because of this pernicious new rule” is nonsense. Limiting resubmissions can’t possibly change the number of “meritorious” applications that go unfunded.

The letter authors seem to have forgotten that–while they may feel put upon that they only get a single resubmission–all their competition also only get a single resubmission. The playing field is still even, but in a context that should make peer review more efficient by substantially reducing “holding pattern” study section behavior. It will also reduce the PI behavior in response to “holding pattern” of submitting half-baked proposals they *know* aren’t fundable in order to “get in line” in the “holding pattern”.

Finally, as far as New Investigators/Early Stage Investigators being disproprortionately affected, that is a red herring. Their grant applications are reviewed separately in study section, and are supposed to be assigned criterion and impact scores by comparison with each other, not with established investigator grants. And even more saliently, all ICs have substantially more generous ESI paylines than for established PIs, in some cases more than *twice* the established PI percentile cutoff.

This new restriction on resubmissions can’t possibly lead to an increase in the number of “meritorious” applications that would have been funded under the previous system now going unfunded. The laws of arithmetic cannot be repealed, and these researchers would be much better advised to devote their energies to lobbying Congress to support the NIH budget, not tilting at irrelevant peer review windmills.

UPDATE: Since there already seems to be substantial confusion about what I mean by “the laws of arithmetic cannot be repealed”, the point is that the number of grants that can be funded is zero-sum. Putting more applications in a holding pattern can’t possibly do anything about that.

The purpose of the one-resubmission rule and the more generous A0 payline is to try to identify the most most meritorious grant applications as efficiently as possible and to get those grants funded as soon as possible. All the A2s that no longer exist and thus cannot be funded means that an exactly equal additional number of A0s and A1s *do* get funded.