after you've written: the NIH application review process

by Janet S. Rasey, Ph.D.
Director, Research Funding Service
University of Washington
Seattle, WA

Introduction
Study Section Psychology 101: Meeting Dynamics
Study Section Psychology 102: The Reviewer
Questions Grant Reviewers Ask
Criteria that Convince a Reviewer to Give Your Grant a Fundable Priority Score
Some Common Faults that Result in a Non-fundable Priority Scores and How to Correct Them
Small Things Can Really Peeve Reviewers
A Few Final Words...

Understanding the process and psychology of grant review helps you write better proposals.

There are some things grant reviewers always look for. Give them what they want. There are some things grant reviewers hate. Avoid them.

Grant reviewers want to do a good job and they want to see good science funded. It is your job to help them avoid making mistakes in evaluating your grant.

Study Section Psychology 101: Meeting Dynamics

Your grant, one of perhaps 100 proposals, will be reviewed in a tense, high pressure three-day Study Section meeting.

Your NIH R01 grant took perhaps 120 hours, or longer, to write. The primary reviewer reads the grant and writes a critique in about seven hours, while other SS members average slightly under one hour to read the proposal. Study Section members spend, on average, a little more than 20 minutes discussing the grant.

Your grant's fate lies primarily in the hands of 2-3 primary reviewers. A competent, well respected Study Section member who is not an assigned reviewer can make comments in discussion that will worsen a priority score; however, he/she cannot counter negative evaluations by the primary reviewers.

The longer the discussion among the reviewers, the poorer the grant's score.

The requirements to review nonscientific aspects of a grant (for example, animal use, gender balance in clinical trials, overlap with other grants) create pressures reviewers hate.

What this means for the applicant:

Communication of good ideas and careful plans is essential. Make your application easy to read and understand the first time through. Make certain the important ideas are laid out clearly, early in the application. Don't leave loose ends or unanswered questions that generate undesired discussion. Don't make the reviewers work to understand you. Do work hard to avoid submitting a grant that mimics others in a "hot" or heavily investigated area. Strive to be outstanding, not to stand out for the wrong reasons.

Study Section Psychology 102: The Reviewer

The mindset of reviewers as well as the dynamics of a Study Section meeting influence the review.

A typical Study Section member is a 40-something full professor faced with his/her own grants, teaching, committees, editorial responsibilities, graduate students, mortgage payments, rebellious teenagers, aging parents, mildewed roses, etc. etc. This person typically is busy, over committed, unable to say "no," and also is very frustrated by the too many grants-too little money syndrome.

A typical Study Section member wants to do a good job; to see really good science get funded (good ideas backed by preliminary data; good methods; focused research plan); to look competent in front of colleagues; and to have as little hassle as possible.

A typical Study Section member does not want to have to choose between several equally good grants in one area; to look stupid in front of colleagues by misinterpreting the applicant's plans or ideas because of fuzzy writing; to work to figure out what the applicant wants to do; or to search the proposal to extract the major idea, priorities for experiments, overall design, or significance of the proposed work.

What this means for the applicant:

Try to put yourself is the reviewer's shoes and make it easy for him/her to read your application and see what is novel, valuable, and exciting about your ideas and plans. Don't ever assume that the reviewers will know what you mean; tell them! Take to heart the lists of "do's" and "don'ts" in this fact sheet. Give the reviewers what they want, avoid the things they hate to see.

Reviewers want to see that you

Think like a scientist
Have good ideas
Focus your writing
State your hypotheses
Propose experiments that test your hypotheses
Have the necessary background and experience to do the work
Have a record of productivity (particularly for renewal grants)

Reviewers don't want to see

Poor scientific rationale (no hypotheses; or experiments that do not test the hypotheses)
Unimaginative or tired ideas
Lack of background and experience
Poor productivity

Questions Grant Reviewers Ask

Grant reviewers consciously or unconsciously ask these questions. There are one or more places in the grant where you can supply the answers.

Q. Who are you? (Interpretation: Can you do the work?)
A. Read my Biographical Sketch to find out my background; look at my preliminary data to get an idea of what kind of scientist I am.

Q. What are you going to do?
A. Read the Specific Aims and also the Research Design and Methods, which begin with an overview of the research strategy for the project.

Q. Why is this worth doing? Why do you want to do it?
A. Read the Background and Significance section to see where my work fits in with what has been done, how it solves a problem, and particularly in what way my ideas are new and important. Examine my preliminary data and Biographical Sketch to see how the planned studies fit logically into my research career.

Q. How are you going to do the work?
A. Read the Research Design and Methods section. The overall strategy as well as the specific methods are clearly described there.

Q. Where will the work lead?
A. The Background and Significance section deals in part with this question. The order of the Specific Aims and the overall research strategy also show how the work will progress into new areas.

Q. How much will it cost?
A. See the Budget and please note that all costs are fully justified and explained in the Budget Justification.

Criteria that Convince a Reviewer to Give Your Grant a Fundable Priority Score

Reviewers want a well thought out, reasonable, and convincing scientific rationale to support the research plan. Quotes from grant reviewers reveal what they want to see:

"A clear hypothesis, cogently developed and addressed"
"Experiments which explicitly test the stated hypothesis"
"Creativity, beauty, and cleverness of experimental design"
"Will the methods and experiments proposed answer the questions?"
"Good organization indicating logical thinking and conclusions"
"Clear relationship of methods to ideas"
"Sound experimental approach/design"
"Logical and well developed aims & objectives"
"Experimental details including pitfalls and alternatives"

Reviewers want to see good new or original ideas. Grant reviewers have stated that they look for:

"A novel approach to an important question"
"A new and exciting idea that is likely to open doors"
"Innovative good ideas"
"An interesting and important question"
"A research plan which is compact, tightly written, thorough, focused, and that provides sufficient detail."

Grant reviewers, in their own words, are impressed by:

"An experimental design which is clearly written to answer specific aims"
"A well developed experimental plan that lacks any significant flaws"
"A well organized, easy to follow (and believe) application"
"The clear development of a rationale and experimental plan"
"A simple, well designed research plan"
"Evidence of background and experience in the essential methodology."

Grant reviewers seek out:

"A good track record if a competitive renewal, or good preliminary data if a new RO1"
"Good preliminary data"
"Record of the applicant"
"Solid methodology and well-documented preliminary data"

Some Common Faults that Result in a Non-fundable Priority Scores and How to Correct Them

Problem: A research plan which is diffuse, rambling, superficial, unfocused, and/or lacking in detail.
Solution: Focus it! Get an intelligent nonexpert (an experienced grant writer from another discipline) to read it. Such a person should immediately see the main points of your proposal. Make sure your main points are stated up front, not at the end of a section. Read other successful applications made to the same grant maker, and if possible, the written critiques of those successful grants.

Problem: Questionable reasoning or scientific rationale
Solution: State clearly the hypotheses that underlie your experiments. Make sure the hypotheses are tenable, based on the literature, and that they pass the scrutiny of other respected scientists. Design experiments that test the hypotheses. Do not set out to "prove" your hypotheses.

Problem: The applicant has little or no background and experience in the essential methodology
Solution: Provide preliminary data that shows you do have the expertise. One picture (i.e., graph) is worth a hundred words. Enlist co-investigators and consultants who can do what you cannot, and make sure they write specific detailed letters of support.

Problem: Attempting to conduct an unrealistic amount of work
Solution: Pare the proposal down! Get an experienced grant writer in your field to read it for you and suggest cuts. Do a detailed budget justification, figuring costs for the various items in each experiment you plan. Also project the number of experiments you can do and need to do in the total grant period. If you apply for a grant with upper budget limits, calculate what you can do with the allowed amount.

Small Things Can Really Peeve Reviewers

Take care to avoid patronizing reviewers; creating extra work in a poorly organized or ambiguous presentation; or making misleading statements in efforts to inflate past research accomplishments.

Some reviewers' pet peeves that meet the above criteria are:

• squeezing too much information into the allowed page limit
• lying about support on other grants
• putting items in the appendix that belong in the proposal
  
• submitting grossly large budgets with justifications the size of a string bikini
• not following the application instructions, especially on revised grants
• ignoring or arguing with reviewers' comments when submitting a revision
• trying to sneak a revised grant in as a new proposal
  
  The reviewer's Kiss of Death: "This grant is a fishing expedition..."

A Few Final Words...

The System helps those who know The System. Knowing The System, including review processes, is a course in continuing education.

There is no substitute for a good idea, but a successful grant is also an exercise in communication

Copyright 1993 by Janet S. Rasey