Hematology Grants Workshop

James L.M. Ferrara, MD^*

Any writer should consider his or her audience. The audience for an NIH grant proposal is the NIH study section (SS), and the initial review process of a grant application is similar to that for a manuscript. At least two SS members evaluate NIH applications: a primary reviewer and a secondary reviewer. A tertiary reviewer may read the application but often is not required to provide a written critique; thus every NIH grant application receives at least two written critiques. These critiques are written prior to the SS meeting and are meant to be completely independent, as are the reviews for a journal manuscript.

From this perspective, the SS may be considered the editorial board. Reviewers present their evaluations to the entire SS, and every SS member (there are usually at least 20) has an equal voice in the final score. Thus, the applicant should consider which SS will vote on the proposal. Investigators can locate SSs at the website http://www.csc.nih.gov/review-irgdesc.htm. Clusters of SSs, termed IRGs (institute review groups), appear first on the website, followed by specific SSs and their membership. If none of the names on the SS roster is familiar to the applicant, he or she should be concerned that the SS may not normally deal with the scientific issues that are pertinent to his or her proposal. Furthermore, the applicant should perform a quick literature search of several SS members to discern their areas of expertise. If one or more have published in an area relevant to the proposal, it is wise to include at least a couple of their papers in the bibliography. Few things are more displeasing to a scholar than the perception that his or her important contributions to a field have been overlooked.

Once a potential SS is identified, the applicant should seek some expert advice. Local experts include faculty members who are, or who have been, members of any NIH SS. These individuals may provide information and insight regarding specific perspectives or biases (e.g., whether a particular SS encourages an animal model as one of the specific aims). It is also often helpful to speak with the SS scientific review administrator (SRA) prior to submission of the grant application. SRAs frequently have insights into the appropriateness of an application for their SS, and an early discussion may save significant time and trouble later in the process. The applicant should always include a cover letter with the grant application, which should refer to the conversation with an SRA. In that letter it is permissible to identify reviewers who might have a conflict with the application; however, it is not advisable to suggest reviewers. Such a suggestion may backfire and actually disqualify that potential reviewer because of the appearance of bias.

During any given cycle, a single SS evaluates 80 or more applications. Each reviewer must provide 6 to 8 written reviews that often entail 20 to 40 hours of work added to an already busy schedule. Reviewers quickly estimate how much time a review will require. For revised applications, the prior SS critiques will be included, and some of the work is therefore already done. The first question that many reviewers ask themselves is: “Will I score this application?” If there are major problems a reviewer may focus on only 1 or 2 issues in the critique to save time because the proposal is unlikely to receive a full discussion.

When the SS convenes, its first order of business is to decide which grants will receive priority scores. NIH scores range from 1.0 to 5.0: 1.0 to 1.5 is outstanding, 1.5 to 2.0 is excellent, 2.0 to 2.5 is very good, 2.5 to 3.0 is good, and below 3.0 is normally unscored. Sample scores and percentiles often fall in a trimodal distribution (see Figure 1 ). A top group of grants (about 15%) lies in the outstanding to excellent range; a second group (approximately 35% to 40%) falls in the very good range; the third group (about 45%) is unscored. SSs do not fund grant applications; institutes fund them. However, SSs decide which applications are likely to be funded. Reviewers thus make a (semi)conscious effort to divide the better grants into a very favorable group and a less favorable group, and many will try to push the best grant in their stack of 6 to 8 into the most favorable group (those likely to be funded). Unscored grants that have at least one if not more major problems (also known as “fatal flaws”) are unlikely to be funded; hence they are not discussed by the full SS.

The decision to not score the lower half of the applications dates back to the early 1990s, when the NIH had (too) little money and the percentile for the fundable applications dipped below 15%. Reviewers felt that not enough time was spent on discussion of applications that were very good but not good enough to fund. Such applications would usually return 8 months later in amended form but without the benefit of a detailed discussion from the review panel. The SS meetings could not be lengthened and the NIH decided not to score—or discuss—the lower half of applications, allowing for fuller discussion of the better half. One downside to this approach is the lack of integration of reviews for applications in the lower half. Reviewers may dislike an application for divergent reasons, and the independent critiques may seem contradictory to the applicant. Nevertheless, this process is very similar to that of a manuscript review for a journal editorial board. An important difference is that an NIH grant application may be amended twice, even if it is unscored (“rejected”) at both initial SS reviews. The revision of an application should be prepared very carefully (see below and Section II).

SS reviewers are instructed to include 5 criteria in their critiques: investigator, environment, significance, approach, and innovation (Figure 2 ). Investigator focuses on productivity (the applicant’s curriculum vitae [CV]) and preliminary data; environment includes consideration of space, resources, and collaborators; approach evaluates whether the experiments will work and what the investigator purposes when some of the experiments do not work (as is inevitably the case); significance includes impact on the field (will it shift any paradigms?) as well as size of the field (is this area basic to several disease processes or more limited in scope?); innovation pertains to both perspectives (e.g., a new approach to an old problem) as well as new techniques or reagents. Although the specific weights for these 5 categories are not predetermined, R01s generally place about 60% of the scoring emphasis on the science and 40% on the investigator, whereas mentored awards (e.g., K awards) reverse the emphasis, with an approximately 30% weighting for the science and 70% for the investigator (with extra considerations for the environment and the qualifications of the mentor).

The merits of an application can be categorized into 4 quadrants using Cartesian coordinates (Figure 3 ) (Martin Cheever, personal communication). On one axis is the quality of the investigator (preliminary data, CV, and the environment); on the other axis is the quality of the science proposed (significance, approach, and innovation). The first and best quadrant (15%) includes proposals with fabulous ideas and the perfect person to test them. The second quadrant (25%) contains good ideas and an applicant not quite able to test them (yet). Into the third quadrant (10-15%) fall not-so-good ideas but an excellent investigator with a strong track record. The last quadrant (50%) contains proposals with bad ideas and an investigator incapable of testing them.

A successful application that falls into the first quadrant often has 4 primary characteristics: it is (1) hypothesis driven, (2) mechanistic, (3) clear, and (4) exciting. A successful application presents a series of related hypotheses focused on a single scientific problem and details the approaches that the investigator will use to test those hypotheses. (An application that does not use the word “hypothesis” in the first page of the experimental design is unlikely to succeed.)

These hypotheses should be mechanistic. “Descriptive” is a pejorative adjective in a reviewer’s lexicon, denoting a weak or insufficiently mechanistic hypothesis. Although the description of the size and shape of a problem is necessary, it belongs in the preliminary data. Many an application falters because specific aims remain descriptive.

A grant application is not science; it is the marketing of science. Clarity is key to any successful marketing strategy. It is therefore imperative to write clearly. Such clarity is challenging for most scientists because in general we write badly, and sometimes very good ideas can seem mediocre because they are not clearly presented. A successful grant also usually excites the reviewer and teaches him or her something. The opportunity to learn motivates many SS members to participate in the arduous process of grant review.

There are many avenues an applicant can take to write a successful grant application. Although by no means exhaustive, the following 7 strategies can help investigators improve their grant applications.

Writing an NIH grant application is perhaps the largest and most complex academic task of any academic scientist, the doctoral thesis notwithstanding. The importance of an early beginning cannot be overemphasized. One should first notify administrators of plans to submit an NIH application because administrative approval often requires layers of signatures. This administrative process can (and should) proceed in parallel with the preparation of the scientific portion of the application; it cannot be accomplished easily at the last minute. Ideally, the investigator should prepare the final draft of science 6 weeks prior to the deadline. He or she should identify a senior colleague who will give the application a meaningful review, which usually requires a minimum of 1 to 2 weeks. During this period it is helpful to put aside an application and not read it; a fresh review after such a respite can provide powerful first impressions and highlight unclear areas. The applicant then has an additional month prior to the deadline to act on any feedback provided by a senior colleague.

A successful application will maximize the use of all institutional resources. For example, experienced senior colleagues who are familiar with good science (but not the applicant’s science) can provide the perspective of the majority of the SS members, and these colleagues’ critiques are thus often exceptionally valuable. Depending on the amount of preliminary data, an applicant may consider holding back a key result for an update, which can then be forwarded to reviewers approximately 1 month before the SS meeting. Strong additional preliminary data can provide a sense of excitement and momentum for the application and demonstrate productivity even though the investigator has not received funding.

First impressions count in grant applications, as in life. By the end of the preliminary data, the reviewer often knows whether the principal investigator can (1) formulate clear hypotheses; (2) test his or her hypotheses; (3) perform careful experiments (using the right controls); (4) accurately interpret his or her experiments; and (5) be excited about his or her science. If 2 or more of these areas are weak or absent, the application is unlikely to fare well.

Proposals should have 3 to 5 specific aims; an application with too many aims can appear unfocused. Hypotheses should be stated prominently at the beginning of each specific aim. Controls should also be explicitly stated; the applicant will receive additional points every time he or she states the controls to be included in a proposed experiment.

The investigator should create narrative flow or a story line, and show only key experiments in the preliminary data, not every experiment ever done. He or she should highlight the design of the experimental approach rather than burying the logical flow in detailed, standard techniques. To this end, an applicant could cluster the detailed methods in a specific section, perhaps at the back. Visual clarity is also important. Wide margins and spaces between paragraphs create a sense of order and make it easier on reviewers who are becoming more presbyopic with each grant application.

Acknowledge competing hypotheses and approaches. To pretend they do not exist usually courts criticism of a less-than-adequate knowledge of a subject. Avoid criticizing the work of others. The scientific community is a very small world, and the target of criticism may be a collaborator or a good friend or colleague of an SS member. The applicant should criticize him- or herself. An essential section of any grant application deals with the pitfalls of a chosen approach and explores alternative strategies. Such alternatives clarify an applicant’s thought processes for the reviewer and may be particularly useful when concerns arise regarding a given approach (which is not uncommon). A well-written “alternatives” section can often sway a reviewer toward a more positive appraisal of an application.

The applicant should acknowledge directly any period of low productivity. Sometimes difficult or labor-intensive techniques have not worked well in the investigator’s hands or a reagent is inconsistent when applied in a new experimental system. Changes of status (either professional or personal) can be distracting and time-consuming and may have contributed to a period of low productivity. In general, it is better to be forthright about such lapses rather than for silence to allow reviewers to come to their own (less charitable) conclusions.

Sloppy writing is often interpreted by reviewers as sloppy experimental technique, particularly for new investigators. Always cross-check the correlations of individual references to the final text. Sections of grant applications are frequently cut and pasted from other documents and the reference chronology disintegrates easily. References can offer important learning moments, especially when a reviewer needs to check a reference about a particular observation or technique. An inaccurate reference gives the impression of sloppiness and can quickly put distance between the applicant and the reviewer. Also, check for accurate correlation of text to figure and table numbers as well as the accuracy of legends and their symbols; correlations erode when materials are cut and pasted from other documents. A textual reference to a Figure 3 that is actually Figure 5 of the application can slow down and annoy a reviewer whose time is precious. This common error is easily correctable in the final draft. Finally, double-check the availability of all proposed reagents. The phantom reagent (e.g., an antibody or a plasmid incorrectly assumed to be standardly available) is a common flaw. Such an oversight may result in a “gotcha” moment that diminishes enthusiasm for the application.

An applicant’s independence is very important and sometimes can be difficult to discern, particularly if the applicant has remained at the institution where he or she received his or her doctoral or postdoctoral training. An applicant should stress last author papers or those without a mentor as well as dedicated space, promotion to a faculty position, and other non-NIH funding, especially if independent publications have not yet appeared. These resources can almost always be identified in a letter from the division chief or chair. A supporting letter should not state that funding of the grant will lead to the applicant’s being given space, a promotion, and so forth. This kind of tactic is never well received by reviewers and is considered by many to be tantamount to blackmail.

This advice pertains particularly to investigators revising their unfunded applications. SS critiques are often written in direct and unvarnished language, and they can be extremely painful to read, even for senior investigators. The grieving process in response to a vigorous critique is virtually inevitable: first denial, then anger, and finally acceptance. The applicant must acknowledge these emotions, work through them, and then reengage in the process of revision. Reengagement is best signified by the respect shown for the critiques. When revising an application, the applicant should list all of the criticisms in the introduction and address them individually. The investigator should thank the SS members genuinely for their criticisms. If he or she cannot express such gratitude without gritting his or her teeth, then it is not yet time to revise and resubmit the application. If a reviewer had correctly predicted a difficulty with an experimental approach, the applicant should say so. Nothing is more pleasing to a reviewer than to be proved right. If the applicant disagrees with a specific criticism, he or she should acknowledge at least the possibility that the reviewer is correct. It is helpful—and necessary—to emphasize interim progress. It is never helpful to be angry or defensive in response to the critiques. The introduction to the revised application is the single section of the grant that the entire SS will read. Its tone should be positive and constructive because it will be evident to all of the SS members who score the application. A senior colleague should thus carefully review the introduction prior to resubmission.

In conclusion, the creation of a successful grant application is a large, complicated, and often daunting task. Few investigators succeed on the first attempt. Peer review of these applications, imperfect and frustrating as it is, is a fact of every investigator’s life. Perhaps Winston Churchill’s comment on democracy applies here as well: “It’s the worst system except for all the rest.”

Alvin H. Schmaier, MD^*

This section will be divided into two parts: (1) a description of the grant review process, in which a grant moves from assignment to a study section and ushered through the review process; and (2) a realistic approach for constructively dealing with an unfunded grant being prepared for resubmission.

The actual grant review process is managed by the Scientific Review Administrator (SRA). This individual organizes the entire review process, from arranging for the study section to review the application, participating in the review, and preparing the postmeeting Summary Statements. In advance of the study section meeting, the SRA participates in the referral of applications to study sections. This individual then identifies and recruits the reviewers for the meeting. He or she interacts with the chair of the study section to determine who specifically reviews the applications and makes the specific assignment of the applications to reviewers. The SRA also administers the review of the applications and in advance of the meeting determines which applications fall above the 50% scoring level (a process called “triage”). The SRA also arranges the logistics of the meeting.

At the meeting, the SRA is the moderator. He or she arranges the order of review of applications and the programmatic and institute groups of proposals to be reviewed. The SRA also makes sure the NIH policy is followed at the meeting concerning review of science, budgetary matters, human studies, animal studies, and so forth. The SRA insures that the meeting is kept moving and he or she is responsible for taking good notes of the discussion of the grant. These notes are then used in compiling the summary statements that will be made available to the applicants. At the time of the meeting, additional NIH administrators are present. Some of these individuals include members from the office of the Program Director who observe the meeting and take notes on the proceedings. The Program Director does not participate in the review process and does not prepare the Summary Statement that gets sent to the applicant. However, these administrators discuss the Summary Statements and are able to interpret and make suggestions on how applicants can improve proposals, if unfunded, on subsequent submissions.

At the conclusion of the study section, the SRA records the scores and budgetary recommendations for the reviewed proposals. The SRA prepares the Summary Statements based upon the written critiques of the reviewers and the summary of the discussion which took place at the meeting. This task is difficult because the SRA is pressured to prepare these summary statements as soon as possible after the meeting and to incorporate information that will help an applicant revise a grant if resubmission is needed. Usually, the SRA does not interact with the applicant; this activity is reserved for the Program Director. However, if the Program Director was not present at the meeting, the SRA sometimes communicates the assessment of the study section to help the applicant with the revision. The SRA may participate in appeals to Council, the overall approving body for grant funding, on behalf of applicants. This individual interacts with present and future members of the study section. Finally, the SRA completes the old business of the previous study section and begins the new business of the next meeting, keeping the grant review process moving.

The most difficult aspect of the grant application process is constructively improving a proposal to make it acceptable to reviewers when it has not been funded. In the two-cycle period from the original submission to resubmission, the applicant needs to redevelop a proposal to address the criticisms of the first review. This process is the same whether the previous application was triaged or had a non-fundable percentile ranking of 25% or higher.

As an initial approach to responding to a negative review of a grant, one has to suppress one’s emotions and read the criticisms from the point of view of the reviewer. Usually there will be a few criticisms indicating that the reviewer did not fully understand one aspect or another of the proposal. This kind of error occurs in both funded and unfunded proposals and likely reflects time constraints the reviewer has in preparing the review. However, these misconceptions in the review are usually a minority of criticisms. If misconceptions account for the majority, however, the applicant has to consider how carefully he or she originally presented the material. An applicant can never do enough to make an application clear to readers. It is helpful for applicants to extract each criticism and specifically respond to it in a direct and concise manner. If this is done soon after receiving the review, it can serve as an outline to formulate the revised application. This response also serves as the Introduction (3-page limit) to the revised application. It should be as well written and detailed as any response to a review that one prepares for a manuscript resubmission. Upon receipt of the revised application, the reviewer usually looks at the quality and level of detail of the response to his or her review. It is flattering to any reviewer to have criticisms taken seriously and responded to with new experiments or data, or with a modulated, respectful disagreement based on sound scientific reasoning.

In addition to addressing all criticisms, the resubmitted proposal should contain evidence of substantial progress in the work on the project. New progress probably results in revision of hypotheses and the preparation of a more sophisticated proposal. It is important for applicants to remember that only published work can be considered as completed work. Unpublished work, even if it comprises a few manuscripts, has to be considered preliminary. This latter work should be presented in detail to indicate the depth of “preliminary” data the applicant has for the proposal. Substantial progress also needs to be supported with publications. On NIH study sections, a few high-quality publications count more than many publications in less critical journals. Applicants should aim to publish in only top journals. It is difficult for reviewers to ignore an applicant who has 5 articles in the Journal of Biological Chemistry, for example, in the last granting period.

Grant writing, in general, requires a special style. It is always important to remember that the presentation of the grant should be pleasing to the reviewer. A proposal should be well organized, for instance. The applicant should consider providing a “Rationale” heading at the beginning of each section in the Methods to synthesize the background and preliminary data leading to the proposed plan of investigation. Re-emphasizing hypotheses and plans in the body of the proposal helps to reorient the reviewer to the focus of the proposal. Above all, it is imperative that the applicant continually try to make the proposal clear and concise in presenting the project’s plan. It is important to remember that grant writing should be as precise and clear as the writing style used in final version of manuscripts. High-quality writing in grants is usually difficult since there is always a time constraint. It is important that applicants spend enough time on proposals, polishing them as much as possible. Very few people can prepare a readable document in a single draft. Usually, the least experienced individual needs the most time to prepare the proposal but usually starts at the latest date. A carefully prepared proposal requires weeks of work. It is best to have a first draft finished 1 month before the deadline so that there is sufficient, unhurried time to critically review the material before submission.

Occasionally, a clear response to the review, new data, significant publications, and a well-written proposal still does not result in receiving a fundable priority. It is possible that there is no one in the study section reviewing the grant who is interested in the research that you are doing. Lack of interest in a proposal does not necessarily mean that the work is unimportant, however. It is not unusual to need to “shop around” to find a study section where there are at least two individuals who have a background and interest in the work being proposed. Careful review of NIH study section rosters may help applicants find these individuals. Efforts should be made to route a grant to the study section with these individuals on it. It is more important to find appropriate reviewers than to stay within the study section that usually reviews the proposal’s content area. To have a variety of study sections that can review an application, the project’s scope must be sufficiently broad that it has appeal to a wide audience. Changing 1 of the 3 specific aims of a proposal can be helpful to tailor a grant such that it may be acceptable to one study section versus another. If more than one application is pending, repetitive specific aims can be listed as alternatives so that there is a clear distinction between overlapping submissions.

In sum, the grant writing and revision process is not an established protocol. Interests and directions in science change, just like the rosters of study sections. Applicants are advised to be able to evolve their style and substance as the research climate changes among study sections and the research environment. Being able to adapt allows members of the “researcher” species to survive and to pursue their chosen areas of scientific investigation.