Editor's Note: This series of edited transcripts is from Symposium A-14 at the Nashville M&M meeting on August 10, 2011, organized on behalf of the Facility Operation and Management Focused Interest Group, co-chaired by Owen Mills and Christopher Gilpin. This is the fifth of six talks on this topic; the remaining article will be published in the next issue.

As I have been listening to the previous speakers, I have been thinking through my slides. We easily could talk about the Major Research Instrumentation (MRI) program for an hour. So during my time, I will run through the rules of road and things you can do to increase your chances of success and the caveats that go with that. My big take-home message is that this is not something where you can just read the MRI solicitation beforehand and write a good proposal. You really have to think about this and do your homework. Get to know program officers and understand the program. I will try to show you enough of the nuances of the program so that it will encourage you to do that. I'll show you some of the pitfalls, and you'll be able to see you need more information. We're in Arlington, VA, and the old-fashioned way to do this was to go visit us in the NSF building; however we have technology to do that now. We have websites, webcasts (one will be held in December), phone calls, etc. There are a lot of ways to electronically reach out and get information.

The first thing you should know about MRI is that it is a foundation-wide program. Why do you care? Basically this means there are two heads to the MRI monster. I represent one head, the Office of Integrative Activity (OIA), and we (my colleague Randy Phelps and I) coordinate the overall program for the foundation. We do this because the MRI program has directives. OIA is in the Office of the Director, and we have strategic goals given to us by the director of NSF and Congress that are not discipline-focused. The other head of the monster is the technical discipline program officers who work with us. They are the technical coordinators who review the proposals and who make recommendations on your proposal. They care about their discipline-focused goals and strategies. MRI is a program with strategic goals that are bigger than one technical discipline; while each Directorate/Office at NSF participates to serve their own communities, an overarching structure is maintained to provide consistent implementation of the program. Get to know us in OIA, as well as the technical program officer. In OIA we own the strategy on behalf of the director and we kind of own the rules of the road to make sure strategies are enabled. We do not review your proposals. We direct them into the correct programs to be reviewed, and the program officers with the technical knowledge coordinate the reviews.

The first strategic MRI goal is to support the acquisition or development of major instrumentation for disciplinary/inter-disciplinary shared use by the nation's scientists, engineers, undergraduate/graduate students, researchers, etc. Most often acquisition involves “plug and play,” where a vendor provides a quote for an already-fabricated instrument. Sometimes the instrument doesn't exist, so we have a development track in MRI whereby an instrument is designed and built to provide new capabilities. The strategic goal is not only to have an advanced research instrument, but also to train the next generation of experimentalists to be able to do research and keep the instrument running, and, in the case of development, to train the next generation of instrument builders.

Enabling academic departments to integrate research and education is probably the most misunderstood goal of all. People take those words about research training and think of taking the instrument in the classroom to instruct the students on how to use the instrument, to show what it can do, to get students interested in using it, and to prepare them to do research. The integration we are talking about is you take the student out of the class and put them into the lab to actually do research. That is our definition of research training. If you look at our solicitation, research training is taking the student out of the classroom and into the laboratory; an MRI proposal, if it supports integration of research and education, means getting real research occurring in an educational environment. It does not mean taking an instrument into the classroom for instructional purposes. That's allowed, but if that's your primary reason, you are discussing a different kind of impact (a “Broader Impact”). We see proposals coming in saying they will use the instrument in this class and that class, and that's fine, but it is not what we mean by research training. We want to see you get those guys out of the classroom and do real hypothesis-driven research using the same tools that you would use in your independent research. That's what research training means to us. They're doing research, not necessarily independently, but under your guidance.

There are other goals. The first is cyberinfrastructure; this applies to every discipline. OIA's sister organization is the Office of Cyber Infrastructure (OCI), which is an organization within the Office of the Director that also supports NSF-wide cyberinfrastructure activities. We promote OCI-related activities in our program because it is an NSF-wide initiative to bring cyberinfrastructure into science and engineering. Going back to what I said about preparing the next generation of experimentalists, another goal is the involvement of commercial partners in development efforts to better enable potential commercialization of novel instrumentation. We encourage partnerships with private industry, but we don't want to see a “custom acquisition” whereby development activities are outsourced to the commercial partner; we don't want to see you write the specifications for something that doesn't exist, give it to a company, and let them build it and bring it back. We want to see you and your institution actively involved in the decision of how to build the thing, what theories and specifications are needed, and what concepts are used so that you and your institution become better at developing and building instruments. We want to see you teach your students to do this so they become the next generation of instrument builders. In short, “custom acquisitions”—not interested.

Okay, so what do we not support? MRI is a partnership with the institutions. We do one part, you need to do the other part. You need to have the people, the building, the electricity, and some ancillary research equipment that is in the lab and ready to go. We provide the big expensive piece of instrumentation that is too expensive or inappropriate to obtain any other way. We support an instrument that is a major research instrument, but we are not supporting infrastructure, early phase research that is needed to decide on a development pathway, or the research, education, and outreach activities that are enabled by the instrument. Also, if you are going to do technology development, albeit really expensive, for millions of dollars of development work to come up with a $10 sensor, that's technology development that leads to a product, which we don't support. We do major capital equipment as the end product.

So here are the basics. There are some restrictions. Only three proposals can be submitted by an institution to any one competition. If three are submitted, at least one has to be a development proposal. If there are no development proposals from the institution, only two may be submitted. Someone on your campus needs to know this, and many institutions have internal competitions to decide which proposals can be submitted. There is cost sharing, and cost sharing is not “matching.” For MRI cost sharing is 30% of the total project cost. This really means for every dollar NSF gives you, you have to contribute 43 cents or so. That is a 30% cost share. The wrong way to think of cost share is to say, if I'm going to get this money from NSF I'm going to do some complementary something over here and count that as cost sharing. No! You need to think as follows: for the allowable duration of my project, and based on eligible costs I can ask for as described in the MRI solicitation, what is this instrument going to cost? The total cost. First figure that out. Note this is just the cost to get it on the floor and make it available. We pay 70% and you pay 30%, if you are required to cost share. It's that simple. If it's not eligible to be paid for by NSF, it's not eligible to be cost shared either. Think of it in terms of total costs to get the instrument out of the vendor, plugged in, on the floor, and available for the maximum term of the award. Again, we'll pay 70%, you pay 30%. Some institutions are exempt from cost sharing. Academic institutions that award no or only a few Ph.D.s aren't required to cost share—see the MRI solicitation for details. (By the way, we set the strategic goals, but Congress sets this one. We have no control over these cost-sharing rules.)

Merit review consists of both an external reviewer component and review by a program officer. Remember that. When you look at your reviews remember that is only the external-reviewed part of merit review. You need to talk to your program officer if you need more information on how the final award/decline decision was made. The reviews often speak for themselves and show your weaknesses. But they also might be confusing or you may not be sure what they mean. Remember, external review is only one part of the merit review process. The program officer, in the context of NSF goals and their technical disciplinary goals, will look at the proposals and judge which proposals have the most merit from their portfolio perspective.

Remember there are restrictions on the types of organizations that can apply. We don't want commercial, for-profit companies to end up owning taxpayer-supported instruments: so they can't submit. Institutions of higher learning, which I think most of you are from, can almost certainly apply, as can many non-profits. The solicitation provides guidance you need to understand. MRI allows consortia to apply. A consortium specifically means joint ownership. A legally incorporated consortium can ask for instruments because they've already done all the paperwork. You can also band together as an informal, non-legal, binding consortium and ask for an instrument, but ownership typically resides at a submission-eligible consortium member. Look at the solicitation. It tells you how to follow the rules to construct an MRI-defined consortium. The Management Plan within your proposal is the place to describe how the consortium will be structured and managed and outlines the roles and responsibilities.

We are actually finishing up the current MRI competition, and my colleague Randy Phelps is probably swamped, because Friday is the last day for him to sign off on the award recommendations for this year. That said, if you have submitted an MRI proposal and don't hear about your proposal by Friday, then that is not good news. The bright side is that we will soon start the whole procedure again. [Editor's note: The next competition deadline is expected to be in January of 2014.] The solicitation will be out 90 days before that. We expect $90 million, but that is not yet established. For your information, the average size of an award is about $600,000 per. You may ask from NSF anywhere from $100,000 to $4 million. (This range is set by Congress.) There are exceptions for particular cases, so check out the solicitation. We get nearly 1,000 proposals and end up awarding 150 or 170. So it is an 18–19% success rate. Historically it is getting smaller because the number of proposals is staying the same, but the average request size is going up and our money is staying the same.

As a preview to the discussion later, I will tell you two simple things. First, I will tell you how a proposal may fail before it is even reviewed. That is “return without review,” and this is basically caused by not following the rules. You might have not followed the rules in the past and gotten away with it, but it is just like traffic. If you run a red light and you don't get caught, then good for you. Next time you might. Every year we have areas of emphasis, and I'm not really supposed to tell you what those areas of emphasis would be (but since another speaker happened to mention that one, pay attention to small fonts). Secondly, you probably would like to know a proposal fails during the review. A simple statement: your proposal has strengths and weaknesses and when your weaknesses completely overwhelm any strengths, you've failed because the weaknesses become fatal. In general weak proposals fail during the review when they are not clear, do not quantify things, leave things open for obvious questions, and/or leave unanswered big questions. A big mistake is to mention things you are going to do but then not say how you are going to do them. You've got to be clear and fully justify everything you are doing. When you are confusing or leave holes, those are weakness that will rise up to the degree to be fatal. Your number of strengths will not matter. A simple takeaway is to put yourself in the place of a reviewer. If you ask a question about your proposal, so will the reviewer.

So what makes a proposal successful? This is a hard question. I can tell you what not to do, but I can't easily say what to do. I'll tell you the last little questions of dualism. We can talk about this in the discussion, and I can go on for a long time about this. I can tell you when you are not in Nashville, but I can't tell you whether that's good or not. Stated differently, I can tell you don't be in Nashville (but why would I want to?!); that's what I just told you, but the opposite of being in Nashville is an undiscovered country with, many, many possibilities that depend on your intuitional situation and what directorate and culture you're in. I can't possibly tell you where to be. I can just tell you not to be in Nashville. So we can talk in the discussion about how best to do that, but it comes down to knowing the program, knowing your discipline, and knowing your program officer to figure out, if you're not supposed to be in Nashville, where you should be.

Question to all. A question I think everyone might be interested in. Transmission electron microscopes, in particular those with aberration correctors, are getting to a price range that is pushing the upper limits of what all of you have discussed. Are there any plans you can share with us about how to deal with this situation? If you take a shared instrumentation grant, there's a maximum of $600K. You almost cannot buy a TEM for $600K, even a basic 120 kV. When we talk about high-end equipment, how do we convince the funding agencies that a standard 120 kV TEM with a decent digital camera is high-end equipment? In terms of science, there could be an argument that they may never fund the basic TEM.

Answer NSF. For MRI, we have a little more cap than the $4M might suggest, simply because if you're required to cost share, then the total value of a project that we would fund could be $5.6 or $5.7M with us putting $4M in it, so we still have a little bit of head room on aberration-corrected TEMs and things like that. NSF is looking into the question of mid-scale research, facilities, and equipment that costs more than can be funded with the MRI program. It's how you get the $10M, $20M, $30M dollar thing, whether it be a full-up facility or a very expensive piece of equipment to fill in the gap between MRI and the major research, equipment, and facilities that produces the major telescopes that are tens and hundreds of millions of dollars. That discussion is going on; look for opportunities for academic and non-profit organizations to have input into that. MRI also has a congressionally mandated trigger; if we get more than $125 million, our cap is moved to $6 million by legislation. This will automatically trigger a new cap by law.

Answer NIH. At NIH, probably the $2M for the HEI is going to stay, but I see a lot of applications, for example, biomedical imagers, very expensive, that are $5–6 million. But because sometimes they get very good discussions going between the agency and the institution, they are willing to pitch in the difference. If you get funded and the institution is willing to pay the difference, you will be able to purchase the equipment. Some applications work out this way and get things going, or else institutions won't purchase that kind of equipment for you.

Question to NSF. Regarding the institutional commitment, can you clarify what you are looking for? Because there is no cost share required, what is the situation you allude to such that if the institution does commit to some money, it is a bonus in the review?

Answer NSF. Voluntarily committed cost sharing is prohibited by the National Science Board. You can only cost share the 30% if you are required to. No more, no less. The reviewers won't ask how much an institution can cough up because it leaves smaller institutions behind. They just don't have the resources to give that kind of support. You need institutional commitment to make sure the equipment will be maintained after it is installed, and that is not covered by this funding mechanism. This will make it a stronger application. Reviewers do want to see a letter of commitment attached.

Question to all. Why did you decide to limit resubmissions to two?

Answer NIH. This is NIH policy. In terms of research, if you get it the second time, they want you to think in another direction to fix the problem. This doesn't apply to the instrumentation, so we have been working to help people bypass this hurdle. We encourage making a new application.

Answer NSF. I'm rotating out of NSF soon. This resubmission issue does not currently apply for MRI, but there is some discussion of imposing it. We sometimes see the same thing over and over, and it's never been fixed. NSF has a hard time dealing with 55,000 proposals a year. It's not efficient to review the same thing over and over again, especially if it's flawed. We spend a good portion of our time talking to communities about outreach and resubmission. We have the conversations and point out that it's a competition each time within the MRI program. You can get better reviews but still not get funded. Also, we don't officially allow any resubmissions. It has to be a new submission, but it can have the same title and PI. It has to address the feedback to count as a new submission. The program officers are aware of having reviewed something because we have a lot of specializations. The one weakness that reviewers are hesitant to address or clearly say is: “This proposal has no weaknesses, but it has no strengths.” The lack of strengths is its weakness. It's the fundamental weakness. It can be written perfectly and clearly but doesn't get the reviewer excited. Need is not enough; all of them have need. You have to be meritorious. Ask yourself, “Is it because my proposal has no strengths?” It could be a weakness to one reviewer, and there can be conflicting reviews, but there is really no reason to fund it. Convey an excitement. Tell a story, a compelling story, write a novel. If you follow all the rules, make no misuse of words, and are clear with the meaning of every single word, you create a dictionary. When a person creates a beautiful novel, it will outweigh the spelling mistakes. Don't write a dictionary, write a novel. Just like a novel, what plays in western literature doesn't play in Asian literature. I can't tell you what novel to write. It depends on your institutional situation. When you get the comments, read them, digest them, take a deep breath, and ask yourself, is it because there are no strengths? If you honestly look at it, you'll find it. You can't fault the reviewers; they can't read your mind. Don't fail to convey information.

Question to NSF or NIH. We have a situation where MRI is the major, really the only, program in order to get major equipment, but for many of us, major equipment isn't the $600k or $1 million piece of equipment, but a $200,000 piece of equipment. When there are only two proposals going out from an institution, it's very unlikely that they will waste one on a smaller piece of equipment. There used to be a program that was specifically geared to the small pieces of equipment that were in the $40–200K range. Is there any discussion of bringing something like that back where that was not part of a maximum submission by an institution? You could have as many of those as you wanted from an institution.

Answer NSF. MRI is not the only instrumentation program at NSF. A lot of programs have very different spaces and cost ranges. Besides those strategic goals, MRI is about funding major instrumentation that is too expensive for other programs to fund. There are other programs that do exist. We have geo scientists, chemistry, vision, bio, field stations—so there are programs. They are listed at the end of the MRI solicitation. I know of no comprehensive solution to the problem being discussed. The conversations are mostly on the other side of that discussion in terms of larger needs. Bring this lower gap to the attention of NSF and program officers, but first look at all avenues of NSF programs, not just MRI, and discuss with your program officers about using a research grant to get some of the equipment.