Appendix C Lessons Learned
ALMA is by no means the last international science mega-project to be built by astronomers. In radio astronomy alone we have the Next Generation Very Large Array (ngVLA) being developed by NRAO, and the Square Kilometer Array Observatory (SKAO), approved for construction in Australia and South Africa, with operation planned for the late 2020s. At optical wavelengths, we find the Thirty Meter Telescope (TMT), Giant Magellan Telescope (GMT), and Extremely Large Telescope (ELT), all at varying stages of development and construction. It is worth asking if there are any lessons to be learned from the ALMA experience that should be passed along. The authors have asked past ALMA project managers and directors this question. The synthesis that follows comes primarily from the response by Tony Beasley, the ALMA project manager who called for re-baselining the project at the start of his tenure and managed much of the construction. Comments in the list are also based on conversations with and material1 from others with experience in ALMA management.
Carry on in the face of disappointment. The loss of the 25 Meter Project was a bitter defeat, but it led to the MMA. Subsequently, the long delay in funding the MMA was discouraging and frustrating. But consider the alternative: if the NSF had funded the MMA in the early 1990s, NRAO would now be in possession of a facility hardly worth operating. During the delay, the concept grew to become ALMA, something vastly more powerful.
Recognize that politics will have a role in the funding of big projects. This is simply a consequence of democratic government; the taxpayers are entitled to a voice and the support of their representatives is essential to a project’s success.
Do not underestimate the importance of cultural differences. Indifference to cultural differences causes problems that cost time and money to correct, as well as missed opportunities by not recognizing the value that each partner uniquely brings to the collaboration. Goodwill is helpful but not sufficient. Some differences are deeply embedded and cannot be changed. For example, the process of soliciting bids and selecting contractors is very different in Europe, the United States, and Japan, as are the terms and conditions of contracts. The processes can take different amounts of time. Planning is required to take all of this into account. Patience is certainly an asset. Social behaviors, even in the workplace, also differ from culture to culture, and being conscientious of such differences helps to avoid giving offense. Even something seemingly as straightforward as time zones requires extra effort, for example, scheduling worldwide virtual meetings so that the partners take turns attending at three in the morning. Remember that it always sounds worse in writing than you intended it to be. Despite steadily improving technology to connect the world, face-to-face meetings are often necessary. It certainly was the case with ALMA that in-person interactions were better.
Cultural Differences
In February 2003 the Bilateral ALMA Agreement between North America and Europe was signed. The board chair would rotate every two years and Europe would start. In December 2002, I was elected president of ESO Council and thus became the first chairman of the ALMA Board. After two years, Bob Dickman took over, and I was vice-chairman for another year until my term of president of ESO Council ended.
It was a fascinating, but very busy and definitely tumultuous period. What made it special was that Europe and North America were equal partners. The United States was clearly not accustomed to being in that role, but for Europe, being on equal terms with the United States in major scientific undertakings was also unusual. In Europe, submillimeter astronomy was an extension of optical/infrared to longer wavelengths, while in America it was radioastronomy at higher frequencies. Our communities had different backgrounds, but that did not cause the worst headaches. Often practical, but important matters differed: funding cycles, terms of employment, or project management.
Problems took time to solve and too often compromising was seen as losing face. Early on the Board had to choose an ALMA logo. The design (actually adopted) showed some dishes with four stars arranged as the Southern Cross. The American side at first felt it looked too much like ESO’s logo, which also contains that constellation. Then there was the location of the Santiago offices, for which the Vitacura premises of ESO were an obvious option, but only acceptable to the United States if the address would be a different street. These, to Europeans, unimportant issues were resolved but took much time and emotional energy.
There were clashes between personalities, sometimes emotions running very high. No individual, institution, or organization was fully in charge. Persons that had worked hard to realize a millimeter array, saw it develop differently from what they had had in mind. Shaping the collaboration with Chile required diplomacy. We had to deal with cost overruns, deciding on antenna designs, and formalizing the collaboration with Japan into a partnership with yet another culture. The latter made us all minority partners.
Incredibly, in the end every problem was overcome, even the apparently unsolvable ones. It was a great experience working with dedicated persons; I am extremely grateful ALMA came to be and to have been part of it.
Effective professional personnel management is critical. Large projects like ALMA have diverse workforces. The employees have a large range of skills and abilities. Effective personnel management requires a dedicated, professional human resources team if the employees are to be treated fairly. Poor workforce morale can be deadly to a project, or at least imply lower efficiency and potentially slowing progress. Employees should be held to a high standard of behavior if they are to work together effectively. Bad behavior should not be tolerated, even in geniuses.
Expect deliveries to be late. It is inevitable that some, perhaps most, deliveries will be late. Contingency plans should be made with this in mind.
Invest heavily in project management, project engineering, and system engineering. The ALMA project discovered rather late that it needed far more systems engineering than had been planned. NRAO had in past projects left systems engineering to a few talented individuals who mainly conversed with one another and produced modest amounts of documentation. This was inadequate for ALMA. ESO had experience in large project management with the VLT, where they relied on massive amounts of documentation to tie the elements of the project together. Reconciling the two approaches took time. Robust document management software with a top-notch search feature greatly facilitates systems engineering.
Remember that there is no substitute for experience – seek knowledge elsewhere. Construction of a complex facility in a high-elevation remote location will encounter unusual situations and questions. It is a mistake to rely on your own staff to solve every knotty problem. The world is full of experts and there is sure to be one with experience in the most arcane area you encounter. Their advice is worth the cost. ALMA learned high-altitude safety standards from the mining industry and high-altitude medicine from Dr. John B. West, a world expert on the subject.
Make decisions promptly, focusing on the big issues and watching risks. Delay leads to increased costs. It is a mistake to put off a decision until every detail has been examined.
Central control of budget and decision-making is critical. ALMA has a flawed management structure. The central office in Chile responsible for ALMA construction and operations is not a legal entity. All purchases and contracts are made by the ALMA Executives: ESO, NRAO, and NAOJ. It required a director and project manager, who were not only capable directors and managers but possessed the diplomatic skills to secure the support of the Executives for their plans and needs. The structure is cumbersome rather than efficient. That it worked at all, and it did, is a testament to the skill, hard work, and goodwill of those involved.
Accept that some decisions will be made politically. No project the size of ALMA can avoid politics. National interests will be asserted and force some decisions. Learn to live with it.
Recognize that the information volume is too high for good external reviews and reporting. No group of experts, however experienced they may be, can fly in for a three-day project review and grasp all that needs to be understood. High-level reviews are essential to maintain commitments to funding, and as such are unavoidable. One can only hope that a review will focus on the high points and issue an endorsement of the project. Accept that detailed advice might miss the mark.
Funding by the partners should be synchronized. Funding for project construction or operation should arrive from all the partners at the same time. Asynchronous funding requires work arounds that can be difficult to achieve. The delay in the availability of funds from the Japanese government greatly complicated the entrance of Japan into ALMA. It was the goodwill and persistence of all those involved that led to success, but at a significant price in time and effort.
Always keep in mind that, in the end, it’s all about the science! There will be many occasions – dealing with difficult management, administration, human resources, procurement, and funding issues – that will be extremely frustrating and demoralizing. One of senior management’s primary responsibilities is to always exhibit positive leadership to the staff no matter the circumstances, reminding them of the end game, because, “It is all about the science!”
The management of big science projects presents substantial scientific, engineering, informational, and human resource challenges. When multiple institutions are involved, these challenges become even more complex; when the institutions are from various countries and regions of the world, the challenges can be overwhelming. Like people, organizations have their own personalities, habits, and idiosyncrasies. Those participating in ALMA were no exception. NAOJ is a governmental organization, ESO a treaty-based entity, and AUI a private corporation. None of the Executives shared any common legal basis. These very different legal regimes, in turn, gave rise to very different policies and procedures relating to contracting, human resources, scientific staff, and public outreach.
A successful ALMA project required people to work at a level outside their comfort limits and to effect fundamental changes within their organization. To this end, ALMA was most fortunate. While ALMA certainly did not lack for its share of contentious issues, a core of individuals within each ALMA Executive stepped up to develop creative solutions to seemingly insurmountable problems. These individuals developed effective relations with their counterparts, resulting in a reservoir of personal trust and goodwill which would often sustain them during difficult times. Early in ALMA’s existence these small cores of managers recognized that the worst place to thrash out issues were high level meetings, such as the ALMA Board, where protecting an organization’s parochial interests was part of the meeting dynamics. Prior to such meetings, people would meet informally, with their counterparts in the other Executives, to explore where the real boundary conditions were so that options for solutions to difficult issues could be devised prior to the actual meetings. Backchanneling made meetings more efficient and often avoided what otherwise might have been embarrassing situations. Formalized agreements were achieved that could not have been envisioned as even possible had people not challenged themselves and their respective organizations to break out of their normal comfort zones.
Certainly, having to meet a construction schedule helped to spur people into actions and risks that they might otherwise not have taken. But without the right set of people in the right places, ALMA’s eventual success would have been much riskier. In the operations phase of big projects, the need for a core team of creative thinkers willing to go outside of their personal comfort zone and push against their own organization’s normal reflexes continues to be essential. Unfortunately, the need for these qualities often goes unrecognized and under-appreciated.
Note
1 A memorandum, Lessons Learned from ALMA, by E.J. Schreier and J. Webber, September 2010, is an accounting of the reasons behind the cost increases revealed in the re-baselining of the project. It can be found at NAA-PVB, ALMA, ALMA: The Story of a Science Mega-Project. https://science.nrao.edu/about/publications/alma.
Open access