by Barry Fulton, Ph.D.
SMART – the acronym for State Messaging and Archive Retrieval Toolset – is a new suite of applications developed to enhance communication within the Department of State. SMART integrates cables and e-mail on Outlook and automatically archives messages of lasting value to a searchable database. The author is a retired Foreign Service Officer who is serving as a senior management consultant on the SMART program.-The Editor
BRIEF HISTORY
“Success has many fathers, but failure is an orphan.” As SMART has passed the tipping point with successful deployment in more than 40 posts, its parentage can be traced to three studies and a digitally-comfortable Secretary of State. In late 1998, both CSIS and the Stimpson Center released studies recommending that the State Department modernize its information technology (IT). Within a year, a third study conducted by the State Department endorsed the CSIS and Stimpson studies by calling for increased resources for information technology.
When Secretary Colin Powell took office, he moved expeditiously to seek funding for modern technology to support America’s diplomats. Congress agreed, and the Department undertook several initiatives. First of all, it provided all employees an access to the Internet through its unclassified network, known as OpenNet. Secondly, the classified network, ClassNet, was substantially upgraded to provide better connectivity around the world. Both of these projects required a substantial investment in hardware and network capability, but little or no interaction with general users. The office of eDiplomacy was launched to introduce emerging technologies to the Department of State. And, after an aborted attempt to integrate several collaborative applications, the SMART office was established to integrate “legacy” messaging systems with off-the-shelf IT applications.
To appreciate the complexity of the program, consider that the Department processes more than one million cables every year – and that the ratio of e-mail to cables is 800 to one. Every cable is distributed based on employees profiles, simultaneously stored in the Department’s archive, and – if warranted by the nature of the cable – transferred to the National Archives after 25 years. Every e-mail, on the other hand, goes to named addressees – and faces the imminent danger of extinction. As one senior officer recently observed, “you mean they are not saved; some of my best writing has disappeared.” Notwithstanding a Department directive to print and save e-mails, there are relatively few who are fully compliant. SMART has been designed to capture not only cables, but also e-mails of lasting value.
In addition to its internal messaging requirements, the State Department legacy cable system interacts with some 60 departments and government agencies. So, developers and testers had to ensure that every message to or from another agency would be translated into the SMART format. This would prove to be one of the most demanding technical challenges.
Equally demanding was the requirement that unclassified messages could be sent between the classified and unclassified networks. The SMART team discovered an application developed for the Air Force and approved by NSA that would allow this transfer between networks — if the vendor could customize it to ensure guaranteed message delivery. The product has been tested, installed, and is currently operational in SMART — the first known application in the Federal Government to routinely allow messages to pass securely between enclaves and ensure message delivery.
STATE DEPARTMENT GOVERNANCE
This article will chronicle the path from conception to production of SMART, designed as the Department envisioned in 2003: A simple, secure and user-driven system to support the conduct of diplomacy through modern messaging, dynamic archiving and information sharing. Because it was conceived and designed to enhance the messaging experience of general users, a senior Steering Committee of some 20 Assistant and Deputy Assistant Secretaries was formed to establish requirements and monitor progress. The Committee was chaired by retired Ambassador Joe Lake, whose career as a political officer was technologically enhanced by senior assignments in the Executive Secretariat and the Office of Information Resource Management.
One of the first tasks of the Steering Committee was to seek an inventory of messaging and archiving systems in other government departments, the private sector, and major foreign ministries. After several months of interviewing and witnessing demonstrations, the Department inventoried numerous best practices in government and the private sector – but found none that successfully integrated command-and-control messaging and archiving across two proprietary unclassified and classified enclaves. If there was no pot of gold at the end of this rainbow, there were inspired solutions to similar problems that helped the Department establish its requirements.
After developing a proof of concept and conducing user testing internally, the Department elected to invite private sector bids on a four-phase program – including designing, piloting, deploying, and managing the system. The Department sought a corporation with extensive IT experience to deliver a system that would integrate traditional cables and e-mails, centrally archive the messages, and make them available in real time through a commercial search engine.
The contract was awarded to a major defense contractor, which labored under the close eye of a small team reporting to Under Secretary for Management Grant Green. Both the contractor and the State Department came to realize that the contractual timeframe for building a new system to accommodate State’s messaging requirements was impossible to meet. Why? Because the new technologies had to be seamlessly integrated with legacy systems that were still operating on DOD standards established in World War II. There was no proven or easy means to link the old and the new, a necessity until all internal and external cable users were converted to SMART.
After several delays and a resource outlay by the contractor that far exceeded the earned payment from the government, both parties decided to terminate the contract at the end of Phase 1. Although a functional system had not been delivered, the Department took ownership of equipment procured by the contractor and prepared itself to go forward with a host of lessons learned. State then urged OMB to give it the green light to continue building the system under its own management. OMB rightly insisted on an independent survey to demonstrate that the Department could successfully undertake management of a complex program that a major contractor had failed to deliver. After several months of a rigorous audit of the Department’s managerial and technical expertise, the Gartner Group recommended to OMB that the funds to continue the program should be released.
So, the Department of State assumed full responsibility for designing, testing, piloting, and implementing the SMART program. With the clock running, the kickoff date was December 2006. Glen Johnson, a SES manager with years of experience in Information Technology was tapped to manage the program. His deputy was a senior Foreign Service officer with rich experience in overseas operations. Division chiefs were a mix of GS and FS. And scores of contract developers and testers from some 12 different firms were hired to work in a makeshift laboratory ten miles south of the Department in Newington, Virginia.
The Steering Committee had been meeting monthly since the program was conceived. Three years had passed, and still most attended every meeting, understanding the complexity of the challenge and the need for the Department to offer employees the best tools for reporting and communicating. There was an obvious sense of pride that the Department was found to have the talent to design and deploy a program of such complexity. The previous delays, of course, were frustrating – as were some yet to come under the Department’s management.
USER INTERACTION
As SMART was developed “user-driven” became more than a throwaway term. From the first proof of concept through the last of 16 posts involved in piloting, user reaction was carefully monitored to ensure user satisfaction. There are four reasons that users might not like a new product: it doesn’t have the desired features; it‘s cumbersome to use; it has defects; and it’s different.
User reaction has varied from a pained query of “Why are we abandoning cables” to an enthusiastic “I love it.” The first iteration at the early pilot posts provoked numerous requests for changes from terminology to functionality. For example, when the SMART team initially decided to abandon the term “cable,” some senior overseas officers charged the program with heresy for ignoring tradition. After the issue was thoroughly debated, the term cable was retained, but not in the appearance, format, or technology that are currently associated with that term.
User reaction to early iterations also revealed dissatisfaction with the customized Outlook screen. And to compound user frustration, the Department upgraded Outlook from version 2003 to the 2007, with changes that were independent of SMART. However, the new Outlook client afforded the SMART developers the opportunity for a customized product that addressed the usability issues. Pilot users have increasingly expressed satisfaction with the ease of use of the new SMART customized Outlook form.
A host of other issues came to light, some of which surprised the SMART developers: What are the standards for classifying a message? For marking it SBU (Sensitive but Unclassified)? For satisfying the requirements of the Privacy Act? The brief answer: the standards have not been uniformly trained or understood. Furthermore, new Federal rules are in the works on classification and sensitivity markings – and privacy concerns are increasing. SMART has tiptoed through the current and evolving standards to build a user-friendly product, and with a few exceptions has largely succeeded.
Is SMART free of defects? Well, no. There are few IT applications that don’t have known defects, particularly in the early years of release. In the first pilots, users found that some military addresses could not be found, that printing under some conditions was troublesome, that cables carefully formatted by the senders arrived with extra spacing or awkward formatting. There were, as well, defects invisible to general users, which initially increased the workload for technicians responsible for managing the system. As time has passed and the defects have been recognized, many have been satisfactorily resolved. And, too, user acceptance has improved.
The final frontier is user resistance to change. A few of us are just plain stubborn, and are not prepared to adopt anything new. But, resistant comes in many forms. Some of our colleagues are so busy – 10 or 12-hour days to work the visa line or complete a reporting cable – that the time required for training and mastery of a new system must be carved out of scarce personal hours. Others, who may suffer from a touch of technophobia, worry that they won’t be able to easily navigate a new system, having struggled with what they currently have on their desktop. Still others feel obliged to respond to a supervisor whose excessive enthusiasm for the new or defense of the status quo challenges their orderly adoption of a new process. SMART has seen all of these – and over time, most of the resistance has vanished. There remain, of course, a few holdouts.
DESIGN, DEVELOPMENT AND TESTING
“Why has it taken so long – it’s just Outlook, isn’t it?” That’s what one curious colleague insisted on knowing. Even if this paper doesn’t convincingly answer that, it should set you on the path to further discovery – and give you an inside view of a process that has involved, directly or indirectly thousands of State Department employees and the integration of eighteen different products. Outlook is the user interface in front of the most complex IT system developed by the Department of State.
SMART started with the vision, as described earlier in the paper, conducted an exhaustive external survey of available products, consulted with industry leaders, established some 40 high-level requirements through the work of a senior Steering Committee – and then turned the requirements over to design and development engineers. The system not only had to interact reliably with the legacy cable system (to be phased out when SMART is completely installed), but also must satisfy all Federal requirements for security and records management. On top of that, it must serve the interests of the 46,000 American and national employees who use the Department’s OpenNet and ClassNet services.
Whether the requirements are given to an outside contractor (as SMART was originally conceived) or given to State Department FS and GS employees to manage (as it eventually was), the processes are similar. High-level requirements are translated into detailed requirements, and dependencies among them are identified. Design documents are drawn up, from which schedules and budgets are produced.
A staff is assembled, contractors and consultants are hired, physical space is located, and hardware and software licenses are purchased. Only then can the demanding work of development, testing, piloting, and deployment begin.
Software developers on the SMART program are a mix of fulltime employees and consultants, some with decades of State Department experience, others who are barely out of college. They include those who have mastered the cable system and others who have never heard of a cable. They became, over a period of several months, a team dedicated to a single goal: building a user-friendly system to enhance the conduct of diplomacy.
SMART adopted a methodology knows as agile development, in which developers strive to write code for a new “build” (i.e., enhancements of the previous day’s product) every day. The developers run a “smoke test” (language borrowed from an earlier day when deficient products went up in smoke), and if it passes muster, they turn it over to a Test and Implementation team.
Testers next do their best to break the product, that is, to find any shortcomings measured against the technical, functional, performance, and usability requirements. Most days, they succeed in identifying defects — and return the product to the developers for another build. This iterative cycle continues until the defects with the highest severity have been eliminated and the others have been reduced to a manageable number. (The elusive goal of zero defects is never met — but such is life.)
Then, the product is connected to the live network for Quality Assurance testing to confirm that it operates as designed on the network on which it will be deployed. Validating that, SMART goes in to production and is made available to selected users. In the early stages of the SMART process, pilot posts agreed to accept and critique a product that was still somewhat less than mature.
The reactions of pilot users — positive, negative, or indifferent — were then summarized and subjected to an internal review to correct newly discovered defects or to add desired functionality. And so it went on and on — with the promise that each iteration would be an improvement on the former.
How did this process differ from a development process at Microsoft or Google or Intel? The SMART staff of some 120 full- and part-time employees and contractors is much smaller and less experienced in product development, but the processes used were industry best practices. Indeed, it was Microsoft engineers who guided the process. And it was veteran State Department Foreign Service and Civil Service employees who guided the developers and testers to success.
SMART FUNCTIONALITY
When SMART is fully deployed, cables as they have been known will disappear. The new cable format will be more readable and allow tables, charts, and graphics to be included. For e-mail users, this is hardly an innovation – but e-mails are limited to named addressees and unavailable to other colleagues or to history. The SMART cable from an Economic Officer with a spreadsheet and a graph will be immediately disseminated and available to all other SMART users through a search of the Archive.
What about e-mail? The day-to-day non-substantive, non-policy business of the Department of State will continue to be conducted by e-mail. In addition, SMART is introducing a hybrid between working e-mail and cables called record e-mail. Like cables, record e-mails are deposited in the archive for search and sharing. Like working e-mails, they are addressed by name to individuals or mailing lists.
The State Department has worked closely with the National Archives and Record Administration (NARA) for the past several years, not only to ensure compliance with Federal standards, but also to blaze a trail for electronic records management that may be adopted across the Federal government. Complementing the SMART messaging system are several search options for users. The first, available to both OpenNet and ClassNet users, is a Microsoft search engine. The second application allows users to freeze a search and receive a daily alert for new messages (cables and record e-mail) that satisfy their search criteria. And the third is the Google search engine, being deployed initially on OpenNet.
That’s not all. To enable sharing and collaboration, SMART also configured, tested, and deployed a product called Communicator, the Microsoft version of Instant Messaging. It allows instant communication within an office or among those working on a common project. Complementing this application was a collaborative tool called SharePoint, which was offered as an option to State Department offices. Although not yet widely used, it provides an efficient means for short-term document storage and collaboration, particularly on a large-scale project.
To summarize, SMART operates on a platform familiar to all State Department employees: Outlook. It integrates cable and e-mail. It offers real-time search options. It provides unclassified connectivity between OpenNet and ClassNet. It supports mobile communication from OpenNet.
Furthermore, it satisfies all security and records management requirements. And it puts the Department of State in the forefront of the digital revolution by providing state-of-the-art tools to employees. In short, SMART offers a combination of messaging, collaboration, and archiving applications on unclassified and classified networks not currently available in any other Federal agency.
FUTURE CONSEQUENCES FOR DIPLOMACY
SMART, of course, is not a panacea for well-established diplomatic practices. It will not replace keen observation, sound analysis, or skilled drafting. For a few who may be tempted to add extraneous graphics to their reporting, SMART might even distract from good writing. However, as the new toolset is deployed, both habit and caution should ensure that the standards of reporting are maintained. When complementary to the substance of a message, drafters will be encouraged to add supplemental documents (Word, Excel, PowerPoint), multimedia (photos, audio, or video), and hyperlinks to existing records in the Archive.
As officers who have grown up with information technology join the Foreign Service, they will inevitably take full advantage of the new SMART tools. The tradition of vertical communication supported by the cable culture will be further eroded through the SMART collaboration tools including instant messaging, SharePoint, and search. The typical message sent from drafter to clearers to approver to a handful in Washington will be just as easily available to one’s colleagues around the world.
If you are among those who fear that the flood of new cables will overwhelm you, SMART can ratchet down your dissemination to the messages you must have to do your job. On the other hand, if something is missing, you can search the archive with as much ease as you now conduct a search on the Internet.
With the ease with which information will move horizontally, SMART invites rich collaboration among policy specialists across offices, bureaus, and borders. The cable culture that has protected information will give rise to a culture that honors sharing. Indeed, one of the precepts of SMART was to provide post-9/11 tools to facilitate a shift from a culture of “need to know” to one of “need to share.”
The Department of State was among the first to articulate this shift, which, several years later, was recognized throughout the Federal Government in a White House directive establishing the Information Sharing Environment.
The worldwide deployment of SMART will be more than 20 percent completed by the end of 2009. If the current pace of deployment is maintained – one post per day – every Embassy and office will be using SMART before the end of 2010.
It is not too early to think beyond SMART, as new technologies become available. Virtualization of the SMART system is already being planned for a second site to ensure Continuity of Operations. Cloud computing is on the horizon. Internet 2 offers nearly unlimited bandwidth. Integrating GPS cannot be far away. And shouldn’t we consider a SMART interface for the iPhone? In another decade, we will be well past the current Twitter, Google, and SMART applications to provide technology to better connect diplomats with relevant information. By then, even SMART cables may be regarded as historical artifacts or contemporary anomalies.
How will the introduction of SMART and other new technologies affect the conduct of diplomacy? Looking backward provides evidence that changes are unpredictable. David Nickles, a State Department historian, has examined the introduction of the telegraph in his book Under the Wire: How the Telegraph Changed Diplomacy.
Change began with the first encrypted cable sent to Paris by Secretary Seward in 1866, resulting in a cost of $19,540.50 – three times Seward’s annual salary! Adoption was understandably slow. Even though costs dropped dramatically, it wasn’t until World War I that the Department accepted cable traffic as a normal means of communication. Diplomats, used to relative autonomy, resisted, ignored, and occasionally sabotaged the use of cables. Nickles attributes their resistance to cultural inertia. Eventually, the authority of Ambassadors was circumscribed, as decision-making shifted to Washington. The introduction of the telegraph, Nickles concludes, did not necessarily render diplomacy more efficient. It did, however, lead to increased speed, greater centralization, and a larger bureaucracy in Washington.
Barry Fulton is a management consultant at the U.S. Department of State and teaches at the School of Media and Public Affairs at George Washington University. A former Associate Director of the United States Information Agency, during his 30-year career as a Foreign Service Officer with the Agency, he served in Brussels, Rome, Tokyo, Karachi, and Islamabad. Fulton holds a Ph. D. in communications from the University of Illinois, a master’s degree in broadcasting and bachelor’s in electrical engineering from Penn State. He has taught at the Foreign Service Institute, at various American universities, and the Pakistani Information Academy. He is the author of Leveraging Technology in the Service of Diplomacy: Innovation in the Department of State and project director and author of the CSIS study, Reinventing Diplomacy in the Information Age.