Creating a New Technology: The Technology of Software Production An Editorial on the 25th Anniversary of UNIX [Editor's Note: The following editorial is an effort to encourage a discussion of the significance of UNIX. We welcome alternative viewpoints, comments, etc. on the issues raised in this editorial.] In his book Ancient Society, Louis Henry Morgan, who has been called the father of anthropology, described the important role that the creation of iron played in the advance of human civilization. He wrote: "When the barbarian, advancing step by step, had discovered the native metals, and learned to melt them in the crucible and to cast them in moulds; when he had alloyed native copper with tin and produced bronze; and, finally, when by a still greater effort of thought he had invented the furnace, and produced iron from the ore, nine tenths of the battle for civilization was gained."(1) "Furnished with iron tools," continued Morgan, "capable of holding both an edge and a point, mankind were certain of attaining to civilzation."(2) Morgan called the production of iron, "the event of events in human experience ... Out of it," he wrote, "came the metallic hammer and anvil, the axe and the chisel, the plow with an iron point, the iron sword; in fine, the basis of civilization which may be said to rest upon this metal." With the birth of the modern computer, the development of a new technology has been put on the agenda for our times. This technology will not be forged in furnaces as were the tools of our ancestors. It is not even possible to grab or hold this technology.(3) This technology is the technology of software production. The challenge for our society is to be able to develop a software production technology. By the early 1960's, Bell Labs researchers realized that there would be an ever increasing role that computers would play in the operations of a large utility like the U.S. telephone system. And the telephone service crisis of the late 1960's showed that indeed AT&T had to automate to be able to meet its obligations to the public in the U.S. To accomplish this automation, they realized they would have to put software production on a more rational basis. Writing about this challenge, researchers explained: "One might think that because typing is easier than soldering, it should be easier to change software than to change hardware. However, the ease of changing software depends on the language at hand, the quality of the editor, the file system, structure, etc."(4) The UNIX time sharing system was created 25 years ago at Bell Labs to help to fulfill this need. One of the important contributions of UNIX in its 25 years of development has been the role that UNIX has played to put the production of software on a more rational and scientific basis. Describing the problem facing computer software pioneers, Evan L. Ivie, a researcher at Bell Labs, writes: "Although the computer industry now has some 30 years of experience, the programming of computer-based systems persists in being a very difficult and costly job. This is particularly true of large and complex systems where scheduled slips, cost overruns, high bug rates, insufficient throughput, maintenance difficulties, etc., all seem to be the rule instead of the exception. Part of the problem stems from the fact that programming is as yet very much a trial and error process."(5) Ivie explains that there is "only the beginnings of a methodology or discipline for designing, building and testing software. The situation is further aggravated," he adds, "by the rapidly changing hardware industry and by the continuing evolution of operating systems which continues to nullify much of the progress that is made in the development of programming tools. What can be done," he asks, "to move the programming industry toward a more professional and stable approach to software development?"(6) After enumerating several possible alternatives, he proposes "a very different approach to improving the development process." The approach he proposes is that "the programming community develop a program development 'faculty' (or facilities) much like those that have been developed for other professions." And he cites as examples a "carpenter's workbench," or a "dentist's office," or an "engineer's laboratory." This approach, he maintains, "would help focus attention on the need for adequate tools and procedures; it would serve as a mechanism for integrating tools into a coordinated set; and it would tend to add stability to the programming environment by separating the tools from the product (the current approach," he notes, "is equivalent to carpenters leaving their tools in each house they build.)"(7) Ivie's proposal was carried out in what came to be known as the Programmer's Workbench. A set of UNIX and C software development tools were created and were made available to programmers, even though they were working on different machines and with different operating systems. P.J. Plauger, one of the UNIX pioneers who helped to define the concept of a software development tool writes that "the programmer working as a tool builder finds that his impact extends far beyond the task he may originally have set out to solve."(8) Doug McIlroy, one of the creators of UNIX, explains that the term "software tools" was still unnamed around the circle of UNIX pioneers until Brian Kernighan and Plauger wrote the book Software Tools. "The idea nevertheless became a 'guiding principle,' " writes McIlroy. It was only with the "liberation of the grep pattern matching program from within the ed editor, that the "unarticulated notion of software tools ... was finally brought home."(9) "More than any other program," McIlroy explains, "grep focused the viewpoint that Kernighan and Plauger christened and formalized in 'Software Tools': make programs do one thing and do it well, with as few preconceptions about input syntax as possible."(10) Kernighan and Plauger give the following definition of a software tool: "It uses the machine; it solves a general problem not a special case; and it's so easy to use that people will use it not build their own."(11) Describing one of the important problems that UNIX was called on to solve, Dick Haight, another of the pioneers, explained the problem facing AT&T, one of the largest corporations in the U.S., in the early 1970s. "We had a real problem to solve," Haight elaborates in an interview. "For one thing, we had a fairly large group of software developers at the Labs working on several different mainframes. The biggest group ... consisted of people working on the IBM 360s or 370s. The programmers working on the IBM 360 or 370 had to contend with batch processing and IBM's Job Control Language (JCL) problems. Other programmers were working with the Univac and another group with Xerox computers. Only a few were using early but expensive time sharing like TSO (IBM's timesharing) and Univacs Remand Service." Haight explains that "these systems not only offered very different programming environments but proved to be very expensive to use and very unfriendly." As part of a group formed in the Business Information Systems Program, (BISP), Haight and his colleagues were charged with the task of creating a more rational programming environment for the AT&T programmers. "Basically," he explains, "we ended up trying to give all these people a cheap text editing front end for interactive program entry." Haight and his colleagues had experience with the UNIX program development system and they used it to create the PWB - a Programmer's Workbench facility that eventually included over 300 tools. The Programmer's Workbench (PWB) created at Bell Labs in response to this need, encouraged the development of machine independent programming tools. "Each tool," they maintained, "must now function for programmers developing code for a number of different vendor machines. ... One is thus forced into a more stable and generalized software development approach which should be more applicable to new machines."(12) The PWB was conceived of in mid April 1973, installed on the first workbench computer in October 1973 (PDP 11/45) and by 1977 the Programmer's Workbench computers were serving 1000 developers. Eventually, it was used to write the thousands of lines of computer code needed for the development of the 5ESS switch that AT&T was installing.(13) The contribution of UNIX to the creation of such advances in the programming profession needs to be studied and built on. The problems of software development are the difficult problems that the computer revolution has thrust on center stage. Writing in a time of similar technological change, Denis Diderot, who was the editor of the Great French Encyclopedia (Encyclopdie, ou dictionaire raisson des sciences, des arts et des mtiers) realized the need to catalog and graphically present drawings of the tools and industrial processes in use in industry up to that time. Though he was attacked for revealing the secrets of the trades, this work made it possible for others to study the level of tool development in use and improve on it. In his book The History of the Machine (NY, 1979), Sigvard Strandh describes how tool production had been at a standstill from the middle ages until the early days of the industrial revolution. He explains that there were previous plans for building some of the components of the steam engine, but the tools that would make doing so possible were not yet available. The work of those like Diderot to publish descriptions of what knowledge was known in the production of tools and industrial processes, helped to advance the state of the art of the technology of tool production, as they made it possible to build on what had been achieved. In an article describing how the PWB was ported to the IBM System/370, the authors write that there were at the time 300 UNIX or C tools that were part of the Programmer's Workbench. Sadly, many of those tools no longer seem to be available. Commenting on the state of availability of UNIX tools, another UNIX pioneer, John Mashey observed, "Our ability to organize software and make it available has lagged our ability to write it." (14) Also, the concept of a software component catalog, first mentioned by Doug McIlroy in 1968 before the creation of UNIX, and then reintroduced in the Bell Labs Journal articles(15) needs to be reviewed and reestablished as a goal. The scientific principle that Plaugher emphasizes, is that the only way to make substantial progress in any field is by building on the work of others. Just as UNIX was built on the lessons that Ritchie and Thompson and others learned from the experience of CTSS and the early Multics collaboration, so it is helpful to learn from the experience of UNIX in order to make any further advances. That is the challenge that the 25th anniversary of UNIX puts on the agenda for those who want to advance. As Henry Spencer and Geoff Collyer wrote in their article "News Need Not Be Slow": "To know how to get somewhere, you must know where you are starting from." Notes (1) Louis Henry Morgan, Ancient Society, Chicago, 1877, pg.4 (2) Ibid., pg. 43 (3) Fred Brooks Jr. explains that "Software is invisible and unvisualizable .... The reality of software is not embedded in space." from "No Silver Bullets," UNIX Review, Nov. 1987, pg. 41. (4) "Microcomputer Control of Apparatus, Machinery, and Experiments" by B. C. Wonsiewicz, A. R. Storm, and J. D. Sieber, "The Bell System Technical Journal," July-August 1978, vol 57, no. 6, pt 2, pg. 2211. (5) "Programmer's Workbench A Machine for Software Development," Communications of the ACM, Oct. 1977, vol 20, no 10. pg. 746 (6) Ibid. (7) Ibid. (8) "Minicompilers, preprocessors and other tools," in AFIPS Conference Proceedings, vol 44, 1975, pg. 281. (9) "UNIX on My Mind," Proc. Virginia Computer Users Conference, vol 21, Sept 1991, Blacksbury, pg. 1-6. (10) From E-mail correspondence. (11) "Interview with Dick Haight," UNIX Review. May 1986. (12) "Programmer's Workbench," pg. 749. (13) See "A UNIX System Implementation for System/370" by W.A. Felton, G. L. Miller, and J. M. Milner, Bell Laboratories Technical Journal, October, 1984, (14) "UNIX Leverage - Past, Present, Future", Usenix Winter 1987 Conference Proceedings, pg. 8 (15) See for example "Software Tools and Components" by R. F. Bergerson and M. J. Rochkind and "Cable Repair Administration System," by P. S. Boggs and J. R. Mashey, pg. 1275 in Bell System Technical Journal, July-August 1982 ---------------------------------------------------------------- Reprinted from the Amateur Computerist Vol 6 no 1 Winter/Spring 1994