Three
Levels of Professional Software Engineering:
the
moral responsibility of software developers
Donald
Gotterbarn
East
Tennessee State University
Abstract: Software engineering curricula need to include an increased emphasis
on professional conduct and software engineering ethics. The reasons for not
doing this are based on a misunderstanding of the different levels of
professional ethics. The levels of professional ethics are distinguished and a
model of software engineering ethics has been developed. This model is of
manageable scope, has practical resolution techniques, and is directly relevant
to the activities of the software professional. This concept of software
engineering ethics makes it an integral part of the TECHNICAL curriculum,
rather than a tangential frill. This discussion is followed by a description of
ways to introduce the study of this level of professional ethics into the
curriculum.
1. INTRODUCTION
In the past few years there have been several significant developments in software engineering education. Graduate curricula have been defined under the sponsorship of the DOD through the SEI. Undergraduate emphases in software engineering have also been developed. Several organizations, both military and commercial, are offering training courses in software engineering. These advances in education have been matched by the evolution of the principles of software engineering. New software development models have emerged and levels of software development have been carefully delineated.
There is, however, a surprising lacuna in software engineering education. One of the primary foci in software engineering education is the preparation of software professionals for the commercial market. Software engineering education has been responsive to most of the needs of industry, and yet there is at least one significant place where that responsiveness is wanting and industry has had to fend for itself. Professional responsibility and professional ethics is not included as a subject in any of the software engineering curricula, although it has been included in most other professional curricula [Bok,123].
This absence is surprising because industry perceives a need for training in software engineering ethics. At a recent ACM panel on what software engineering knowledge and skills should be required in a typical baccalaureate-computing program, Dennis Fairley of Texas Instruments declared that ethics was the most essential skill that industry wanted in the students. The significance that industry gives to this is further evidenced by the place of ethics officers in corporations, e.g., the ethics officers of Boeing Computer Services, Raytheon, and Texas Instruments are all CEOs at the vice presidential level. More than 25% of the Fortune 1000 have ethics officers at this level. Many corporation's commitment to ethics education is underlined by their publication of weekly newsletters, texts and video training materials, and the personnel devoted to ethics training. In several major companies, there are ethics officers at each division location. The corporate commitment to professional ethics is underlined by the fact that all new employees receive ethics training from their line managers.
This training is not directed specifically at software engineers or any particular skill group, but it is designed to address the ethical issues that arise within that particular industry. Industry presumes that their employees are basically moral human beings but that they are not acquainted with the moral rules and problems which are unique to that particular industry or profession.
The need for this type of ethics education is recognized in other professions such as medicine and engineering. Both of those disciplines require professional ethics courses in their curricula. In computer science there is a renewed emphasis in this direction. The Computer Science Accreditation Board requires some discussion of ethics in the curriculum for a school to be accredited. The IEEE/ACM Curriculum Task Force curriculum also requires ethics training. Some organizations, such as the American Board of Pediatrics, have even added ethical skills items to their certification examinations.
Why have we in software engineering education failed to recognize or failed to respond to this need which has been recognized by our peers and by those who employ our students? There are many reasons for this. Part of the answer, I think, lies in the fact that we are still a maturing discipline and are pre-occupied with defining the technical standards of software engineering. This single-mindedness has blinded us to the ethical elements of our applied science.
Software engineering matured in computer science departments rather than in engineering schools. This has led to an emphasis on theoretical elements. In theoretical sciences, ethics is not considered a relevant subject. The pure sciences are interested in the elegance of solutions. Software engineering is really more like engineering in that it is an applied science. Everything that an engineer develops has a direct impact on mankind. Ethics and professional responsibility is a significant element of engineering.
Engineering education has been concerned with ethics. Most engineering curricula have courses in engineering ethics. ABET engineering accreditation requires that engineering ethics be addressed throughout the engineering curriculum. A similar interest in ethics is not mirrored in pure sciences like mathematics. In pure sciences, like mathematics, one deals with formal systems and not with people. In the applied sciences one has to be concerned with people who will ultimately use its products. If the design and development of a product in the applied sciences is simply treated as an interesting mathematical puzzle which has no purpose or application then the product will be inferior and there are likely to be ethical problems. A programmer treated the development of a controller to raise and lower an X-ray machine as an interesting exercise in specifying loop invariants. The elegant and efficient program successfully moved a 3,000-pound X-ray machine above an X-ray table. Shortly after it was installed, an X-ray technician told a patient to get off the table and then set the height of the device to 0 (the table top). The program executed all of its elegant instructions, but the patient did not get off the table in time. If there had been any thought of the professional responsibility we have because of the human interaction with our products, the programmer would have programmed a pause in the loop which would only resume after confirmation that the patient was no longer under the descending machine. The programmer did not intend this result, but the view of software design merely as an interesting theoretical exercise contributed to this disaster.
Software engineering is an applied science. Every product of software engineering involves people and so at any stage of product development the users of the intermediate products and final products have to be kept in mind. We have obligations to the users of our products which include not only the implemented system, but also the requirements, software project management plans, specifications, designs, documentation, test suites, and programs.
2. A Broad View of Ethics
The media's presentation of computer ethics leaves us uncertain about the nature of the subject. Computer ethics is treated as any misuse of computer technology, from stock fraud to using the company word processor to write Christmas letters or it is portrayed as any significant social issue involving computers. Even a cursory glance at the broad concept of software engineering ethics or computer ethics used by the media reveals its superficiality. The media coverage falsely implies that computer ethics is only concerned with moral or legal abuses committed with a computer. Textbooks include stories of stock fraud committed with a computer and skimming interest from bank accounts done by a computer. Since these actions use a computer they are considered problems in COMPUTER ethics. This view is absurd. These same acts, when committed with a fountain pen were not considered FOUNTAIN PEN ethics; nor is beating someone to death with a law book a case of legal ethics. An early attempt to narrow the concept to a more meaningful scope was offered by James Moore --"The mark of a basic problem in computer ethics is one in which computer technology is essentially involved... [Moore, 267]". This definition continues the confusion about computer ethics, a confusion which ties the concept of computer ethics to the machine rather than to the practices and behaviors of individuals.
Admittedly, there are sets of ethical puzzles which are generated by the new applications of computer technology. For example, sociologists are puzzling about computer monitoring of employees. We should be concerned about this, but it is not an issue in computer ethics. It is the same kind of ethical issue involved in monitoring employees through one-way mirrors. Most of these so-called issues in computer ethics have analogies with general ethical issues and are resolved when we find an appropriate analogous situation. Fraud committed with the aid of a computer is fraud and nothing more.
3. A Model for Software Engineering Ethics
Software engineering ethics can best be understood when we use models from engineering ethics, legal ethics or medical ethics. These are all instances of different professional ethics. They are each directly related to the daily activity of the practicing professional and emphasize the process of each profession. Using some of the elements of professional ethics as our model has the advantage of bringing software engineering ethics with the horizon of action of the MORAL individual software engineer.
Ethics can be discussed on several levels. One can talk about ethics in general as a means to facilitate the interactions of people living in a society by placing obligations of action and avoidance of action on them. In any given society ethics regulates individual behavior in those circumstances where it cannot be or, as yet has not been regulated by law. Ethics depends entirely on voluntary acceptance of established rules and is not enforced by the state. We presume that all people have this sense of ethics in varying degrees.
There is a sense of ethics, which is required of all professionals, regardless of their particular profession. This broad sense of professional ethics is derived from the concept of a profession. There are several marks of a professional. They generally include: a special skill or knowledge to produce a product or service, a commitment to public service, a commitment to cause no harm, membership in a representative organization, being licensed by a representative organization, and autonomous judgement based on this specialized knowledge [John1]. A profession is permitted by society to lay claim to a body of knowledge because through the utilization of the knowledge, society as a whole gains. This claim to exclusive knowledge is only justified if indeed society does gain. This in turn means that a profession assumes an obligation to serve society. These obligations are generally articulated in professional codes of ethics. Although software engineering does not meet all of these marks of professionalism, it does meet those marks that are significant for understanding how ethics is related to software engineering. The primary marks of similarity are: that there is a special skill and training used to develop products which directly affect many lives and affects the way software artifacts are used in society, and by virtue of that training one gains the power to have a significant effect on society. The effect of software engineering upon the public's well being is a vital one.
When talking about medical ethics and engineering ethics we refer to "Professional Ethics" as if it were a single simple concept. There are, however, variations in professional ethics. Each profession operates in a different context with different tools and addresses different problems. Thus, at this level they employ different ethical principles and give these principles different priorities. The physician has a significant obligation to patient confidentiality, while this has little relevance for the mechanical engineer. These ethical concerns and how to respond to them is determined by consensus within a profession. Society trusts that they are best equipped to determine these principles because of their special knowledge. This is the basis for self-regulation of the professions. This is the sense in which software engineering ethics is unique.
This professional responsibility to the public can be divided into a third level of professional ethics. There are two distinct types of professional responsibility -- societal responsibility and technical responsibility. Societal responsibility involves the consideration of the impact of our software artifacts and computing products on society at large. Technical responsibility is concerned with the approach taken by professionals while they are solving technical problems. Sometimes this is subsumed under terms like 'quality' or 'professionalism'. This involves the consequences of knowing and following good professional standards.
Although there is a strong sense of individual responsibility here, this sense of professional ethics should not be confused with merely following the law. We are aware of many technical solutions to problems. These solutions vary in their adequacy to solve a particular problem. Although a product we develop does not meet the ideological goals of the profession there is no cause for legal action until some damage has occurred. There are three basic "requirements for a tort lawsuit: (1) a duty is owed; (2) the duty is breached; and (3) damages are caused by the breach of duty."[West 363] Attempts to legally define technical professional responsibility have been unsuccessful. Industry wants people who feel an ethical responsibility to approach these ideological goals.
In the professions, our technical skill guides processes; processes of building bridges, healing patients, of building software artifacts. The performance of these processes involves numerous ethical issues. The physician's choice of which medicine to dispense involves a technical judgement about its curative powers, but it also involves questions about the side effects of the medicine. Is it too expensive for the patient to purchase? Is it addictive? Is it likely to cause other problems? Technical decisions in software engineering are analogous to this. The choice of which life cycle to use will lead to radically different products, although they both might satisfy the requirements.
Ethical difficulties arise when the professional senses a conflict between opposing goods (or harms) and there is no course of action that realizes both of the goods or avoids both of the harms. There are varying degrees of ethical difficulty, ranging from easy to resolve to a dilemma. The degree of difficulty is determined by the commitment to the competing values in a particular context. For example, there is no question that lying about completing the testing of a product in order to play golf is not consistent with the ideals of our profession. There are also harder decisions. Suppose, your work on the vote tabulating program has not been completely tested and it is the day before elections. The choice between 1) releasing the product and continuing testing while it is being used and 2) failing to release the product and thereby disenfranchising people is not nearly as clear. Opinions of fellow software engineers will vary, and they will be influenced by the quality of the testing that has been completed. The existence of these hard decisions is frequently used as evidence that there are no answers in ethics. Pointing to the existence of some hard cases and concluding that there can be no answers in software engineering ethics is like pointing to the irrational number that results when dividing 100 by 3 and saying that because this is a hard problem, there are no solutions in mathematics.
The senses of ethics that need to be discussed in software engineering include both professional and technical ethics. Technical ethics involves moral commitments to standards of software production, to the process and professional ethics involves commitment to using these special skills in the service of society, a society that includes all other participating in the software development process. Both of these approaches to ethics give positive guidance for our behavior as software practitioners. Software professionals have a set of standards they can follow and recent work has shown that there is significant agreement about these technical and ethical standards among software professionals. Within the realm of professional ethics there is a surprising degree of agreement [Gott5]. Although there are technical standards, there are a variety of solutions available for most problems presented to us. Our choice of which solution to give is based on our professional values. These professional values have a direct impact on the way we develop applications and the quality of those artifacts.
It is precisely because industry has recognized that the production of software artifacts is not a value free enterprise that has led them to emphasize professional ethics. If there is no sense of professional service and one simply has a "let the buyer beware" attitude then the software product will be of lower quality. For example, we might produce a hand held anti-tank missile launcher that effectively destroys tanks, but on occasion generates so much heat that it kills the person firing the missile. Without a sense of values, we could argue that the specifications only talked about the "tank kill efficiency" and they didn't say anything about user safety. A case similar to this is currently before the courts. A failure of technical ethics also leads to problems. For example if the standards of configuration management are not maintained, then even if there are no problems in the initial deployment of an artifact, there is likely to be difficulty modifying the product during its post-deployment phase.
Software engineering ethics consists of two major elements. One element, called technical ethics, consists of doing a technically competent job at all phases of the software development process; the other element is the use of a set of moral values to guide the technical decisions.
In general, all professional ethics -- engineering ethics, legal ethics and medical ethics-- are only distinguished by the context to which they apply moral rules. These professional ethics are applied ethics. We are familiar with these rules and do not have to learn a unique set of ethical rules to do software engineering ethics. There are of course, differences in the professional ethics. The contexts bring out different ethical problems. Because different contexts raise different ethical concerns, the order in which the moral rules are applied varies for each application domain. Consider the different ways informed consent is treated. This rule seems to have different priorities in different stages of the software life cycle. During the requirements phase of life-critical software, informed consent--understanding agreement-- is an important rule. During testing, principles about not deceiving and not cheating are very important but informed consent is not a major concern. Professional ethics not only consists of the context, it is also the order in which the moral rules are consulted. The professional side of software engineering ethics is the application moral rules in the software engineering context [Gott2]. There are elements of software engineering ethics which are unique to the discipline and there are elements which are common to all professional ethics.
Professionalism maintains a proactive posture. We expect software professionals to develop project plans which anticipate and plan for risks. It is hopped that because of the professional's special knowledge that they will avoid many of the risks. If we anticipate the difficulty ahead of time and plan for it, the difficulty will be reduced or eliminated. The anticipatory approach produces better software more efficiently, and the same is true for ethics. By adopting this attitude of the professional, the software engineer can avoid many of the ethical problems that develop in the process of building software artifacts. When properly understood, software engineering ethics does not consist of a litany of mistakes that can be made in software engineering but it is a positive guide to behavior.
With this understanding of the way a computing professional functions, we can see that our current software engineering education is incomplete and that we need to include material on professional ethics.
The recognition of the need to include this material immediately raises the question of exactly what should be included, how it should be taught, who should teach it, and what important technical skills should be dropped in order to include this material.
4. How to teach software engineering ethics.
4.1 Who should teach it?
We often delegate the teaching of professional ethics to humanities and social science departments. In numerous studies it has been shown that these departments are not familiar enough with the technical details of a discipline to help with technical professional ethics. What is more surprising is that studies in engineering ethics have shown that they also fail to make the connection between technology and society.[Herk]
Some argue that only a philosopher should teach ethics. When we say this I think we have bought into the philosopher's presumption that THEIR moral theories can be used to solve moral problems. So in works on computer ethics there are philosophy sections on teleological theories and deontological theories[John1]. Behind these two theories lies two approaches, one emphasizes results or consequences of actions (teleology) and the other emphasizes motive or duties(deontology). Armed with these theories we are supposed to solve the practical ethical problems which confront us daily. A mistake made by the philosopher is to portray ethics as "Pick your theory and then reason to an answer." They believe that different theories will lead to different sets of answers. Advocating this model of reasoning reinforces the view that all ethics discussion is fruitless because in philosophy there are as many answers as there are theories.
It has been shown that these theories primarily conflict on the level of theory and justification rather than on the level of practice and action[DeGe]. These moral theories are like all theories-merely alternate descriptions of how we move through our daily lives responding to roles, standards, purposes, and duties.
Using the example of professional ethics described above, there is an emphasis of a set of standards accepted by the professional community engaged in the software development process. Following this accepted set of standards is an obligation of the professional and it is of little practical consequence whether that is done out of a sense of duty or because one has a contractual relation with a customer. One might ask whether the description of software engineering I have offered is based on consequentialism--doing things because of the significance of the consequences-- or deontologism--doing things because they are the Right thing to do. I think such a question has little relevance to the moral life of a software engineer. We emphasize the standards of the process and following them could be described in terms of duty or deontology; but we emphasize the process because we believe the product will be better. This could be described in terms of consequentialism. The moral theory used to describe the event has little impact on the moral life. The philosopher has no special competence here.
We do not need to be a philosopher to discuss software engineering ethics. Our aim in software engineering ethics is not the acquaintance with complex ethical theories but it is recognizing role responsibility and awareness of the nature of the profession. A few hours reading about these theories is adequate. Everyone has to act in the world whether or not they are trained in moral philosophy.
4.2 What should be taught?
We are interested in teaching how to anticipate and avoid ethical problems within the profession. We are also interested in providing techniques or methodologies which can guide our behavior when a problem does arise. Given the theory of ethics outlined above, this is best done within the context of our technical curriculum. This can easily be done in most of our courses. For example in a requirements class one could look at a case like the following: Suppose you were asked to develop a system which would collect the length of time for ambulance trips to accident scenes and back to hospitals starting from different ambulance services. This data was to be used to redesign ambulance service districts to reduce the time spent getting patients to hospitals. The requirements are developed, and during prototyping it is discovered that there are a significant number of trips for which no time is recorded. In determining how to handle these zero times it is discovered that in most of these cases no time was recorded because very critical patients were being transported and the paramedic was too busy to record the time. At this stage in development it is discovered that the most significant times to the study are the times that are not recorded. The discussion of the technical and professional options in this situation is teaching software engineering ethics.[For a full analysis of this case see Gott5.] They are learning how to handle a morally significant situation in an application area.
The moral reasoning involved here is not generated from some esoteric theory which requires a trained philosopher to understand, but rather the moral reasoning here is based on reasoning by analogy, where we can examine the technical alternatives and attempt to anticipate their morally significant outcomes. It is the technical knowledge that enables us to understand the potential consequences. The judgements involve considering the technically viable alternatives and making judgements guided by both our technical skills and professional values. The technical discussion in class, deciding which are the better solutions and why, is teaching software engineering ethics. The use of detailed technical examples in the class is a way to develop the skill of anticipating some of these ethical problems.
One can use cases which are less technical to emphasize what I have called the value side of software engineering ethics. Here are some issues that can arise at each phase of the software development life cycle.
In the testing phase, if funds are exhausted before the testing is satisfactorily completed and there is no possibility of further funds, you have several options. Whichever option you take it must be conditioned by moral rules such as, "don't deceive", "keep promises", and "act professionally." Depending on the type of software being tested, rules like "don't cause pain" and "don't kill" might also come into play.
In the design phase, other moral rules might get applied first. Consider designing a journal file for a library check-out system to determine the popularity of books and the number of additional copies that should be ordered, if any. The association of the patron's name with the book checked out has a potential for the violation of several moral rules, such as the violation of privacy and the deprivation of pleasure because one does not feel free to read what one wants, and possibly causing psychological pain. Notice that there are no laws violated by this design.
In the design phase, the choice of a language for a life-critical system can have moral implications. If the language is hard to debug, write, or modify, then one puts people at risk and violates principles of good system design in the choice of that language[Gott3]. The ethical issues can come from two directions here. If the designers were ignorant and didn't know a better language, they are morally culpable for undertaking a project which was beyond their skill. If on the other hand, they were well aware of these significant differences in languages but decided to stick with the more dangerous language for other reasons such as profit, then they have violated professional ethics. It is unlikely, nonetheless, that they would lose a tort suit.
What we want to do is to teach students to transfer moral paradigms to decisions they make as professional software engineers.
4.3 Reasoning about ethical issues.
I also believe the student needs some general acquaintance with effective ways to reason about ethics. They need to understand how different ethical values compete and how they sometimes only have limited application. They need to see how these values get prioritized and how that affects decisions. This can be accomplished by having the student read articles in professional ethics on analogous issues. When discussing the ambulance case one can read articles in [Mart] [Stev] or [John2]. It is important to choose these discussions carefully. The point of the reading is to have the students gain an understanding of ethical reasoning within their profession. Unfortunately, many textbooks in computer ethics are merely lists of moral abuses committed with a computer. They also tell exciting tales of computer disasters. I think these texts are dangerous for two reasons. First they contain no model of ethical reasoning. Students don't get to see the way moral reasoning works. The second danger is professional ethics is presented as a subject which merely deals with proscribed activity -- don't embezzle, don't commit fraud, etc. The majority of our students do not intend to be crooks so they perceive this as irrelevant to them. They will not take kindly to being constantly equated with bumbling fools or crooks. Material like this does little to forward professional ethics. There is a third danger with these "war stories". Because they are fun to tell, we easily fall into the trap of thinking we are discussing professional ethics just by telling the story.
The story is useful only if it meets several conditions.
1. It is not told because it is impossible to resolve.
2. It has enough detail to be able to do a technical analysis.
3. It does not deal with a professional who is morally bankrupt.
4. It is related to an issue in the profession.
5. It can be discussed using moral values.
I think it is important to be careful when discussing major computing disasters. If these disasters are emphasized then the student is given the impression that ethical problems will be rare in their lives. Dealing with large social issues such as providing computers to terrorist countries is also a problem. These issues shift the emphasis from individual decision making to some form of social philosophy.
Try to develop a proactive attitude. The stories should be directly relevant to the class topic. There are some examples of these in [Gott4]. If we learn to think about the ethical issues before design, then some potential ethical issues will be avoided.
The pedagogical goals of discussing ethics in the technical curriculum are the development of the following set of skills:
1. The ability to identify correctly the potential for an ethical problem in a particular context, and the ability to identify what moral rules are being compromised.
2. The ability to identify the cause of these issues, determine several alternate forms of action consistent with morality in that context and for each of these possible actions to determine expected outcomes and reasons for taking or not taking those actions.
3. The ability to select a workable solution and work through the situation either TECHNICALLY or morally.
Teaching these skills does not require learning new theories. These are process skills, the emphasis should be on a process that can be applied to changing contexts. These skills, like other abstract skills, are learned by practice. A single class meeting or a single chapter of a textbook discussed in one course will not develop these skills. To teach a single ethics course or have a special professor who is the ethics professor reinforces the mistaken notion that ethics and the practice of software engineering are distinct. The approach taken by engineering, of discussing ethical concerns in every course, is the most effective. A student needs practice several times in a course and practice in several software engineering courses. A case study methodology does these best, with each case addressing one or more specific phases of the software development process. This methodology involves giving a brief description of a professional situation that might involve an ethical problem. The class discusses the situation trying to identify if there is an ethical issue. If they find a situation which involves a violation of moral rules, then they should try to determine alternate approaches which would eliminate or at least reduce the moral difficulty. The case methodology has been shown to be most effective when professors and students discuss the case as peers. This reduces the need for the software engineering professor to be an ethics specialist [Gott2].
Issues that might be discussed include: designing test suites for life preserving products, developing questionable software for foreign governments, work on projects that they discover are non-functional at a late stage of the project, doing shoddy work because the project is behind schedule and will not be ready by the end of the semester[Gott1, Gott4].
I think the introduction of this material is necessary if educators are to be responsive to the needs of the software engineering community. We have to have software engineers internalize the values of the profession and persuade them to act as professionals.
REFERENCES
[Bok] Derek Bok, Beyond the Ivory Tower:Social Responsibilities for the Modern University (Cambridge:Harvard University Press, 1982),p123
[DeGe] Richard DeGeorge, Business Ethics(New York, 1982).
[Gott1] Donald Gotterbarn, "A Capstone Course in Software Engineering Ethics," in Proceedings of the National Conference on Computing and Values,(New Haven:1991).
[Gott2] Donald Gotterbarn and Lionel Deimel, "Teaching Software Engineering Ethics," unpublished manuscript :
[Gott3] Donald Gotterbarn "Software Engineering Ethics Workshop," in The Journal of Systems and Software, March 1990.
[Gott4] Donald Gotterbarn, "Ethical Considerations in Software Engineering," in Proceedings of the 13th International Conference on Software Engineering,(IEEE Press, 1991).
[Gott5] Donald Gotterbarn, "Professionalism and Ethics," a video tapped lecture in Software Project Management for the Software Engineering Institute's video dissemination project. April 9th, 1991.
[Herk] J.R. Herkert & B.V. Viscomi, "Introducing Professionalism and Ethics in Engineering Curriculum," Journal of Professional Issues in Engineering Education and Practice,117,4,(October 1991) p. 383-388.
[John1] Deborah G. Johnson, Computer Ethics (New Jersey: Prentice-Hall,1885)
[John2] Deborah G. Johnson and John W. Snapper, Ethical Issues in the Use of Computers (Belmont California:Wadsworth Publishing Company,1985).
[Mart] Mike W. Martin & Roland Schinzinger, Ethics in Engineering, (New York:McGraw Hill, 1989).
[Mill] Keith Miller, "Integrating Computer Ethics into the Computer Science Curriculum," in Computer Science Education,1, p37-52,(1988).
[Moor] James Moor, "What is Computer Ethics?," Metaphilosophy,16,4, 267-275.
[Stev] J.T. Stevenson, Engineering Ethics: Practices and Principles (Toronto:Canadian Scholars' Press 1987).
[West] L.B. West ,"Professional Civil Engineering: Responsibility," Journal of Professional Issues in Engineering Education and Practice, 117,4, (October 1991).