Occidental Engineering Case Study: Part 6

In practice when we are faced with an ethical dilemma, we seldom think it through from first principles. We usually draw on existing experience and past judgments. Making a sound ethical judgment is difficult. It involves consideration and weighing of many factors, the sifting through of different positions, careful reasoning and argument, and much experience of the implications of different actions, what works and what does not. It is helpful to be able to apply the wisdom so dearly bought by applying the judgments to new situations. There are a number of ways in which we do that.

Analogy

In ethics as in law, we often decide a case by finding a similar case that has already been decided and applying the same judgment. When Wayne in the Occidental case protests that delivering flawed safety-critical software is like selling a car with defective brakes, he is reasoning by analogy. We are well aware of issues of safety in the automobile industry and the importance of engineers maintaining the highest standards for critical systems. Most would agree that it is wrong for engineers knowingly to release a vehicle with brakes that could fail. In the same way, Wayne argues, it would be wrong for him knowingly to release safety-critical software that could fail. The cases are analogous according to his analysis because they both involve engineered systems whose failures could cost lives, and both are being released with known flaws.

The use of analogy and arguing by cases goes back at least as far as the ancient Greek philosophers. For example in Plato's Gorgias, Socrates wishes to discredit sophists because of the shallowness of their flattering oratory, so he sets up the following analogy: "what pastry making is to medicine, oratory is to justice."³³ In other words, just as pastry gives the body an immediate sensation of well-being, though it is in fact unhealthy, whereas medicine produces real health, so the sophists' oratory seeks only the appearance of good, while justice is the true good of the state. The implication is that because there is something misleading and unhealthy about pastry, in the same way there is something untrue and corrupting about sophistry.

The ultimate justification for using analogy in ethical judgment is the principle of universalizability noted above as one of the principles derivable from Kant's categorical imperative. According to this principle if Case A and Case B are ethically analogous, meaning that they are governed by the same ethical principles and do not differ in any ethically significant way, then whatever judgment is attached to A should also be attached to B. If we have already arrived at a judgment for A, we can use it for B as well. The hard part is deciding when cases are analogous, and especially what differences are significant enough ethically to invalidate the analogy. For example, one might counter Wayne's use of the defective brakes analogy by pointing out that, while brakes have physical defects that cause them to fail randomly, the air traffic control program has a logical defect that fails in a predictable manner, and that removing a program bug is a routine procedure that can be done with a regularly scheduled program update, whereas retrofitting brakes is a major operation that can only be done through an extraordinary recall. But are these differences enough to invalidate the judgment based on a fundamental similarity, that both involve knowingly releasing a safety-critical system with a potentially dangerous flaw? For that matter, are software bugs really that predictable and controllable?

The above example shows that analogies can be enlightening, but they can never simply be taken at face value. They must always be examined very carefully. Nevertheless, even though analogies seldom give easy answers to ethical problems, they are often worth pursuing because of the insight they can provide. Thinking through the similarities and differences with known cases highlights the ethical issues and the ethically relevant factors, and suggests applicable ethical principles. At the very least comparative case analysis is a good way of exploring new issues and cases. For example, to what extent do our categories and rules about ownership of tangible property apply to software? What are the relevant similarities and differences? Can software be "owned" in the same way that a car or house can? Is copying software "stealing" or "sharing, " two competing analogies with quite different ethical implications.

Laws

Law is one of the ways in which society orders its common life. It concretizes some of society's expectations for justice, honesty, fidelity, and respect for rights. Thus the law reflects, in some complex and imperfect way, society's ethical requirements.

While there is a strong relationship between law and ethics, they are not identical. It is a mistake to think that when something is legal, it is necessarily ethical, or worse yet, that "It's ok as long as you get away with it." The law, at its best, represents the minimum expectations necessary to maintain a civilized society. There are many areas of private life it does not seek to regulate, and other areas that, while they touch public life, are left alone because it would be too oppressive to try to control them. Even in the areas that it can and should be concerned with, the law is a less than perfect representation of ethics, both because of disagreements and misunderstandings about what is right and because powerful interests often block legislation that would serve the public good at their expense. Therefore there are many actions that are legal but still not ethical. It may not be illegal in some cases to make misleading claims about a product, but it is unethical. It may not be illegal to take an unpublished idea from a colleague and publish it as one's own, but it is certainly wrong. In most places it is still not against the law to force employees to spend all day at a keyboard working under conditions that are likely to lead to very painful injuries, but it is still a violation of the duty not to harm others. Those who take the law as the ultimate standard of what is right and wrong are in an arrested stage of moral development.³⁴

On the other hand, not everything illegal is also immoral. There some laws whose requirements are arbitrary; they have nothing to do with right and wrong. For example, there is nothing fundamentally wrong with driving on the left side of the road. Yet in many countries the law forbids it. Some such law is necessary to impose at least a minimum of order on the flow of traffic, but the choice of right over left as the preferred side is arbitrary; other countries do as well driving on the left.

Other laws are simply wrong. The laws that maintained slavery in the United States, for example, were unjust, and those who violated them by refusing to return escaped slaves were not necessarily acting unethically. An even more striking example is the whole system of laws meant to facilitate the extermination of the Jews in Nazi Germany. Those who broke them by hiding Jews or helping them escape were not immoral. In fact they may have been the only people acting justly in that society. It can never be taken for granted that the law is right. There is always a responsibility to examine and critique it, and, if it is found to be seriously deficient, to try to change it.

Nevertheless, there is generally a presumption in favor of obeying the law, once it is established. There are a number of reasons for this. First, laws, imperfect as they are, often represent an attempt to codify society's standards of ethical behavior. In many cases a law is the product of a serious debate over ethical issues and an attempt to balance and reconcile competing values and interests. Also laws develop over time, as people learn more about their effects and reflect on their meaning. Therefore laws embody a certain collective wisdom. This in itself is enough to command serious consideration, if not uncritical acceptance. Second, there are usually penalties for violation of the law. To break a law is to put oneself, and often one's community or institution, in danger of fines and imprisonment, embarrassment and scandal, and other losses. Someone would have to have very grave reasons for taking on this risk. Civil disobedience is sometimes an important weapon in the struggle for justice; but it is not to be taken lightly. Third, laws, even when they are arbitrary, are needed to keep order in society, and to violate them is to violate that order. To drive on the left is not wrong in itself, but to drive on the left when the law says to drive on the right is to disrupt traffic flow unnecessarily and to endanger oneself and others. Finally, the law, when it works properly, defines a certain set of constraints under which everyone operates. When individuals or groups act outside the law, it gives them an unfair advantage over those who observe it. For example part of the cost of an industrial process is usually the disposal of wastes. If a company can simply dump them into the air or water, it is really transferring some of the costs to public agencies that must clean up the waste and the public, who pay in terms of a greater threat to their health and enjoyment. Clean air and water standards are meant to avoid that. When a company breaks these laws, it is avoiding its fair share of the cost of maintaining a livable environment and gaining an unfair advantage over companies that keep them.

When Wayne in the Occidental case protests that releasing the flawed software would be illegal, presumably because it involved fraud and violated the terms of their contract with the government, he is raising a very serious objection, even if the law is overly restrictive, as Deborah seems to think. They would be exposing themselves and the company to the risk of very serious consequences, including fines, imprisonment, disgrace, loss of customers, and loss of future government contracts. Furthermore, because the laws exist in part to keep the bidding process fair and make sure the bidders deliver what they promise, it would be an injustice to violate the law, for the reasons listed earlier in the section on justice.

Laws, therefore, by their very existence, create ethical obligations to observe them. At the same time, these laws are not above ethical analysis and criticism. When we examine an ethical issue, therefore, we must both understand what the law requires and ask whether the law is right or if it needs to be changed.

Norms

Norms are ethical "rules of thumb." They are guidelines and rules that have been developed for analyzing and deciding cases in a particular area of ethics, such as medical ethics or political ethics. They are less fundamental, and therefore less general and authoritative, than the basic duties listed earlier. Their authority comes from their having been found useful in practice and from the fact that they somehow embody our judgments about what is right in a given situation. Therefore they are a way of formalizing, generalizing and communicating ethical wisdom and experience in a specific field.

A good example of a set of norms is the so-called "just war theory." This is a set of tests that are to be applied to determine whether a particular war can be justified.³⁵ There are two parts to it. The first governs the decision to enter into a war. This is tolerable only if the following conditions are met:

1. There must be a just cause;
2. There must be a right intention;
3. It must be a last resort;
4. There must be a reasonable hope of success;
5. The good to be achieved or the evil avoided by the war must be in proportion to the evil done by the war;
6. The decision to enter into the war must be made by legitimate authority; and
7. There must be a declaration of war with a clear statement of what it intends to achieve. The second part defines the legitimate means for pursuing the war. The means used must be proportional to the ends to be achieved, and there must be no direct attacks on civilians. The just war theory is thus an intermediate position between the two absolutist positions that claim, on the one hand, that all war is wrong, and, on the other, that in war anything is justified. Because that middle position tolerates some use of violence, but only in extreme situations and only to a limited extent, it requires careful analysis and judgment in its application. The norms of the just war theory provide the necessary framework. While the just war theory is by no means universally accepted, it is widely respected as a solid and sensible set of minimal guidelines for analyzing the morality of war or a particular war. It has to be part of any serious and well-informed discussion on that topic.

In the computer field, an example of norms is the "Ten Commandments of Computer Ethics" published by the Computer Ethics Institute:

1. Thou shalt not use a computer to harm other people.
2. Thou shalt not interfere with other people's computer work.
3. Thou shalt not snoop around in other people's computer files.
4. Thou shalt not use a computer to steal.
5. Thou shalt not use a computer to bear false witness.
6. Thou shalt not copy or use proprietary software for which you have not paid.
7. Thou shalt not use other people's computer resources without authorization or proper compensation.
8. Thou shalt not appropriate other people's intellectual output.
9. Thou shalt think about the social consequences of the program you are writing or the system you are designing.
10. Thou shalt always use a computer in ways that insure consideration and respect for your fellow humans.

These norms are not as specific or precise as the just war theory, nor do they have the weight of tradition or the test of time behind them. Nevertheless they do represent a consensus among many computer ethicists on what constitutes ethical computer use, and can provide some general guidance to users.

Norms are different from laws in that they are meant to guide judgment rather than to regulate behavior. Norms are not meant to be enforceable and do not have the authority of government behind them, although in some cases, norms can be incorporated into laws. In general, however, norms must stand on their own; their authority comes from their own inherent wisdom and the moral authority of the community that has formulated and accepted them.

An important task of ethics is to formulate or recognize norms for particular areas of concern. This is far more valuable than endorsing or condemning specific acts. A good set of norms gives a person guidance in understanding new situations, in knowing what questions to ask and what factors to consider, and in formulating an ethical response.

Professional Codes of Ethics

Many professions, including doctors, lawyers, and the military, have their own specific codes of ethics. These are needed because these groups are entrusted with special responsibilities in the community, so that a higher degree of care and dedication to the common good is expected of them, and because in their work they deal with special issues and problems that ordinary citizens do not usually face. For example, physicians have had codes of ethics going all the way back to the Hippocratic Oath that had its origins in ancient Greek society. Now they are covered the codes of their professional societies, such as the code of the American Medical Association or the code of the British Medical Association. There is also an International Code of Medical Ethics. These are all attempts to define the physician's duty to the patient, the profession, and the society. They include the duties to preserve life, to act always for the welfare of the patient, to respect confidentiality, to act honestly and justly, and so on.³⁶

Professional codes of ethics tend to come from within their professions; part of the definition of a profession is that it is self-governing. In some cases the codes are enforceable. Serious violations can lead to suspension or expulsion from the profession and the dishonor that comes with it. However the moral authority of a professional code comes from the fact that it represents a consensus of the traditions of the profession and the judgments of its most respected members on the duties specific to their particular calling.

Computer scientists and engineers represent a relatively young field compared to law, medicine and other well-established professions. Nevertheless they have a number of professional organizations that have developed codes of ethics. These include the codes of the Association for Computing Machinery (ACM), the International Federation for Information Processing (IFIP), and the Data Processing Management Association (DPMA). The Institute of Electrical and Electronics Engineers (IEEE), which includes many computer professionals in its ranks, also has a code of ethics; and the IEEE Computer Society (IEEE-CS) and the ACM have jointly developed and approved a Software Engineering Code of Ethics and Professional Practice.

Of these the ACM code and the joint ACM/IEEE-CS code for software engineering are the most comprehensive. The ACM Code³⁷ , revised in 1992, contains a preamble and four parts. The first is a set of "general moral imperatives" that link the responsibilities particular to computer professionals to fundamental ethical principles such as the duties to be honest and fair, to contribute to the good of society and humanity and avoid harm, and to respect privacy, confidentiality and intellectual property rights. Section 2 has more specific responsibilities of individual computer professionals. These include the obligation to understand and take responsibility for the consequences of their work, to honor contracts, laws and other obligations, to help educate the public, and to avoid unauthorized access to computers and communications systems. Section 3 balances the individualist perspective of section 2 by giving the obligations of those who have roles of leadership in organizations. These include the responsibility to see that the organization serves the needs, welfare and dignity of workers, users of its products, and others affected by its work. Finally there is a short section on compliance. This is seen as mainly voluntary, but there is a possibility of the ACM taking action against a member for serious code violations. The code proper is accompanied by a set of so-called Guidelines that serve as a commentary on and explanation of the code.

The joint ACM/IEEE-CS Software Engineering Code of Ethics and Professional Conduct³⁸ contains a number of principles to guide the judgment and conduct of software engineers, organized around the stakeholders to whom they owe responsibility and other professional concerns, including the public at large, the client and employer, the product, professional judgment, management, the profession, colleagues and personal development. The code makes it clear that protecting the public interest takes priority over all other concerns. Under each principle is a list of specific obligations that illustrate how that principle is to be put into practice. For example, under the first principle, which states that "software engineers shall act consistently with the public interest," one of the specific obligations (1.06) requires that software engineers shall "Be fair and avoid deception in all statements, particularly public ones, concerning software or related documents, methods and tools." This is particularly relevant to the Occidental case discussed here. So is 5.11 under Management, which says that anyone managing a software engineer shall "Not ask a software engineer to do anything inconsistent with this Code."

These codes are not meant to be a comprehensive definition of ethical behavior for computer professionals, although the ACM Code has been applied in a number of cases.³⁹ Nor do they give an answer to every ethical dilemma; that is neither possible nor desirable. There must be respect for the ethical and professional autonomy of the individual member, and awareness of the need for individual judgment in particular cases. As the joint ACM/IEEE-CS Code states in its Preamble:

Ethical tensions can best be addressed by thoughtful consideration of fundamental principles, rather than blind reliance on detailed regulations. These Principles should influence software engineers to consider broadly who is affected by their work; to examine if they and their colleagues are treating other human beings with due respect; to consider how the public, if reasonably well informed, would view their decisions; to analyze how the least empowered will be affected by their decisions; and to consider whether their acts would be judged worthy of the ideal professional working as a software engineer. In all these judgments concern for the health, safety and welfare of the public is primary; that is, the "Public Interest" is central to this Code.

What these codes do provide is a set of shared expectations and obligations that can help define the profession and its commitment to serve the public. That in itself is important.

---

33. Plato, Gorgias, trans. Donald J. Zeyl, Indianapolis, IN: Hackett Publishing Co., 1987, p. 26.
34. Lawrence Kohlberg, The Philosophy of Moral Development, New York: Harper & Row, 1981.
35. See James R. Childress, "Just-War Theories," Theological Studies, Vol. 39, 1978, pp. 427-445.
36. See John Arras and Robert Hunt, Ethical Issues in Modern Medicine, Second Edition, Palo Alto, CA: Mayfield Publishing Co., 1983, pp. 3-6.
37. http://www.acm.org/about/code-of-ethics. See Ronald E. Anderson, "The ACM Code of Ethics: History Process and Implications," in Chuck Huff and Thomas Finholt, eds., Social Issues in Computing: Putting Computing in its Place, New York: McGraw-Hill, 1994, pp. 48-62.
38. http://www.acm.org/about/se-code
39. Ronald E. Anderson, et al., "Using the New ACM Code of Ethics," Communications of the ACM, Vol. 35, No. 5, May, 1992, pp. 94-99.

Michael McFarland, S.J., a computer scientist, is the former president of College of the Holy Cross and was a visiting scholar at the Ethics Center.