|
Volume 25, Nos. 3 & 4 (2001) reprinted by permission Legal Studies Forum THE PALE CAST OF THOUGHT: ON THE LEGAL STATUS OF SOPHISTICATED ANDROIDS R. GEORGE WRIGHT* “Why do you cry, Gloria? Robbie was only a machine, just a nasty old machine. He wasn’t alive at all.” I. INTRODUCTION This article will, in a rather loose sense, test these alternative approaches by examining some of the direct and vicarious experiences we have had with the best-known sophisticated android, Lieutenant Commander Data of the Starship Enterprise. In particular, I focus on the occasion in which Data’s legal personhood5 was explicitly called into question and judicially resolved. In the course of our inquiry, it will turn out that the greater one’s personal interaction with Data, the more likely one is, generally, to think of Data as a person. We notice how difficult it is for anyone to deny or suspend belief in Data’s personhood without engaging in what appears to be something like an actual performative contradiction. Of course, any kind of contradiction can be resolved in more than one way. It’s possible that someone might decide, in the face of their own self-contradictory behavior, to consistently treat Data as a non-person. But just such a choice can then itself be judged on various grounds. We can then ask and investigate how a decision to treat Data as a non-person affects the treatment of other, utterly uncontroversial persons. This kind of consideration is loosely suggested by Kant’s discussion of the idea of duties toward animals and machines,6 but is also a pragmatic concern. We are asking at this stage how regarding Data as a non-person would tend to affect the moral dimensions of that person’s relationship with other persons, and, if we wish to broaden the inquiry, with non-persons other than Data. If it turns out that those who regard Data as a non-person tend to be specially insensitive toward or prejudiced with respect to some classes of uncontroversial persons, we might want to infer that morality requires treating Data as a person. If it were possible to perform an observational test,7 with a clear and objective outcome, of the being in question to determine personhood, recourse to performative contradictions would not be necessary. But there does not seem to be any such test.8 Nor, for that matter, can we simply decide minimally close cases of personhood in favor of the putative person. A bias toward progressive inclusion has appeal. But every decision in favor of inclusion within personhood unavoidably leaves a new borderline case to decide. It is not obvious, despite the moral value of personhood inclusiveness, that morality counsels us to limitlessly expand the personhood category through endless incorporation of each new borderline case. There might admittedly be moral value in deciding close cases in favor of personhood.9 But as the class of personhood expands, the resulting moral conflicts and general moral costs increase. Moral and legal personhood, despite their intrinsic moral value, also confer relative status in the event of conflicts of interest. As we expand the class of recognized moral and legal persons, we confer not just desirable rights in general, but rights as against other sorts of persons, including those who are most clearly persons. At some point, as with protozoans or wheat grains, the costs of conferring personhood and its associated rights may come at the substantial expense of the interests of animals, including humans.10 Would we want cows to starve rather than, say, violate the rights of wheat? At some point, the limitless expansion of personhood begins to lose its appeal. Of course, we may, for all we know, have much distance to travel in expanding the scope of the category of personhood before severe and unattractive tradeoffs begin to arise. Let us turn, then, to the case of the particular sophisticated android with whom, it is fair to say, we have the most vivid and detailed set of experiences, Commander Data. The scripted version of the episode11 opens at a holographic, swimming pool created aboard the Starship Enterprise. Data joins his colleagues and fellow officers Riker, Geordi, Worf, Dr. Pulaski, and Wesley Crusher. Data is preparing to swim for the first time, and is, apparently, anxiously attempting to reassure himself through reciting an Archimedean mantra concerning specific gravity.12 Ironically, Data is at this point emphasizing what he shares even with mere inert substances, rather than, in classic Aristotelian13 or Thomistic14 terms, how he might claim to possess categorically higher capacities. Dr. Pulaski immediately calls attention to Data’s status by referring to “CPR . . . or whatever you do to an android.”15 Data’s swim is comically unsuccessful, due to his apparent miscalculation, and the humans, along with Worf, manifest various sorts of genuine concern for his welfare. Data, however, eventually emerges unscathed. Picard’s relationship with Captain Phillipa Louvois, Data’s ultimate judge, is sufficiently rich to raise a possible recusal issue, which, given the limitations of her new Starbase, no one seems inclined to pursue. Their prior relationship is, as it turns out, linked to the legal adversary process as an engine for discovering the truth. Dr. Maddox, backed by Star Fleet’s Admiral Nakamura, announces his intention to refit, upgrade, and multiply Data. Dr. Maddox consistently refers to Data as an “it,”16 unlike virtually everyone else who regularly interacts with Data. Data suggests that Dr. Maddox’s proposed experimental procedures pose risks, including to his identity that Dr. Maddox rejects as absurd and unrealistic.17 It seems clear that were Data human, Data’s informed consent to the proposed procedures would be required.18 Captain Picard has, early on, perhaps understandably focused on his own prerogatives, the welfare of his crew, and the efficiency of ship operations. The deeper issues are first alluded to by Data’s friend Geordi. Picard’s focus changes a bit in response to Admiral Nakamura’s belief that Data is only a machine, and that “an army of these machines”19 may, if Maddox is right, be in the offing. Picard, in response, includes Data among his “people,” and when Admiral Nakamura insists again that Data is “a machine,” Picard adds firmly, if rather hazily, “and so are we, just a different kind of machine.”20 It is unclear whether Picard’s view at this point is that we are all just machines, or that some machines, including Data and humans, are also ensouled machines with an essential spiritual element. Admiral Nakamura breezes past this crucial distinction, countering, as do the personified Laws in Plato’s Crito,21 that we as Data’s builders retain certain rights over him. Picard responds that making or building something does not necessarily confer broad, permanent ownership rights.22 Later, Data attempts to articulate his reservations to Picard. Data, who has on many occasions put his life at serious risk for crew and mission,23 reports that he considers Dr. Maddox’s proposed procedure to be “too much like dying,”24 even if the procedure were in some sense a success. Picard responds by opining that humans do have, or themselves are, spirits, but these spirits depend on constant containment within an assigned bodily container, where as Data’s does not.25 At this point, Data articulates his concern for a possible loss of the continuity of his identity, of who he is, even if Maddox’s experimental upgrades are successful. While software and programmed information can be temporarily stored elsewhere and then returned to a rebooted unit, Data fears, however inarticulately, that the rebooted self will be a different, truncated self.26 At this point, Picard observes, oblivious to its implications, that Starfleet officers all take a binding, solemn oath to serve.27 Data points out the clear superiority of Geordi’s artificial eyes to the natural biological human eye and wonders why humans cannot be compelled to undergo upgrades. Picard has no answer. Doubtless humans have a certain attachment to much of their original equipment, except in the case of cosmetic surgery or a purportedly enhanced appearance. Data draws the conclusion that he would not be required to undergo the procedure if he were human. Picard is left speechless. Picard, disturbed by the encounter with Data, confesses to Riker a “distressing tendency toward bigotry”28 in the case. By themselves, words like bigotry, prejudice or discriminatory are ambiguous. Picard might mean that he has assumed Data to be a merely mechanical non-person, and that Picard now sees that such a conclusion is wrong on the merits. Or Picard might mean that he had assumed Data to be merely a mechanical non-person, but that he should have held this as an open, unresolved question. Picard himself confesses to some uncertainty as to his own attitude.29 Is Data as a new life-form or a mere machine? Picard then goes to his nemesis/neutral arbiter Captain Phillipa Louvois, who seems to regard Data as a machine, and who like Picard fails to recognize the contradiction in holding Data responsible for risks entailed by his voluntary and free choice to enter Starfleet30 and his oath as a Starfleet officer while holding to the notion that Data is a machine incapable of choice and oath. Phillipa’s initial assumption that Data has a valid legal right to resign from Starfleet and thus avoid Maddox’s experiment bespeaks yet another contradiction. Toasters, or computers, cannot voluntarily resign. At this point, Picard speaks in the language of fairness, justice, and rights held by Data. In a charged scene, Data then responds to what he apparently takes to be Maddox’s impertinent intrusion by pointing out that “[i]t is customary to request permission before entering an individual’s quarters.”31 Data thereby implicitly asserts the dignity of a person, as opposed to the sheerly instrumental, unfeeling, insensate status of a complex mechanism. Their desultory conversation is of interest for Data’s assertion that “living is not composed merely of the accumulation of information, but by the very experience of life,”32 and, more dramatically, that there is an “ineffable quality” to his memory and an untransferable soul or heart of his experiences that cannot simply be downloaded as Maddox imagines.33 Maddox announces, fervently if ambiguously, that he does not care how Data “feels.” While Maddox continues to refer to Data as a human appearing “it,”34 he is not above the tendency to anthropomorphism, as when he refers to his anticipated legions of Datas as “our faithful automatons.”35 At this point, Phillipa responds by perhaps inadvertently including Data within the class of “people” not subject to seizure in the name of scientific advance.36 Those who do not know Data refer to him as an “it,” but then gradually use more personal language as they gain more direct experience with Data.37 Maddox argues that the Enterprise’s central computer would not have the right to refuse an upgrade even if that computer were somehow so disposed.38 Picard has begun to focus on the familiar administrative law principle that Starfleet, like any other similar authoritative institution, should not “ignore its own regulations when they become inconvenient.”39 It will turn out to be this general principle of consistency, commitment, or estoppel40 that gives bite to the contradictions in which Starfleet and Data’s antagonists regularly ensnare themselves. It is more difficult than they realize to consistently think of Data as a machine. Data clearly has the capacity to at least mimic, in a typically endearing way, human emotional responses and human foibles.41 Such behavior largely drives the inquiry into whether Data is, if not in any sense human; a “new life-form” as opposed to a machine;42 whether he has emotions or a soul,43 whether he is sentient,44 conscious,45 or self-conscious;46 whether he has interests;47 or genuine intelligence;48 or is merely a lower-order entity that may belong, as property,49 to a person or group. The judicial hearing itself is to be presided over, and the facts and legal outcome to be determined, by Captain Phillipa Louvois. She has, contrary to general administrative law principles,50 structured the hearing as a sort of de novo appeal from her own summary ruling that Data is not a sentient being, but mere property, and therefore lacks the legal right either to refuse Maddox’s refit or to resign from Starfleet. Phillipa argues that a judgment in Data’s favor would result in collateral estoppel51 or res judicata52 with the effect that, in Phillipa’s words, “he’ll never need fear the Doctor Maddoxes any longer.”53 Again, we are left to wonder whether Phillipa’s reference to Data’s ‘fear’ is mere casual, convenient anthropomorphism, of the sort Daniel Dennett reports of Maine loggers with regard to trees,54 or yet another instance of the contradictions in which Starfleet finds itself ensnared. By regulation, it falls to Commander Riker, the direct subordinate of Captain Picard, to judicially represent the Starfleet view that Data is non-sentient and a non-person, with no legal rights beyond, of course, those procedural rights that could not be denied without making the hearing meaningless. Commander Riker is profoundly disturbed at being placed in this position, not from the conflict inherent in opposing Captain Picard, who will be defending Data, but from Riker’s horror at playing a role in any official denial of Data’s personhood, in which Riker has come to entertain no doubt. Data, ever-reflective, echoes in conversation the classic arguments of Plato’s Crito. In particular, Data observes that “in many ways I have always considered myself to belong to Starfleet. You activated me, rescued me, educated me, employed me.” Ironically, it is just such grateful sentiments on Data’s part that typically lead us to believe that Data is not only a person, but a responsible person, even worthy of emulation. Data’s admission of his many social dependencies, and his avoidance of false claims of self-sufficiency, make him sound all the more human.55 His detachment seems the detachment of responsible moral reflection, not of mechanical isolation. As the discussion continues, Picard’s defense of Data’s personhood is premised on his qualities of intelligence and sentience and that Data can “belong to no one” but himself.56 A critic might argue that Data cannot have meaningful rights unless he also has genuine interests, on the view that interests are prior to rights. Whether Data has interests is an open question, and one that perhaps should be central to the judicial inquiry. Data himself seems at least casually to assume that he does possess, as in his response to Picard that he, Data, has “complete confidence in your ability to represent my interests.”57 Meanwhile, Riker’s preparation for Data’s hearing is dogged by conflicts and ambivalence. His faith in the logic and practice of adversarial litigation, at least in this case, is utterly nonexistent. He recognizes that the official Starfleet view have been met with veiled to overt hostility on the part of the crew. There is sentiment among the crew that Riker’s adversarial role involves a betrayal of his friend Data. Again, the language is revealing. We do not betray toasters, or even complex machinery. The idea of betrayal suggests the violation of a faith and confidence properly engendered in a conscious, interested being to whom something better is owed.58 Riker calls Data to the witness stand. No oath of truth-telling is administered, allowing Riker and Starfleet to evade a potentially devastating contradiction: if an oath of truth-telling amounts only to Data’s mechanical speech act predicting that he will not objectively mislead the tribunal, no contradiction need be involved. But if an oath of truth-telling must be undertaken knowingly and of one’s own free will, as a solemn commitment to choose one’s answers with integrity, or to deliberate before answering, would concede that Data is capable of understanding, agreeing to, and adhering to such an oath and would torpedo the case against him. Instead, Data is scanned on the witness stand, and the computer voice begins to announce his name, rank, assignment, and service record.59 Riker interrupts to object to the recitation, and Picard insists that the recitation continue. Phillipa Louvois overrules the objection, and we hear, interestingly, that among Data’s awards is something called the Starfleet Command Decoration for Gallantry.60 Again, a contradiction looms. Does Starfleet, or anyone else, solemnly and officially credit machines, sophisticated or otherwise, with the quality of gallantry? Has the Enterprise’s computer, given its life-saving work performances, ever been seriously considered for any comparable award? The emotional highlight of the hearing, comes when Riker removes Data’s hand, irreverently toys with it, and then, without apparent consent, melodramatically presses Data’s on-off switch, leaving him, as Riker has it, a Pinocchio with his strings cut, built by an inventor and now temporarily deactivated by a mere man.61 Riker’s theatrics aside, he has established nothing that was in contest, or that could not have been shown in less degrading fashion. While Riker thereby dramatically shows how Data differs from most humans who cannot be dissembled or switched off. But we cannot limit the concept of a person to only those beings who very closely resemble most human beings in even the most superficial respects. We can certainly imagine a genuine person with a detachable limb or even an off switch. We can also imagine Riker’s theatrics backfiring if an adjudicator saw those tactics as an assault on Data’s real or at least arguable fundamental dignity and right to bodily integrity.62 After a court recess and an initially despairing consultation with Guinan,63 Picard begins his rebuttal by noting the limited significance of Riker’s demonstration. Picard then seeks to metaphorically humanize Data by establishing that Data has kept his Starfleet medals not from any narrowly utilitarian or functional motive, but, perhaps, from sheer vanity, a distinctly familiar human foible.64 Picard gradually overcomes Data’s reticence to reveal his intimacy with former crewmate Tasha Yar. Phillipa Louvois is at least momentarily startled by this revelation; whether it plays much of a role in her ultimate disposition of the case is left to conjecture. From there, Picard borrows Riker’s tactic of establishing the uncontested. Based on Data’s sophisticated verbal responses, it appears to any neutral observer that Data is aware of his physical and social surroundings, and of the nature of his legal jeopardy.65 Data appears to manifest sufficient self-awareness to qualify for personhood. Picard then closes with the parlor stunt of establishing that Commander Maddox cannot clearly and objectively prove that Commander Riker is himself a genuinely rational and intelligent person.66 Maddox retreats to a bare, insistent intuitionism, which Picard immediately compares to the discredited intuitionist method of recognizing pornography. Phillipa, acting as the judge, pauses only briefly after Picard’s closing argument. She finds Data to be a legal person rather than property, and to possess, among other rights, the right to refuse Maddox’s experimental refit. The script then concludes as Phillipa claims partial vindication of the adversarial system, and as Riker reconciles with the apparently understanding, forgiving, modest, and grateful Data.67 The Data story presents most of the familiar considerations in determining who should count as a person.68 The adversarial battle has focused on considerations such as intelligence, consciousness or self-consciousness, sentience, and even ensoulment. We would, however, probably not want to say that Data is a person, even if he possesses all the above, if nothing that Data does, that happens to, or that is done to Data ever genuinely matters to Data. On this point, we can perhaps look to moments such as Data’s awkward expression of the loss he will feel in missing his friend and crewmate Geordi, or his understandable desire to keep his holocube portrait of Tasha Yar. As far as we can tell, Data seems not only genuinely self-conscious, but to have genuine subjective and objective interests as well, including the desire not to be riskily dismantled. in Data,69 and concedes that this may be due either to the sheer alienness of Data’s mind,70 or to its utter nonexistence. A problem arises in Starfleet’s institutional decision to administer and accept Data’s sworn oath to Starfleet as a free and knowledge act. Presumably Starfleet does not administer or receive oaths from toasters, computers, or other entities, incapable of such oaths. In now claiming that Data is and presumably always has been incapable of a solemn oath, Starfleet seems to be placing itself in a rather awkward logical bind. Pointing out the existence of something akin to a performative contradiction can be a powerful rhetorical and logical weapon. It is difficult, for example, to convincingly argue one’s own current non-existence.71 It is also difficult for a free and voluntary inquirer to contend that broad-based knowledge is not a good thing.72 Historically, the institution of slavery has embodied and been discussed in terms of various sorts of performative contradictions.73 Consider, by way of loose example, Frederick Douglass’ account of a practice engaged in by one of his slavemasters: When I carried to him my weekly wages, he would, after counting the money, look me in the face with a robber-like fierceness, and ask, “Is this all?” He was satisfied with nothing less than the last cent. He would, however, when I made him six dollars, sometimes give me six cents, to encourage me. It had the opposite effect. I regarded it as a sort of admission of my right to the whole. The fact that he gave me any part of my wages was proof, to my mind, that he believed me entitled to the whole of them.74 It is difficult, if not impossible, for slaveowners to avoid grotesque logical embarrassments. Nor is the problem of performative contradiction in the legal sphere confined to contexts as invidious as that of slavery as Jürgen Habermas makes clear in his work on communicative ethics.75 Any contradiction, however, admits of two possible resolutions. Starfleet might now contend, for example, that its only error was in originally believing Data to be capable of a genuine oath. There is thus no guarantee that any performative contradiction of which Starfleet is guilty would have to be resolved in favor of acknowledging Data’s personhood. And since it also seems at least possible that even prolonged, careful empirical observation of Data will not tell us whether he is conscious,76 we may well need a more comprehensive test for personhood, perhaps of the sort suggested by Immanuel Kant. According to Kant, we cannot owe moral duties to non-human animals.77 But Kant does not conclude from this that what we do to animals is of no moral significance. Treating animals kindly or cruelly, Kant assumes, tends indirectly over time to shape and be reflected in the moral quality of our treatment of genuine persons.78 Thus Kant concludes that “[t]ender feelings toward dumb animals develop humane feelings towards mankind.”79 Kant would presumably be open to the idea that even if we consider Data as a complex machine, our treatment of Data may somehow come to affect the morality of our actions toward persons.80 We need not assume that Data is, or is not, a person. We would instead assume that given the closeness of Data’s case, some persons would choose to consider Data as a person, and others would not. As our experiences accumulate, we would compare the decision as to Data’s personhood with their treatment of uncontroversial persons. So we might ask of animal rights activists: Do they tend to act in a more praiseworthy way toward other human beings than do persons who are not animal rights activists? We should not assume, without evidence, that those who regard Data as a person will tend to be more moral generally than those who do not. One might speculate that those inclined to expand the class of persons might also tend to dilute the duties owed toward all persons. (We can imagine a dog-exalting misanthrope.) Professor Colin McGinn has recently raised the issue of the possible conflicts of interest between robots and uncontroversial persons. Should they [robots] be allowed to get married and reproduce themselves? How do their rights compare to the rights of animals? Are some robots “more equal than others?” Will it ever be okay to sacrifice a single human to save a thousand innocent robots?81 In the absence of satisfactory empirical tests for the qualities we choose to deem relevant,82 we may wish to address such questions through each of the steps described above, in succession. Of course, a communicative ethics presumably accords great moral significance to the present or future ability to “speak” or otherwise communicate in demarcating the bounds of the moral community. See, e.g., Jürgen Habermas, Moral Consciousness and Communicative Action, supra, at 89 (referring to “every subject with the competence to speak and act”); Jürgen Habermas, Justification and Application, supra, at 50 (referring to “an ideal communication community encompassing all subjects capable of speech and action”); Thomas McCarthy, THE CRITICAL THEORY OF JÜRGEN HABERMAS 306 (1996) (1978); Simone Chambers, “Discourse and Democratic Practices,” in The Cambridge Companion to Habermas, supra, at 233, 238 (“no one with the competency to speak may be excluded from discourse”); Karl-Otto Apel, “Regulative Ideas or Truth-Happening?: An Attempt to Answer the Question of the Conditions of the Possibility of Valid Understanding,” in THE PHILOSOPHY OF HANS-GEORG GADAMER 67, 86 (Lewis Edwin Hahn ed. 1997)(referring to “partners in the discourse”); Robert Alexy, “A Theory of Practical Discourse,” in The Communicative Ethics Controversy, supra, at 151, 166 (referring to Habermas’s view that “[a]nyone who can speak may take part in discourse”). See also Daniel Dennett, “Conditions of Personhood,” in THE IDENTITIES OF PERSONS, supra note 52, at 178. Actually it is far from clear whether Habermas would want his general emphasis on the ability to speak, or even to communicate, to translate directly into the context of our own concerns. Should the ability to communicate, let alone to speak, constitute either a necessary or a sufficient condition of moral personhood? See, e.g., Daniel C. Dennett, KINDS OF MINDS 12 (1996)(“[t]alking . . . is not necessary for having a mind”). Could there be a clearly non-conscious robot, clearly lacking personhood, who nonetheless can speak or communicate? Do some or all animals who can feel pain also have the ability to communicate? Habermas himself seems to extend his focus to include all those who will somehow be affected by application of a particular norm. See Habermas, Moral Consciousness and Communicative Action, supra, at 65 (“valid norms must deserve recognition by all concerned”)(emphasis in the original); David Couzens Hoy & Thomas McCarthy, supra at 55 (“as Habermas conceives of it, practical discourse demands of participants that they take into consideration the needs and interests of all those who will be affected by the outcome of their deliberations”). Habermas may thus not intend a communicative ethics to answer the question of who counts, beyond including all communicative human persons. 76. See, e.g., Amélie O. Rorty, Slaves and Machines, 22 Analysis 118, 120 (1962) (following Kant in arguing that willing, resolving, and making moral decisions are not subject to empirical observation). See also supra note 6 and accompanying text, as well as Professor Steve Pinker’s discussion of the robot Alicia in Steve Pinker, Can a Computer be Conscious?, 123 U.S. News & World Report 63 (Aug. 18, 1997)(no experimental test for sentient consciousness possible). 77. See Immanuel Kant, LECTURES ON ETHICS 239 (Louis Infield trans. 1963)(1930). 78. See id. at 239-40. 79. Id. at 240. 80. For Kant’s brief extension of his discussion of animals to the context of inanimate objects, see id. at 241. 81. See McGinn, supra 2, at 177. 82. Beyond questions of self-consciousness we might, consider the capacity of a being for having “second-order” desires. These would include desires to maintain or to change one’s basic priorities or one’s present desires, as when we wish to break a habit. See, e.g., Harry G. Frankfurt, “Freedom of the Will and the Concept of a Person,” in John Martin Fischer (ed.), MORAL RESPONSIBILITY 65, 67 1986)(“[n]o animal other than man . . . appears to have the capacity for . . . the formation of second-order desires). Cf. Daniel Dennett, “Conditions of Personhood,” in THE IDENTITIES OF PERSONS, supra note 54, at 181 (“[n]ow are human beings the only second-order Intentional system so far as we know? I take this to be an empirical question”). This kind of inquiry, assuming its moral significance, only takes us so far. In some sense, Data clearly appears to have and indeed emphasize second order desires, most especially to become more fully or more nearly human, or to better appreciate what it is like to be human, as in a remarkable conversation between Data and Mr. Spock, in which the two confess to directly opposing attitudes toward the role and value of emotion. But this leaves open the crucial question: does Data really have second order desires, or has he merely been designed and programmed to effectively mimic the behavioral dimension of genuinely having second order desires? |
