From the Bridge: Chatbot Confidential: Machine Magic and the Power of Language
“In the beginning was the word. Language is the operating system of human culture. From language emerges myth and law, gods and money, art and science, friendships and nations — even computer code. A.I.’s new mastery of language means it can now hack and manipulate the operating system of civilization. By gaining mastery of language, A.I. is seizing the master key to civilization, from bank vaults to holy sepulchers.” From “If We Don’t Master A.I., It Will Master Us,” Yuval Harari, Tristan Harris, and Aza Raskin, NYT Opinion Page, 3/24/23.
Note the range of reference in the cautionary paragraph above. We move from religious allusion to technological language and sweeping pronouncements, with “operating systems” spanning realms of law, commerce, human relationships, and spirituality. There’s jargon (“hack”), threat (“manipulate”), and panoramic scope regarding the shaping cultural power of language, with the master key “seized” by a machine. While we have the invocation of “human culture,” we don’t have human beings. Agency belongs to language and AI. Now listen to a couple of human beings musing in words for a living:
“I could barely sleep that night. To observe an A.I. system — its software, microchips and connectivity — produce that level of originality in multiple languages in just seconds each time, well, the first thing that came to mind was the observation by the science fiction writer Arthur C. Clarke that ‘any sufficiently advanced technology is indistinguishable from magic.’” From “Our New Promethean Moment,” Thomas Friedman, NYT Opinion Page 3/21/23.
“Our ancestors knew that open flame was a feral power, which deserved reverence and even fear. The same technology that made civilization possible also flattened cities. The ancient myths about fire were never simple. When Prometheus stole it from the gods, he transformed the life of mortals but was doomed to live in agony. The people building artificial general intelligence today don’t need media mythmaking to inflate their ego; they already clearly believe in the humanity-altering potential of their invention. But it is a complex thing, playing at Prometheus. They have stolen from the realm of knowledge something very powerful and equally strange. I think this technology will expand our minds. And I think it will burn us.” Derek Thompson, “Work in Progress” in The Atlantic Daily, 3/21/23.
Friedman and Thompson, working with language, consider human agency and human feeling. Friedman can’t sleep after observing AI in action. He shares a well-known observation from Clarke about technology and magic. Thompson ponders the ambition and agony of Prometheus, the egotism of techies following in Promethean footsteps, and the implications for himself and his readers.
I want to contemplate various pronouncements about and implications of this moment by connecting back to a conversation about language that unfolded some 2400 years ago, when writing qualified as a new technology, the written record providing us a window on the building of a (not the) rhetorical tradition, though one that remains much with us in universities. Let’s talk Gorgias, Plato, and Aristotle.
Gorgias—Speech Is a Powerful Lord
Gorgias of Leontini traveled from Italy to Greece as an emissary from his government to Athens. According to Bizzell and Herzberg in their biographical notes about him in their magisterial (and inching toward inclusivity with each new edition) The Rhetorical Tradition, he may qualify as one of the first improvisers, composing speeches (bits?) on the spot from audience suggestions. Imagine an Athenian calling out, “Helen and that Trojan War thing,” leading Gorgias to deliver his “Encomium on Helen,” one of the few surviving texts attributed to him. The orderly four-part argument and structure of the encomium suggest that Gorgias did not improvise his defense of Helen. He addresses in turn four forces that compelled her to leave home for Troy: “Either by the wishes of Fortune and plans of the gods and decrees of Necessity she did what she did, or abducted by force, or persuaded by speeches, <or conquered by Love>.” I draw on the Donovan translation here rather than the Kennedy, though the section heading comes from Kennedy’s translation. In the section on persuasion by speech, Donovan translates the phrase as “discourse is a great potentate,” which to my ear doesn’t have the same zing. For both, though, Gorgias clearly equates the power of language with the power of the gods, physical force, and love (which Gorgias notes comes from the gods). In the section on speech, Gorgias continues his description of the power (italics mine):
Persuasion belonging to discourse shapes the soul at will: witness, first, the discourses of the astronomers, who by setting aside one opinion and building up another in its stead make incredible and obscure things apparent to the eyes of opinion; second, the necessary debates in which one discourse, artfully written but not truthfully meant, delights and persuades a numerous crowd; and third, the competing arguments of the philosophers, in which speed of thought is shown off, as it renders changeable the credibility of an opinion.
The power of discourse stands in the same relation to the soul's organization as the pharmacopoeia does to the physiology of bodies. For just as different drugs draw off different humors from the body, and some put an end to disease and others to life, so too of discourses: some give pain, others delight, others terrify, others rouse the hearers to courage, and yet others by a certain vile persuasion drug and trick the soul.
So, while recognizing the seemingly limitless power of language, Gorgias in this analysis of how speech works offers us warnings regarding deceitful speakers in debates in civil and philosophical contexts. His warnings here surface again in Plato’s Gorgias, but voiced by Socrates rather than the character Gorgias.
A final note—one distinctive feature of the encomium concerns what I’ll call the absence of agency, mostly in relation to Helen but also general audiences. Gorgias gives her no choices to make in his attempt to absolve her of blame. Neither does Gorgias develop the character of Paris the abductor. While he likely belongs in one of the denigrated categories Gorgias itemizes above, speech has the power more than any individual named Paris. Speech uses him as an instrument more than he controls speech, at least in how Gorgias describes the events and lays out a philosophy of language.
Plato’s Lament—That’s Some Bad Magic There
Plato portrays Gorgias as a doddering fool in the Gorgias, easily dispatching him to silence early in the dialogue by getting him to admit that rhetoric is akin to cooking (persuading people to crave and consume tasty food over healthy food) and that rhetoric endangers the wellbeing of its victims with the example of Gorgias persuading a patient to follow his bad advice rather than the good advice of a doctor with actual knowledge. (An early draft of the dialogue disparages him as Gorgias George, a peripatetic Greco-Roamin’ wrestler known for his flowing tresses and goading addresses hurled at opponents in the ring—all that remains in the version we have is Gorgias claiming that the teacher of wrestling does not bear responsibility for the mayhem wrought by a student turned villain.) Callicles, a calculating Machiavellian politician, serves as the main antagonist in the Gorgias, reveling in how his prowess with language helps him dupe the populace into giving him power and wealth. He embodies one category Gorgias identifies in the encomium, “delighting and persuading the numerous crowd” for his own selfish ends. Why this choice, to sideline Gorgias and engage at length with Callicles? What does this suggest about Plato’s anxiety about the power of speech highlighted in the encomium?
At the end of the Gorgias, after Callicles falls silent, sullenly, not because Socrates has persuaded him of anything regarding the harm to himself (more specifically his soul) and others stemming from his evil use of rhetoric, Plato has Socrates offer a story about the soul after death, which must appear “naked” in a place of judgment, stripped of all worldly things though displaying the marks of actions performed while alive. He makes a similar narrative move in the Phaedrus, his other dialogue focused on rhetoric, there including the image of the soul as a chariot drawn by two horses, one good and one evil, with the charioteer struggling to keep to a path that moves toward the gods and heaven. These concluding narratives go further than the analogies drawn earlier, used to disparage rhetoric by comparing it to flattery, cooking, and wrestling, not an art and not really a techne either, just a knack. The stories rely on the power of metaphorical language to persuade (and the reading audience’s imaginative engagement with narrative), not the dialectical logic of Socratic dialogue or the analogies drawn from everyday life.
Thus Plato’s rhetorical legacy offers us more than one rhetorical strategy. The painstaking questioning of the Socratic method persists in many classrooms (and merits mention in many teaching philosophy statements). The narratives work on us at what we might call the soul level, engaging our imaginations and emotions. Perhaps Plato silences Gorgias preemptively in the dialogue bearing his name because Plato too believes in the overwhelming power of language in human interactions, and he wants to reserve that power for Socrates to wield in his narratives regarding the soul. The Phaedrus even concedes the possibility of a rhetoric aimed at true engagement with the soul, rhetorical power used for good.
Aristotle—Watch Me Pull a Rhetoric Out of My Hat
Aristotle, Plato’s student, becomes a teacher of rhetoric, requiring him to respond to his teacher’s objections to the knack. He takes pains to lay out a rhetorical system, brimming with taxonomies describing (prescribing?) the contexts for rhetoric, canons of rhetoric, rhetorical appeals, general and specialized means of argument, the parts of a speech, and a compendium of schemes and tropes.
Unwittingly, Aristotle designed a system that could be put to use by a future educational industrial complex (EIC), complete with a testing/sorting apparatus and a range of gatekeeping mechanisms. The system turns both teachers and students into rhetoric machines and removes human interaction in the dialogic/dialectic sense from the mass-schooling system. A Plato-endorsed system centered on dialectical conversation survives in the Socratic method in relation to dialectical inquiry aiming to establish absolute truth via logic, though the EIC also turns that into a gatekeeping mechanism; more prosaically, we can discern traces of Plato’s preferences in both the Oxford tutorial system and the seminar classroom format privileging conversation without the same level of rigor re definitions and syllogistic structures.
None of the philosophers-in-conversation (PIC) imagined the EIC and the need to teach rhetoric to masses of young people.
We can tell the story of Aristotelian rhetoric as a domestication of magic, sorting of good and bad magic (Aristotle states early in the Rhetoric that protecting oneself from evil speakers constitutes one reason to learn rhetoric as well as that speakers should only use rhetoric for ethical ends); Aristotle establishes rhetoric as a techne, a well-ordered procedural middle ground providing a guide to everyday practical magic. Despite the definition he gives—the ability to discover the available means of persuasion in any situation—and the placement of invention as the first canon of rhetoric, the taxonomies generate an unfortunate (that’s an understatement) pedagogical side effect when the EIC embraces Aristotle’s system: too practical and streamlined a pedagogy positions language and rhetoric as all surface and little depth, all labels and little substance. Skimming what can be memorized from the taxonomies for lesson planning leads to the displacement of rhetoric as a fundamentally human art—arranging words becomes an exercise in producing language to be ignored, seen through, or denigrated as menial labor divorced from intelligence, thinking, and higher order functioning in the world. Writing and speaking instruction allows the school-based domestication of the power of language, substituting form for substance, surface for art (with creative writing as a realm of meaningless magic and degraded power, though literature retains some glamor and access to fame for a few magical writers).
As noted earlier, Aristotle could not imagine a world of mass education and the potential for his taxonomies to become grist for composition pedagogy and high school AP courses incorporating rhetoric. He designated the three branches of rhetoric as judicial, forensic, and epideictic, offering training for legislators, lawyers, and anyone speaking in the public sphere for official ceremonial purposes. (He leaves out the religious sphere, thus not offering anything about sermonic rhetoric.) As education expanded, educators needed taxonomies, and Aristotle functions as a go-to taxonomist in his orderly rendering of rhetoric, ready-made for translation into templates.
They Say, I Say, We All Say
Amazon blurbs the fifth edition of Birkenstein and Graff’s They Say, I Say, this way: ”Used and loved by millions of students for its lively and practical advice, this is the book that demystifies academic writing and shows how to engage with the views of others. Extensively revised in response to feedback from our community of adopters, this edition of ‘They Say / I Say’ is an even more practical companion for students, featuring a new chapter on research, new exercises, expanded support for reading, and an expanded chapter on Revising.” Jeremy Garber’s five-star Amazon review from May 2022 praises the book this way:
Faculty often think students can’t learn to write better. Gerald Graff and Cathy Birkenstein think otherwise, and show us all a simple way to help them do so through the use of writing templates. These two opening sentences model the way that the authors have successfully coached students in the art of writing academic arguments – ‘they say,’ and ‘I say.’ They begin in their preface by correctly observing that ‘The trouble is that many students will never learn on their own to make the key intellectual moves that our templates represent. While seasoned writers pick up these moves unconsciously through their reading, most students do not.’ The rest of the book provides specific templates that help students correctly and sympathetically summarize the arguments of the texts they’re reading, and then provide their own agreements and/or disagreements with clear and reasoned evidence. The authors also pay careful attention to maintaining students’ own voices in writing (yes, it’s OK to use ‘I’!) and the communal, dialogical nature of research and argument. The book closes by providing models of online discussion and writing for several specific fields, including humanities, science, and social science, and several readings for students to analyze using the tools in this text. Online resources also include online tutorials, quizzes, and resources for particular LMSes. As a graduate-level writing director, I plan to use Graff and Birkenstein’s template on a regular basis – students have found these templates not pedantic or babyish, but eminently useful.
So much to gloss here, but I’ll focus on what Garber quotes from the preface: “many students will never learn on their own to make the key intellectual moves that our templates represent. While seasoned writers pick up these moves unconsciously through their reading, most students do not.” Let’s think about this in relation to generative AI/ChatGPT, which has trained on billions of word, learning not “key intellectual moves” but the patterns of words that signal intellectual work in academic contexts as well as what Garber calls “the communal, dialogical nature of research and argument.” For students who haven’t read billions of words and don’t plan to, the magic of generative AI has both practical and supernatural aspects. The blurb lauds the practical value of the book; the review references Birkenstein and Graff’s fundamental insight that led to a publishing bonanza (I wonder how forthrightly the original book proposal expressed the premise): students don’t read as much or in the way we want them to, so here’s a book of templates to train students to put together the words that give the appearance of them having read more than they have. The book offers a small curated language model for students rather than the super-abundant large language model driving generative AI.
Keeping this in mind, ChatGPT and other generative AI trained on/taught by a vast array of texts influenced by school rhetoric functions as a perfect/average student. ChatGPT doesn’t make grammatical mistakes, generating clean and correct prose. ChatGPT doesn’t make rhetorical choices based on personal lived experience; rather, ChatGPT predicts surface-level rhetorical choices based on the choices of human writers trained in school rhetoric(s). We can consider the errors and hallucinations of ChatGPT a byproduct of the contingent forays human writers make into the world of language, a vapor exhaled by the swirling of billions of words. What happens when the large language model used to train/teach generative AI chatbots incorporates scads of AI-generated text (bound to happen)? What errors and hallucinations will come of machines loading words into machines?
How Generative AI Reinforces the Thinking/Writing Divide—An Aside on a Powerful Enthymeme
Louis Hyman, a historian who considers the intersections of technology and labor, offers this offhand observation about the potential usefulness of ChatGPT to writers in his 4/22/23 New York Times Opinion piece, “It’s Not the End of Work. It’s the End of Boring Work” (italics mine):
Researchers at the Massachusetts Institute of Technology recently conducted an experiment, reported in a working paper, with 444 “college-educated professionals” who were given a “midlevel professional writing task” like drafting news releases or delicate emails. Half were provided with ChatGPT and half weren’t. The participants who were given ChatGPT took less time, wrote better and reported enjoying the task more. Even more important, perhaps, ChatGPT helped “low-ability workers,” meaning that those with weaker writing skills — but perhaps with good ideas — could carry out the task effectively.
Many teaching philosophy statements include assertions that “writing is thinking.” Enter the phrase into Google and you’ll get 1,300,000,000 links. Though she doesn’t use the term in her title, Janet Emig in her oft-cited 1977 CCC article “Writing is a Mode of Learning” influenced the budding discipline of rhetoric and composition and perhaps inspired the more anthemic hook “writing is thinking.” (Imagine the possible rock songs—“Writing is Thinking” likely works better in a stadium or arena setting than “Writing is a Mode of Learning,” which strikes me more as a song delivered by a sensitive singer-songwriter in a coffeehouse--both offer ready rhyming possibilities for the chorus, whether “inking, drinking, blinking, and sinking” or “burning, earning, yearning, churning.”) Anyway, where were we… Oh, I often say that for the purposes of writing pedagogy, we shouldn’t separate writing and thinking at all, instead treating them as so intertwined that teasing them apart does more harm than good, allowing for the objectifying of language as somehow separate from the work of world-building.
ChatGPT feeds into that separation by untangling writing and thinking. Let the machine do the writing, I’ll do the thinking. ChatGPT exposes the stark limits of school rhetoric with its arbitrary rules and expectations—a machine can generate prose that pleases (and fools) the teacher. My thinking, though, remains my own, safe from the teacher’s judgments, corrections, and suggestions.
Are We Stochastic Parrots?
Elizabeth Weil’s March 2023 TIC-adjacent New York Magazine piece “You Are Not a Parrot” features this cast of characters: Emily Bender, Computational Linguist; Timnit Gebru, AI Ethicist (fired by Google); Margaret Mitchell, AI Ethicist (fired by Google); Sam Altman, OpenAI CEO; Christopher Manning, Computational Linguist, Head of Stanford Artificial Intelligence Lab; and Judith Butler, Critical Theorist, UC Berkeley.
Weil: Stochastic means (1) random and (2) determined by random, probabilistic distribution. A stochastic parrot (coinage Bender’s) is an entity “for haphazardly stitching together sequences of linguistic forms … according to probabilistic information about how they combine, but without any reference to meaning.” In March 2021, Bender published “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” with three co-authors. After the paper came out, two of the co-authors, both women, lost their jobs as co-leads of Google’s Ethical AI team. The controversy around it solidified Bender’s position as the go-to linguist in arguing against AI boosterism.
Altman tweet: i am a stochastic parrot, and so r u.
Bender: This is one of the moves that turn up ridiculously frequently. People saying, “Well, people are just stochastic parrots.” People want to believe so badly that these language models are actually intelligent that they’re willing to take themselves as a point of reference and devalue that to match what the language model can do.
Butler: There’s a narcissism that reemerges in the AI dream that we are going to prove that everything we thought was distinctively human can actually be accomplished by machines and accomplished better. Or that human potential — that’s the fascist idea — human potential is more fully actualized with AI than without it... Some people say, “Yes! Isn’t that great!” Or “Isn’t that interesting?!” “Let’s get over our romantic ideas, our anthropocentric idealism,” you know, da-da-da, debunking… But the question of what’s living in my speech, what’s living in my emotion, in my love, in my language, gets eclipsed.
Consider your favorite go-to bits of language, whether mottos, bedrock beliefs, valued premises from which all else proceeds, jokes, exhortations. Where did they come from? What magic blending with your life experience led to the creation of your most cherished, most world-changing, most self-realizing bits of language?
Domesticated Magic and School Rhetoric
As a fledgling WPA at Arizona in the 1980’s, I had the great good fortune of having a role in inviting and hosting a number of Writing Studies scholars for colloquium talks to the Composition Program. Arizona at the time employed about 200 writing instructors, an amalgam of graduate student instructors (more than half the total instructional group), adjunct lecturers, and the UA equivalent of career-track lecturers. Along with the colloquium talk, guest scholars often participated in less formal conversations with small groups of interested instructors and some of us involved in program administration. On one of these occasions, Sharon Crowley, visiting from Northern Arizona, surveyed the crew gathered around the table (this happened in the program’s small conference room, Modern Languages 382, where I spent much of my administrative life in those years). She offered that “people like us,” including herself and her eager young colleagues, qualified as what she termed “hyperliterate”—by this I think she referred to a couple of things about “us”: first, we read a lot, by choice; and second, we choose to live in and through language, believing in its value and trusting in its power to affect the world more than most. She offered also that we needed to understand that most of our students had no desire to achieve hyperliteracy, either through voluminous voluntary reading or through deep belief in the centrality of language to personal, academic, civic, and professional life. I’ve remembered that table talk for nearly forty years now; I heard about “hyperliteracy” from Sharon Crowley before the crashing/caressing waves of personal computers, the internet, smartphones, social media, troll farms, and now generative AI swept over and enveloped all of us.
Let’s consider some possible significant differences between teachers and students in writing classes in relation to the data sets on which each has been “trained”; if we can call most writing teachers hyperliterate, they have likely processed many millions of words in their lives and incorporated all that word data into the storehouse they draw on to write themselves as well as to read texts they assign to students and the texts students generate. Students have processed fewer words, most likely, partly because of spending fewer years alive, partly because of ascribing less value to the experience of engaging with words as data and world, and often because of the era in which they came of age; they may have surpassed their teachers in screen time and processed more words and images on screens; they may have more literacy experience of certain kinds than their teachers. But words, words, words—why all the fuss? How does this translate into the classroom community and interactions between teachers and students (and between students themselves)? What does this mean in relation to the crashing/caressing wave of generative AI powered by large language models?
Generative AI/ChatGPT—Hyperliteracy as/and Magic
Let’s bring the concepts of hyperliteracy and magic into conversation. Let’s think about them in relation to surface and depth, or the appearance of fluency and the epistemological world-building work of language we try to cultivate in our teaching. Many look to writing programs to teach surface correctness (extending from grammar and syntax to citation practices), leaving the work of teaching content knowledge and critical thinking to departmental/disciplinary guardians of guild knowledge and specialized ways of thinking. From an outside perspective, we have responsibility for teaching the knack of fluency, the pleasing surface that allows thought to shine through without noise getting in the way. Few think of writing programs pursuing the business of bringing out the latent powerful lord in every student writer. Few think of a writing teacher’s hyperliteracy beyond its service function of quickly discerning and correcting “error” (am I the only one here who feels surface “error” physically?). The university and wider culture have consigned us to an enclosure of domesticated and marginalized magic—one of the great ironies of the “universal” writing requirement has to do with the continued persistence of the required course despite the casting out of rhetoric as a discipline, life practice, and cultural glue. Many refer to getting the requirement “out of the way,” many question why “good writers” must complete a “basic” course beneath their level of fluency—few think of the requirement as a course in practical and limitless magic. Students at Hogwarts have required courses; many of them likely wanted to get the spells course with its strict attention to the right words in the right order (and the wand, the wand) “out of the way.”
Enter ChatGPT as machine/technology magic, machine hyperliteracy that has the power to undermine or even destroy the structure of domesticated magic taught in schools and practiced in the professions, particularly law and journalism; ChatGPT replaces world-building, relationship, and good word magic with an algorithm with omniscient programming about the surface patterns of language—ChatGPT knows nothing of the depths of epistemology or the winding joys of exploration. So, what do we do to balance magic/world-building and skills/surfaces in a rhetorical education? What do we do about the complex question of agency, when everyone potentially wields the power of language while simultaneously being subject to bad magic on one front (powerless) and using language in a culture that distrusts, fears, and even hates language for its potential for world-building, world-changing, and providing access to magic for all?
What About Helen?
ChatGPT exists as magic and machine, reinforcing two ways of thinking about language that seem in opposition to each other; first, that language is magic and can take agency away from people through manipulation, turning us all into Helen; second, that language is menial work that can be automated and assigned to a machine, leaving people free to focus on other things like thinking, being creative, being smart. Putting words in the right order can thus function as a high form of magic with profound consequences and as a rote skill best assigned to machines as drudge work.
