THEY ONLY THINK THEY'RE HALLUCINATING

JACOBIN
ChatGPT Is an Ideology Machine

                                                           

BY LEIF WEATHERBY       

On February 16, Vanderbilt University’s office for equity, diversity, and inclusion issued a statement on the shooting at Michigan State University. The statement was boilerplate... The only remarkable thing… was that a footnote credited ChatGPT with producing its first draft. The office apologized one day later, after an outcry.

This curious incident throws the most recent panic-hype cycle around artificial intelligence into stark relief…

These debates, including the exhibitionist scaremongering, are mostly vapor. But the systems themselves should be taken seriously. They may supplant low-level tasks in both writing and coding, and could lead to a mass cognitive deskilling, just as the industrial factory disaggregated and immiserated physical labor.…

But however the next phase of technological capitalism plays out, the new AI is intervening directly in the social process of making meaning at all. GPT systems are ideology machines.

Language Models Are the First Quantitative Producers of Ideology

The three main takes on GPT systems are that they are toys, that they are harmful, and that they present a major change in civilization as such. Noam Chomsky thinks they are toys, writing in the New York Times that they have no substantial relationship to language, a human neural function that allows us to divine truth and reason morally. Emily Bender and Timnit Gebru think they are harmful, calling them “stochastic parrots” that reflect the bias of their “unfathomably” large datasets, redistributing harm that humans have already inflicted discursively.

 Henry Kissinger thinks they are societal game changers, that they will change not only labor and geopolitics, but also our very sense of “reality itself.”

Kissinger is right, alas: GPT systems, because they automate a function very close to our felt sense of what it means to be human at all, may produce shifts in the very way we think about things. 

Control over the way we think about things is called “ideology,” and GPT systems engage it directly and quantitatively in an unprecedented manner.

“GPT” stands for “generative pretrained transformer,” but “GPT” also means “general purpose technology” in economic jargon. This highlights the ambition behind these systems, 

What GPT systems spit out is language, but averaged out around a selected center of words. It’s a mush with vague conceptual borders…

Hegemony and Kitsch

Ideology is not just political doctrine. When Marx wrote of the “German Ideology,” he meant his fellow socialists’ implicit belief in the power of ideas, to which he countered the power of material forces. 

But Marxists slowly took up the problem of the power of discourse and representation, acknowledging that what we are able to think, imagine, and say is a crucial political issue. Antonio Gramsci called the dominant set of ideas “hegemony,” arguing that these ideas conformed to the dominance of the ruling class while not being about that dominance. Literary critic Hannes Bajohr has warned against privatized GPT systems in just this sense, saying that “whoever controls language controls politics.”

Hegemony and kitsch are combined in the output of GPT systems’ semantic packages, which might miss aspects of ‘the world’ but faithfully capture ideology.