ChatGPT and Lobachevski

What made ChatGPT the genius that it is today? What brought Sam Altman and his team their new wealth and fame?

What is the secret of the commercial triumph of artificial intelligence (AI) and its breakthrough success ChatGPT, recently released by OpenAI in a quaternary “Turbo” version updating its data to April 2023?

What made them the fastest-growing program ever launched (until Instagram “Threads”)?

For one plausible answer, I will consult my childhood guide Tom Lehrer and his canonical song, “Lobachevski.”

Whatever the question, Lehrer confesses, “This I know from nothing.” ChatGPT could say the same.

The all-purpose miracle method in Lehrer’s song is: “Plagiarize! Let no one else’s work evade your eyes.”

For ChatGPT and its output, “every chapter I stole from somewhere else.” The Index I got from the Worldwide Web.

I just change all the words into numbers and map them in algebraic patterns. I disguise my tracks by a “Generative Pretrained Transformer (GPT) that adds a bit of randomness that preens as ‘creativity.’”

Is this a revolution or merely a mirage, with randomness substituting for invention, and authoritative sounding prose substituting for accuracy, and statistics standing in for logical conclusions and explanations?

That’s how it looks to me: ChatGPT is mostly “plagiarize.com”.

Last week, we met the “Mona Lisa enigma”: How image AI—Midjourney, DALL-E, Stable Diffusion, Jasper—which supposedly mean imagistic “artificial intelligence,” stupidly destroy the essence of a work of art. The first iteration just repeats a Google Search. Mistaking randomness for creative entropy, subsequent “improvements” jump back across the “uncanny valley” of verisimilitude and simply destroy the quiddity or creativity. They subtract information—the Mona Lisa smile—which is defined in information theory as “unexpected bits” or “surprisal.” Actual information—unexpected bits—are the nemesis of a probability engine like Midjourney or GPT.

We also explored the phenomenon of “model collapse” in ChatGPT. As an AI program exhausts its database and increasingly feeds on its own fecal matter, it suffers a collapse of its initial plausibility, when it was more reliably plagiarizing from the web.

Perhaps the most brilliant mathematician-physicist-software inventor in the world is Stephen Wolfram, of Mathematica, Wolfram Alpha, and Wolfram Language, among many other creations.

His new book “What is ChatGPT Doing…and Why Does It Work,” might be more truthfully entitled, “Why it Doesn’t Work.” Instead, most of this pithy and insightful book disguises its deadly findings by touting ChatGPT’s “remarkable” and “unexpected” successes in simulating human language.

For each token (next word or phrase) produced by the system, more than 175 billion calculations are needed. While conceding the essential mystery of these massively parallel computations and their effectiveness, Wolfram contends that the entire multitudinous contraption is “a potentially surprising scientific discovery” [that is able] “to capture the essence of what human brains manage to do in generating language.”

Wolfram’s true purpose here becomes clear in the final two chapters, when he shows with a host of amusing examples that ChatGPT cannot get correct answers for problems in mathematics and physics. Not knowing what it doesn’t know, it lies plausibly and authoritatively.

These issues of mechanistic dementia go by such labels as hallucination, model collapse, data concentration, computational irreducibility, dysmathia, Gödel’s uncertainty, and model mendacity (or euphemistically “overconfidence”). Related issues are the end of Moore’s Law, the hyperscale datacenter “heat crisis,” Power Grid fragility, and even civilizational decadence in which surprising information and creativity gives way to mass stultification.

Wolfram’s answer is essentially to reduce ChatGPT to a useful interface to Wolfram Language and Wolfram Alpha, which accurately perform mathematics and physics. As such a human-oriented input-output layer, AI does have many uses. Wolfram may exaggerate the degree to which his “Ruliad” of simple laws can engender and explain all the complexities of mathematics and physics. But he is on target in showing that these new AI systems are essentially just new I-O systems for computers.

Here, I must make a confession. I did not believe that I had much to learn from Noam Chomsky, MIT’s far-leftist linguistics scholar.

But let me quote from him here: AI in its current form will “degrade our science and debase our ethics by incorporating into our technology… a fundamentally flawed conception of language and knowledge… The human mind is not, like ChatGPT and its ilk, a lumbering statistical engine for pattern matching, gorging on hundreds of terabytes of data and extrapolating the most probable conversational responses or most probable answer to a scientific question…

“A young child acquiring a language is developing—unconsciously, automatically and speedily from minuscule data—a grammar, a stupendously sophisticated system of logical principles and parameters… The child’s operating system (OS) is completely different from a machine learning program.”

True intelligence is demonstrated in the ability to think and express improbable but insightful things.

In information theory terms, ChatGPT, Google BARD et. Al. may not convey any information—unexpected, high entropy bits at all—because they are biased toward low entropy probabilities rather than high entropy revelations.

So, let’s return to our opening questions: What made AI the genius it is today? What brought fame and fortune to Sam Altman and his OpenAI-Microsoft teams?

We presented the real answers at our COSM conference early this month in Bellevue, Washington. There we converged with human beings with surprising ideas galore, such inventors Wolfram and Ray Kurzweil, new Turing Prize winner Bob Metcalfe, and Kyoto Prize winner Carver Mead, distributed data center entrepreneur Stephen Balaban, and disruptive investor Cathie Wood and encountered the real new revolution in technology.

This new breakthrough is not AI, which is merely a new computer interface platform, which in its current form closely resembles the centralized mainframe computers of the IBM era of the mid-20th Century. Today’s revolution is the nanocosm of materials science, bringing us the Graphene Moment, a new substance one carbon atom thick 200 times stronger than steel and a thousand times more conductive than copper. Now producible by the ton under the guidance of Rice inventor James Tour and his students, graphene will transform every industry, including AI, more radically even than the invention of the integrated circuit transformed electronics and computation in the Silicon Moment fifty years ago.

We will be covering this real breakthrough in all our newsletters to come.