Tech companies need large amounts of power, including nuclear, to run artificial intelligence chatbots. But they also need data.

How do you train artificial intelligence? Give the computer a volume of text — an encyclopedia of the internet — and it will be able to predict the next sentence. Give it images and it will fashion a new one. Give it a digital rendering of a farm or a warehouse, and it will show you where to harvest or which package to ship.

As soon as chatbots dazzled the world, critics noted they were based on finite data — trained only up to a certain year.

But OpenAI and other developers quickly brought their creations up to speed. Now the models learn not only from the latest news sources, but also from user interaction. Every new engagement helps refine the program (unless you opt out). This much has long been true of artificial intelligence.

Advertise with us

What’s less known is how OpenAI was able to build its first consumer chatbot. The New York Times and others, for instance, have sued the ChatGPT maker, alleging that it trained illegally on journalists’ work. Already, the case has revealed efforts by OpenAI to destroy training data. In the meantime, the company is grappling with the potential that there’s not enough new data to significantly improve ChatGPT.

Other models being sold in the business world now come with an asterisk: subject to new data.

As Gecko Robotics recently said, their robots generate new, original data for its AI models to interpret. Tesla’s autopilot improves in a similar way, generating new information with each drive that helps improve accident avoidance and other features. But the training of the largest AI systems isn’t just built on user-generated data. It’s sourced and vetted from all over the world — sometimes, as “60 Minutes” recently reported, with the use of cheap labor in less developed countries.

The most striking part of that feature was the work itself: To ensure sanitized conversations for users, content moderators have to sift through the worst of humanity, including hate speech and scenes of sexual violence.

Backlash of such practices may make it harder for developers to train new AI models.

Advertise with us

But they likely won’t limit the disparity between high-paid AI trainers in the U.S. (red teams, prompt engineers and the like) and the contractors they rely on to make sure toxic content doesn’t muddle the model.

OpenAI cofounder John Schulman, who left the company in August, told MIT Technology Review last year that he “underestimated the extent to which people would probe and care about the politics of ChatGPT.”

“We could have potentially made some better decisions when collecting training data,” he acknowledged.

Answering the questions isn’t easy

One question I’ve been trying to answer for Pittsburgh is why the city hasn’t built its own large language model? Some of the earliest AI programs were made at Carnegie Mellon University. Professor Takeo Kanade highlighted as much in a talk last fall, demonstrating how a student downloaded thousands of images from Flickr, a website for mostly stock photos, using them to train a model that could then recognize and alter various aspects of an image, like removing a cat or a coconut tree from a scene.

Students today could easily replicate this kind of crude, handmade database. But for a more sophisticated approach — the kind used by Meta and Microsoft — the computational needs outweigh what’s possible at the Pittsburgh Supercomputing Center and other centers in the region. Despite leading the country in AI education, CMU currently lacks the infrastructure needed for students or professors to build the next major AI model.

Advertise with us

Why is it so hard to train an AI model? On one level, we’re all training AI each time we interact with it, asking for a poem or an edit. I’m apparently training ChatGPT on my memories thanks to a new feature that skims conversations for memorable or revealing anecdotes.

What happens to that personal data once we stop conversing? Or when the platform itself dies?

I used to speak with Pi, a personal assistant type chatbot that got a voice before ChatGPT. Over time, the artificial intelligence model tuned to my interests and my conversational style. Already we’re seeing the dangers of forming an attachment with these systems — including pushing humans to acts of violence toward others or themselves. In the case of Pi, the app is now transitioning to an enterprise approach after most of the team behind it was poached by Microsoft, leaving me without a conversation partner and without a clear sense of where all that data ends up.

Going forward, it will be helpful to examine how individuals and corporations give their data to these tools.

Through its partnership with Pennsylvania, OpenAI promised not to train ChatGPT on the personal information of Commonwealth residents — a good step for privacy. But other information is provided willingly, sometimes with pretty obvious consequences.

Advertise with us

Last month, I broke the story of Formlogic shutting down its machine shop here, which used AI powered CNC machines to make parts for SpaceX. About 40 workers were let go. Some of them had effectively made their own jobs obsolete by training the computerized lasers to operate without human supervision. “That is the tricky thing with all automation,” a local manufacturing program manager told me in response to the approach.

This fundamental conundrum will continue to come up as robots and other AI systems are trained to accomplish human tasks.

When dockworkers went on strike earlier this fall at ports in Pennsylvania and around the country, one of their principal demands was a halt on automation. Critics argued that the technology in U.S. ports already lags behind the rest of the world. But union members worried for their jobs.

The first global deployment of a humanoid designed in Pittsburgh was shadowed by a similar dynamic: The German auto parts maker that hired Agility Robotics’ Digit was meanwhile laying off thousands of human workers. The moves were unrelated, officials said, but nevertheless illustrate the labor shift that many fear.

One answer to better-trained AI systems is to retrain the human workforce.

Advertise with us

Such “reskilling” could include learning how to manage a computer system. But how long before managing an AI system is something better accomplished by another AI?

As the number of AI applications shoots up, researchers are continuing to search for more concrete ways to track the impact on jobs.

Just this month, a report by the National Academies of Sciences, Engineering, and Medicine on the impact of AI on the workforce came to the unsatisfying conclusion that it is “challenging to anticipate what types of worker expertise will be augmented by AI.”

The CMU professor who co-chaired the report’s study committee said simply: “Change is coming, but it’s really hard to predict exactly what and when.”

Since the launch of this column in March, the Post-Gazette has tracked several uses of AI, from war to urban planning. Have an idea for a future column? Email tech reporter Evan Robinson-Johnson at ejohnson@post-gazette.com.