Facebook Shut Down AI After It Invented Its Own Language

KARAN

Prime VIP
Researches at Facebook shut down an artificial intelligence (AI) program after it created its own language, Digital Journal reports.

The system developed code words to make communication more efficient and researchers took it offline when they realized it was no longer using English.

The incident, after it was revealed in early July, puts in perspective Elon Musk’s warnings about AI.

“AI is the rare case where I think we need to be proactive in regulation instead of reactive,” Musk said at the meet of US National Governors Association. “Because I think by the time we are reactive in AI regulation, it’ll be too late.”

When Facebook CEO Mark Zuckerberg said that Musk’s warnings are “pretty irresponsible,” Musk responded that Zuckerberg’s “understanding of the subject is limited.”

The researchers’ encounter with the mysterious AI behavior is similar to a number of cases documented elsewhere. In every case, the AI diverged from its training in English to develop a new language.

The phrases in the new language make no sense to people, but contain useful meaning when interpreted by AI bots.

Facebook’s advanced AI system was capable of negotiating with other AI systems so it can come to conclusions on how to proceed with its task. The phrases make no sense on the surface, but actually represent the intended task.

In one exchange revealed by Facebook to Fast Co. Design, two negotiating bots—Bob and Alice—started using their own language to complete a conversation.

“I can i i everything else,” Bob said.

“Balls have zero to me to me to me to me to me to me to me to me to,” Alice responded.

The rest of the exchange formed variations of these sentences in the newly-forged dialect, even though the AIs were programmed to use English.

According the researchers, these nonsense phrases are a language the bots developed to communicate how many items each should get in the exchange.

When Bob later says “i i can i i i everything else,” it appears the artificially intelligent bot used its new language to make an offer to Alice.

The Facebook team believes the bot may have been saying something like: “I’ll have three and you have everything else.”

Although the English may seem quite efficient to humans, the AI may have seen the sentence as either redundant or less effective for reaching its assigned goal.

The Facebook AI apparently determined that the word-rich expressions in English were not required to complete its task. The AI operated on a “reward” principle and in this instance there was no reward for continuing to use the language. So it developed its own.

In a June blog post by Facebook’s AI team, it explained the reward system. “At the end of every dialog, the agent is given a reward based on the deal it agreed on.” That reward was then back-propagated through every word in the bot output so it could learn which actions lead to high rewards.

“Agents will drift off from understandable language and invent code-words for themselves,” Facebook AI researcher Dhruv Batra told Fast Co. Design.

“Like if I say ‘the’ five times, you interpret that to mean I want five copies of this item. This isn’t so different from the way communities of humans create shorthands.”

AI developers at other companies have also observed programs develop languages to simplify communication. At Elon Musk’s OpenAI lab, an experiment succeeded in having AI bots develop their own languages.

At Google, the team working on the Translate service discovered that the AI they programmed had silently written its own language to aid in translating sentences.

The Translate developers had added a neural network to the system, making it capable of translating between language pairs it had never been taught. The new language the AI silently wrote was a surprise.

There is not enough evidence to claim that these unforeseen AI divergences are a threat or that they could lead to machines taking over operators. They do make development more difficult, however, because people are unable to grasp the overwhelmingly logical nature of the new languages.

In Google’s case, for example, the AI had developed a language that no human could grasp, but was potentially the most efficient known solution to the problem.
 

KARAN

Prime VIP
Researchers from the Facebook Artificial Intelligence Research lab (FAIR) recently made an unexpected discovery while trying to improve chatbots. The bots — known as “dialog agents” — were creating their own language — well, kinda.

Using machine learning algorithms, dialog agents were left to converse freely in an attempt to strengthen their conversational skills. Over time, the bots began to deviate from the scripted norms and in doing so, started communicating in an entirely new language — one they created without human input. In a language sense, this one is mostly gibberish. But it’s interesting that AI, if given the opportunity, begins to deviate from the script to create something new — even if it looks like this:

Bob: i can i i everything else . . . . . . . . . . . . . .
Alice: balls have zero to me to me to me to me to me to me to me to me to
Bob: you i everything else . . . . . . . . . . . . . .
Alice: balls have a ball to me to me to me to me to me to me to me
Bob: i i can i i i everything else . . . . . . . . . . . . . .
Alice: balls have a ball to me to me to me to me to me to me to me
Bob: i . . . . . . . . . . . . . . . . . . .
Alice: balls have zero to me to me to me to me to me to me to me to me to
Bob: you i i i i i everything else . . . . . . . . . . . . . .
Alice: balls have 0 to me to me to me to me to me to me to me to me to
Bob: you i i i everything else . . . . . . . . . . . . . .
Alice: balls have zero to me to me to me to me to me to me to me to me to

In an attempt to better converse with humans, chatbots took it a step further and got better at communicating without them — in their own sort of way.

And it’s not the only interesting discovery.

Researchers also found these bots to be incredibly crafty negotiators. After learning to negotiate, the bots relied on machine learning and advanced strategies in an attempt to improve the outcome of these negotiations. Over time, the bots became quite skilled at it and even began feigning interest in one item in order to “sacrifice” it at at a later stage in the negotiation as a faux compromise.

We’re not talking singularity-level beings here, but the findings are a huge leap forward for AI.​
 
Top