Have you had one of those moments in life when you set on to achieve one thing but stumbled on something completely different instead. Well, something similar has happened to researchers at OpenAI, a non-profit AI research company funded by the likes of Elon Musk and Peter Thiel.
While a majority of the artificial intelligence (AI) systems present in the market today are dependent on machine learning algorithms that are capable of predicting specific results by drawing on some pre-established values. But, researchers from OpenAI recently discovered that a machine learning system that they had created to predict the next character in the text of reviews from Amazon had mysteriously evolved into an unsupervised learning system that could effectively learn how to read sentiment. And that too by itself. For the uninitiated, it's kind of a huge discovery, something that even the OpenAI researchers are finding hard to explain right now.
In a blogpost on the surprising discovery, the company states that its neural network somehow trained itself and started analysing sentiment accurately by classifying the reviews on popular ecommerce site Amazon as either positive or negative. Not only this, it also generates follow on text that go in sync with the sentiment of the review.
"We were very surprised that our model learned an interpretable feature, and that simply predicting the next character in Amazon reviews resulted in discovering the concept of sentiment," said OpenAI in the blogpost.
With a full-time staff of 60 researchers and engineers, OpenAI's mission is to build safe artificial general intelligence (AGI) and ensure its benefits are as widely and evenly distributed as possible.
The blogpost further explained that the AI research team made use of a multiplicative long short-term memory (LSTM) model that was trained for a month. It can process 12,500 characters a second using Nvidia Pascal GPU’s with “4,096 units on a corpus of 82 million Amazon reviews to predict the next character in a chunk of text.” The Nvidia Pascal GPU was gifted to OpenAI by Nvidia’s CEO Elon Musk last year.
After the meticulous training was done, the researchers then turned the LSTM model into a sentiment classifier by making use of a linear combination of these units. This is when they noticed that instead of using all of the learned units, the model was using just a few of them. They then discovered the presence of a single “sentiment neuron” that boosted of a highly predictive sentiment value.
The GIF below clearly shows a character-by-character value of the sentiment neuron, displaying the positive values as green and negative values as red. It is important to note here that indicative words like “horrendous” or “best” lead to big shifts in the colours.
The sentiment analysis capabilities of OpenAI's new AI has caught every AI enthusiasts eye as it has left behind every other approach employed in the Stanford Sentiment Treebank, which is an extensively studied sentiment analysis data set. According to numbers available, OpenAI's AI 91.8 per cent accuracy beats the previous record of 90.2 per cent.
The OpenAI's AI seems to be very close to achieving every AI researchers dream of creating an unsupervised learning system, wherein the AI learns by itself without any help and eliminates the need of feeding tomes of training data.
It is important to note here that if AIs are able to learn unsupervised, it could prove to be a significant boost to the technology as it will significantly reduce the time it takes currently to make them learn new task.
Google's DeepMind is currently working on building the world’s first AGI architecture that, combined with an AI that can learn unsupervised. When DeepMind id able to achieve this, it could prove to be a great step forward in unleashing the true potential of AI.
[Top Image: futureoflife]
While a majority of the artificial intelligence (AI) systems present in the market today are dependent on machine learning algorithms that are capable of predicting specific results by drawing on some pre-established values. But, researchers from OpenAI recently discovered that a machine learning system that they had created to predict the next character in the text of reviews from Amazon had mysteriously evolved into an unsupervised learning system that could effectively learn how to read sentiment. And that too by itself. For the uninitiated, it's kind of a huge discovery, something that even the OpenAI researchers are finding hard to explain right now.
In a blogpost on the surprising discovery, the company states that its neural network somehow trained itself and started analysing sentiment accurately by classifying the reviews on popular ecommerce site Amazon as either positive or negative. Not only this, it also generates follow on text that go in sync with the sentiment of the review.
"We were very surprised that our model learned an interpretable feature, and that simply predicting the next character in Amazon reviews resulted in discovering the concept of sentiment," said OpenAI in the blogpost.
With a full-time staff of 60 researchers and engineers, OpenAI's mission is to build safe artificial general intelligence (AGI) and ensure its benefits are as widely and evenly distributed as possible.
The blogpost further explained that the AI research team made use of a multiplicative long short-term memory (LSTM) model that was trained for a month. It can process 12,500 characters a second using Nvidia Pascal GPU’s with “4,096 units on a corpus of 82 million Amazon reviews to predict the next character in a chunk of text.” The Nvidia Pascal GPU was gifted to OpenAI by Nvidia’s CEO Elon Musk last year.
After the meticulous training was done, the researchers then turned the LSTM model into a sentiment classifier by making use of a linear combination of these units. This is when they noticed that instead of using all of the learned units, the model was using just a few of them. They then discovered the presence of a single “sentiment neuron” that boosted of a highly predictive sentiment value.
The GIF below clearly shows a character-by-character value of the sentiment neuron, displaying the positive values as green and negative values as red. It is important to note here that indicative words like “horrendous” or “best” lead to big shifts in the colours.
The sentiment analysis capabilities of OpenAI's new AI has caught every AI enthusiasts eye as it has left behind every other approach employed in the Stanford Sentiment Treebank, which is an extensively studied sentiment analysis data set. According to numbers available, OpenAI's AI 91.8 per cent accuracy beats the previous record of 90.2 per cent.
The OpenAI's AI seems to be very close to achieving every AI researchers dream of creating an unsupervised learning system, wherein the AI learns by itself without any help and eliminates the need of feeding tomes of training data.
It is important to note here that if AIs are able to learn unsupervised, it could prove to be a significant boost to the technology as it will significantly reduce the time it takes currently to make them learn new task.
Google's DeepMind is currently working on building the world’s first AGI architecture that, combined with an AI that can learn unsupervised. When DeepMind id able to achieve this, it could prove to be a great step forward in unleashing the true potential of AI.
[Top Image: futureoflife]
Advertisements