Are you interested in receiving more in-depth technical education about language models and NLP applications? Subscribe below to receive relevant updates. In the eyes of most NLP researchers, 2018 was a year of great technological advancement, with new pre-trained NLP models shattering records on tasks ranging from sentiment analysis to … [Read more...] about OpenAI GPT-2: Understanding Language Generation through Visualization
Natural Language Processing
Deconstructing BERT, Part 2: Visualizing the Inner Workings of Attention
This is the second part of a two-part series on deconstructing BERT. In part 1, Deconstructing BERT: Distilling 6 Patterns from 100 Million Parameters, I described how BERT’s attention mechanism can take on many different forms. For example, one attention head focused nearly all of the attention on the next word in the sequence; another focused on the previous … [Read more...] about Deconstructing BERT, Part 2: Visualizing the Inner Workings of Attention
Deconstructing BERT: Distilling 6 Patterns from 100 Million Parameters
The year 2018 marked a turning point for the field of Natural Language Processing, with a series of deep-learning models achieving state-of-the-art results on NLP tasks ranging from question answering to sentiment classification. Most recently, Google’s BERT algorithm has emerged as a sort of “one model to rule them all,” based on its superior performance over a wide variety of … [Read more...] about Deconstructing BERT: Distilling 6 Patterns from 100 Million Parameters
How To Solve 90% Of NLP Problems: A Step-By-Step Guide
Text data is everywhere Whether you are an established company or working to launch a new service, you can always leverage text data to validate, improve, and expand the functionalities of your product. The science of extracting meaning and learning from text data is an active topic of research called Natural Language Processing (NLP). NLP … [Read more...] about How To Solve 90% Of NLP Problems: A Step-By-Step Guide
Novel Methods For Text Generation Using Adversarial Learning & Autoencoders
Just two years ago, text generation models were so unreliable that you needed to generate hundreds of samples in hopes of finding even one plausible sentence. Nowadays, OpenAI's pre-trained language model can generate relatively coherent news articles given only two sentence of context. Other approaches like Generative Adversarial Networks (GANs) and Variational … [Read more...] about Novel Methods For Text Generation Using Adversarial Learning & Autoencoders
What Every NLP Engineer Needs to Know About Pre-Trained Language Models
Practical applications of Natural Language Processing (NLP) have gotten significantly cheaper, faster, and easier due to the transfer learning capabilities enabled by pre-trained language models. Transfer learning enables engineers to pre-train an NLP model on one large dataset and then quickly fine-tune the model to adapt to other NLP tasks. This new approach enables NLP … [Read more...] about What Every NLP Engineer Needs to Know About Pre-Trained Language Models
State-Of-The-Art Methods For Neural Machine Translation & Multilingual Tasks
The quality of machine translation produced by state-of-the-art models is already quite high and often requires only minor corrections from professional human translators. This is especially true for high-resource language pairs like English-German and English-French. So, the main focus of recent research studies in machine translation was on improving system performance for … [Read more...] about State-Of-The-Art Methods For Neural Machine Translation & Multilingual Tasks
We Summarized 14 NLP Research Breakthroughs You Can Apply To Your Business
UPDATE: We've also summarized the top 2019 and top 2020 NLP research papers. Language understanding is a challenge for computers. Subtle nuances of communication that human toddlers can understand still confuse the most powerful machines. Even though advanced techniques like deep learning can detect and replicate complex language patterns, machine learning models still lack … [Read more...] about We Summarized 14 NLP Research Breakthroughs You Can Apply To Your Business