NeurIPS is the largest machine learning conference held every December. It brings together researchers in computational neuroscience, reinforcement learning, deep learning, and their applications such as computer vision, fairness and transparency, natural language processing, robotics, and more.Our team reviewed the papers accepted to NeurIPS 2020 and shortlisted the most … [Read more...] about NeurIPS 2020: Key Research Papers in Natural Language Processing (NLP) & Conversational AI
Natural Language Processing
GPT-3 & Beyond: 10 NLP Research Papers You Should Read
NLP research advances in 2020 are still dominated by large pre-trained language models, and specifically transformers. There were many interesting updates introduced this year that have made transformer architecture more efficient and applicable to long documents.Another hot topic relates to the evaluation of NLP models in different applications. We still lack evaluation … [Read more...] about GPT-3 & Beyond: 10 NLP Research Papers You Should Read
Natural Language Processing in Production: 27 Fast Text Pre-Processing Methods
Estimates state that 70%–85% of the world’s data is text (unstructured data) [1]. New deep learning language models (transformers) have caused explosive growth in industry applications [5,6,11].This blog is not an article introducing you to Natural Language Processing. Instead, it assumes you are familiar with noise reduction and normalization of text. It covers text … [Read more...] about Natural Language Processing in Production: 27 Fast Text Pre-Processing Methods
The Relationship Between Perplexity And Entropy In NLP
Perplexity is a common metric to use when evaluating language models. For example, scikit-learn’s implementation of Latent Dirichlet Allocation (a topic-modeling algorithm) includes perplexity as a built-in metric.In this post, I will define perplexity and then discuss entropy, the relation between the two, and how it arises naturally in natural language … [Read more...] about The Relationship Between Perplexity And Entropy In NLP
3 NLP Interpretability Tools For Debugging Language Models
With constant advances and unprecedented performance on many NLP tasks, language models have gotten really complex and hard to debug. Researchers and engineers often can’t easily answer questions like: Why did your model make that prediction? Does your model have any algorithmic biases?What kind of data samples does your model perform poorly … [Read more...] about 3 NLP Interpretability Tools For Debugging Language Models
Highlights of ACL 2020
ACL Trends Visualization by Wanxiang CheWith ACL becoming virtual this year, I unfortunately spent less time networking and catching up with colleagues, but as a silver lining I watched many more talks than I usually do. I decided to share the notes I took and discuss some overall trends. The list is not exhaustive, and is based on my research interests. I recommend also … [Read more...] about Highlights of ACL 2020
Best Research Papers From ACL 2020
ACL is the leading conference in the field of natural language processing (NLP), covering a broad spectrum of research areas in computational linguistics. Due to the COVID-19 risks, ACL 2020 took place 100% virtually, similar to other big academic conferences of this year.However, as always, it was the best place to learn about the latest NLP research trends and … [Read more...] about Best Research Papers From ACL 2020
8 Leading Language Models For NLP In 2020
The introduction of transfer learning and pretrained language models in natural language processing (NLP) pushed forward the limits of language understanding and generation. Transfer learning and applying transformers to different downstream NLP tasks have become the main trend of the latest research advances.At the same time, there is a controversy in the NLP community … [Read more...] about 8 Leading Language Models For NLP In 2020
Reformer, Longformer, and ELECTRA: Key Updates To Transformer Architecture In 2020
The leading pre-trained language models demonstrate remarkable performance on different NLP tasks, making them a much-welcomed tool for a number of applications, including sentiment analysis, chatbots, text summarization, and so on. However, good performance usually comes at the cost of enormous computational resources that are not accessible by most researchers and business … [Read more...] about Reformer, Longformer, and ELECTRA: Key Updates To Transformer Architecture In 2020
The Best NLP Papers From ICLR 2020
I went through 687 papers that were accepted to ICLR 2020 virtual conference (out of 2594 submitted - up 63% since 2019!) and identified 9 papers with the potential to advance the use of deep learning NLP models in everyday use cases.Here are the papers found and why they matter.ELECTRA: Pre-training Text Encoders as Discriminators Rather Than GeneratorsKevin … [Read more...] about The Best NLP Papers From ICLR 2020