TOPBOTS

The Best of Applied Artificial Intelligence, Machine Learning, Automation, Bots, Chatbots

Language Models

Can Too Much BERT Be Bad for You?

by

BERT and GPT-2: we all love language models…I mean, who doesn’t? Language models like BERT and GPT-2 (and GPT-3) have had an enormous impact on the entire NLP field. Most of the models that obtained groundbreaking results on the famous GLUE benchmark are based on BERT. I, too, have benefited from BERT, since I released a library for topic modeling and some HuggingFace … [Read more...] about Can Too Much BERT Be Bad for You?

Fine-tune Transformers in PyTorch Using Hugging Face Transformers

by

This notebook is designed to use a pretrained transformers model and fine-tune it on a classification task. The focus of this tutorial will be on the code itself and how to adjust it to your needs.This notebook is using the AutoClasses from transformer by Hugging Face functionality. This functionality can guess a model’s configuration, tokenizer and … [Read more...] about Fine-tune Transformers in PyTorch Using Hugging Face Transformers

Pretrain Transformers Models in PyTorch Using Hugging Face Transformers

by

This notebook is used to pretrain transformers models using Hugging Face on your own custom dataset.What do I mean by pretrain transformers? The definition of pretraining is to train in advance. That is exactly what I mean! Train a transformer model to use it as a pretrained transformers model which can be used to fine-tune it on a specific … [Read more...] about Pretrain Transformers Models in PyTorch Using Hugging Face Transformers

Discovering the Encoded Linguistic Knowledge in NLP Models

by

This article is authored by Keyur Faldu and Dr. Amit Sheth. This article elaborates on a niche aspect of the broader cover story on “Rise of Modern NLP and the Need of Interpretability!”At Embibe, we desiderate answers to the open questions while we build the NLP platform to solve numerous problems for the academic content.Modern NLP models (BERT, GPT, … [Read more...] about Discovering the Encoded Linguistic Knowledge in NLP Models

The Curious Case of Developmental BERTology

by

This essay is written for machine learning researchers and neuroscientists (some jargons in both fields will be used). Though it is not intended to be a comprehensive review of literature, we will take a tour through a selection of classic work and new results from a range of topics, in an attempt to develop the following thesis:Just like the fruitful interaction between … [Read more...] about The Curious Case of Developmental BERTology

The Dark Secrets Of BERT

by

This blog post summarizes EMNLP 2019 paper Revealing the Dark Secrets of BERT by researchers from the Text Machine Lab at UMass Lowell:  Olga Kovaleva (LinkedIn), Alexey Romanov (LinkedIn), Anna Rogers (Twitter: @annargrs), and Anna Rumshisky (Twitter: @arumshisky).Here are the topics covered:A brief intro to … [Read more...] about The Dark Secrets Of BERT

Why Choosing a Heavier NLP Model Might Be a Good Choice?

by

From Google’s 43 rules of ML.Rule #4: Keep the first model simple and get the infrastructure right.With some opinions floating in the market, I feel it’s a good time to spark a discussion about this topic. Otherwise, the opinions of the popular will just drown other ideas.Note: I work in NLP and these opinions are more focussed towards NLP applications. Cannot … [Read more...] about Why Choosing a Heavier NLP Model Might Be a Good Choice?

Dissecting The Transformer

by

We saw how attention works and how it improved neural machine translation systems (see the previous blogpost), we are going to unveil the secrets behind the power of the most famous NLP models nowadays (a.k.a BERT and friends), the transformer.In this second part, we are going to dive into the details of this architecture with the aim of getting a solid … [Read more...] about Dissecting The Transformer

  1. 1
  2. 2