The introduction of transfer learning and pretrained language models in natural language processing (NLP) pushed forward the limits of language understanding and generation. Transfer learning and applying transformers to different downstream NLP tasks have become the main trend of the latest research advances.At the same time, there is a controversy in the NLP community … [Read more...] about 10 Leading Language Models For NLP In 2021
Top
BERT Inner Workings
I created this notebook to better understand the inner workings of Bert. I followed a lot of tutorials to try to understand the architecture, but I was never able to really understand what was happening under the hood. For me it always helps to see the actual code instead of just simple abstract diagrams that a lot of times don’t match the actual implementation. If you’re like … [Read more...] about BERT Inner Workings
Pretrain Transformers Models in PyTorch Using Hugging Face Transformers
This notebook is used to pretrain transformers models using Hugging Face on your own custom dataset.What do I mean by pretrain transformers? The definition of pretraining is to train in advance. That is exactly what I mean! Train a transformer model to use it as a pretrained transformers model which can be used to fine-tune it on a specific … [Read more...] about Pretrain Transformers Models in PyTorch Using Hugging Face Transformers
Top Applications of Graph Neural Networks 2021
At the beginning of the year, I have a feeling that Graph Neural Nets (GNNs) became a buzzword. As a researcher in this field, I feel a little bit proud (at least not ashamed) to say that I work on this. It was not always the case: three years ago when I was talking to my peers, who got busy working on GANs and Transformers, the general impression that they got on me was that I … [Read more...] about Top Applications of Graph Neural Networks 2021
Leading AI & Machine Learning Research Trends 2021
To help you stay well prepared for 2021, we have summarized the latest trends across different research areas, including natural language processing, conversational AI, computer vision, and reinforcement learning.We also suggest key research papers in different areas that we think are representative of the latest advances.Subscribe to our AI Research mailing list at the … [Read more...] about Leading AI & Machine Learning Research Trends 2021
TOPBOTS Guide to NeurIPS 2020
This year’s Annual Conference on Neural Information Processing (NeurIPS 2020) is held 100% virtually from December 6th to 12th, 2020. Historically NeurIPS sells out quickly and can be a challenge to register for. But, this year’s virtual format welcomes everyone with an affordable ticket price of $100.2020 is our third year of TOPBOTS covering the conference. To … [Read more...] about TOPBOTS Guide to NeurIPS 2020
2020’s Top AI & Machine Learning Research Papers
Despite the challenges of 2020, the AI research community produced a number of meaningful technical breakthroughs. GPT-3 by OpenAI may be the most famous, but there are definitely many other research papers worth your attention. For example, teams from Google introduced a revolutionary chatbot, Meena, and EfficientDet object detectors in image recognition. Researchers … [Read more...] about 2020’s Top AI & Machine Learning Research Papers
GPT-3 & Beyond: 10 NLP Research Papers You Should Read
NLP research advances in 2020 are still dominated by large pre-trained language models, and specifically transformers. There were many interesting updates introduced this year that have made transformer architecture more efficient and applicable to long documents.Another hot topic relates to the evaluation of NLP models in different applications. We still lack evaluation … [Read more...] about GPT-3 & Beyond: 10 NLP Research Papers You Should Read
Novel Computer Vision Research Papers From 2020
Will transformers revolutionize computer vision like they did with natural language processing? That’s one of the major research questions investigated by computer vision scientists in 2020. The first results indicate that transformers achieve very promising results on image recognition tasks.Beyond transformers in vision applications, we also noticed a continuous … [Read more...] about Novel Computer Vision Research Papers From 2020
Graph Convolutional Networks (GCN)
In this post, we’re gonna take a close look at one of the well-known graph neural networks named Graph Convolutional Network (GCN). First, we’ll get the intuition to see how it works, then we’ll go deeper into the maths behind it.Why Graphs?Many problems are graphs in true nature. In our world, we see many data are graphs, such as molecules, social … [Read more...] about Graph Convolutional Networks (GCN)