Large Language Models (LLMs) are Deep Learning models trained to produce text. With this impressive ability, LLMs have become the backbone of modern Natural Language Processing (NLP). Traditionally, they are pre-trained by academic institutions and big tech companies such as OpenAI, Microsoft and NVIDIA. Most of them are then made available for public use. This plug-and-play … [Read more...] about Choosing The Right Language Model For Your NLP Use Case
NLP Tutorials
On The Gap Between Adoption And Understanding
This blog post describes our recent paper: Federico Bianchi and Dirk Hovy (2021). On the Gap between Adoption and Understanding in NLP. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. The main focus of this work is to describe issues that currently affect NLP research and hinder scientific development. NLP is driven by … [Read more...] about On The Gap Between Adoption And Understanding
Topical Language Generation With Transformers
Full Paper Codes Large-scale transformer-based language models (LMs) demonstrate impressive capabilities in open text generation. However, controlling the generated text’s properties such as the topic, style, and sentiment is challenging and often requires significant changes to the model architecture or retraining and fine-tuning the model on new supervised data. We … [Read more...] about Topical Language Generation With Transformers
To ROUGE Or Not To ROUGE?
In this article, we will learn about … … the difference between extractive and abstractive text summarization. … what the ROUGE score is. … why and where it fails. Text Summarization We refer to text summarization as the process of training an Artificial Intelligence (AI) model to produce a smaller chunk of text out of a bigger chunk of text. Where “smaller … [Read more...] about To ROUGE Or Not To ROUGE?
BERT Inner Workings
I created this notebook to better understand the inner workings of Bert. I followed a lot of tutorials to try to understand the architecture, but I was never able to really understand what was happening under the hood. For me it always helps to see the actual code instead of just simple abstract diagrams that a lot of times don’t match the actual implementation. If you’re like … [Read more...] about BERT Inner Workings
GPT2 For Text Classification Using Hugging Face Transformers
This notebook is used to fine-tune GPT2 model for text classification using Hugging Face transformers library on a custom dataset. Hugging Face is very nice to us to include all the functionality needed for GPT2 to be used in classification tasks. Thank you Hugging Face! I wasn’t able to find much information on how to use GPT2 for classification so I … [Read more...] about GPT2 For Text Classification Using Hugging Face Transformers
Extractive Text Summarization Using Contextual Embeddings
Text Summarization is a process of generating a compact and meaningful synopsis from a huge volume of text. Sources for such text include news articles, blogs, social media posts, all kinds of documentation, and many more. If you are new to NLP and want to read more about text summarization, this article will help you understand the basic and advanced concepts. The … [Read more...] about Extractive Text Summarization Using Contextual Embeddings
From Text to Knowledge: The Information Extraction Pipeline
I am thrilled to present my latest project I have been working on. In this blog post, I will present my implementation of an information extraction data pipeline, following my passion for combining natural language processing and knowledge graphs. Later on, I will also explain why I see the combination of NLP and graphs as one of the paths to explainable AI. If this in-depth … [Read more...] about From Text to Knowledge: The Information Extraction Pipeline
Building a Complete AI Based Search Engine with Elasticsearch, Kubeflow and Katib
Building search systems is hard. Preparing them to work with machine learning is really hard. Developing a complete search engine framework integrated with AI is really really hard. So let’s make one. ✌️ In this post, we’ll build a search engine from scratch and discuss on how to further optimize results by adding a machine learning layer using Kubeflow and Katib. This … [Read more...] about Building a Complete AI Based Search Engine with Elasticsearch, Kubeflow and Katib
Natural Language Processing in Production: 27 Fast Text Pre-Processing Methods
Estimates state that 70%–85% of the world’s data is text (unstructured data) [1]. New deep learning language models (transformers) have caused explosive growth in industry applications [5,6,11]. This blog is not an article introducing you to Natural Language Processing. Instead, it assumes you are familiar with noise reduction and normalization of text. It covers … [Read more...] about Natural Language Processing in Production: 27 Fast Text Pre-Processing Methods