This notebook is used to pretrain transformers models using Hugging Face on your own custom dataset.What do I mean by pretrain transformers? The definition of pretraining is to train in advance. That is exactly what I mean! Train a transformer model to use it as a pretrained transformers model which can be used to fine-tune it on a specific … [Read more...] about Pretrain Transformers Models in PyTorch Using Hugging Face Transformers
Natural Language Processing
From Text to Knowledge: The Information Extraction Pipeline
I am thrilled to present my latest project I have been working on. In this blog post, I will present my implementation of an information extraction data pipeline, following my passion for combining natural language processing and knowledge graphs. Later on, I will also explain why I see the combination of NLP and graphs as one of the paths to explainable AI.If this in-depth … [Read more...] about From Text to Knowledge: The Information Extraction Pipeline
Building a Complete AI Based Search Engine with Elasticsearch, Kubeflow and Katib
Building search systems is hard. Preparing them to work with machine learning is really hard. Developing a complete search engine framework integrated with AI is really really hard.So let’s make one. ✌️In this post, we’ll build a search engine from scratch and discuss on how to further optimize results by adding a machine learning layer using Kubeflow and Katib. This … [Read more...] about Building a Complete AI Based Search Engine with Elasticsearch, Kubeflow and Katib
Linguistics Wisdom of NLP Models
This article is authored by Keyur Faldu and Dr. Amit Sheth. This article elaborates on a niche aspect of the broader cover story on “Rise of Modern NLP and the Need of Interpretability!”At Embibe, we focus on developing interpretable and explainable Deep Learning systems, and we survey the current state of the art techniques to answer … [Read more...] about Linguistics Wisdom of NLP Models
Discovering the Encoded Linguistic Knowledge in NLP Models
This article is authored by Keyur Faldu and Dr. Amit Sheth. This article elaborates on a niche aspect of the broader cover story on “Rise of Modern NLP and the Need of Interpretability!”At Embibe, we desiderate answers to the open questions while we build the NLP platform to solve numerous problems for the academic content.Modern NLP models (BERT, GPT, … [Read more...] about Discovering the Encoded Linguistic Knowledge in NLP Models
NeurIPS 2020 Papers: Takeaways for a Deep Learning Engineer
Advances in Deep Learning research are of great utility for a Deep Learning engineer working on real-world problems as most of the Deep Learning research is empirical with validation of new techniques and theories done on datasets that closely resemble real-world datasets/tasks (ImageNet pre-trained weights are still useful!).But, churning a vast amount of … [Read more...] about NeurIPS 2020 Papers: Takeaways for a Deep Learning Engineer
The Curious Case of Developmental BERTology
This essay is written for machine learning researchers and neuroscientists (some jargons in both fields will be used). Though it is not intended to be a comprehensive review of literature, we will take a tour through a selection of classic work and new results from a range of topics, in an attempt to develop the following thesis:Just like the fruitful interaction between … [Read more...] about The Curious Case of Developmental BERTology
NeurIPS 2020: Key Research Papers in Natural Language Processing (NLP) & Conversational AI
NeurIPS is the largest machine learning conference held every December. It brings together researchers in computational neuroscience, reinforcement learning, deep learning, and their applications such as computer vision, fairness and transparency, natural language processing, robotics, and more.Our team reviewed the papers accepted to NeurIPS 2020 and shortlisted the most … [Read more...] about NeurIPS 2020: Key Research Papers in Natural Language Processing (NLP) & Conversational AI
GPT-3 & Beyond: 10 NLP Research Papers You Should Read
NLP research advances in 2020 are still dominated by large pre-trained language models, and specifically transformers. There were many interesting updates introduced this year that have made transformer architecture more efficient and applicable to long documents.Another hot topic relates to the evaluation of NLP models in different applications. We still lack evaluation … [Read more...] about GPT-3 & Beyond: 10 NLP Research Papers You Should Read
Natural Language Processing in Production: 27 Fast Text Pre-Processing Methods
Estimates state that 70%–85% of the world’s data is text (unstructured data) [1]. New deep learning language models (transformers) have caused explosive growth in industry applications [5,6,11].This blog is not an article introducing you to Natural Language Processing. Instead, it assumes you are familiar with noise reduction and normalization of text. It covers … [Read more...] about Natural Language Processing in Production: 27 Fast Text Pre-Processing Methods