All you need to know about Positional encodings in Transformer

Photo by corina ardeleanu on Unsplash

In RNN, LSTM the words are fed in sequence, and hence it understands the order of words. Recurrence in LSTM will require a lot of operations as the length of the sentence increases. But in transformer, we process all the words in parallel. This helps in decreasing the training time. To keep in mind the order of words, the concept of positional encodings is introduced. It’s a kind of encoding that denotes the position of words. …


This blog post will get into the nitty-gritty details of the Attention mechanism and create an attention mechanism from scratch using python

Photo by Robert Katzki on Unsplash

Before beginning this blog post, I highly recommend visiting my earlier blog post on an overview of transformers. To get the best out of this blog, please check my previous blog post in the following order.

  1. Transformers — You just need Attention.
  2. Intuitive Maths and Code behind Self-Attention Mechanism of Transformers.
  3. Concepts about Positional Encoding You Might Not Know About.

This blog post will get into the nitty-gritty details of the Attention mechanism and create an attention mechanism from scratch using python. The codes and intuitive maths explanation go hand in hand.


Photo by Paul Skorupskas on Unsplash

Natural language processing or NLP is a subset of machine learning that deals with text analytics. It is concerned with the interaction of human language and computers. There have been different NLP techniques and the latest one is the state-of-the-art Transformers.

This is going to be the very first series of blog posts on Transformers architecture. Transformers have proved revolutionary in the field of Natural language processing(NLP). Ever since its invention, it has replaced all other Natural language processing(NLP) architectures such as Recurrent Neural Network(RNN), Convolutional Neural Network(CNN), and Long- Short term memory(LSTM). …


Improving the Self- attention mechanism

In my last blog post , we have discussed about Self Attention. I strongly recommend going through that before understanding Multi-Headed Attention mechanism. Now , let’s see how Multi Headed attention could be of help.

Say we have a sentence:-

“I gave my dog Charlie some food.” . As we can see, there are multiple actions going on .

  • “I gave” is one action.
  • “to my dog Charlie” is second action .
  • “What did I gave(some food)” is third action.

To keep a track of all these action we need Multi headed attention.

As…


Attention Mechanism is the State of the Art implementation in the world of NLP. It is being used to solve almost every major problem in NLP.

This blog post is inspired by Rasa Algorithm Whiteboard Series. Make sure to watch and subscribe the channel.

Why Attention?

Before Attention Mechanism came into existence , there were CNN and RNN. While CNN isn’t suitable for NLP tasks, RNN also had their own limitations. RNN wasn’t able to make connection between words in longer statements. While RNN gave more importance to words that are in close proximity, Attention Mechanism don’t give any importance to…


2- Classifying and Localizing Objects in Images with the help of Bounding Box.

(Read Part 1 here)

Welcome to the Part 2 of fast.ai . This is the 8th lesson of Fastdotai where we will deal with Single Object Detection . Before we start , I would like to thank Jeremy Howard and Rachel Thomas for their efforts to democratize AI.

The 2nd part assumes to have good understanding of the first part. Here are the links , feel free to explore the first Part of this Series in the following order.

  1. Dog Vs Cat Image Classification
  2. Dog Breed Image Classification
  3. Multi-label Image Classification
  4. Time Series Analysis using Neural Network
  5. NLP- Sentiment Analysis…

1 - Drawing the bounding box around the largest object in an Image. It is about getting the Image Data ready for analysis.

Welcome to the Part 2 of fast.ai. where we will deal with Single Object Detection . Before we start , I would like to thank Jeremy Howard and Rachel Thomas for their efforts to democratize AI.

This part assumes you to have good understanding of the Part 1. Here are the links , feel free to explore the first Part of this Series in the following order.

  1. Dog Vs Cat Image Classification
  2. Dog Breed Image Classification
  3. Multi-label Image Classification
  4. Time Series Analysis using Neural Network
  5. NLP- Sentiment Analysis on IMDB Movie Dataset
  6. Basic of Movie Recommendation System
  7. Collaborative Filtering from…


Datathon= Data +Hackathon. Superb initiative by Yesbank.

“A big idea is great, but putting that big idea into action has the power to change the world.”-Tae Yoo

The above quote shows exactly the same passion, grit and tenacity of the top 20 finalist teams out of 6000 participants that I had an opportunity to interact with, in an event that was organized by Yesbank which was India’s first Bank led Datathon event.

!!! You don’t have to believe me, see it for yourself !!!

About Yesbank

Yesbank is India’s fourth largest Private Sector Bank. It is founded by Rana Kapoor and Ashok Kapur in 2004.

Trending AI Articles:

1. Paper…

Ashis Kumar Panda

Democratizing AI | Looking for challenging problems | Connect @ https://machinelearningmarvel.in/

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store