Transformers — Attention is all you need

Photo by Somchai Kongkamsri:

RNN — Recurrent Neural Network


Encode Decoder Architecture


Model Design

Self Attention

Positional embedding

Model Training : Feed Forward Neural Network

Score as Probability Distribution

Context Vector for each word vs Single Context Vector

Score Calculation




Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store


Passionate about Data Science and applying Machine Learning,Deep Learning algorithms