Transformers — Attention is all you need

Photo by Somchai Kongkamsri: https://www.pexels.com/photo/red-and-black-robot-statue-185725/

RNN — Recurrent Neural Network

LSTM/GRU

Encode Decoder Architecture

Transformers

Model Design

Self Attention

Positional embedding

Model Training : Feed Forward Neural Network

Score as Probability Distribution

Context Vector for each word vs Single Context Vector

Score Calculation

Conclusion

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Solomon

Solomon

Passionate about Data Science and applying Machine Learning,Deep Learning algorithms