A comprehensive guide to understanding and implementing LSTM layers for natural language processing with Python
This work is a continuation of my article about RNNs and NLP with Python. A natural progression of a deep learning network with a simple recurrent layer is a deep learning network with a Long Short Term Memory (LSTM for short) layer.
As with the RNN and NLP, I will try to explain the LSTM layer in great detail and code the forward pass of the layer from scratch.
All the codes can be viewed here: https://github.com/Eligijus112/NLP-python
We will work with the same dataset¹ as in the previous article:
# Data wrangling
import pandas as pd# Reading the data
d = pd.read_csv('input/Tweets.csv', header=None)
# Adding the columns
d.columns = ['INDEX', 'GAME', "SENTIMENT", 'TEXT']
# Leaving only the positive and the negative sentiments
d = d[d['SENTIMENT'].isin(['Positive', 'Negative'])]
# Encoding the sentiments that the negative will be 1 and the positive 0
d['SENTIMENT'] = d['SENTIMENT'].apply(lambda x: 0 if x == 'Positive' else 1)
# Dropping missing values
d = d.dropna()
Remember, that SENTIMENT=1 is a negative sentiment, and SENTIMENT=0 is a positive sentiment.
We need to convert the text data into a sequence of integers. Unlike in the previous article though, we will now create a sequence not of words but of individual characters.
For example, the text “Nice Game” could be converted to the following example vector:
[1, 2, 3, 4, 5, 6, 7, 8, 3]
Each individual character, including whitespaces and punctuations, will have an index.
def create_word_index(
x: str,
shift_for_padding: bool = False,
char_level: bool = False) -> Tuple[dict, dict]:
"""
Function that scans a given text and creates two dictionaries:
- word2idx: dictionary mapping words to integers…