How to use a bidirectional LSTM layer for text classification in Keras?
Published on Aug. 22, 2023, 12:19 p.m.
To use a bidirectional LSTM layer for text classification in Keras, you can use the Bidirectional
layer wrapper provided by Keras. This wrapper takes a recurrent layer (e.g. LSTM) as an argument, and creates a new layer that passes the input sequence through the recurrent layer in both forward and backward directions.
Here’s an example of how to use the Bidirectional
layer wrapper for text classification:
from keras.models import Sequential
from keras.layers import Dense, LSTM, Bidirectional, Embedding
model = Sequential()
model.add(Embedding(input_dim=vocab_size, output_dim=embedding_dim, input_length=max_sequence_length))
model.add(Bidirectional(LSTM(128)))
model.add(Dense(num_classes, activation='softmax'))
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model.summary()
In this example, we first add an Embedding
layer to represent the input text as dense vectors, with vocab_size as input dimension and embedding_dim as output dimension.
We then add a Bidirectional
layer that wraps an LSTM layer. The LSTM layer has 128 units, which is the number of hidden states in the LSTM. The Bidirectional
layer processes the input sequence in both forward and backward directions, and concatenates the outputs.
Finally, we add a Dense
layer with num_classes units and a softmax activation function, which generates the probability distribution over the output classes.
We compile the model with adam
optimizer and categorical_crossentropy
loss, and accuracy
as a metric to monitor during training.