Generating Poetic Texts with Recurrent Neural Networks in Python

Introduction

In this tutorial, we will walk through the process of creating an AI project that generates text similar to that of Shakespeare using a recurrent neural network (RNN) implemented in Python. Although it won’t replicate Shakespeare’s quality perfectly, it should produce text that resembles his style. This project will involve using TensorFlow to build and train our neural network.

Libraries Required

Before we dive into the code, we need to take note of the libraries we will require:

• random: A basic Python library for generating random numbers.
• numpy: This library allows us to handle arrays efficiently.
• tensorflow: The core library that will facilitate our neural network creation and training.
• tensorflow.keras.layers: To import specific neural network layers like LSTM and Dense.

import random
import numpy as np
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense

Preparing the Data

Our next step involves gathering the text we want to train our model with. We will utilize Shakespeare's text, which can be obtained through TensorFlow directly. We’ll save the text in a local file and read it into our script.

file_path = tf.keras.utils.get_file("shakespeare.txt", "LINK_TO_SHAKESPEARE_TEXT")
with open(file_path, 'rb') as file:
    text = file.read().decode('utf-8').lower()  # Convert to lowercase

Converting Text to Numerical Format

To prepare the input for our neural network, we must convert each character into a numerical representation. This involves creating a dictionary for mapping characters to indices and vice versa.

characters = sorted(set(text))
char_to_index = (c: i for i, c in enumerate(characters))
index_to_char = (i: c for i, c in enumerate(characters))

Creating Training Samples

To assist our model in learning, we will create sequences of characters (i.e., inputs) and their corresponding next characters (i.e., outputs).

sequence_length = 40
step_size = 3

sentences = []
next_characters = []

for i in range(0, len(text) - sequence_length, step_size):
    sentences.append(text[i: i + sequence_length])
    next_characters.append(text[i + sequence_length])

Following this, we convert our character sequences into a NumPy array format that is suitable for training.

X = np.zeros((len(sentences), sequence_length, len(characters)), dtype=bool)
y = np.zeros((len(sentences), len(characters)), dtype=bool)

## Introduction
for i, sentence in enumerate(sentences):
    for t, char in enumerate(sentence):
        X[i, t, char_to_index[char]] = 1
    y[i, char_to_index[next_characters[i]]] = 1

Building the Neural Network

Now that our data is prepared, we can construct our RNN. It will consist of an LSTM layer followed by a Dense layer and an activation layer.

model = Sequential()
model.add(LSTM(128, input_shape=(sequence_length, len(characters))))
model.add(Dense(len(characters), activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='rmsprop')

We can now train the model. To avoid retraining every time we run the script, we save the model after training.

model.fit(X, y, batch_size=256, epochs=10)  # Adjust epochs as necessary
model.save('text_generator.h5')

Generating Text

Once our model is trained, we can use it to generate text based on a starting sequence. The generate_text function will help us in predicting next characters through the model.

def generate_text(length, temperature):
    start_index = random.randint(0, len(text) - sequence_length)
    generated_text = ''
    sentence = text[start_index: start_index + sequence_length]

    for _ in range(length):
        x = np.zeros((1, sequence_length, len(characters)))
        for t, char in enumerate(sentence):
            x[0, t, char_to_index[char]] = 1

        predictions = model.predict(x, verbose=0)[0]
        next_index = sample(predictions, temperature)
        next_char = index_to_char[next_index]

        generated_text += next_char
        sentence = sentence[1:] + next_char

    return generated_text

You can experiment with different temperatures to generate varied styles of text.

print(generate_text(300, 0.2))

Conclusion

In this article, we covered setting up a recurrent neural network for generating poetic text similar to Shakespeare's works. Remember, while the generated text may not always make logical sense, it demonstrates the learning capability of neural networks based on the input data.

---

Keywords

• Recurrent Neural Network (RNN)
• TensorFlow
• Keras
• LSTM
• Text Generation
• Character Mapping
• Training Data

FAQ

Q1: What is the purpose of using LSTM in this project?
A1: LSTM (Long Short-Term Memory) networks are ideal for sequential data like text because they can maintain memory over long sequences, allowing them to recognize contextual patterns in text.

Q2: How do temperature settings affect the generated text?
A2: A lower temperature results in safer, more predictable text outputs, while a higher temperature encourages more creative, but potentially nonsensical, text generation.

Q3: Can I use texts other than Shakespeare's?
A3: Absolutely! You can use any text data, such as your WhatsApp chat history, to train the model to generate output similar to that source material.

Q4: How long does it take to train the model?
A4: Training time varies based on hardware and dataset size, but this implementation aims to complete training within an hour on standard systems.