Skip to main content
Crowdfunding
Python + AI for Geeks
Practice

Structure and Principles of Recurrent Neural Networks (RNN)

RNN (Recurrent Neural Network) is a type of neural network that processes data while maintaining the sequence of inputs.

In this lesson, we will explore how an RNN is constructed and the principles behind how it remembers order and processes information.


How is an RNN structured?​

An RNN is composed in a repeating form of neural network cells with the same structure.

Each cell receives input and the output from the previous time step, processes them, and then passes the output to the next time step.

RNN Flow Structure
Input X1 → [RNN Cell] → Output H1 →
Input X2 → [RNN Cell] → Output H2 →
Input X3 → [RNN Cell] → Output H3 → ...

The output at each time step utilizes the output (H) from the previous step, remembering it.

An RNN repeats the same computational structure along the time axis, updating information at each step.

For example, when processing a sentence, inputting words sequentially from the first word to the last, the information from earlier words can influence how later words are processed.

RNN Structure Over Time t
[t=1] → [t=2] → [t=3] → ...
 X1      X2      X3
  |       |       |
  ↓       ↓       ↓
 H1 →    H2 →    H3

Thanks to this structure, sequential data such as words, speech, and numbers can be naturally understood and predicted.


Relationship of Input, Hidden State, and Output​

An RNN cell consists of three main components:

  • Input (Xt): The input data at the current time step.

  • Hidden State (Ht): The internal state that stores and summarizes past information.

  • Output (Yt): The final output or prediction at the current time step.

An RNN calculates a new hidden state (Ht) by using the previous hidden state (Ht-1) along with the current input (Xt).

This hidden state is reused as memorized information when moving on to the next input.


In the next lesson, we will discuss the Long-term Dependency Problem, which is a representative issue in RNNs.

Want to learn more?

Join CodeFriends Plus membership or enroll in a course to start your journey.