Simplified Recurrent Neural Network Structure, GRU
GRU (Gated Recurrent Unit) is a structure created to solve the limitations of RNNs. It offers similar functionality to LSTM but as a simplified recurrent neural network.
GRU addresses the long-term dependency problem by retaining essential information and discarding irrelevant data.
Why was GRU developed?
LSTM has the advantage of retaining long-term information, but its complex structure can lead to slower learning speeds.
GRU was developed to preserve the performance of LSTM while offering a simpler, faster architecture.
Like LSTM, GRU uses gates but has fewer gates with simpler calculations.
This makes it easy to implement and also quicker to train.
Key Structure of GRU
GRU is composed of the following two gates:
-
Update Gate: Decides how much of the past information to retain. It regulates the amount of information to remember.
-
Reset Gate: Determines how much of the past information should be ignored. It controls how much of the previous state to combine with the current input.
These two gates work together to maintain important information and discard unnecessary information. Consequently, GRU can effectively process information in a sequence over time.
How does GRU operate?
GRU functions through the following process:
-
It calculates both the update gate and reset gate based on the current input and previous state.
-
The reset gate determines the extent to which past information is reflected.
-
The update gate decides how much the new state should be reflected.
-
Finally, it calculates the new state and passes it to the next time step.
In this way, GRU can sequentially process information like an RNN, while effectively remembering even older information with fewer calculations.
In the next lesson, we will explore the Transformer structure, which is often compared with recurrent neural network-based models.
Want to learn more?
Join CodeFriends Plus membership or enroll in a course to start your journey.