Most neural network models have the property that they store incoming information over time sequentially, and then use additional mechanisms to weigh information by importance and learn connections between the serially stored information. Both transformer and GRU (LSTM) type networks use this architecture.
They store information in a common network, without modularization. Additional adjustments attempt to update internal connections and adapt their weights to order and structure the material. This can be seen as a method to impose a modularized structure onto the material.
