Memory in LLM: Feature-State-Driven Memory Mechanism
The model is endowed with memory capabilities during inference, enabling information storage, retrieval, and forgetting through a differentiable module. It leverages hierarchical abstractions and nonlinear representations within neural networks, while incorporating data-dependent adaptive forgetting factors and adaptive learning rates to dynamically regulate memory retention strength.
Longterm dependencies are preserved via gated updates, whereas novel knowledge can be flexibly integrated in accordance with the characteristics of the input distribution. Furthermore, a sparse memory module is introduced to enhance the model’s overall memory capacity.
Brain-like activation mechanism: Significantly reducing computational redundancy
Simulating neuronal activation patterns in the brain to process complex data and tasks more effectively, significantly improving computational efficiency and accuracy, and providing new tools for solving real-world complex problems.
Patent Layout
