NH
Method

A structural controller for low-rank continual adaptation.

NH-LoRA explores parameter-efficient continual learning where a frozen Vision Transformer is adapted through structurally controlled low-rank modules.

Problem statement

In rehearsal-free class-incremental learning, the model receives new classes sequentially and cannot store previous task data. The method must learn new classes while preserving prior knowledge without replay.

Core intuition

Instead of assigning one static adapter design to every task, NH-LoRA treats adapter structure as a controllable state. The system senses the current task, estimates novelty and conflict, then allocates low-rank capacity where it is needed.

Architecture

The NH-LoRA control loop.

The diagram below is migrated from the original research assets and presented as the primary architecture reference.

NH-LoRA architecture overview
Core components

Shared memory, expandable capacity, sparse routing, and planning.

Reuse

Shared Core LoRA

A shared low-rank memory path captures reusable structure across tasks and offers a stable base for later adaptation.

Expand

Expandable Task Slot Bank

A bank of task-specific low-rank slots expands or activates capacity when new evidence suggests that shared memory is insufficient.

Selector

Instance Router

A lightweight router selects sparse slot pathways per sample, limiting unnecessary adapter interference during inference.

Planner

Horizon Planner

A planner consumes task-state signals and predicts layer-wise novelty, conflict, rank budget, shared contribution, and consolidation pressure.

Training

Training-time structural flow.

  1. 1

    Freeze the Vision Transformer backbone and initialize compact shared low-rank memory.

  2. 2

    Run warm-up sensing on the incoming task to summarize feature statistics, gradients, similarity, and uncertainty.

  3. 3

    Use the Horizon Planner to decide where to reuse, expand, open, freeze, merge, or prune capacity.

  4. 4

    Train NH-LoRA modules and the incremental classifier head while applying retention-oriented objectives when history is available.

  5. 5

    Consolidate stable updates after each task and store the task summary in the history bank.

Inference

Sparse path selection at runtime.

  1. 1

    Encode the sample with the frozen backbone.

  2. 2

    Use the Instance Router to activate a sparse subset of relevant task slots.

  3. 3

    Combine shared low-rank memory with selected slot updates.

  4. 4

    Predict through the incremental classifier head with minimized slot interference.

Glossary

Core terms for reading the method.

Rehearsal-Free

A continual learning setup where previous task samples are not stored or replayed.

Low-Rank Adapter

A compact trainable update that approximates a larger weight change through low-rank matrices.

Task-State Encoder

A module that summarizes the current task into signals used for structural planning.

Homeostasis

Post-task regulation that keeps useful capacity and removes or merges redundant structure.