Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

Fast Multi-dimensional Refusal Subspaces via RFM-AGOP

Source

arXiv

arxiv.org/abs/2607.02396v1

Read the source

Source

Plain-Language Brief

The text discusses a new method for identifying multi-dimensional refusal subspaces in Large Language Models (LLMs), which are used for safety and interpretability. The proposed method adapts the Recursive Feature Machine (RFM) algorithm to identify these subspaces quickly and efficiently, even on models that produce long reasoning traces. The new method showed better performance than alternatives in an ablation task.

What the Document Says

  • The text is about a study related to steering and monitoring activations in Large Language Models (LLMs).
  • It mentions that early work assumed behaviors are encoded along single linear directions, but recent findings suggest complex behaviors are in multi-dimensional subspaces.
  • Existing methods for extracting these subspaces are computationally expensive, especially on reasoning models producing long reasoning traces.
  • The authors propose to adapt the Recursive Feature Machine (RFM) algorithm to identify multi-dimensional refusal subspaces quickly and efficiently.
  • The RFM algorithm is said to work well even on non-reasoning models.
  • The study shows that while RFM allows for faster subspace identification, it also performed better in an ablation task than its alternatives.

Key Concepts Mentioned

  • Large Language Models
  • Steering and monitoring activations
  • Refusal subspaces
  • Recursive Feature Machine (RFM) algorithm
  • Ablation task

Claims Made by the Document

  • Complex behaviors live in multi-dimensional subspaces.
  • Existing methods for extracting these subspaces are computationally expensive.
  • The adapted RFM algorithm can identify multi-dimensional refusal subspaces quickly and efficiently.
  • RFM showed better performances on the ablation task than its alternatives.

Evidence Used

  • The text mentions complex behaviors in LLMs, but it's not stated where this information comes from.
  • The text states that existing methods for extracting subspaces are computationally expensive.
  • The text explains how the authors adapted the RFM algorithm to identify multi-dimensional refusal subspaces quickly and efficiently.
  • The text mentions that RFM showed better performances on the ablation task than its alternatives.

Methods / Approach

  • Adaptation of the Recursive Feature Machine (RFM) algorithm

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Steering and monitoring activations in Large Language Models (LLMs) are increasingly used for both safety and interpretability. Early work assumed behaviours are encoded along single linear directions, but recent findings suggest complex behaviours, such as the refusal to answer harmful queries, live in multi-dimensional subspaces. However, existing methods for extracting these subspaces are computationally expensive, which becomes prohibitive on reasoning models who produce long reasoning traces. By adapting the Recursive Feature Machine (RFM) algorithm -- which can be computed efficiently -- with a probe-informed initialization, we are able to identify the multi-dimensional refusal subspace in seconds, on reasoning (Qwen 3) and non-reasoning (Qwen 2.5) models. While RFM allows for faster subspace identification, it also showed better performances on the ablation task than its alternatives. More work is planned to better understand the relations between subspaces found by different methods. If confirmed, RFM could be a cheap and scalable complement to existing subspace-extraction methods in LLMs.