Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

Latent Actions from Factorized Transition Effects under Agent Ambiguity

Source

arXiv

arxiv.org/abs/2606.30544v1

Read the source

Source

Plain-Language Brief

The research introduces Observed Transition Factorization (OTF), a method to decompose transitions in multi-object or distracting scenes into reusable transition primitives. These primitives are used to form action-like latents more robustly. The paper proposes OTF-LAM and OTF-LAM-Dino, models that abstract motion primitives and predict future states. The primitives transfer well across different conditions and perform competitively in policy learning under complex transition ambiguity.

What the Document Says

  • Latent Action Models (LAMs) learn action-like proxies from observation transitions, but these visual effects can be ambiguous in multi-object or distractor-rich scenes due to mixed agent motion, camera dynamics, and background changes.
  • OTF decomposes each transition into a sparse set of observed transition primitives, acting as the transition interface.
  • OTF-LAM abstracts motion primitives into action-like latents within the standard inverse-forward dynamics framework.
  • OTF-LAM-Dino predicts future states in a frozen DINOv2 representation space, without a decoder.
  • The OTF primitives transfer well across controlled carrier and morphology shifts, demonstrating reusability.
  • Downstream policy learning results match or outperform baselines under complex transition ambiguity.

Key Concepts Mentioned

  • Latent Action Models (LAMs)
  • Observed Transition Factorization (OTF)
  • transition primitives
  • inverse-forward dynamics framework
  • DINOv2 representation space
  • policy learning

Claims Made by the Document

  • OTF provides an intermediate representation from which action-like latents can be more robustly formed.
  • OTF primitives transfer well across different conditions.
  • Downstream policy learning results match or outperform baselines under complex transition ambiguity.

Evidence Used

  • The paper demonstrates the reusability of OTF primitives across controlled carrier and morphology shifts.
  • Empirical results show that downstream policy learning matches or outperforms baselines under complex transition ambiguity.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Latent Action Models (LAMs) learn action-like proxies from observation transitions. However, in multi-object or distractor-rich scenes, these visual effects mix agent motion with distractors, camera dynamics, and background changes, making the underlying action source ambiguous without supervision. Structuring this mixture as reusable transition effects provides an intermediate representation from which action-like latents can be more robustly formed. We introduce Observed Transition Factorization (OTF), which decomposes each transition into a sparse set of observed transition primitives. Using these primitives as the transition interface, we propose OTF-LAM, which abstracts motion primitives into action-like latents within the standard inverse-forward dynamics framework, and OTF-LAM-Dino, a decoder-free variant that predicts future states in a frozen DINOv2 representation space. Empirically, OTF primitives transfer zeroshot across controlled carrier and morphology shifts, showing reusability. Furthermore, downstream policy learning results match or outperform baselines under complex transition ambiguity.