NoContactNoWorries: Estimating Contact through Vision and Proprioception for In-Hand Dexterous Manipulation

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Soham Patil, Avirup Das, Sourabh Bhosale, Spandan Roy
• Venue: arXiv: cs.RO
• Published: 2026-06-23
• Collected: 2026-06-24
• URL: http://arxiv.org/abs/2606.24450v1
• PDF: https://arxiv.org/pdf/2606.24450v1
• DOI: UNKNOWN

Plain-Language Brief

Researchers have developed a transformer-based multimodal framework named NoContactNoWorries that allows robots to infer binary contact states from RGB-D vision and the robot's proprioception. This approach offers a scalable alternative to tactile hardware for binary contact estimation, addressing practical challenges in cost, fragility, and integration.

What the Document Says

• The research focuses on perceiving physical contact during dexterous manipulation.
• Robots typically use dedicated tactile sensors but humans integrate visual information with their body's pose and movement to infer contact.
• NoContactNoWorries is a transformer-based multimodal framework that combines RGB-D vision with proprioception to infer binary contact states as a pseudo-tactile signal for hand-object interactions.
• The model was validated by training on multiple objects and demonstrating support for downstream reinforcement learning agents in object reorientation, generalizing to novel objects.
• Experiments in simulation and on a real-world robot validate the approach.

Key Concepts Mentioned

• perception
• robots
• dexterous manipulation
• tactile sensors
• visual information
• proprioception
• transformer-based multimodal framework
• binary contact estimation
• RGB-D vision
• reinforcement learning
• hand-object interactions

Claims Made by the Document

• Robots can learn to infer contact from vision.
• NoContactNoWorries can fuse RGB-D vision and proprioception for binary contact state inference.

Evidence Used

• The text describes a framework that combines vision and proprioception for contact inference

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Perceiving physical contact is fundamental to dexterous manipulation. While robots often rely on dedicated hardware tactile sensors, humans exhibit a remarkable ability to infer contact by integrating visual information with an innate sense of their body's pose and movement. Inspired by this embodied perceptual skill, we investigate whether a robot can learn to infer contact from vision, an approach that also offers a scalable alternative to tactile hardware specifically for binary contact estimation, which faces practical challenges in cost, fragility, and integration. We present NoContactNoWorries, a transformer-based multimodal framework that fuses RGB-D vision with the robot's proprioception to infer binary contact states as a pseudo-tactile signal for hand-object interactions. We validate by training a single contact prediction model on multiple objects and show that the inferred contact signal supports downstream reinforcement learning agents for in-hand object reorientation, generalizing to novel objects. Experiments in both simulation and on a real-world robot validate our approach, highlighting the feasibility of inferring contact from vision and proprioception. Project Page: https://soham2560.github.io/no-contact-no-worries/