FIELD NOTES TOPIC
gui grounding
2 notes, newest first.
Hallucination-Free GUI Grounding via Regression-Free Layout-Aware Matching
Researchers propose a new framework for grounding user instructions in graphical user interfaces (GUI). The framework uses a frozen multimodal large language model (MLLM) to parse instructions and a dedicated grounding…
Uncertainty Quantification for Computer-Use Agents: A Benchmark across Vision-Language Models and GUI Grounding Datasets
Researchers have developed a benchmark called Argus to test the reliability of computer-use agents that convert vision-language model predictions into executable clicks. The study compares 27 methods across four…
