Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Tasks

2503.21696

Wenqi Zhang et el.

🤗 2025-03-28

TL;DR
#

Deep thinking models excel in math and coding but struggle with embodied tasks needing constant environment interaction. This paper introduces Embodied-Reasoner, which brings 01-style reasoning to embodied search. Unlike logic-based math reasoning, embodied tasks need spatial understanding, temporal reasoning, and reflection using interaction history. It includes 9.3k Observation-Thought-Action trajectories with 64k images and 90k thinking processes for analysis and planning.

The model trains in three stages: imitation learning, self-exploration through rejection sampling, and self-correction using reflection tuning. Embodied-Reasoner outperforms visual reasoning models like OpenAI 01, 03-mini, and Claude-3.7 by +9%, 24%, and +13% respectively. It reduces repeated searches and logical errors, excelling in complex tasks. Real-world tests prove its superiority with fewer repeated searches and logical inconsistencies.

Key Takeaways
#

Why does it matter?
#

This paper offers a valuable dataset and model for embodied AI research. It tackles the challenges of interactive reasoning, paving the way for more intelligent agents in complex environments. It opens doors for future work in long-horizon tasks and real-world applications.

Visual Insights
#

🔼 The figure illustrates an embodied interactive task where an agent searches for objects within an unfamiliar room. The Embodied-Reasoner model is introduced, showcasing its ability to engage in spontaneous reasoning and interact with the environment. Before each action, the model generates diverse thoughts (e.g., self-reflection, spatial reasoning), creating an image-text interleaved trajectory. This approach results in consistent reasoning and efficient search strategies. In contrast, the figure highlights how OpenAI’s o3-mini model frequently performs repetitive searches and demonstrates logical inconsistencies, leading to a higher failure rate in completing the task.
read the caption
Figure 1: We design an embodied interactive task: searching for objects in an unknown room. Then we propose Embodied-Reasoner, which presents spontaneous reasoning and interaction ability. Before each action, it generates diverse thoughts, e.g., self-reflection or spatial reasoning, forming an image-text interleaved trajectory. It shows consistent reasoning and efficient search behaviors, whereas OpenAI o3-mini often exhibits repetitive searches and logical inconsistencies with higher failure rates.

Stage	#Trajectory	Source	#Image_all	#Image_max	#Action_avg
Train_1st	1,128	Synthesis	4636	11	4.11
Train_2nd	6,246	Self-Explore	45.8k	26	7.33
Train_3rd	2,016	Synthesis	13.8k	29	8.63
Total	9,390	-	64k	29	7.22
Testset	809	Human	4.9k	29	6.06

🔼 This table summarizes the dataset used for training and testing the Embodied-Reasoner model. The training dataset consists of 9,390 ⟨task, trajectory⟩ pairs, where each pair includes a task instruction and the corresponding sequence of observations, thoughts, and actions taken by an agent in the AI2-THOR simulator. The trajectories were synthesized using a data engine, which automatically generates coherent and diverse sequences of actions and accompanying thoughts. In addition to this synthesized training data, 809 novel testing tasks were manually annotated with key actions to enable a comprehensive evaluation of the model.
read the caption
Table 1: We synthesize 9.3k𝑘kitalic_k ⟨⟨\langle⟨task, trajectory⟩⟩\rangle⟩ for training. Also, we manually annotate key actions for 809 novel testing tasks.

In-depth insights
#

Embodied Logic
#

Embodied logic focuses on how reasoning is integrated with physical interaction and perception. It’s about understanding how agents, whether robots or humans, make decisions and solve problems while actively engaging with their environment. This involves a synergy of perception, cognition, and action, where logical deductions are informed by real-time sensory input. Unlike abstract reasoning, embodied logic considers spatial understanding, temporal awareness, and the agent’s interaction history. Models employing embodied logic need to address challenges such as extended multimodal interaction, where reasoning is based on a continuous stream of visual and textual data. Further, they must handle diverse reasoning modalities, such as situational analysis, spatial reasoning, and self-reflection, which are essential for navigating and manipulating the physical world. Embodied-Reasoner addresses this by synthesizing coherent observation-thought-action trajectories. The ultimate goal is to create AI agents that can reason, plan, and act effectively in complex, real-world scenarios, similar to how humans do.

Visuomotor Tuning
#

While ‘Visuomotor Tuning’ isn’t explicitly in the paper, the concept is interwoven within the embodied AI task. The paper leverages visuomotor coordination through its three-stage training: imitation, exploration, and reflection. Imitation learning initiates basic visuomotor skills, exploration refines action selection based on visual feedback, and reflection corrects errors, essentially tuning the policy. The framework emphasizes spatial reasoning and memory. The agent uses observations and previous actions to plan and adjust strategies, indicating visuomotor tuning is essential for navigation, object manipulation, and more. Error correction further hones this by learning from failures. By incorporating real-time visual processing with reasoned actions, the model exhibits a form of adaptive visuomotor tuning, crucial for long-horizon tasks.

Data Engine
#

The paper introduces a data engine to synthesize Observation-Thought-Action trajectories, crucial for training an embodied agent. It automates the generation of coherent and diverse datasets, addressing the limitations of existing datasets. The data engine leverages task templates and an affiliation graph to ensure constraint satisfaction and derive key actions. It integrates LLMs to diversify instructions and synthesize reasoning tokens, creating a realistic interactive experience. This methodology is essential to equip the embodied agent in planning and decision making and the ability to exhibit behaviors when interacting with novel situations and environments.

Iterative Refine
#

Iterative refinement, a cornerstone in diverse fields, emphasizes incremental improvements. In the context of research papers, it likely denotes a methodology where initial results are progressively refined through multiple cycles of experimentation, analysis, and adjustment. This process is crucial for robustness and accuracy. It helps in eliminating errors, and refining models for optimal performance. Such an approach contrasts with single-pass methods, highlighting a commitment to thoroughness and nuanced understanding by continually revisiting and improving on prior work. This dedication often leads to a more credible result, better-optimized parameters, and a higher degree of confidence in the findings. Further research is needed to fully validate the results.

Beyond Robotics
#

While the research paper primarily focuses on synergizing visual search, reasoning, and action within embodied interactive tasks, the concept of ‘Beyond Robotics’ suggests exploring broader implications and future directions. Traditional robotics often emphasizes task execution in structured environments, but this paper implicitly pushes for greater adaptability and cognitive capabilities in robots operating in dynamic and unstructured settings. Envisioning the future, we anticipate robots that seamlessly integrate with human environments, understanding nuanced instructions, adapting to unexpected changes, and even exhibiting a degree of creativity in problem-solving. By endowing robots with advanced perception, reasoning, and self-reflection mechanisms, we move beyond mere task completion and toward genuinely intelligent and helpful robotic companions. This could mean deploying similar models in more high-stakes environments such as search and rescue or hazardous material handling.

More visual insights
#

More on figures

🔼 The figure showcases the Embodied-Reasoner’s superior performance in handling complex, interactive tasks compared to traditional Vision-Language Models (VLMs). It highlights the model’s ability to generate spontaneous and coherent thoughts (analysis, reflection, planning) across multiple steps. Specifically, it demonstrates how Embodied-Reasoner effectively analyzes the environment (#1, #3), considers previously missed information (#4), reasons using the latest observations (#5), and recalls previous cues to create efficient plans (#9). This contrasts with traditional VLMs, which often struggle with long-horizon tasks, leading to inconsistent or illogical actions, such as forgetting tasks or repetitive searching.
read the caption
Figure 2: Embodied-Reasoner exhibits spontaneous thinking behaviors, e.g., analyzing environmental states (#1,3), reflecting on missed details (#4), reasoning based on the latest observations (#5), and recalling cues for efficient planning (#9). These thoughts remain coherent and logically consistent despite spanning multiple rounds. In contrast, general VLMs lacking thinking abilities struggle with long-horizon interactive tasks and produce unreasonable actions, e.g., forget tasks or repetitive searching.

🔼 Figure 3 illustrates the process of creating a dataset for training an embodied reasoning model and the three-stage training process used. The left side shows how instructions are generated from task templates and an affiliation graph, which represents relationships between objects, is built. Exploratory actions and interleaved thoughts are then added to create interactive trajectories. The right side depicts the three-stage training recipe: 1) imitation learning using the synthesized trajectories, 2) self-exploration through rejection sampling to enhance exploration abilities, and 3) self-correction by adding anomalous states and reflective thoughts to refine the model’s behavior. The final outcome is the Embodied-Reasoner model.
read the caption
Figure 3: Left: Data Engine for <<>> synthesis. First, we synthesize instructions from task templates, and build an affiliation graph from scene’s meta-data. It enables us to derive key actions needed for task. We add exploratory actions and insert thinking thoughts between observation and actions. Right: Three-stage training recipe. ①We fine-tune on synthesized trajectory to develop interaction skills. ②We sample multiple trajectories on novel tasks and evaluate their correctness. The successful ones are used for developing its exploring abilities. ③We continue to sample trajectories using updated model, injecting anomalous states and reflective thoughts in successful cases and correcting errors in failed ones. This self-correction training yields Embodied-Reasoner.

🔼 This figure visualizes the frequency of five different types of thought processes (Situation Analysis, Task Planning, Spatial Reasoning, Self-Reflection, and Double Verification) within the generated embodied reasoning trajectories. It also shows the dynamic transitions between these thought types, highlighting their flexible and interconnected nature within the problem-solving process. This demonstrates the model’s ability to adapt its reasoning approach depending on the task’s demands and the current situation.
read the caption
Figure 4: We analyze the frequency of five types of thoughts and their flexible transition relationships in all trajectories.

🔼 Figure 5 illustrates the relationship between task complexity, model performance, and the number of reasoning tokens generated. The x-axis represents task length (number of key actions required), indicating increasing complexity. The y-axis shows two key metrics: success rate and the number of reasoning tokens produced by the model. The figure demonstrates that as task complexity increases (longer task lengths), the success rate of baseline models drops significantly. However, the Embodied-Reasoner model maintains high success rates even for complex tasks, achieving this by generating a substantially larger number of reasoning tokens. This suggests that the model leverages more extensive reasoning to tackle more challenging problems, showcasing the effectiveness of its deep-thinking mechanism.
read the caption
Figure 5: Relations between task length and success rate, and output token number. As task complexity increases, our model generates more reasoning tokens to maintain high success rates.

🔼 This figure illustrates the results of evaluating the models’ tendency to repeatedly explore the same areas during a search task. The x-axis represents different task types (search, manipulate, transport, composite, overall), while the y-axis shows the percentage of repetitive explorations. The bars indicate the repetitive exploration rate (RER) for various models including the authors’ proposed Embodied-Reasoner and Embodied-Explorer, as well as several baseline models (GPT-40, Claude 3.5-Sonnet, Gemini-2.0 Flash Thinking, Qwen-VL-Max, GPT-03-mini, and two versions of Qwen2.5-VL-72B). The lower the RER, the more efficient the search strategy. This figure highlights that the proposed models significantly reduce repetitive searches compared to baseline models, demonstrating the efficiency of their planning and self-reflection capabilities in avoiding unnecessary exploration.
read the caption
Figure 6: Repetitive Exploration Rate measures repetitive search issues, which are often observed in baselines. Our models reduce repetitive searches by recalling and reflecting on past trajectories.

🔼 The figure showcases a comparison of the success rates achieved in real-world experiments across three different models. Embodied-Reasoner demonstrated a significantly higher success rate (56.7%) compared to OpenAI’s O3-mini (43.4%) and O1 (50%). This highlights the model’s improved performance in real-world settings for object searching tasks.
read the caption
Figure 7: Real-world experiments. Our model achieves a higher success rate (56.7%) than OpenAI o3-mini (43.4%) and o1 (50%).

🔼 This figure shows a breakdown of the 9,390 samples in the training dataset. It visualizes the proportions of four main task types (Search, Manipulate, Transport, Composite) and their further subdivisions into ten sub-task types. The sizes of the sections in the circular diagram represent the relative number of samples belonging to each category. This provides a clear overview of the dataset’s composition and the distribution of various task complexities.
read the caption
Figure C1: The distribution of the training dataset with 9,390 samples, including 4 task types and 10 sub-task types.

🔼 This figure shows a breakdown of the 809 tasks used in the test set of the Embodied-Reasoner model. It details the distribution of tasks across four main task types (Search, Manipulate, Transport, Composite) and further specifies the distribution within each main type according to 11 sub-task categories. This provides a visual representation of the complexity and diversity of tasks the model was evaluated on.
read the caption
Figure C2: The distribution of the test set with 809 tasks, including 4 task types and 11 sub-task types.

🔼 Figure C3 shows a breakdown of the actions taken within the training dataset’s trajectories. It displays the frequency of eight distinct interaction types: ’navigate to’, ‘pickup’, ‘open’, ‘close’, ‘put in’, ‘observe’, ‘move forward’, and ’toggle’. The bar chart visually represents the number of times each action was performed across all trajectories in the training dataset, providing insights into the distribution of actions within the embodied interactive tasks.
read the caption
Figure C3: The distribution of the training set interactions, including 8 interaction types in trajectories: navigate to, pickup, open, close, put in, observe, move forward, and toggle.

🔼 Figure C4 shows the frequency of six different interaction types within the key actions of the test dataset. The six interaction types are: ’navigate to’, ‘pickup’, ‘open’, ‘close’, ‘put in’, and ’toggle’. The chart visually represents the number of times each action occurred in the test set’s key action sequences, providing insights into the relative frequency of different action types during task execution.
read the caption
Figure C4: The distribution of the test set interactions, including 6 interaction types in key actions: navigate to, pickup, open, close, put in, and toggle.

🔼 This figure shows the distribution of trajectory lengths for different task types in the training dataset. The x-axis represents the length of the trajectory (number of actions taken), and the y-axis shows the number of tasks with that trajectory length. Each colored bar represents a different task type: Search, Manipulate, Transport, and Composite. The figure highlights that Search tasks tend to have shorter trajectories (mostly between 1 and 9 actions), Manipulate tasks have slightly longer trajectories (between 2 and 11 actions), Transport tasks are longer still (between 3 and 14 actions), and Composite tasks have the longest trajectories, often exceeding 23 actions.
read the caption
Figure C5: The quantity distribution of trajectory lengths in the training set and the corresponding task type composition, where Search Task is mainly within 1-9, Manipulate Task within 2-11, Transport Task within 3-14, and Composite Task above 8, extending beyond 23.

🔼 Figure C6 shows the distribution of the number of key actions needed to complete tasks in the test dataset. The x-axis represents the length of the action sequence, and the y-axis shows the count of tasks. Each bar is further divided into four colors representing the four task types: Search, Manipulate, Transport, and Composite. The figure reveals that Search tasks generally require 1–2 actions, Manipulate tasks 2–5, Transport tasks 4–7, and Composite tasks 8 or more, with some extending beyond 19 actions.
read the caption
Figure C6: The quantity distribution of key action lengths in the test set and the corresponding task type composition, where Search Task is mainly within 1-2, Manipulate Task within 2, 4-5, Transport Task within 4-7, and Composite Task above 8, extending beyond 19.

🔼 This figure shows the frequency distribution of the top 32 most common object types across all trajectories in the training dataset. The ‘Others’ category encompasses the remaining 62 less frequent object types, examples of which include Bread, Book, and DeskLamp. The visualization helps to understand the prevalence of different object types within the simulated environments used for training the embodied reasoning model.
read the caption
Figure C7: The quantity distribution of the top 32 object types in the training dataset’s trajectories, with Others representing the remaining 62 categories, such as Bread, Book, DeskLamp, etc.

🔼 Figure C8 shows the frequency distribution of the top 32 most frequently appearing object types within the key actions of the test set. The chart visually represents the number of times each object type is involved in the key action sequences during testing. The category ‘Others’ encompasses the remaining 44 object types that did not rank within the top 32, including items like watches, pencils, cups, etc. This visualization helps to understand the prevalence of various object types in the tasks and the overall composition of the test dataset.
read the caption
Figure C8: The quantity distribution of the top 32 object types in the test set’s key actions, with Others representing the remaining 44 categories, such as Watch, Pencil, Cup, etc.

🔼 This figure visualizes a step-by-step example of Embodied Reasoner completing a complex task. It showcases the model’s ability to generate coherent reasoning tokens (thoughts), plan actions, and execute them successfully in a simulated environment. Each step includes an image from the agent’s perspective, followed by the model’s reasoning process and the selected action. The process demonstrates capabilities like spatial understanding, planning, and self-reflection.
read the caption
Figure F9: Trajectory Case for Embodied Reasoner

🔼 This figure shows a step-by-step breakdown of GPT-01’s performance on the task of placing a laptop on a sofa and then a cellphone in a drawer. It highlights GPT-01’s struggles with task completion due to issues like forgetting the task objective, getting stuck in action loops, and failing to appropriately respond to feedback regarding illegal actions or unavailable objects. The figure contrasts with Figure F9, which showcases Embodied-Reasoner’s superior performance on the same task.
read the caption
Figure F10: Trajectory Case for GPT-o1

🔼 This figure showcases a real-world application of the Embodied Reasoner model. The task is to place a carton of milk on a coffee table. The figure depicts a sequence of images showing the robot’s actions and the model’s thought process at each step. The model begins by locating the milk in the refrigerator, retrieves it, and then, after a moment of re-evaluation of the environment (to make sure the coffee table’s location is clearly identified), places it on the coffee table and concludes the task.
read the caption
Figure F11: Trajectory Case for Embodied Reasoner in Real World

Model			Success Rate $\uparrow$	Search Efficiency $\uparrow$	Task Completeness $\uparrow$	Success Rate for SubTasks $\uparrow$
Model			Success Rate $\uparrow$	Search Efficiency $\uparrow$	Search	Manipulate	Transport	Composite
		Qwen2.5-VL-7B-Instruct [4]	12.38%	10.87%	27.53%	6.45%	23.55%	7.56%	0.95%
	Qwen2-VL-7B-Instruct [45]	14.79%	11.97%	38.67%	23.33%	25.50%	2.82%	0.00%
General-	Qwen2.5-VL-72B-Instruct [4]	31.75%	22.61%	50.62%	52.14%	38.89%	21.90%	0.00%
purpose	Qwen2-VL-72B-Instruct [45]	39.00%	28.88%	54.56%	50.00%	52.36%	33.19%	0.00%
VLMs	Claude 3.5-Sonnet [2]	45.35%	28.05%	64.12%	54.25%	50.51%	51.22%	3.84%
	Qwen-VL-Max [40]	49.81%	36.28%	68.39%	63.87%	63.21%	45.16%	1.90%
	GPT-4o [29]	66.67%	41.68%	79.07%	69.03%	79.26%	71.95%	14.42%
		QVQ-72B-Preview [34]	7.54%	6.39%	36.33%	4.35%	7.50%	10.53%	0.00%
	Kimi-K1.5^† [38]	46.00%	-	-	-	-	-	-
	GPT-o3-mini [31]	56.55%	26.93%	67.41%	78.57%	59.32%	66.67%	0.00%
Visual	Gemini-2.0 Flash Thinking [10]	56.74%	43.01%	71.70%	71.05%	75.60%	40.67%	8.89%
Reasoning	Claude-3.7-Sonnet-thinking [3]	67.70%	37.95%	78.63%	69.12%	75.88%	71.94%	13.79%
Models	GPT-o1 [30]	71.73%	43.06%	82.49%	78.42%	79.10%	67.36%	13.16%
	Embodied-Interactor-7B (ours-1st)	25.46%	24.75%	53.67%	30.97%	27.09%	29.20%	3.81%
	Embodied-Explorer-7B (ours-2nd)	65.39%	46.25%	77.73%	60.00%	75.92%	72.24%	26.67%
	Embodied-Reasoner-7B (ours-3rd)	80.96%	55.07%	86.30%	65.16%	93.31%	87.20%	54.29%

Model	Success Rate (%)
Qwen2.5-VL-72B-Instruct	43.3
OpenAI o1	50.0
OpenAI o3-mini	44.0
Embodied-Reasoner	56.7

TL;DR#

Key Takeaways#

Why does it matter?#

Visual Insights#

In-depth insights#

Embodied Logic#

Visuomotor Tuning#

Data Engine#

Iterative Refine#

Beyond Robotics#

More visual insights#

Full paper#