Abstract

Operating robots in open-ended scenarios with diverse tasks is a crucial research and application direction in robotics. While recent progress in natural language processing and large multimodal models has enhanced robots' ability to understand complex instructions, robot manipulation still faces the procedural skill dilemma and the declarative skill dilemma in open environments. Existing methods often compromise cognitive and executive capabilities. To address these challenges, in this paper, we propose RoBridge, a hierarchical intelligent architecture for general robotic manipulation. It consists of a high-level cognitive planner (HCP) based on a large-scale pre-trained vision-language model (VLM), an invariant operable representation (IOR) serving as a symbolic bridge, and a generalist embodied agent (GEA). RoBridge maintains the declarative skill of VLM and unleashes the procedural skill of reinforcement learning, effectively bridging the gap between cognition and execution. RoBridge demonstrates significant performance improvements over existing baselines, achieving a 75% success rate on new tasks and an 83% average success rate in sim-to-real generalization using only five real-world data samples per task. This work represents a significant step towards integrating cognitive reasoning with physical execution in robotic systems, offering a new paradigm for general robotic manipulation.

Framework

Experiment

Conclusion

In this paper,we introduce RoBridge, a novel hierarchical intelligent architecture designed to enhance robotic manip ulation by bridging the gap between high-level cognitive planning and low-level physical execution. The architecture integrates a high-level cognitive planner, an invariant operable representation, and a generalist embodied agent, demonstrating significant advancements in task generalization and execution robustness. Through extensive experiments, RoBridge has shown superior performance and strong zero shot generalization capabilities in unknown environments and novel tasks.