具身智能行业正面临一场昂贵的浪费。同样的任务，每换一种机器人，就可能需要重新采集数据、训练模型。如果未来有上万种机器人，难道就要训练上万颗大脑。如今，一脑多形、一脑多态成为行业探索的新方向，试图让不同机器人共享智能。但身体千差万别的机器人，究竟能共享哪些能力，一颗大脑又如何适应不同的身体。为了寻找答案，虎嗅和香港具身智能实验室联合总监、香港中文大学助理教授李钟毓聊了聊。
The embodied intelligence industry faces an expensive waste. The same task may require re-collecting data and retraining models every time the robot changes. If there are tens of thousands of robots in the future, must we train tens of thousands of brains. Today, "one brain, multiple forms" and "one brain, multiple states" have become new directions the industry is exploring, attempting to let different robots share intelligence. But what capabilities can robots with vastly different bodies actually share, and how can one brain adapt to different bodies. To find answers, Huxiu spoke with Li Zhongyu, Joint Director of the Hong Kong Embodied Intelligence Lab and Assistant Professor at the Chinese University of Hong Kong.
李钟毓1996年出生，本科毕业于浙江大学机械电子工程专业，2025年获得加州大学伯克利分校机械工程博士学位。他早期主要研究双足机器人的运动控制，后来逐渐将研究延伸至不同机器人之间的数据、模型与能力迁移。他所在的香港具身智能实验室，也在尝试将跨构型研究推向产业。目前，实验室已与24家产业伙伴、投资机构及科创企业展开合作，包括智元机器人、宇树科技、云深处、仙工智能、舞肌科技等。
Li Zhongyu was born in 1996, graduated from Zhejiang University with a bachelor's degree in mechatronic engineering, and earned a Ph.D. in mechanical engineering from UC Berkeley in 2025. He initially focused on bipedal robot motion control, later extending research to data, model, and capability transfer between different robots. His Hong Kong Embodied Intelligence Lab is also attempting to push cross-embodiment research toward industry. Currently, the lab has cooperated with 24 industry partners, investment institutions, and tech startups, including AgiBot, Unitree, Deep Robotics, Seer Intelligent, and Wuji Technology.
李钟毓说，我们最开始研究人形机器人，看中的就是它的通用性。人有两条腿、两条手臂和两只灵活的手，可以去各种各样的地方，完成各种各样的工作。所以理论上只要把人形机器人的模型和运动控制能力做好，它就能够适应不同的任务和环境。但我们也很清楚，人形机器人现在还不够成熟。即使再过五年、十年，也很难说会有某一种机器人，能够把所有事情都做了。现实世界中的任务，对机器人的身体有不同要求。有的需要两米长的机械臂，有的要求重载，有的要把东西举得很高，有的需要移动得很快。所以未来更可能出现的情况，是多种构型的机器人长期共存，各自在专业场景中完成不同的工作。
Li Zhongyu said: "We initially studied humanoid robots for their generality. Humans have two legs, two arms, and two dexterous hands, can go to all kinds of places and complete all kinds of work. So theoretically, if the humanoid robot's model and motion control capabilities are well developed, it can adapt to different tasks and environments. But we are also clear that humanoid robots are not yet mature. Even in five or ten years, it is hard to say that one kind of robot will be able to do everything. Real-world tasks have different requirements for robot bodies. Some need two-meter robotic arms, some require heavy loads, some need to lift things high, some need to move fast. So the more likely future is multiple robot configurations coexisting long-term, each completing different work in specialized scenarios."
这也带来一个问题，如果每一种机器人都要单独开发一套大脑和小脑，重新采集数据、训练模型，整个行业的成本会非常高。更大的问题是，这些投入很难积累到一起。比如，一家公司做机械臂，另一家公司做人形机器人，它们采集的数据不同，模型的输入输出不同，对任务的理解和动作表示也可能不同。即使两者完成的是类似任务，积累的经验也很难直接共享。机器人越来越多，每一类却都在各练各的，就很难形成我们希望看到的Scaling效应。所以跨构型是一个必须尽早考虑的问题。
"This also raises a problem: if every kind of robot needs its own brain and cerebellum, re-collecting data and retraining models, the entire industry's cost will be very high. The bigger problem is that these investments are hard to accumulate together. For example, one company makes robotic arms, another makes humanoid robots; the data they collect differs, the models' inputs and outputs differ, and their understanding of tasks and action representations may also differ. Even if both complete similar tasks, accumulated experience is hard to share directly. With more and more robots, each category practicing on its own, it is hard to form the Scaling effect we hope to see. So cross-embodiment is a problem that must be considered early."
为什么最近大家又开始重视跨构型。一个重要原因是，我们发现，单一构型模型训练出来的部分能力，已经开始表现出复用和迁移的可能性。过去，跨构型最大的难点之一，就是输入和输出对不上。不同机器人拥有不同的自由度、关节结构和传感器。摄像头的位置不同，看到的画面就不同；机械臂的关节数量不同，模型输出的控制指令也不能直接照搬。但随着数据规模扩大，我们开始发现，如果能够在不同构型之间找到一些共性，模型就有机会学习到可以迁移的能力。比如，机器人要把一个杯子放到桌子的另一边。无论是人形机器人、轮式双臂机器人，还是固定机械臂，它们都需要理解杯子在哪里、目标在哪里，以及应该完成什么任务。至于具体哪个关节转多少度、轮子往哪个方向走，则取决于它们各自的身体结构。
"Why has everyone recently begun to value cross-embodiment again. An important reason is that we found some capabilities trained by single-configuration models have begun to show reusability and transferability. Previously, the biggest difficulty in cross-embodiment was input-output mismatch. Different robots have different degrees of freedom, joint structures, and sensors. Cameras at different positions see different images; arms with different joint counts cannot directly copy model output control commands. But as data scale expands, we began to find that if commonalities can be found between configurations, models have a chance to learn transferable capabilities. For example, a robot needs to put a cup on the other side of a table. Whether humanoid, wheeled dual-arm, or fixed arm, they all need to understand where the cup is, where the target is, and what task to complete. As for which joint rotates how much and which direction the wheel moves, that depends on their respective body structures."
所以，跨构型的一个重要思路，就是先把要做什么和具体怎么做尽量分开。我们可以让模型学习一种更抽象的表示，例如任务意图，再由不同机器人根据自己的身体条件，把这种意图转化成具体动作。这有点像人学习不同的交通工具。一个人先学会骑自行车，再去学骑摩托车、开汽车。具体操作方式一直在变，但过去的经验可以帮助他更快掌握新的操作方式。但这里还有一个关键问题，迁移能力最好在预训练阶段就开始考虑。如果模型最开始只围绕某一种构型训练，到了后训练阶段才要求它适应另一种机器人，就可能出现所谓的灾难性遗忘。也就是说，新任务学会了，原来的一些能力反而丢掉了。因此，我们需要在模型设计和训练的早期，就考虑不同构型之间有哪些能力可以共享。
"So an important cross-embodiment idea is to separate 'what to do' from 'how to do it' as much as possible. We can let models learn a more abstract representation, such as task intent, and then different robots convert this intent into specific actions based on their own body conditions. This is a bit like a person learning different vehicles. First learn to ride a bicycle, then learn to ride a motorcycle or drive a car. Specific operations keep changing, but past experience helps faster mastery of new operations. But there is a key problem: transferability is best considered at the pretraining stage. If a model is initially trained only around one configuration and only later required to adapt to another robot, catastrophic forgetting may occur—new tasks are learned but some original capabilities are lost. Therefore, we need to consider which capabilities can be shared between configurations early in model design and training."
尽管机器人的身体一直在变，但我们可以从中找到一些相对不变的东西。除了任务意图，还有两种能力非常重要。第一种，是对世界的理解。这与世界模型有关。不同机器人虽然身体不同，但它们观察的是同一个物理世界。比如，一个杯子放在桌子上，不管是人形还是轮式机器人，都需要理解杯子的位置、周围环境，以及自己的动作可能带来什么变化。机器人能够采取的动作不同，但世界本身的很多规律是共通的。因此，世界模型的预测能力和空间、物理的理解能力，可以成为跨构型的一个共同基础。第二种，是对任务进度的判断。人在完成一项任务时，通常会对进度有一个大致的判断。我们未必能准确说出已经完成了百分之多少，但通常知道自己做到了哪一步、还有哪些事情没做、是不是快完成了。机器人也可以学习这种能力，也就是进度模型。
"Although robot bodies keep changing, we can find some relatively invariant things. Besides task intent, two other capabilities are very important. First, understanding the world. This relates to World Models. Different robots have different bodies but observe the same physical world. For example, a cup on a table—whether humanoid or wheeled robot, both need to understand the cup's position, surrounding environment, and what changes their actions may bring. Robots can take different actions, but many laws of the world itself are common. Therefore, world models' prediction capability and spatial/physical understanding can become a common foundation for cross-embodiment. Second, judging task progress. When humans complete a task, they usually have a rough sense of progress. We may not accurately say what percentage is done, but we usually know which step we're at, what remains, and whether we're nearly finished. Robots can also learn this capability—a Progress Model."
跨构型还有一个现实问题，数据从哪里来。数据规模对模型能力至关重要，但要让机器人掌握各种各样的技能，需要的数据量非常庞大。如果完全依靠机器人自身采集，成本很高，而且不同构型的数据也很难直接复用。所以，我们认为，人类的第一视角数据非常重要。比如，人可以佩戴摄像头或其他采集设备，记录自己拿取物品、整理桌面、完成装配任务的过程。这些数据包含了人对环境的观察，以及人与环境交互的经验，可以帮助模型学习一些更通用的能力。在我们看来，人类操作经验可以成为不同构型机器人学习的共同参照。
"Cross-embodiment has another practical problem: where does data come from. Data scale is crucial to model capability, but for robots to master diverse skills, the required data volume is enormous. If relying entirely on robots' own collection, costs are high, and data from different configurations is hard to reuse directly. So we believe human first-person data is very important. For example, a person can wear cameras or other collection devices to record their process of picking up items, organizing desks, and completing assembly tasks. This data contains human observations of the environment and human-environment interaction experience, helping models learn more general capabilities. In our view, human operation experience can become a common reference for different robot configurations to learn from."
当然，人类数据只是一个起点。机器人最终还是要用自己的身体完成任务，因此还需要结合不同构型的真机数据进行训练。一种可行的路径是，先利用大量人类第一视角数据进行预训练，让模型掌握相对通用的基础能力，再加入不同构型的真机数据，通过微调或继续训练，让模型适应不同的机器人。更重要的是，模型训练完成后，一定要重新部署到真机上，在真实环境中产生新的数据，再用这些数据继续训练。随着机器人部署规模扩大，未来的数据来源有机会逐渐减少对人类示范的依赖，更多来自机器人自身的交互。
"Of course, human data is only a starting point. Robots ultimately must complete tasks with their own bodies, so training must also combine real-machine data from different configurations. One feasible path is to first use large amounts of human first-person data for pretraining, letting models master relatively general foundational capabilities, then add real-machine data from different configurations through fine-tuning or continued training to adapt models to different robots. More importantly, after model training, it must be redeployed to real machines, generating new data in real environments, and using that data for continued training. As robot deployment scale expands, future data sources have the opportunity to gradually reduce dependence on human demonstrations and come more from robots' own interactions."
数据多样性很重要。比如，将同一段视频重复十万遍，即使累计出十万小时的数据，也不会增加多少有效信息。相比之下，不同构型、不同环境和不同交互方式产生的数据，能够让模型接触更多样化的任务和操作经验。如果始终围绕单一构型采集数据、训练模型，数据的覆盖范围和模型能力都会受到一定限制。而将不同构型的数据整合起来训练，就有机会让模型从中找到更多共性，学习可以在不同机器人之间迁移的能力。所以，未来的数据采集，也不应该局限于某一种机器人，或者某一个专门的数据采集中心。当然，这件事很难由几所高校或少数企业独立完成。高校擅长模型、算法和训练方法的研究，但很难长期覆盖大量机器人本体和真实场景，单个企业能够接触的构型和任务同样有限。因此，我们希望更多本体公司、模型企业、数据服务商和实际场景方参与进来，让不同构型的机器人在真实部署中持续产生数据，再将这些经验用于共同训练。跨构型最终能否实现规模化，关键在于不同机器人产生的数据和能力，能否真正流动和积累起来。
"Data diversity is important. For example, repeating the same video 100,000 times, even accumulating 100,000 hours of data, adds little effective information. In contrast, data generated from different configurations, different environments, and different interaction methods lets models more diverse tasks and operation experience. If data collection and model training always revolve around a single configuration, data coverage and model capability will be limited. Integrating data from different configurations gives models a chance to find more commonalities and learn transferable capabilities. So future data collection should not be limited to one type of robot or one dedicated data collection center. Of course, this is hard for a few universities or a few enterprises to complete independently. Universities excel at models, algorithms, and training methods but struggle to long-term cover large numbers of robot bodies and real scenarios; individual enterprises have limited configurations and tasks. Therefore, we hope more body companies, model companies, data service providers, and actual scenario parties participate, letting robots of different configurations continuously generate data in real deployment and using this experience for joint training. Whether cross-embodiment ultimately scales depends on whether data and capabilities generated by different robots can truly flow and accumulate."
