Proactive Human-Robot Collaboration

Human-robot collaboration (HRC) has attracted strong interests from researchers and engineers for improved operational flexibility and efficiency towards mass personalization. Nevertheless, existing HRC development mainly undertakes either human-centred or robot-centred manner reactively, where operations are conducted by following the pre-defined instructions, thus far from an efficient integration of robotic automation and human cognitions. The prevailing research on human-level information processing of cognitive computing, the industrial IoT, and robot learning creates the possibility of bridging the gap of knowledge distilling and information sharing between onsite operators, robots and other manufacturing systems. Hence, a foreseeable informatics-based cognitive manufacturing paradigm, Proactive HRC, is introduced as an advanced form of Symbiotic HRC with high-level cognitive teamwork skills to be achieved stepwise, including: 1) inter-collaboration cognition, establishing bi-directional empathy in the execution loop based on a holistic understanding of humans and robots’ situations; 2) spatio-temporal cooperation prediction, estimating human-robot-object interaction of hierarchical sub-tasks/activities over time for the proactive planning; and 3) self-organizing teamwork, converging knowledge of distributed HRC systems for self-organization learning and task allocation.

Figure. An evolvement pathway towards Proactive HRC

Vision Language Model-based Human-Robot Interactive Navigation and Manipulation
Deep reinforcement learning-based robotic safe motion planning with human-in-the-loop
VR-based Dexterous Robotic Teleoperation
Mutual Cognitive Human-Robot Collaboration System

Main funded Projects

  • 01/2024 – 12/2026 ‘基於擴增實境和數位孿生技術的人機協作製造系統’, 莱茵科斯特智能科技有限公司, HK$2,000,000, PI
  • 01/2024 – 12/2026 ‘A Vision-based Human Digital Twin Modelling and Scene Understanding Approach for Adaptive and Natural Task Planning in Human-Robot Collaboration’, GRF, HK$990,198, PI
  • 12/2023 – 11/2025‘基于自主学习的沉浸式自动化制孔在线质量评估’, 中國商飛上海飛機製造有限公司, CNY 1,495,000, PI
  • 01/2023 – 12/2025 ‘A multimodal intelligence-enabled strategy learning approach for cognitive human-robot collaborative assembly‘, GRF, HK$1,036,905, PI
  • 01/2022 – 12/2025 Wong Tit Shing Endowed Young Scholar in Smart Robotics, HK$1,200,000, PI
  • 09/2020 – 08/2024 ‘Big data analytics for product and service design’ (AiDLab-RP2-1), AIR@InnoHK Project by Innovation and Technology Commission (ITC), ~HK$2.5 Million, Co-I
  • 09/2020 – 12/2021 ‘A Novel CNN-based Approach for 3D Printed Complex Surface Inspection’ (G-UAHH), DGRF project by The Hong Kong Polytechnic University, HK$250,000, PI
  • 06/2020 – 06/2022 ‘Joint research and development of key technology of intelligent design of cutting machine’ (BZ2020049), by Jiangsu Science and Technology Plan (Hong Kong/Macau/Taiwan Joint Programme), 200,000 CNY (Total amount: 700,000 CNY), Hong Kong PI