Robots and automated systems depend on three closely linked elements: sensors that gather information, control and AI algorithms that turn that information into decisions, and actuators that act on the physical world. At HKU AIR Lab, we develop new technologies across all three and integrate them into robotic systems that work from the micro/nano scale to the millimetre scale. Our research is organised into three directions, with applications in healthcare, advanced manufacturing and construction.
Miniature and Soft Robots
Robots only a few millimetres in size, or smaller, cannot carry conventional motors, batteries or processors. We therefore study how to deliver power and motion from outside the robot body, mainly through magnetic and smart-material actuation. We design magnet arrangements, field sources and control strategies that let a single external system drive, steer and coordinate one or several small robots, including selective control of individual robots sharing the same field. We also explore new locomotion mechanisms, such as momentum-driven and repulsion-based motion, that let small robots move efficiently across rough and changing terrain and produce useful force despite their size.
Many of our designs draw on ideas from nature. By studying how animals and collective systems move and deform, we derive principles for soft, compliant and origami-structured bodies that fold and change shape, fabricated with methods such as origami folding and soft 3D printing. Recent and ongoing work includes bio-inspired swimming robots with folded structures driven by near-field magnetic actuation, and soft capsule-type robots that travel through pipes and tubes.
Because these robots usually operate out of direct view, we combine onboard force sensing, external tracking with magnetic sensor arrays and closed-loop control. This allows miniature robots and steerable magnetic catheters to work reliably in confined spaces, such as inside the human body for minimally invasive procedures, and inside pipes.
Key technologies
- Magnetic actuation and field control for untethered small robots
- Smart-material actuation for soft and shape-changing bodies
- Bio-inspired locomotion mechanisms for rough and changing terrain
- Soft, compliant and origami-structured robot bodies
- Magnetic tracking with sensor arrays, and embedded force sensing
- Closed-loop control of single and multiple miniature robots and catheters
Representative work
- “Magnetically actuated momentum-driven millirobots” — M. Wang et al. and J. Liu, Nature Communications, 2025. Demonstrated millirobots that use magnetic actuation to generate momentum-driven locomotion.
- “Near-Field Driven Origami-Based Bio-Inspired Jellyfish Robot” — S. Wang et al. and J. Liu, IEEE International Conference on Robotics and Automation (ICRA), 2026 (accepted). Introduced an origami-based, jellyfish-inspired swimming robot driven by near-field magnetic actuation.
Robotic Micromanipulation and Biomedical Vision
Handling single cells, embryos and other micro-scale objects demands precise motion while the scene can only be seen through a microscope. We close the loop between vision and motion through microscope-based visual feedback that tracks cells and tool tips, accurate positioning of micro-tools such as micropipettes, and automated procedure design that turns multi-step manual protocols into reliable robotic sequences. We also explore non-contact manipulation, using acoustic and flow-based effects to arrange particles and to transport and mix micro-samples on chip.
Vision is equally central to our biomedical work. We develop deep learning models that detect, count and segment small, crowded or low-contrast objects in microscopic and medical images, and that remain reliable when images vary between instruments and laboratories. Because medical data are often held by different hospitals and cannot be pooled, we develop federated learning methods that let models learn across institutions without sharing raw data. We also build easy-to-use software so that clinicians and biologists with limited programming experience can apply deep learning to cell classification and analysis.
Applications include cell injection and characterisation, embryo vitrification, reproductive-cell handling such as sperm analysis and selection, medical image analysis, and image guidance for minimally invasive interventions.
Key technologies
- Microscope-based visual feedback and tool-tip localisation
- Precise positioning and automated control of micro-tools
- Automated design of multi-step micromanipulation procedures
- Non-contact (acoustic and flow-based) handling of micro-objects
- Deep learning for detecting, counting and segmenting small objects
- Federated learning across institutions without sharing raw data
Representative work
- “SonicPlex: Simultaneous Arrangement of Massive Particles through a Simple Acoustic Micromanipulation Platform” — J. Zhou et al. and J. Liu, International Conference on Manipulation, Automation and Robotics at Small Scales (MARSS), 2023. Presented a simple acoustic platform for arranging large numbers of particles simultaneously.
- “Feature-aligned cell detection for heterogeneous microscopic images with focal attenuated distance transform” — R. Liu et al. and J. Liu, IEEE Transactions on Automation Science and Engineering, 2026. Introduced a deep learning method that aligns features across heterogeneous microscopic images to detect cells.
Intelligent Perception for Human–Robot Interaction and Manufacturing
For robots to work alongside people and within real production processes, they must understand what is happening around them. We develop image and video understanding methods that recognise, localise and predict human actions and motion, from hand gestures and sign language to whole-body movement, so that robots can anticipate and respond to people. We also study active perception, in which a service robot decides where to look and remembers context to find objects over long periods.
Alongside vision, we develop wearable and flexible sensors that capture touch, force and gestures, including where and in which direction contact occurs, as well as non-contact interfaces that sense a nearby hand before any touch. Such sensors let robots respond to changes in their surroundings in real time, which is essential for safe, intuitive interaction, for example by sensing human proximity to avoid collisions.
In manufacturing, AI offers new approaches to design, measurement, analysis and prediction. We develop point-cloud and vision models that infer geometry and quality from measurement data, such as predicting the assembly accuracy of rotating components or detecting loosened bolts from camera images, and combine them with planning of smooth, time-efficient and accurate robot motion for tasks such as robotic milling. These technologies support human–robot collaboration, service robots, precision assembly, robotic machining and system diagnosis.
Key technologies
- Video understanding for human action recognition, detection and prediction
- Active perception with memory for long-term object search
- Wearable, flexible and non-contact sensors for touch, force and gestures
- Point-cloud deep learning for geometry and quality prediction
- Vision-based measurement and inspection
- Smooth, time-optimal robot motion and path planning
Representative work
- “Information-Bottleneck Guided Hybrid Neural Architecture Search for Temporal Action Detection in Untrimmed Videos” — Y. Tang et al. and J. Liu, IEEE Transactions on Image Processing, 2026. Showed how an information-bottleneck-guided architecture search can design hybrid networks that detect when actions occur in long, untrimmed videos.
- “Liquid Metal-Based Flexible Sensor for Perception of Force Magnitude, Location, and Contacting Orientation” — M. Wang et al. and J. Liu, IEEE Transactions on Instrumentation & Measurement, 2023. Developed a flexible sensor that perceives the magnitude, location and orientation of contact forces.
