A Tianmouc chip (left) and a perception system (right). Tsinghua University
Scientists from Tsinghua University in China have created the world’s first brain-inspired vision chip, which is capable of sensing 10,000 frames per second.
Called Tianmouc, the “world’s fastest vision chip” not only overcomes the performance bottlenecks of traditional visual sensing paradigms but also efficiently handles various extreme scenarios, ensuring system stability and safety.
The Center for Brain-Inspired Computing Research (CBICR) at Tsinghua University was behind the development of the chip, which brings human eye-like perception to machines. The chip reduces bandwidth by 90%.
Team has developed high-performance software
Researchers introduced a complementary sensing paradigm inspired by the human visual system. This paradigm involves breaking down visual information into basic representations and then combining these to create two distinct pathways: one for precise cognition and another for rapid response.
To realize this paradigm, they developed Tianmouc, incorporating a hybrid pixel array and a parallel-and-heterogeneous readout architecture.
Leveraging the Tianmouc chip, the team has developed high-performance software and algorithms and validated their performance on a vehicle-mounted perception platform operating in open environments.
In various extreme scenarios, the system demonstrated low-latency, high-performance real-time perception, showcasing its immense potential for applications in the field of intelligent unmanned systems.
Significant breakthrough
Scientists have called it a significant breakthrough in the field of visual sensing chips. It not only provides strong technological support for the advancement of intelligent revolution but also opens new avenues for crucial applications such as autonomous driving and embodied intelligence, according to researchers.
Combined with CBICR’s established technological foundation in brain-inspired computing chips like “Tianjic,” toolchains, and brain-inspired robotics, the addition of Tianmouc will further enhance the brain-inspired intelligence ecosystem, accelerating the progress of artificial general intelligence.
“We demonstrate the integration of a Tianmouc chip into an autonomous driving system, showcasing its abilities to enable accurate, fast and robust perception, even in challenging corner cases on open roads,” said researchers.
The primitive-based complementary sensing paradigm helps overcome fundamental limitations in developing vision systems for diverse real-world applications.
“This is a perception chip, not a computational one, based on our original technical route,” said project leader Shi Luping, a professor with the university’s Centre for Brain-Inspired Computing Research, according to SCMP.
“Firstly, it balances speed and dynamic performance in vision chips and introduces a novel computational method that diverges from existing machine vision strategies. Secondly, this approach mimics the human visual system’s dual pathway, enabling decision-making without complete clarity.”
Researchers claimed that the chip would open new doors for advancements in autonomous driving and defense sectors and could lead to the development of various novel applications.
RECOMMENDED ARTICLES
Their research paper is published in the journal Nature on May 30.
0COMMENT
NEWSLETTER
The Blueprint Daily
Stay up-to-date on engineering, tech, space, and science news with The Blueprint.
By clicking sign up, you confirm that you accept this site's Terms of Use and Privacy Policy
ABOUT THE EDITOR
Prabhat Ranjan Mishra Prabhat, an alumnus of the Indian Institute of Mass Communication, is a tech and defense journalist. While he enjoys writing on modern weapons and emerging tech, he has also reported on global politics and business. He has been previously associated with well-known media houses, including the International Business Times (Singapore Edition) and ANI.