Researchers created a robotic sensor that uses artificial intelligence algorithms to read braille at a rate about twice that of most human readers. The University of Cambridge researchers utilized machine learning algorithms to train a robotic sensor to quickly slide across lines of braille text. The robot was able to read the braille at 315 words per minute with about 90% accuracy.
Although the robot braille reader was not designed as an assistive technology, the researchers believe the great sensitivity necessary to read braille makes it an appropriate test for the creation of robot hands or prosthetics with human-like sensitivity. The findings are published in the journal IEEE Robotics and Automation Letters.
Human fingertips are extraordinarily sensitive, allowing us to absorb information about our surroundings. Our fingertips can sense minute changes in the texture of a substance or tell us how much force to apply while grabbing an object, such as picking up an egg without shattering it or a bowling ball without losing it.
The softness of human fingertips is one of the reasons we’re able to grip things with the right amount of pressure. For robotics, softness is a useful characteristic, but you also need lots of sensor information, and it’s tricky to have both at once, especially when dealing with flexible or deformable surfaces.
Parth Potdar
Reproducing that level of sensitivity in a robotic hand while remaining energy efficient is a significant engineering problem. In Professor Fumiya Iida’s lab at Cambridge’s Department of Engineering, researchers are working on answers to these and other talents that humans find simple but tough for robots.
“The softness of human fingertips is one of the reasons we’re able to grip things with the right amount of pressure,” said Parth Potdar from Cambridge’s Department of Engineering and an undergraduate at Pembroke College, the paper’s first author. “For robotics, softness is a useful characteristic, but you also need lots of sensor information, and it’s tricky to have both at once, especially when dealing with flexible or deformable surfaces.”
Braille is an ideal test for a robot ‘fingertip’ as reading it requires high sensitivity since the dots in each representative letter pattern are so close together. The researchers used an off-the-shelf sensor to develop a robotic braille reader that more accurately replicates human reading behavior.
“There are existing robotic braille readers, but they only read one letter at a time, which is not how humans read,” said co-author David Hardman, also from the Department of Engineering. “Existing robotic braille readers work in a static way: they touch one letter pattern, read it, pull up from the surface, move over, lower onto the next letter pattern, and so on. We want something that’s more realistic and far more efficient.”
The researchers utilized a robotic sensor with a camera in its ‘fingertip’ that reads utilizing information from both the camera and sensors. “This is a hard problem for roboticists as there’s a lot of image processing that needs to be done to remove motion blur, which is time and energy-consuming,” says Potdar.
The team created machine learning techniques so that the robotic reader could ‘deblur’ the images before the sensor attempted to recognize the letters. They trained the system on a batch of sharp Braille images with a false blur. After the algorithm learnt how to deblur the letters, they utilized a computer vision model to recognize and classify each character.
Once the algorithms were incorporated, the researchers tested their reader by sliding it quickly along rows of braille characters. The robotic braille reader could read at 315 words per minute at 87% accuracy, which is twice as fast and about as accurate as a human Braille reader.
“Considering that we used fake blur the train the algorithm, it was surprising how accurate it was at reading braille,” Hardman added. “We found a nice trade-off between speed and accuracy, which is also the case with human readers.”
“Braille reading speed is a great way to measure the dynamic performance of tactile sensing systems, so our findings could be applicable beyond braille, for applications like detecting surface textures or slippage in robotic manipulation,” Potdar went on to say.
In the future, researchers hope to scale the technology to the size of a humanoid hand or skin. The Samsung Global Research Outreach Program provided some funding for this study.
