Soon, computers will read handwriting

When you type something, it's clear to you and your computer -- but until now, handwriting has been about as easy for a computer to understand as it is for a human to read a document written in the Wingdings font.

Handwriting recognition science is still in its relative infancy.  It's a big field, and one with a lot of commercial potential. Researchers worldwide are working away at getting machines that can understand handwriting recognition, including many Australian scientists.

The development of this particular science is the subject of a new book by Associate Professor Brijesh Verma of CQU University and Dr Michael Blumenstein of Griffith University, "Pattern Recognition Technologies and Applications: Recent Advances". With a title like that, we feel it's probably not going to worry the Harry Potters of this world in terms of sales numbers, but interesting stuff from an IT perspective nonetheless.

APC spoke to Dr Blumenstein about the state of play in handwriting recognition, and where the future might lie. You might think that, given most of the handwriting recognition on objects like Tablet PCs and PDAs is pretty accurate that this is a refined area of science, but that isn't so, according to Blumenstein. "Handwriting recognition on Tablet PC and PDAs -- that type is quite easy in one sense -- because you do train it. It also takes all these easy to capture things like speed of handwriting, and even the direction you're entering strokes into in terms of reading your input."

"What we're after is the ability to recognise unconstrained handwriting from any source -- it could be from hundreds of years ago -- and make machines able to read it", he said.

Blumenstein's work has to date focused on the Roman alphabet, working in English "because of the simplicity from our perspective." It's interesting to note, however, that the technology doesn't seem to be limited to English only. "We recently had a visitor to our labs from India, and we were able to get some Hindi writing samples from him. Our approach with Hindi at this stage isn't too bad -- we were getting around 85% accuracy."

The most computationally intensive part of getting machines to read handwriting doesn't come from the individual documents, according to Blumenstein, but from the initial training. "We do use training, similar to the way that PDAs and Tablets do, but ours is more universal", he said. "We've used everything from simple PCs to Supercomputers to do the actual data set crunching, but once but once it's trained with a large data set, the actual character recognition is quite fast."

One of the challenges that even ordinary people struggle with is in trusting what it is that they're reading is being correctly interpreted, and Blumenstein feels that this level of trust is even more important when it comes to machine reading. "When you look at the type of applications we're targeting -- from reading postal addresses to signature verification, if you read them incorrectly it could cost you millions of dollars."

"The system we're working on, if it can't recognise something with a very good level of confidence, then it will automatically reject that item so it can be read by a human. Any system that can reject on its own is a much better system, if there is any doubt."

"At the moment our character recogniser accuracy is above 90%. Even humans, if they don't see the context that characters are in, can typically only manage about 85%. For unconstrained words we'd be ideally looking at about 90 percent, maybe 95%. It sounds like a copout to say that the system can reject things it can't read, but in terms of trusting the system, it's quite vital. "

So how far away from having that kind of technology are we? Blumenstein told APC that "There's been a lot of good advances. We've had excellent results in handwriting recognition for single characters. When it comes to unconstrained cursive, there's a long way to go."