A supermarket trial run of our ShopTalk system at Lee's Market Place in Logan, Utah. ShopTalk is a proof-of-concept
wearable system designed to assist visually impaired shoppers with finding shelved
products in supermarkets. The system is based on a simple conceptual formula:
Independent Blind Shopping = Verbal Route Directions + Shelf Barcode Scans.
This individual is a cane user. He has selected a product, and is now
following a set of verbal instructions automatically generated by the system
from the topological map of the supermarket. The the first leg of the route
starts at the store's lobby and ends at the aisle where the selected
product resides.
When the shopper scans any shelf barcode, the system begins to use its
database, called the barcode connectivity matrix (BCM). The database
associates each shelf barcode with an aisle number, the side of the aisle,
a shelf section number, a shelf number in that shelf section, and a relative
position on the shelf.
The system's hardware consists of a computational unit, an OQO model 01, a Belkin
numeric keypad, a wireless barcode scanner and its base station, and a USB hub that
connects all components. The user wears a small CamelBak backpack to carry the
equipment. The equipment is very light. The numeric keypad is attached by a velcro
strip to one of the backpack's shoulder straps.
ShopTalk does not handle product identification so it is possible that a shopper
could be at the correct location for a target product and yet
retrieve an incorrect product. We have started to address vision-based product
identification in ShopMobile, the mobile version of ShopTalk that runs on
an Android phone. A video of ShopMobile is available at
http://www.youtube.com/watch?v=LW_mEVlFpmg&feature=channel
What was quite surprising and interesting to us in these trial runs is that lack
of product identification did not turn out to be a major obstacle to visually
impaired shoppers. Well-managed stores, such as Lee's Market Place, maintain
their shelves in order because their business depends on it, and employees
periodically inspect the shelves to remove misplaced items and restock the
sold items. In time, the shopper learns valid audio, touch, and smell cues to
identify correct products. For example, a peanut butter jar can be easily
distinguished from a can of corn by touch. In the case of identical containers,
shaking or verification at checkout could be used to resolve product questions.
The system is designed for mobile individuals who have sufficient O&M skills to
navigate indoor environments independently and have no other impairments that
could potentially impede navigation, such as serious cognitive or physical
disabilities. ShopTalk is not designed to address physical limitations that may
prevent individuals from grocery shopping, such as not being able to reach high
or low shelves or being unable to carry heavy items. The level of visual impairment
is not a factor in that ShopTalk can be used by people with complete or partial
vision loss.
We were following each shopper with the Lufkin wheel, a distance measuring device.
We used this wheel to systematically estimate the distance walked by each participant.
We noticed that the more the participants used the system the smaller was the
total distance they walked in the store.
A key finding was that verbal route directions were sufficient for our
sample of independent travelers to navigate this supermarket reliably and
retrieve various products. The more they used the system, the less they
requested verbal route directions. As soon they heard the aisle of a
selected product, they would just go there independently.
If this finding generalizes to the entire population of independent
visually impaired travelers, it suggests that verbal route directions
may well be sufficient. The important point of ShopTalk is that the
store may not need to be instrumented with any external sensors, such
as RFID tags, Wi-Fi routers, IR transmitters, etc.
We would like to take this opportunity to thank the late Mr. Lee Badger,
the founder of Lee's Market Place. We met with Mr. Badger in early 2004
when this system was just bunch of drawings. He grasped everyting right
away and gave us permission to use his store in Logan, Utah, for our
field experiments. Without Mr. Badger's support ShopTalk could have never
become what you see it to be in this video.
For more information go to http://digital.cs.usu.edu/~vkulyukin/vkweb/research/accessible_shopping.html
Video Narration: Vladimir Kulyukin
