Accessible Document Layout: An Interface for 2D Tactile Displays

Reading and processing documents is a challenging task for people with blindness and visual impairment (BVI). Despite various methods and research being conducted on converting text and multimedia content to an accessible format, the task of layout navigation remains under-explored. Within a document, a demanding task for people with BVI is to understand the layout and navigate, so as to form a proper reading order. This is specifically challenging in documents with complex layouts, such as those with multiple columns and arbitrarily distributed elements. Traditional methods, such as screen readers, can be limited in their ability to navigate complex layouts and unreliable for various types of documents such as newspapers and slides. A new tactile layout reader has been developed in this work to enhance document navigation by providing pinpointed audio-tactile explanations. Our approach, as shown in Figure 1, uses state-of-the-art AI object detection to create a high-level abstract of the document structure, and an optimized interface suitable for both 2D refreshable displays and braille-embossed documents, with both audio and tactile representations. In order to enable the research community and contribute to the field, we will make our codes, models, and annotated data publicly available.