Open access
Date
2011Type
- Report
ETH Bibliography
yes
Altmetrics
Abstract
We present a novel framework for the multimodal display of words using topological, appearance, and auditory representations. The methods are designed for effective language training and serve as a learning aid for individuals with dyslexia. Our topological code decomposes the word into its syllables and displays it graphically as a tree structure. The appearance code assigns color attributes and shape primitives to each letter and takes into account conditional symbol probabilities, code ambiguities, and phonologically confusable letter combinations. An additional auditory code assigns midi events to each symbol and thus generates a melody for each input string. The entire framework is based on information theory and utilizes a Markovian language model derived from linguistic analysis of language corpora for English, French, and German. For effective word repetition a selection controller adapts to the user’s state and optimizes the learning process by minimizing error entropy. The performance of the method was evaluated in a large scale experimental study involving 80 dyslexic and non-dyslexic children. The results show significant improvements in writing skills in both groups after small amounts of daily training. Our approach combines findings from 3D computer graphics, visualization, linguistics, perception, psychology, and information theory. Show more
Permanent link
https://doi.org/10.3929/ethz-a-006809497Publication status
publishedJournal / series
Technical Report / ETH Zurich, Department of Computer ScienceVolume
Publisher
ETH, Department of Computer ScienceSubject
Coding; Learning; Information Theory; Visualization; Dyslexia; LANGUAGE INSTRUCTION (SPECIAL SUBJECTS); MULTIMEDIA (INFORMATION SYSTEMS); Visual Information; Natural Language; COMPUTERUNTERSTÜTZTER UNTERRICHT + NEW MEDIA (UNTERRICHT); MULTIMEDIA (INFORMATIONSSYSTEME); SPRACHUNTERRICHT (FACHUNTERRICHT); COMPUTER-AIDED INSTRUCTION + NEW MEDIA (INSTRUCTION); EntropyOrganisational unit
02150 - Dep. Informatik / Dep. of Computer Science
Notes
Technical Reports D-INFK.More
Show all metadata
ETH Bibliography
yes
Altmetrics