Information processing and semantic correlation in multimodal, multilingual, human computer interfaces

One long-term goal in human computer interaction (HCI) study has been to migrate the "natural" means that humans employ to communicate with each other into the HCI system. Any HCI system that aspires to have the same naturalness as that of an interaction between humans should be multimodal, thus requiring a need for developing a multimodal HCI system. Multilingual collaboration through a multimodal human computer interaction (HCI) interface needs to handle differences among the multiple human languages used by the collaborators. The difficulties of handling the multilingual information are partially caused by the linguistic differences in grammar, vocabulary, idiom, expression, word-formation, and the existence of unique homonym, polysemy, and homophone in each human language. Due to the difficulties, a semantic extraction and correlation process among the multilingual information becomes a further difficult task in a multimodal HCI system. In this dissertation, a deterministic information processing framework for multilingual collaboration in the context of geo-spatial collaborative crisis management is proposed. The deterministic information processing framework comprises an information unit augmentation step, called object verification, and operating mechanism for the augmented information unit, called polymorphic operator, as a part of the proposed approach to process the multilingual information. In the subject of the multilingualism, the deterministic information modeling approach is an attempt to reduce the complexity in handling the multilingual information by replacing the language modeling process with the information modeling approach. In this dissertation, the term "rule" is defined as an agreement or guideline based on the intention for achieving a goal among the users of a particular embodiment of the approach in a specifically predefined domain for defining a set of relationships among the verified information units that are used in the specifically predefined domain. It is an objective of the proposed approach to see whether the deterministic information processing approach can efficiently handle the multilingual collaboration and semantic correlation with feasible computational complexity and processing time through algorithm analysis and discussion of exemplary experimentation. The proposed approach is concerned with the reduced corpus of the multilingual information in the specified domain. The proposed approach is also restricted to the multilingual collaboration and understanding in the medium of a multimodal HCI system.