Lists
Lists
Covaci, Alexandra, B.Saleme, Estevao, Mesfin, Gebremariam, Hussain, Nadia, Kani-Zabihi, Elahe 
ORCID: https://orcid.org/0000-0002-5679-8512 and Ghinea, Gheorghita
(2019)
How do we experience crossmodal correspondent mulsemedia content?
IEEE Transactions on Multimedia, 22 (5).
pp. 1249-1258.
ISSN 1520-9210
Preview |
PDF
Kani-Zabihi_etal_IEEE_ToM_2019_How_do_we_experience_crossmodal_correspondent_mulsemedia_content.pdf - Accepted Version Download (10MB) | Preview |
Abstract
Sensory studies emerged as a significant influence upon Human Computer Interaction and traditional multimedia. Mulsemedia is an area that extends multimedia addressing issues of multisensorial response through the combination of at least three media, typically a non-traditional media with traditional audio-video content. In this paper, we explore the concepts of Quality of Experience and crossmodal correspondences through a case study of different types of mulsemedia setups. The content is designed following principles of crossmodal correspondence between different sensory dimensions and delivered through olfactory, auditory and vibrotactile displays. The Quality of Experience is evaluated through both subjective (questionnaire) and objective means (eye gaze and heart rate). Results show that the auditory experience has an influence on the olfactory sensorial responses and lessens the perception of lingering odor. Heat maps of the eye gazes suggest that the crossmodality between olfactory and visual content leads to an increased visual attention on the factors of the employed crossmodal correspondence (e.g. colour, brightness and shape).
| Item Type: | Article |
|---|---|
| Identifier: | 10.1109/TMM.2019.2941274 |
| Additional Information: | This paper was partially funded by the European Union’s Horizon 2020 Research and Innovation programme under Grant Agreement no. 688503. This study was also part financed by the Coordenacão de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 88881.187844/2018-01. Estêvão B. Saleme also thankfully acknowledges support from the Federal Institute of Espírito Santo. © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. This paper appears in IEEE Transactions on Multimedia. First published on-line, 13 September 2019, in IEEEXplore (https://ieeexplore.ieee.org/Xplore/home.jsp) |
| Keywords: | mulsemedia, QoE, crossmodal correspondence, heart rate, eye gaze |
| Subjects: | Computing > Innovation and user experience > Computing interaction design Computing > Innovation and user experience > Global software Computing > Innovation and user experience > Usability |
| Related URLs: | |
| Depositing User: | Elahe Kani-Zabihi |
| Date Deposited: | 10 Sep 2019 08:18 |
| Last Modified: | 04 Nov 2024 11:52 |
| URI: | https://repository.uwl.ac.uk/id/eprint/6369 |
Downloads
Downloads per month over past year
Actions (login required)
 |
View Item |
