Skip to main content

MAMEM - Multimedia Authoring and Management using your Eyes and Mind

Loss of the voluntary muscular control while preserving cognitive functions is a common symptom of neuromuscular diseases leading to a variety of functional deficits, including the ability to operate software tools that require the use of conventional interfaces like mouse, key-board, or touch-screens.  As a result, the affected individuals are marginalized and unable to keep up with the rest of the society in a digitized world.

MAMEM's goal is to integrade these people back into society by increasing their potential for communication and exchangein leisure (e.g. social networks) and non leisure context (e.g. workplace).  In this direction MAMEM delivers the technology to enable interface channels that can be controlled through eye-movements and mental commands.  This is accomplished by extending the core API of current operating systems with advanced functions calls, appropriate for accessing the signals captured by an eye-tracker, an EEG-recorder and bio-measurement sensors.  Then, pattern recognition and tracking algorithms are employed to jointly translate these signals into meaningful control and enable a set novel paradigms for multimodal interaction.

These paradigmas will allow for low (e.g. move a mouse), meso (e.g. tick a box) and high-level (e.g. select n-out-of-m items), control interface applications through eyes and mind.  A set of persuasive design princilpes together with profiles modeling the users (dis-)abilities will be also employed for designing adapted interfaces for disabled.  MAMEM will engage three different cohorts of disabled (i.e. parkinson's disease, neuromuscular diseases, and tetraplegia) that will be asked to test a set of prototype applications dealing with multimedia authoring and management.

MAMEM's overaching goal is to integrate people with disabilities back into society by endowing them with the critical skill of managing and authoring multimedia content using novel and more natural interface channels.  These channels will be controlled by eye-movement and mental commands, significantly increasing the potential for communication and exchange in leisure (e.g. social networks) and non-leisure context (e.g. workplace).

MAMEM's final objective is to assess the impact of this technology in making these people more socially integrated by, for instance, becoming more active in sharing content through social networks and communicating with their friends and family.

OBJECTIVES

  • Capture, record and make available at the necessary scale, real-time and accurate information about eye-movements, brain electric signals and bio-measurements.
  • Develop the necessary algorithms for translating this information into meaningful control that will take the form of sematic widgets.
  • Implement a middleware sitting on top of current operating systems so as to make these semantic widgets available as elementary building blocks for implementing multimedia-related interfaces.
  • Model users based on their (dis-)abilities, interaction behavior, emotions and intentions and follow a set of persuasive design principles to developinterfaces that will effectively stimulate users to use them and encourage their behavioral change.
  • Design, implement and evaluate a set of prototype interface applications that rely on MAMEM's middleware to execute the multimedia-related usage scenarios through the user's eyes and mind.
  • Enable the use of prototype interface applications in non-controled environments, such as the home environment of the user.
  • Assess the degree of success in bringing disabled people back to the society as a result of their newly acquired ability to manage and author multimedia content.

 

CONSORTIUM

  1. Centre for Researc and Technology Hellas - Information Technologies Institute (CERTH - ITI) - www.iti.gr
  2. University of Koblenz - Landau (Ukob), WeST Intitute - www.west.uni-koblenz.de
  3. SensoMotoric Instruments GmbH (SMI) - www.smivision.com
  4. The Medical Research Infrastructure Development & Health Services Fund by the Sheba Medical Center (SMC - SHEBA) - www.sheba.co.il
  5. EB Neuro S.p.A. (EBN) - www.ebneuro.biz
  6. 3rd Dept. of Neurology of the Aristotle University of Thessaloniki (AUTH) - www.med.auth.gr
  7. Eindhoven University of Technology (TUe) - www.tue.nl
  8. Muscular Dystrophy Association of Greece (MDA Hellas) - www.mdahellas.gr

 

 

 

 

 

http://www.mamem.eu

Funded by Horizon 2020,
The EU Framework Programme for Research and Innovation

Muscular Dystrophy Association