Multimodal Intelligent inteRactIon for Autonomous systeMs

Welcome to the MIRIAM project page.

MIRIAM enables operators to interrogate the AUV's actions and status in real time through natural language chat.
Natural language is intuitive and requires no memorisation of commands or traditional training.

MIRIAM builds off REGIME for report generation in-mission and post mission.


  • Collaborative decision-making where humans and machines are effective parts of the same team
  • Increase transparency between operator and autonomous system
  • Increase situation awareness
  • Reduce information overload

Related Publications

  • Helen Hastie, Francisco J. Chiyah Garcia, David A. Robb, Pedro Patron and Atanas Laskov (2017). MIRIAM: A Multimodal Chat-based Interface for Autonomous Systems. In Proceedings of ICMI. Glasgow, UK [link]
  • Helen Hastie, Xingkun Liu and Pedro Patron (2017). Trust Triggers for Multimodal Command and Control Interfaces. In Proceedings of ICMI, Glasgow, UK [link]
  • Helen Hastie, Xingkun Liu, Pedro Patron and Yvan Petillot (2017). AUVs That Talk: Automatic Natural Language Generation of Mission Updates. In Proceedings of OCEANS'17 MTS/IEEE, Aberdeen, UK
  • Helen Hastie, Xingkun Liu and Pedro Patron "A Demonstration of Multimodal Debrief Generation for AUVs, Post-mission and In-mission". In Proceedings of ICMI. Tokyo, Japan [to appear, 2016]
  • Zhuoran Wang and Helen Hastie (2015). A Prototype for AUV Post-mission Debrief Generation from Metadata (Demonstration). In Proceedings of AAMAS [pdf]
  • Helen Hastie (2015). Challenges of Military Applications for Data-to-Text Generation. In Proceedings of the 1st International workshop on Data-to-text Generation (D2T). Edinburgh, UK (non-refereed). [pdf]


This work is funded under under the Dstl Defence and Security Accelerator Programme for revolutionise the human information relationship for Defence (ACC101939, 2017); and under the Dstl Autonomous Systems Underpinning Research (ASUR) programme (0913_C2_Ph1 009/0913_CH2_PH2_009, 2014-2016); and the RAEng/Leverhulme Trust Senior Research Fellowship Scheme (Hastie/LTSRF1617/13/37, 2017-2018).


Do not hesitate to contact us if you would like more information or are interested in the project.
We also provide a login to try out the simulations.

Contact: Prof. Helen Hastie
Email address: h.hastie@hw.ac.uk

Visit the INCRA lab page for more information about other related projects.