Online Augmentation to
"The Contributions of Multimodality and Polymorphic Fictions:
Beyond Narrative and Games Studies"
by Christy Dena


This minisite is an online augmentation of a chapter published in the special volume edited by Ruth Page: New Perspectives on Narrative and Multimodality (Routledge, 2009). The abstract of the publication is:
The contributors in this collection question what kinds of relationships hold between narrative studies and the recently established field of multimodality, evaluate how we might develop an analytical vocabulary which recognises that stories do not consist of words alone, and demonstrate the ways in which multimodality brings into fresh focus the embodied nature of narrative production and processing. Engaging with a spectrum of multimodal storytelling, from ‘low tech’ examples encompassing face-to-face stories, comic books, printed literature, through to opera, film adaptation and television documentary, stretching beyond to narratives that employ new media such as hypertext, performance art, and interactive museum guides, this volume examines the interplay of semiotic codes (visual, oral, aural, haptic, physiological) within each case under scrutiny, thereby exposing both points of commonality and difference in the range of multimodal narrative experiences.

The abstract of my chapter, titled 'The Contributions of Multimodality and Polymorphic Fictions: Beyond Narrative and Games Studies':


Polymorphic fictions present a unique opportunity to stretch current notions of the relations between media, narrative and game, among other factors. At present, though, theories that directly describe these practices are attending to parts of the multimedial aspect, and not the multimodal. The semiotic theory of multimodality, on the other hand, offers strategic methodological approaches to recognise and analyse polymorphic fictions. This chapter discusses the contributions multimodality makes to the study of polymorphic fictions and what both of these contribute to narrative, game and media studies.
The following are additions to the printed chapter. While there are some recapitulations offered here for the sake of clarity, most of the ideas discussed here are not versions but expansions of the main essay in the book.


The remit of both narrative and games studies is the nature of their own modal characteristics. They are therefore methodologically structured to be monomodal inquiries. Both narrative and game studies also share similar concerns when it comes to medial elements. There are, however, glimpses of attempts to reach beyond. Of the many possible inquiries to review here, the following theories are juxtaposed here for their representativeness and relevance: transmedial narrative and transmedial game; intermediality and intermodal relations.

Transmedial Narrative and Transmedial Game

Transmedial Narrative

Variously described as ‘transmedial narratology’, ‘transmedial narrative’ and ‘narrative media studies’, the recent burgeoning of research in this area focuses on media…but it concentrates on what the consideration of media can tell us about the nature of narrative. The spirit of transmedial narratology, David Herman explains, is what the structuralists championed but were unable to achieve: ‘the study of narratives of all sorts, irrespective of origin, medium, theme, reputation, or genre’ (Herman 2004: PAGE). But, as Ryan notes, the relationship between media and narrative has only recently been addressed with rigor:
Nearly forty years later, in a period of swelling interest in both comparative media studies and narrative […], the question of how the intrinsic properties of the medium shape the form of narrative and affect the narrative experience can no longer be ignored. (Ryan 2004: 1)
Research questions within transmedial narratology are various, but tend to debate the status of narrative in relation to medium-specificity and medium-independence. In transmedial narrative studies, there are multiple media considered, but narrative is the only mode under investigation.

Transmedial Game

Game studies emerged with force in response to (among other things) the interest in understanding digital games, how games are different to narrative, and how digital games differ to non-digital games. Games have for a long time been considered transmedial in that they have traits that are non-medium specific. For the sake of pith, consider Juul’s transmedial definition of games:
A game is a rule-based system with a variable and quantifiable outcome, where different outcomes are assigned different values, the player exerts effort in order to influence the outcome, the player feels emotionally attached to the outcome, and the consequences of the activity are negotiable. (Juul 2005: 36)
However, like transmedial narrative, the area is positioned to understand the nature of relationships between media and content, a single mode (game), and does not appear to be (at this stage) concerned with combinations of media and modes.

Intermedial & Intermodal Relations


In 1982, Gérard Genette’s book on ‘transtextuality’ was published (Genette 1997 [1982]). Although there are a small selection of what Genette describes as auto-hyperartistic practices (same-creator, different artform relations), the predominant concern of the book are co-presences between (literary) texts by different authors. One year after Genette’s book, Aage A. Hansen-Löve (Hansen-Löve 1983) introduced ‘intermediality’ to augment transtextuality with a theory that would ‘capture relations between literature and the visual arts’ (Wolf 2005: 252). The area has since, however, developed to encompass all so-called ‘heteromedial relations’:
[I]ntertextuality is a variant of ‘intramediality’ and refers exclusively to ‘homomedial’ relations between verbal texts or text systems. Intermediality, in contrast, applies in its broadest sense to any transgression of boundaries between media and thus is concerned with ‘heteromedial’ relations between different semiotic complexes or between different parts of a semiotic complex. (Wolf 2005: 252)
Intermediality is a broad area, including studies of narrative in different medial systems, but predominately the co-presence of different media systems in different works. In intermediality, the remediation of a medium in another is not only recognised (unlike intertextuality), but is studied as a rhetorical strategy. Continuing the key insight of transtextuality, however, it is predominately concerned with different-creator relations. In intermediality, all medial representations within a work are semiotic. Although the area is by definition concerned with heteromedial relations, due to the large range of interdisciplinary interest in the area there is the possibility of the interrogation of heteromodal relations as well (since most of the writings in this area are in languages I cannot read, this may already be the case).

Intermodal Relations

It is in games studies, an area that has worked quite hard to explain that a game mode is a mode distinct from narrative, that intermodal relations have been provisionally explored.

In 2001, game scholar Jesper Juul argued that it is important to understand the difference between games and narrative:
Using other media as starting points, we may learn many things about the construction of fictive worlds, characters ... but relying too heavily on existing theories will make us forget what makes games games: Such as rules, goals, player activity, the projection of the player’s actions into the game world, the way the game defines the possible actions of the player. (Juul 2001)
Juul interrogated this idea through the concept of translation, and pondered ‘the problem of what we actually mean by saying that something can be translated from one medium to another’ (ibid.). To illuminate the point, Juul offered a table of ‘narrative - game translations’, with examples such as the translation of a character in a cut-scene (non-interactive video) to player position in a game (ibid.). In 2005, game scholar Markku Eskelinen continued this exploration of intermodal relations (Eskelinen 2005). Like Juul, Eskelinen argues that what is missing from content-media studies is an understanding that ‘when bits and pieces of content move across »media« they change context, function and position which may affect and usually also affects their modal status’ (ibid.). To Eskelinen, ‘transmedial storytelling’ and ‘transmedial gaming’ are modal ecologies. Eskelinen also offers a chart, and explains that transformations can occur within a mode (intramodal), between modes (intermodal), and can either be homomodal (retaining their modal status in transformation) or heteromodal (elements changing their modal status in transformation). Examples of intermodal transformations, Eskelinen offers, between story/narrative elements and game elements are the shift from an episode to a cut-scene (homomodal), or a film character to a player-character (heteromodal). It should be noted that this intermodal interrogation was not the main concern of Eskelinen’s essay, but a necessary precursor to his exploration of an ‘inter- or transludic networks in game studies’ (ibid.).

In summary, mediality has obviously been a primary concern of theories that consider the relations between media and content. Game scholars, however, in their attempt to distinguish game as a mode, have started exploring intermodal transformations. Combinations of media and modes for meaning has not been a concern for either fields as yet. This is perhaps one of the reasons why the combined medial and modal characteristics of polymorphic fictions have hitherto been unrecognised. The semiotic theory of multimodality offers a way forward.

In light of these variables, Genette’s ‘transtextuality’ can be described as the study of allographic homomedial homomodal extracompositional relations (Genette 1997 [1982]). One year after Genette’s book, Aage A. Hansen-Löve (Hansen-Löve 1983) introduced ‘intermediality’ to augment transtextuality with a theory that would ‘capture relations between literature and the visual arts’ (Wolf 2005: 252). The area has since, however, developed to encompass all so-called ‘heteromedial relations’:

Intermediality is a broad area, but each study appears to be concerned with homomodal, heteromedial, (auto) intra or (auto or allo) extracompositional relations. Modal Variations & Inquiries:
Transmodal: elements that can be realised in different modes (Kress and van Leeuwen)
Multimodal: combinations of modes
Intramodal: "within a mode" (Eskelinen)
Intermodal: "between modes" (Eskelinen)
Homomodal: "elements retaining their modal status in transformation" (Eskelinen)
Heteromodal: "elements changing their modal status in transformation" (Eskelinen)

Transmedial: elements (modes) can be realised in different media (Kress and van Leeuwen)
Multimedial: combinations of media
Intramedial: within a media
Intermedial: between media
Homomedial: elements retain their medial status in transformation (after Eskelinen)
Heteromedial: elements change their medial status in transformation (after Eskelinen)



Created: 31 May, 2008
Last Updated: 31 July, 2008
Author: Christy Dena