The contributors in this collection question what kinds of relationships hold between narrative studies and the recently established field of multimodality, evaluate how we might develop an analytical vocabulary which recognises that stories do not consist of words alone, and demonstrate the ways in which multimodality brings into fresh focus the embodied nature of narrative production and processing. Engaging with a spectrum of multimodal storytelling, from ‘low tech’ examples encompassing face-to-face stories, comic books, printed literature, through to opera, film adaptation and television documentary, stretching beyond to narratives that employ new media such as hypertext, performance art, and interactive museum guides, this volume examines the interplay of semiotic codes (visual, oral, aural, haptic, physiological) within each case under scrutiny, thereby exposing both points of commonality and difference in the range of multimodal narrative experiences.
Polymorphic fictions present a unique opportunity to stretch current notions of the relations between media, narrative and game, among other factors. At present, though, theories that directly describe these practices are attending to parts of the multimedial aspect, and not the multimodal. The semiotic theory of multimodality, on the other hand, offers strategic methodological approaches to recognise and analyse polymorphic fictions. This chapter discusses the contributions multimodality makes to the study of polymorphic fictions and what both of these contribute to narrative, game and media studies.The following are additions to the printed chapter. While there are some recapitulations offered here for the sake of clarity, most of the ideas discussed here are not versions but expansions of the main essay in the book.
Nearly forty years later, in a period of swelling interest in both comparative media studies and narrative […], the question of how the intrinsic properties of the medium shape the form of narrative and affect the narrative experience can no longer be ignored. (Ryan 2004: 1)Research questions within transmedial narratology are various, but tend to debate the status of narrative in relation to medium-specificity and medium-independence. In transmedial narrative studies, there are multiple media considered, but narrative is the only mode under investigation.
A game is a rule-based system with a variable and quantifiable outcome, where different outcomes are assigned different values, the player exerts effort in order to influence the outcome, the player feels emotionally attached to the outcome, and the consequences of the activity are negotiable. (Juul 2005: 36)However, like transmedial narrative, the area is positioned to understand the nature of relationships between media and content, a single mode (game), and does not appear to be (at this stage) concerned with combinations of media and modes.
[I]ntertextuality is a variant of ‘intramediality’ and refers exclusively to ‘homomedial’ relations between verbal texts or text systems. Intermediality, in contrast, applies in its broadest sense to any transgression of boundaries between media and thus is concerned with ‘heteromedial’ relations between different semiotic complexes or between different parts of a semiotic complex. (Wolf 2005: 252)Intermediality is a broad area, including studies of narrative in different medial systems, but predominately the co-presence of different media systems in different works. In intermediality, the remediation of a medium in another is not only recognised (unlike intertextuality), but is studied as a rhetorical strategy. Continuing the key insight of transtextuality, however, it is predominately concerned with different-creator relations. In intermediality, all medial representations within a work are semiotic. Although the area is by definition concerned with heteromedial relations, due to the large range of interdisciplinary interest in the area there is the possibility of the interrogation of heteromodal relations as well (since most of the writings in this area are in languages I cannot read, this may already be the case).
Using other media as starting points, we may learn many things about the construction of fictive worlds, characters ... but relying too heavily on existing theories will make us forget what makes games games: Such as rules, goals, player activity, the projection of the player’s actions into the game world, the way the game defines the possible actions of the player. (Juul 2001)Juul interrogated this idea through the concept of translation, and pondered ‘the problem of what we actually mean by saying that something can be translated from one medium to another’ (ibid.). To illuminate the point, Juul offered a table of ‘narrative - game translations’, with examples such as the translation of a character in a cut-scene (non-interactive video) to player position in a game (ibid.). In 2005, game scholar Markku Eskelinen continued this exploration of intermodal relations (Eskelinen 2005). Like Juul, Eskelinen argues that what is missing from content-media studies is an understanding that ‘when bits and pieces of content move across »media« they change context, function and position which may affect and usually also affects their modal status’ (ibid.). To Eskelinen, ‘transmedial storytelling’ and ‘transmedial gaming’ are modal ecologies. Eskelinen also offers a chart, and explains that transformations can occur within a mode (intramodal), between modes (intermodal), and can either be homomodal (retaining their modal status in transformation) or heteromodal (elements changing their modal status in transformation). Examples of intermodal transformations, Eskelinen offers, between story/narrative elements and game elements are the shift from an episode to a cut-scene (homomodal), or a film character to a player-character (heteromodal). It should be noted that this intermodal interrogation was not the main concern of Eskelinen’s essay, but a necessary precursor to his exploration of an ‘inter- or transludic networks in game studies’ (ibid.).
Created: 31 May, 2008
Last Updated: 31 July, 2008
Author: Christy Dena