What are arbitrary elements used to represent things Processes ideas or events?

Int J Hum Comput Stud. Author manuscript; available in PMC 2013 Aug 1.

Published in final edited form as:

PMCID: PMC3383821

NIHMSID: NIHMS365926

Abstract

Predicting whether the intended audience will be able to recognize the meaning of an icon or pictograph is not an easy task. Many icon recognition studies have been conducted in the past. However, their findings cannot be generalized to other icons that were not included in the study, which, we argue, is their main limitation. In this paper, we propose a comprehensive taxonomy of icons that is intended to enable the generalization of the findings of recognition studies. To accomplish this, we analyzed a sample of more than eight hundred icons according to three axes: lexical category, semantic category, and representation strategy. Three basic representation strategies were identified: visual similarity; semantic association; and arbitrary convention. These representation strategies are in agreement with the strategies identified in previous taxonomies. However, a greater number of subcategories of these strategies were identified. Our results also indicate that the lexical and semantic attributes of a concept influence the choice of representation strategy.

Keywords: icon, pictograph, pictogram, iconic communication, pictorial communication, taxonomy

1. Introduction

“A picture is worth a thousand words”, they say. This proverb summarizes both the greatest strength and the greatest weakness in iconic or pictorial communication. By iconic communication, we mean communication through the use of pictures instead of words. Rather than words, the lexical units in iconic communication are pictures. When used to suggest specific meanings, these pictures are referred to as icons, symbols, or pictographs, among other terms. In this paper, we chose to use the term pictograph because this is the most commonly used term in healthcare, our field of study. In the background section, we elaborate on the distinctions among them. The association between a picture and its meaning is a many-to-many relationship. That is, the same picture can suggest many different concepts or ideas and, conversely, a single concept or idea can be conveyed through many different pictorial representations. A pictograph, on the other hand, is intended to represent a specific concept or idea with minimal ambiguity, as in verbal communication. Unlike verbal communication, however, iconic communication seldom relies on pre-established codes or conventions.

Languages allow us to communicate with each other in a relatively effortless way because they rely on many types of conventions (e.g., morphological, syntactic, semantic, phonetic). Although dominant, verbal communication is not the only form of communication that we use on a daily basis. The domain of cartography, for instance, is derived from the fact that, in many contexts, spatial relations are more easily conveyed visually than verbally. The same logic applies to the use of charts and diagrams to communicate quantitative information. Given this dichotomy, one can recognize that iconic communication stands on murky territory. Pictographs do not have the same representational advantages of other graphic formats such as maps or diagrams. At the same time, pictographs do not operate at the same level of codification of verbal communication, save a few exceptions (e.g., traffic signs). Nonetheless, pictographs can be found virtually anywhere: museums, hospitals, shopping malls, airports, and computer desktops. Two rationales support their use in these contexts: legibility and universality.

The legibility rationale means pictographs are more robust to changes in scale, reading speed, and distance than text. That is why pictographs are often used in traffic and orientation signs (see Figure 1). The universality rationale assumes that it is possible to convey certain concepts or ideas without the use of any convention. While the legibility rationale has been empirically demonstrated (Babbitt Kline et al., 1990; Huchingson, 1981), the universality rationale has not. In fact, the universality rationale has been discredited from both a conceptual and empirical perspectives.

Examples of pictographs used in traffic and orientation signs.

The only reliable way to decrease communication ambiguity is through the establishment of a code. Indeed it is possible to convey the concept “dog” to people that do not speak the same language by showing them the picture of a dog. However, this strategy only works for a very narrow class of concepts. Namely, for those concepts related to concrete entities. It will not work reliably for concepts related to conceptual entities (e.g., envy, sloth) or events (e.g., growing up, allergy) and it certainly would not work reliably at the sentence level of discourse. For a more in-depth discussion about the constraints of iconic communication, see King (2000).

From an empirical perspective, one can quote any of the several pictograph recognition studies that have been conducted in the past. In the specific field of healthcare, studies such as the ones conducted by Dowse and Ehlers (2004), Houts. et al. (1998), Kim et al. (2009), and Ngoh and Shepherd (1997) have investigated the recognition levels of specific sets of pictographs. Virtually all of the existing studies show that the recognition of a pictograph’s meaning varies considerably across cohorts. Further, studies such as the one conducted by the Hablamos Juntos initiative (Cowgill and Bolek, 2003) show that pictograph recognition varies considerably even within a set created by the same design team.

Empirical studies on pictograph recognition also have their methodological shortcomings. They can determine how easily recognizable the meaning of any given picture is in a given population. However, their results cannot be generalized to other pictographs. To make the outcomes of such studies more generalizable, researchers must first be able to identify a manageable set of relevant characteristics that are shared by most, if not all, pictographs. More specifically, researchers need a comprehensive taxonomy of pictographs.

Every pictograph is intrinsically composed of two parts: a graphic representation and a referent (i.e., meaning). Thus, a taxonomy of pictographs must necessarily be able to identify and systematically classify all possible representation/referent relations. That is, it must be able to catalogue all possible strategies used to convert a concept or an idea into a picture. Figure 2 shows some possible representations of the concept “water”.

Examples of verbal and pictorial representations of the concept “water”.

A pictograph taxonomy can be used to extrapolate the findings of pictograph recognition studies to other pictographs that were not part of the original study. Figure 3 shows a hypothetical case for extrapolation. Let’s consider that a pictograph recognition study includes two depictions of the concept “milk”. One of them is classified as a pictograph in which the concept (milk) is represented through the familiar shape of its container (milk carton). This pictograph is recognized by 50% of the study’s participants. The other is classified as a pictograph in which the concept is represented through the shape of its container and through its source (cow). The recognition rate this time is 70%. Assuming analog representation strategies yield similar recognition rates, one can predict that, for example, a pictograph in which the concept “coffee” is depicted through a coffee cup and saucer would yield a recognition rate of approximately 50%.

Example of how the findings of pictograph recognition studies based on a taxonomy of representation strategies can be generalized to other pictographs.

Indeed, the predictive power of a pictograph taxonomy that focuses on the types of possible semantic relationships between representation and referent is conditional on the actual weight that these relationships bear on pictograph recognition in naturalistic contexts. Other factors such as readers’ familiarity with the original concept, graphic quality, and representation genre surely influence recognition as well. Consequently, a synergistic relation between taxonomic studies and recognition studies must occur. That is, a pictograph taxonomy can enable the generalization of the findings of pictograph recognition studies. Conversely, pictograph recognition studies can be used to fine-tune the taxonomy.

The study we describe in this paper is part of a larger research project in which we propose to develop a computer application that automatically complements patient instructions with pictographs. Studies have shown that patients often do not fully understand or recall the instructions they receive (Heng et al., 2007; Hwang et al., 2005; Spandorfer et al., 1995). Aside from the instruction’s complexity, factors such as physical and emotional distress and environmental distractions at the moment of discharge affect the comprehension and recall of the instructions. But prior studies also show that simplifying the language of the instructions or complementing them with pictures can increase comprehension and recall (Austin et al., 1995; Delp and Jones, 1996; Jolly et al., 1995). However, manually creating such pictographs is prohibitively time consuming and requires solid drawing skills from the health care team. To address these restrictive factors, we are developing a system that will automatically enhance free-text instructions with pictographs.

In the next sections we present the results of our review of previous taxonomies. We discuss what we consider their main limitation, namely, the lack of granularity. We then describe the development of our proposed taxonomy based on the analysis of a sample of 846 pictographs in three sequential stages: lexical classification, semantic classification, and classification by representation strategy. Content-wise, our study focuses on pictographs used in the health care domain. Finally, we discuss the limitations of our approach and possible future directions.

2. Background

2.1 Sign Taxonomies

Semiotics and semiology are probably the most widely known systems used to classify signs in general. The terms ‘semiotics’ and ‘semiology’ have often been used interchangeably despite their distinct origins. Semiotics (Hartshorne, Weiss, & Burks, 1931–58) was created by Charles Sanders Peirce as part of his full philosophical system, composed of phaneroscopy (phenomenology), normative sciences, and metaphysics. Semiotics was one of his proposed normative sciences. Conversely, semiology (Bally & Sechehaye, 1966) was created by Ferdinand de Saussure and is closely related to his linguistics theories. Both semiotics and semiology were developed by their respective authors in the same period and were devoted to the same subject. The coincidences seem to end here, as these two systems were based on different models.

Saussure modeled a sign as a dyadic chain connecting a signifier (the sign itself) and a signified (the referent). Peirce, on the other hand, envisioned a sign as part of a triadic relation composed of an object (the referent), a sign or representamen, and an interpretant (the logics that can possibly connect object and sign). There is at least one advantage of a triadic model over a dyadic one: the distance between a sign and its referent can be viewed in a more flexible manner. In the triadic model, the interpretant vertex can make the distance between the other two vertexes shorter or longer. This makes sense if we consider that the interpretation of a sign depends on more than the characteristics of the sign and the object alone.

Peirce went as far as proposing ten trichotomies, that is, ten possible triadic classifications of a sign. Among those, the most widely known and used are the ones exploring the relationships between a sign and its object (icon, index, and symbol), between a sign and its interpretant (rheme, dicisign, and argument), and a sign in itself (qualisign, sinsign, and legisign). In the fields of design and visual communication, the trichotomy icon/index/symbol has been the most frequently mentioned. Icons denote a sign-object relationship based on similarity (e.g., a caricature). Indexes denote a real existential connection between a sign and its object (e.g., a fingerprint). Symbols denote a sign-object relationship based on an arbitrary convention (e.g., a national flag). Johansen (1988) has called attention to the fact that the distinctions between a sign, an index, and a symbol can become less clear at a closer inspection. For example, a photograph can be considered an icon since it relates to its object by visual similarity. It can also be considered an index as it is physically influenced by its object (through light). Finally, it can be considered a symbol as the interpretation of a three-dimensional object represented in a two-dimensional medium is a cultural phenomenon.

2.2 Icons, Symbols, and Pictographs

In the previous sub-section, we provided Peirce’s definitions of an icon and a symbol. We also quoted Johansen to demonstrate that the distinction between Peirce’s definitions of these two concepts can be blurred depending on how one approaches the topic.

An examination of the definitions of an icon and a symbol offered by experts in visual communication often mirrors this lack of clear boundaries. For example, Horton considers icons as a subset of symbols. He argues: “The small pictorial symbols used on computer menus, windows, and screens are icons” (Horton, 1994, p. 2). Moreover: “In the computer industry, the term icon is often used as a synonym for any small visual symbol” (Horton, 1994, p. 2). McDougall argues for an inverse subordination between these two concepts: “For the sake of simplicity, icons is the term used in this article to refer to the broad range of icons, signs, or symbols used to help individuals interact with machines and their environment.” (McDougall & Isherwood, 2009, p. 325). Gurak also defines icons as a specific type of symbol: “Icon usually means a symbol that represents, either alone or in combination with some text, a task that users will perform” (Gurak, 1992, p. 33).

Marcus argues for a distinction between an icon and a symbol in terms of the concreteness of the representation: “Icons are signs that are familiar, are easy to understand, and are often concrete representations of objects or people. Symbols are signs that are often more abstract and require specific instruction to learn” (Marcus, 1996, p. 257). Yazdani and Barker, on the other hand, argue that icons can be both concrete and abstract in nature: “we regard an icon as being a graphical representation of some object or process. The object that is represented could have a concrete existence (such as a traffic signal or an airplane) or it could be abstract in nature -- such as a thought, a concept or an idea” (Yazdani & Barker, 2000, p. 7).

Definitions for the term ‘pictograph’ also vary considerably, depending on the author. Bottero offers a broad definition: “The term pictograph is used here to mean a graph that encodes a word (or a linguistic unit) through the medium of depiction” (Bottero, 2004, p. 251). Dewar uses the terms ‘symbol’, ‘icon’ and ‘pictograph’ as synonyms: “The terms ‘symbol’, ‘pictograph’, ‘pictogram’, ‘icon’ and ‘glyph’ all refer to commonly used pictorials. These terms will be used to mean more or less the same thing in the present chapter” (Dewar, 1999, p. 285). For Bocker (1996), the term ‘pictograph’ can be used to refer to both icons and symbols:

Pictograms, icons, and symbols convey information in pictorial form. They are graphical as opposed to textual signs. Pictograms (sometimes called pictographs) form the general class of graphical signs and include icons and symbols. An icon is a simplified pictorial representation which consists of more or less concrete and realistic elements and should be self-explanatory. A symbol is an abstract and often simplified pictorial representation which is not necessarily realistic. A symbol often requires a learning process in order to be understood. (p. 107).

In this paper, we will follow Peirce’s definitions of ‘icon’ and ‘symbol’ and, like Bocker, employ the term ‘pictograph’ to refer to both icons and symbols.

2.3 Pictograph Taxonomies

A few icon taxonomies have been proposed in the area of graphic user interface. However, they are not granular enough to capture differences among representation strategies that can be visually subtle but semantically significant. In the following paragraphs we discuss these previous taxonomies.

The most basic classification systems have focused their analysis on the characteristics of the pictorial representation. The studies by Lindgaard et al. (1987) and Blattner et al. (1989) fall into this category. Lindegaard et al. (1987) identified three classes of icons: abstract, depictive, or mixed. Abstract icons are purely symbolic and lack any resemblance to actual objects. The female symbol (a circle joined to a cross) is an example of an abstract icon. Depictive icons depictive elements. Blattner et al. (1989) proposed overlapping design principles for both icons and earcons (audio messages) in computer interfaces. Three types of icons were identified in this study: representational, abstract, and semi-abstract. Representational icons are representations of familiar objects or operations. Abstract icons are representations created by combining geometric shapes. A semi-abstract icon combines both representational and abstract features.

The focus on the pictorial representation is the major weakness of the two classification systems described above. As previously argued, pictographs are composed of two parts, a visual representation and its referent. The visual representation is supposed to communicate the referent with minimal ambiguity and minimal reliance on pre-established codes or conventions. Thus, a taxonomy of pictographs must assess the relation between concept and representation rather than focus on the merits and weaknesses of the graphic representation alone.

Studies that classify pictographs according to their relation to their intended meaning seem to agree that, broadly speaking, there are three basic ways to graphically represent a concept: directly, indirectly, or arbitrarily (Lodding, 1983; Purchase, 1998; Webb et al., 1989). In direct representations, the relation between pictograph and referent is immediate (as in the case of a portrait of a person). In indirect representations, the relation between pictograph and referent is mediated. That is, the referent is suggested rather than explicitly depicted in the pictograph (as in the case of a picture of a tent that indicates a camping area). In arbitrary representations, the relation between pictograph and referent is established arbitrarily, by convention (as in the case of a country and its national flag).

Lodding (1983) distinguished between three kinds of icons: representational, abstract, and arbitrary. Representational icons are those that contain images that mimic an object. Abstract icons were described as images of objects that displayed associated concepts. A picture of a broken wine glass used to convey the concept “fragile” is an example of an abstract icon. Arbitrary icons were described as images whose associations of meaning are arbitrarily established, for example, the red-cross symbol commonly associated with “first aid”.

Webb et al. (1989) also identified three types of icons: pictorial, symbolic, and sign. Pictorial icons are those that represent objects by resemblance. Symbolic icons operate by analogy, as in a picture of a cupid representing the concept “love”. Signs are abstract depictions that are learned by reinforcement. That is, there is no inherent or intuitive connection between a sign and its referent.

Purchase (1998) proposed a semiotic model of multimedia in which icons were classified as concrete-iconic, abstract-iconic, or symbolic. Concrete icons are associated to their referents by physical resemblance, as in a photograph. Abstract icons still maintain a relation with its referent though that relation is less explicit, as in the case of a diagram. Symbolic icons hold an arbitrary relation to their referents.

Direct and arbitrary representation strategies are relatively straightforward. Indirect representation strategies, on the other hand, can follow many different types of concept association. For example, the picture of a fork and a knife used to represent the concept “restaurant” and the picture of a running faucet used to represent the concept “water” are both examples of indirect representation. Nonetheless, these two examples are quite distinct in regard to the relation between the pictograph and its referent.

The studies by Gaver (1986), Rogers (1989), and Lidwell et al. (2003) attempted to develop more granular classifications of pictures created through indirect approaches.

Rogers (1989) identified four types of icon, according to their relation to their referents: resemblance icons, exemplar icons, symbolic icons, and arbitrary icons. Resemblance and arbitrary icons have self-explanatory titles. The former represent its referent by resemblance and the latter by an arbitrary convention. Exemplar and symbolic icons represent their referents indirectly. Exemplar icons focus on the most representative characteristics of the referent. The picture of a fork and a knife representing the concept “restaurant” is an example of an exemplar icon. Symbolic icons represent concepts that are at a higher level of abstraction. The picture of a broken wine glass representing the concept “fragile” (which Lodding classifies as abstract icon) is an example of a symbolic icon.

The classification system proposed by Lidwell, et al. (2003) mirrors the one created by Rogers (1989). The four iconic categories in this system are: similar icons, example icons, symbolic icons, and arbitrary icons. The definitions are quite similar to Roger’s classification: resemblance icons, exemplar icons, symbolic icons, and arbitrary, respectively.

Gaver (1986) proposed a system to map the relation between representations and their referents. The system was composed of three categories: symbolic mapping, nomic mapping, and metaphorical mapping. Symbolic mappings represent arbitrary relations. Nomic mappings represent direct relations, as the one between a photograph and the scene it depicts. Metaphorical mappings are based on similarities. They are not completely arbitrary and, at the same time, they do not depend on physical causation. The author offers as an example the relation between “genealogy” and “tree”. Metaphorical mappings are further divided into structure mapping and metonymic mapping. Structure mappings explore the similarities between referent and what is actually depicted (e.g., genealogy: tree). Metonymic mappings represent a concept through one of its parts (e.g., horse: horseshoe). Note that Gaver’s metonymic mapping is equivalent to Rogers’ exemplar icon.

Table 1 summarizes the classification systems described above.

Table 1

Existing Icon Taxonomies

The studies presented above generally agree that there are three basic ways to represent a concept graphically:

1. Direct representations that explore the visual similarity between a pictograph and its referent;

2. Arbitrary representations that are established by social convention;

3. Indirect representations that explore semantic relations between a pictograph and its referent.

There are relatively few types of concept that can be represented through the direct and arbitrary approaches. Most conceptual categories require an indirect representation approach, as they cannot be represented by visual similarity or by an existing convention. However, prior studies have not yet produced a classification that is granular enough to distinguish among the different types of indirect approaches. Further, none of these studies have attempted to identify correlation patterns between representation strategy and the semantic characteristics of the referent.

The last taxonomy of pictographs we want to discuss is that proposed by Familant and Detweiler (1993). Of all existing taxonomies, this is probably the most comprehensive and shares many features with the taxonomy we propose in this paper.

Familant and Detweiler define a sign as something composed of two parts, a signal (what we call a representation in this paper) and a referent. Following Peirce’s model, they acknowledge that “what a particular sign denotes is dependent on the agent who gives the sign its meaning, i.e. the sign’s interpreter” (p. 710). They also follow Peirce’s distinctions between an icon and a symbol.

Like the taxonomy proposed here, Familant and Detweiler’s taxonomy distinguishes between direct and indirect relations between a representation and its referent. They distinguish among five types of referent relations (part-part, part-whole, whole-part, identical, disjoint) elaborating on two of them: part-part and part-whole. A part-part relation occurs when “the sign referent’s feature set intersects with the denotative referent’s set, but both sets contain features not found in the other set”, for example the picture of a trash can on a computer desktop to represent the concept ‘delete’ (p. 713). A whole-part relation occurs when “the sign referent’s feature set is a proper subset of the denotative referent’s feature set”, for example the picture of a fork and a knife to represent the concept ‘restaurant’ (p. 714).

A final merit of the taxonomy proposed by Familant and Detweiler is that it recognizes that many signs are composed of more than one type of relation between representation and referent. That is, they are composed through a combination of representation strategies.

In the following sub-section we conclude our review of the literature by discussing a few pictograph characteristics that have been previously investigated.

2.4 Pictograph Characteristics that Have Been Investigated

Indeed, there are many other ways to classify pictographs other than the ones examined in this study. Among those, the studies conducted by McDougall and colleagues are especially worth mentioning as they managed to assess several of those characteristics simultaneously (Isherwood, McDougall, & Curry, 2007; McDougall, Curry, & Bruijn, 1999; McDougall & Isherwood, 2009). We’ll discuss four of them: complexity (amount of detail in a representation), concreteness (the extent to which a representation depicts real objects, materials, or people), semantic distance (how close a representation is to its referent), and familiarity (how often a given pictograph is encountered).

Complexity is a characteristic of the representation in itself. Assessing how visually complex a representation is does not depend on what kind of relation it may hold with its referent. Concreteness is also a characteristic of the representation in itself: the concreteness of a representation is assessed in terms of its relation to a real world object and not to its referent. For example, a picture of a skull representing the concept ‘poison’ can be considered concrete because it visually resembles a skull, not a poison. McDougall, Curry, and Bruijn (1999) demonstrated in their study that, unlike previous studies suggested (Garcia, Badre, & Stasko, 1994), concreteness and complexity are two separate dimensions, with no significant correlation between them. Concreteness was found to be correlated with identification accuracy scores but complexity was not (Isherwood, McDougall, & Curry, 2007).

Semantic distance is a characteristic that takes into consideration both the representation and the referent. However, measuring the semantic distance between two terms is a challenging endeavor due to the lack of consolidated metrics. In their studies, McDougall, Curry, and Bruijn (1999) quantified semantic distance by asking participants to provide ratings (in a five-point scale) for the perceived closeness between symbols and their functions. Thus, perceived semantic distance rather than semantic distance was the object of assessment. In a later study (Isherwood, McDougall, & Curry, 2007), semantic distance was also found to be correlated with identification accuracy scores.

Familiarity is a characteristic that takes into consideration the representation, the referent, and also the interpretant. It is the one characteristic that incorporates the full triadic chain proposed by Peirce. McDougall and colleagues also found significant correlations between familiarity and icon identification accuracy.

Studies by McDougall and colleagues demonstrate that concreteness, semantic distance, and familiarity are dimensions that do influence the interpretation of pictographs. Although our study did not directly employ these dimensions, the evidence suggests they deserve further investigation and integration to pictograph taxonomies.

2.5 Goals

In this study, we propose to produce a more in-depth classification of representation strategies used in the creation of pictographs. More specifically, we propose to:

1. Develop a classification system able to distinguish the representation/referent relations at a finer level.

2. Identify the relation between the lexical and semantic characteristics of a concept and the representation strategies that can be applied to that concept.

3. Method

We conducted a taxonomical analysis of 846 health-related pictographs that were collected from online sources and journal articles (Discovery Education, 2009; Dowse & Ehlers, 2001; Hablamos Juntos, 2009; Hirslanden, 2009; Microsoft, 2009; Moriyama et al., 1994; Picto Online, 2009; Pooviah, 2000; RCJ Pictogram, 2009; Somerset Total Communication, 2009; National Institute of Dental and Craniofacial Research, 1999; U.S. Pharmacopeia, 2009). The logistics of obtaining legal approval to reproduce the analyzed pictographs in a publication would require a counterproductive amount of time and effort, as they were taken from many sources. Therefore, all pictographs shown in this paper are not reproductions of the original pictographs but recreations made by the authors.

Given the taxonomic nature of this study, we chose to adopt a purposive sampling approach. We selected pictographs whose referents were related to health care, such as diseases, symptoms, medical specializations. We also included terms that were likely to appear in patient instructions, such as those related to diet, exercise, drug regimen, etc. Data collection stopped when our sample reached a state of theoretical saturation, the point at which further observations lead to minimal or no new information (Glaser and Strauss, 1967).

We developed the taxonomy based on a cross classification of health-related pictographs based on three dimensions: (1) lexical classification (part-of-speech); (2) semantic classification; and (3) classification by representation strategy. The three classifications are sequential rather than parallel. The semantic classification depends on the results of the lexical classification (which excluded some pictographs such as those associated with sentences and function words) and the classification by representation strategy is performed upon the results of the two prior classifications.

The lexical classification was performed according to the specific grammatical rules of the English language. The semantic classification was performed using the Unified Medical Language System (UMLS) (Bodenreider, 2004).

The taxonomy of representation strategies was created using an analytical approach based on the iterative coding procedures of the constant comparative method (Glaser and Strauss, 1967): open coding; axial coding; and selective coding. Open coding corresponds to the initial development of categories and associated properties from the analysis of the collected data. In axial coding, the categories are integrated in order to form a theoretical framework. At this stage, the coding is refined and the relationships among the categories and sub-categories are defined. Selective coding corresponds to the enhancement of internal validity, that is, the validation of categories, properties, and relationships. As our methodological approach was based on the constant comparative method, all taxonomic categories were created ad hoc, based on the representation patterns that we identified in our pictograph sample. Interrater agreement was measured with a reliability test using a small sample of 50 pictographs.

4. Results

The 846 pictographs in our sample (sources listed in Section 3, ¶ 1) corresponded to 644 distinct concepts. The number of concepts is smaller than the number of pictographs because many concepts had more than one pictographic representation.

4.1. Lexical Classification

Pictographs were initially divided into lexical words (or content words) and function words (or grammatical words). Function words (e.g., prepositions, conjunctions) were excluded from our analyses because their existence is only justified within a verbal communication context, that is, they are not explicitly represented in pictographs. The remaining 832 lexical words were divided into modifiers (adjectives, adjectival phrases, adverbs, adverbial phrases, prepositional phrases) and standalone lexical words (nouns, noun phrases, and verbs). This division is important to our analyses because modifiers represent a special challenge to graphic representation, as they cannot be conveyed graphically without a specific context. That is, they cannot be represented without also representing the concept being modified (which generates issues related to minimal units of meaning). Finally, nouns and noun phrases were separated from verbs. This distinction was strategic for our next analytical round, semantic classification, which was based on the UMLS (Bodenreider, 2004). The UMLS semantic network is effective for the semantic classification of nouns but not for other lexical categories. Figure 4 presents the results of the lexical classification.

Lexical and semantic classification.

With a few exceptions, the transitive senses of verbs and most modifiers cannot be graphically represented on their own. For example, the concept “thin” can only be graphically represented by also representing the modified entity. That is, it is possible to draw a thin person or a thin object but it is virtually impossible to graphically convey the concept “thin” in isolation in a manner that would be easily comprehended. The same applies for the transitive sense of a verb. For example, a snapshot-like picture of a person in a running posture could represent the intransitive verb “run”; however, verbs like “turn on” cannot be represented without also representing its object.

The examples shown above highlight the impossibility of adapting the modularity of verbal communication to pictographic communication. Consequently, a pictographic lexicon will necessarily be far larger than its equivalent verbal lexicon – the minimal unit issue. Linguistically, the modifier “broken” can be directly connected to a large number of entities: “broken arm”, “broken bone”, “broken glass”. In the graphic realm such modularity is seldom possible. A picture of a broken arm would be very different from a picture of a broken glass. Each of them would constitute a minimal unit of meaning. Graphically, most modifiers cannot have an independent existence. Thus, in most cases, a pictographic lexicon must anticipate and create all modifier/modified combinations that are likely to be used.

4.2. Semantic Classification

Nouns and noun phrases were further classified according to the UMLS semantic network. Seventy-eight concepts were classified as Events and 484 as Entities. Of the concepts classified as Event, 29 were sub-classified as Phenomenon or Process and 49 as Activity. Of the concepts classified as Entities, 192 were sub-classified as Conceptual Entity and 292 as Physical Object. Our classification of nouns diverges from the UMLS in one category: Manufactured Objects. The UMLS classifies concepts such as laboratory and hotel as Manufactured Objects. In our system, those concepts are classified as Environmental Objects. The UMLS focuses on the production aspects to classify these concepts as manufactured objects whereas we are focusing on the objects’ scale. Thus, any entity, manufactured or not, that can contain a person (e.g., parking, library, room) was classified as Environmental Object. This distinction becomes relevant to the next classification stage because pictographs of Manufactured Objects and of Environmental Objects are created through distinct representation strategies. Figure 4 presents the results of the semantic classification.

4.3. Classification by Representation Strategy

We identified three basic strategies used to convert concepts into pictographs: visual similarity, arbitrary convention, and semantic association (See Figure 5). Representation through visual similarity can only be applied to nouns. More specifically it can only be applied to concrete objects, but not to conceptual entities or events (activities or processes). Representation through semantic association and through arbitrary convention have been applied to other part-of-speech categories as well. The top level strategies are equivalent to the strategies identified in previous taxonomies (Lodding, 1983; Purchase, 1998; Webb, et al., 1989).

The nested levels of the proposed taxonomy with examples of the three major representation strategies.

The structure of the proposed taxonomy is based on the assumption that there are two nested levels through which one can classify the relation between pictograph and referent. The first level is related to the prior knowledge of the reader, which other authors have referred to as familiarity (Isherwood, McDougall, & Curry, 2007). In this context, familiarity refers to both familiarity with the object being depicted and the relationship of the depiction and its intended meaning. At this level, representations are classified as conventional (arbitrary convention) or non-conventional (visual similarity and semantic association). Conventional representations imply that the relation between pictograph and referent is established through a social contract. Therefore, decoding can only happen if the reader knows the pre-existing code. Non-conventional representations imply that the referent must be inferred solely based on pictorial clues. In this second level, non-conventional representations can be further classified as non-mediated or mediated, or as direct or indirect. Non-mediated representations imply a direct visual relation between pictograph and referent, that is, what you see is what you get. Mediated representations require more inferential power from the reader, as the representation/referent relation is not explicitly depicted. That is, the pictograph stands for a concept other than the one that can be directly inferred. A picture of a scale standing for the concept “weight” is an example of a mediated representation. While non-mediated relations are singular, mediated relations can be mediated in several different ways. Different mediation strategies have been used to create pictographs. A more in-depth description of mediation strategies is presented in 4.3.3.

4.3.1. Representation through visual similarity

In representation through visual similarity there is a direct connection between pictograph and referent. The pictograph is created by reproducing the visual characteristics of the referent (Figure 6). However, only nouns and, more specifically, only physical objects with an easily identifiable shape can be represented through visual similarity.

Examples of representation through visual similarity.

4.3.2. Representation through arbitrary convention

In representation through arbitrary convention there is no real connection between pictograph and referent. The connection is established by reinforcement, like words in verbal communication. The male and female symbols are two examples of arbitrary convention. Traffic signs, national flags, and logos are examples of other types of arbitrary graphic conventions. We identified three subtypes of representation by arbitrary convention: abstract, concrete, and transposed convention (Figure 7).

Examples of representation through arbitrary convention.

Abstract convention

Abstract conventions are mainly geometric (e.g., 3 arrows standing for “recycle”) or verbal (e.g., the letter P standing for “parking”).

Concrete convention

A concrete convention works like an abstract convention. The difference is that it visually resembles a physical object. For example, a picture of a skull with crossed bones conventionally represents the concept “poison”. Many concrete conventions have semantic origins. For example, the caduceus used to represent the concept “medicine” was originally associated with the Greek myth of Aesculapius. Over time, the semantic association faded and the conventional aspect of the representation became dominant. Nowadays, one can identify the caduceus as symbolizing medicine without any knowledge of Aesculapius. One limitation of concrete conventions is that the reader often may not be able to tell whether the pictograph should be read iconically (inferred based solely on the elements depicted in the picture) or symbolically (conditional to prior knowledge of the relation between pictograph and referent).

Transposed convention

This subtype of arbitrary convention differs from the two previous ones in the sense that the convention is in the referent, not in the pictorial representation. Flexing one’s arm to contract the biceps, which is a gesture conventionally associated with the concepts “muscle” and “strong”, is an example of a convention associated with the referent rather than the representation.

Representation by arbitrary convention is an all-or-nothing strategy. It represents a considerable trade-off: for those who know the code, interpretation is instantaneous, but for those who do not know the code, interpretation is impossible. We all can communicate rather effortlessly in our own language because we are proficient in its semantic and syntactic conventions. In verbal communication, the safety of the message is assured by the fact that we are communicating using pre-established conventions. In human communication, the use of convention is the most effective (though imperfect) way to avoid semantic ambiguity. However, only those familiar with the adopted conventions will be able to understand the message. Thus, the use of convention in pictographs must be carefully considered, since one of the reasons behind the use of pictographs in health care communication is to bypass language barriers or, in more general terms, to communicate without having to rely on a code.

Representation by arbitrary convention can also be applied to only specific elements of a pictograph. Examples of specific elements within a pictograph that are coded through arbitrary convention include: a diagonal bar to indicate negation; arrows to indicate direction or sequence; balloons to mark speech or thought; visual reproduction of some gestures like “thumbs up”; and the use of words and letters.

4.3.3. Representation through semantic association

In representation through semantic association, the relation between pictograph and referent is neither direct nor arbitrary, but mediated as in the case of a picture of a clock used to convey the concept “time”. Because there are no visual counterparts to the concept “time”, one can create a pictograph not by visual but by semantic proximity. In this case, a device used to measure time, which has an easily identified form, stands for the concept of time itself. Our analyses are based on the relationship between representation and referent rather than on the representation alone. Thus, a pictograph of a clock could be classified as visual similarity or semantic association, depending on whether it is intended to convey the concept “clock” or the concept “time”, respectively. Semantic association can be broken down into several subtypes (see Figure 5).

Our classification system was developed based exclusively on the direct analysis of our sample of 846 pictographs. None of the classes proposed were developed a priori, based on existing taxonomies. However, two sub-categories of representation through semantic association, metaphor and contiguity, also show a strong parallel to widely used figures of speech, namely, metaphor and metonymy. For example, a picture of a cactus drawn over a tongue to suggest the concept “dry mouth” (Figure 14) is classified as a metaphor in our proposed system and also mirrors verbal metaphors such as “he is no angel”. Similarly, the subcategory contiguity suggests meaning in a manner akin to verbal metonymy. For example, the concept “toothpaste” has often been visually suggested by the characteristic shape of its package (Figure 15). This representation strategy mirrors a common type of verbal metonymy in which the container stands for the contained as in “I already drank two bottles.”

Examples of representation through semantic association: Metaphor.

Examples of representation through semantic association: Contiguity.

Comparison or Contrast

In representation by comparison or contrast, concepts are represented (and are supposed to be interpreted) in relation to other elements included in the pictograph (Figure 8). Conventions such as arrows and highlighting are often used to make the main element stand out from the accessory elements. Adjectives are often represented through this strategy.

Examples of representation through semantic association: Comparison or contrast.

Exemplification

The use of multiple examples is useful to represent general or collective concepts such as “clothes”, “food”, or “toiletries” (Figure 9). Because nesting can occur among concepts (e.g., clothes/underwear), the chosen examples must convey the concept without being overly broad or overly narrow.

Examples of representation through semantic association: Exemplification.

Semantic Narrowing

This representation strategy is similar to representation by exemplification. The main difference is that, instead of using multiple examples of a concept, only one example is depicted. The use of one single example invariably induces a semantic narrowing effect (Figure 10). Singular examples are often used to represent modifiers and transitive senses of verbs. Modifiers are challenging to represent because most of the times they can only be represented in context. That is, one cannot represent a modifier without also representing the modified concept. For example, it is possible to depict anger in someone’s face or to depict someone who looks angry, but it is not possible to depict “angry”. The same applies for transitive senses of verbs. In these cases, the verbs are frequently represented within a specific instance. For example, verbs that require an object are represented with a specific object (e.g., smell = smell flower). The problem with this strategy is the reader cannot possibly know which elements of the pictograph are essential and which are accessories.

Examples of representation through semantic association: Semantic narrowing.

Physical Decomposition

In physical decomposition, a part is used to represent the whole, as in a synecdoche (Figure 11). We subdivided this category into Single Component and Multiple Components because we hypothesized that pictographs depicting multiple components would be more easily recognized. The challenge in using this strategy resides in identifying the part or parts that are the most representative of the referent. Using a single component to represent a concept can often raise the ambiguity level. For example, the picture of a microscope standing for the concept “laboratory” could be interpreted simply as “microscope”. Furthermore, the same component can belong to different concepts. When multiple components are included in the representation, the reader has more clues to interpret the pictograph. Moreover, the pictograph is less likely to be interpreted as something created by visual similarity.

Examples of representation through semantic association: Physical decomposition.

Temporal Decomposition (Snapshot)

This strategy is useful to represent concepts that intrinsically involve a temporal component such as events and verbs. It consists of representing a concept in a snapshot fashion (Figure 12). The symbols of the various sports created for the Olympic Games are examples of representations created by temporal decomposition.

Examples of representation through semantic association: Temporal decomposition (snapshot).

Body Language

This strategy involves the depiction of facial expressions, gestures, and body postures to convey a concept (Figure 13). Pictographs that adopt this strategy can be further subdivided into pictographs that convey their meaning through the exclusive use of facial expressions and pictographs that convey their meaning using facial expressions, gestures, and body postures. Facial expressions are often used to represent adjectives related to mental states such as angry, happy, or puzzled. Although facial expression can be suggestive, they can become ambiguous at a finer-grained level. For example, the depictions for “sad” and “depressed” can be very similar although these concepts are quite distinct, especially in medical terms. The combination of facial expressions, gestures, and body posture, on the other hand, is often used to represent physical symptoms such as back pain, toothache, etc.

Examples of representation through semantic association: Body language.

Metaphor

Like verbal metaphors, visual metaphors connect concepts by similarity. Bells drawn over a person’s ears to represent the concept “ringing in the ears” is an example of a metaphoric representation strategy (Figure 14).

Contiguity

Representation by semantic contiguity is analogous to a metonym. It often involves a substitution. For example, the picture of a sphygmomanometer is commonly used to represent the concept “blood pressure”. Below are the most common strategies based on semantic contiguity. Table 2 shows the metonymic strategies that we have identified in our sample.

Table 2

Subcategories of Representation Strategy by Contiguity

Figure 15 shows examples of representation strategies based on semantic contiguity.

4.3.4. Combination of representation strategies

The representation strategies discussed above are not mutually exclusive. In fact, pictographs are often created by combining two or more strategies. Figure 16 shows a few examples of pictographs created through a combination of representation strategies.

Examples of combined representation strategies.

4.3.5. Interrater agreement

To test our proposed taxonomy, we conducted two distinct interrater reliability studies. The two studies were based on two distinct sets of pictographs. In the first study, we worked with 50 pictographs representing 10 categories of the taxonomy. The selected categories are listed below.

1. Visual Similarity

2. Arbitrary Convention

3. Semantic Association

   • 3.1

     Comparison or Contrast

   • 3.2

     Exemplification

   • 3.3

     Semantic Narrowing

   • 3.4

     Physical Decomposition

   • 3.5

     Temporal Decomposition

   • 3.6

     Body Language

   • 3.7

     Metaphor

   • 3.8

     Contiguity

Two coders (the two authors) applied the 10 categories to the 50 pictographs. Interrater agreement was 90%. This can be considered as a substantial agreement rate, given that this is a new taxonomy being tested for the first time, and it contains 10 different levels, which decreases the possibility of chance agreement.

For the second study, we recruited four Ph.D. students at the Department of Biomedical Informatics, University of Utah. After a 30-minute explanation of the taxonomy, the participants were randomly grouped in pairs, given a set of 96 pictographs, and asked to classify them according to the taxonomy. In this study, the sub-taxa referring to representation through contiguity were also included (physical contiguity, container, source, use, cause of effect, tool, and object). Thus 16 possible classifications were possible. Interrater reliability was 100%. In this second study, participants were also asked to classify an additional 21 pictographs created through a combination of two different representation strategies. Interrater reliability for the second set was 85%. The fact that we were able to achieve interrater reliability levels of 85% and higher is encouraging given that the second study: (1) used a much larger set of pictographs; (2) included pictographs representing the full taxonomy; (3) included pictographs created using more than one representation strategy; and (4) relied on more coders who were less familiar with the taxonomy.

5. Discussion

There have been many studies on pictograph recognition. The greatest limitation of such studies is that their results cannot be generalized. Researchers can test how easily identifiable a given pictograph is, but that conclusion cannot be transferred to other pictographs. This study was an attempt to identify relevant commonalities among pictographs that influence their identification. We started with the assumption that these relevant characteristics reside not in the pictorial representations themselves but in the relation between the representations and their referents. Further, we hypothesized that the diversity of representation/referent relationships were far more diverse than what had been previously reported in the literature. To accomplish our goals, we analyzed 846 pictographs and proposed a classification system based on representation/referent relations.

Three major representation strategies emerged based on our initial assumptions on the two nested levels of pictograph-referent relationships (see 4.3). Representation through visual similarity generates pictographs that are relatively easy to interpret, as there is a direct connection between concept and pictograph. However, it can only be used with concrete entities. Representation through arbitrary convention is an all-or-nothing strategy. Representation through semantic association produces pictographs that tend to be more challenging to interpret because the reader must infer the logic that was used to create the pictograph. As our classification system shows, there can be many indirect ways to convert a concept into a pictograph, which magnify the chances for misinterpretation.

Modifiers are more challenging to represent than standalone lexical units. In fact, modifiers and the transitive senses of verbs are poor candidates for pictorial representation. As we have argued before, graphic units of meaning tend to be larger than verbal units of meaning, which makes graphic lexicons considerably larger than verbal ones. Morphological modularity is an important issue in iconic communication.

Two other general aspects of iconic communication deserve a final discussion: the asymmetry between the coding and decoding of pictographs and cross-cultural issues.

5.1. Asymmetry Between the Coding and Decoding of Pictographs

Perhaps the most fundamental characteristic of a language is the level of codification. The code is the element that synchronizes meanings between the addresser and the addressee. However, rather than a dichotomy, this coding level is more of a range. Computer languages such as C++ are at one end of this range whereas iconic communication is located at the opposite end.

The existence of a code is the only way to assure a minimal level of symmetry between the coding and the decoding of a message in human communication. In iconic communication, we cannot rely on such symmetry except in the case of pictographs created through arbitrary conventions. Arbitrary pictographs, however, do not have the same level of reinforcement as oral or written communication. Thus, the only pictographs that are likely to be correctly decoded are the ones created through visual similarity. In all other cases, the proper decoding is merely a possibility. For example a pictograph depicting an apple could stand for the concepts “apple”, “fruit”, “food”, or even “healthy diet”, depending on the context. The reader cannot know which strategy was actually used for the creation of the pictograph.

The reader does have, however, one interpretation aid: the reading context. The context allows the reader to prune the reading possibilities to the most viable alternatives. For example, a sign depicting a baby in a shopping mall is likely to be interpreted as “changing station” whereas the same sign in a hospital may be interpreted as “neonatal clinic”. However, pictograph designers must still be constantly aware that, in the absence of a code, the proper interpretation is a possibility rather than a certainty and even the reading context is often not enough to ensure the correct interpretation.

5.2. Cross-Cultural Issues

Although visual communication is less culture-bound than verbal communication, it is indeed influenced by cultural conventions. Cultural conventions are often difficult to identify as our thinking and communication capabilities are shaped by the culture we live in. The male and female pictographs used in public restrooms are examples of how the decoding of graphic signs is culture-bound. The female pictograph is distinguished from the male pictograph by the contours of a dress. However, this distinction would be rendered less useful in a culture where men’s and women’s clothing do not have the characteristic shapes depicted in these pictographs.

Due to its subjective and partial nature, a graphic representation is always influenced by the culture to which it belongs. For example, when depicting a person, one must select which features to depict and how to depict them. In ancient Egyptian pictures the different parts of the body were always represented from their most characteristic angle. Faces, legs, and feet were depicted in profile but eyes and torso were depicted from a frontal view, resulting in a somewhat incongruent albeit easily identifiable ensemble (Janson and Janson, 2004). This shows that even pictures that were apparently created through visual similarity can be influenced by cultural conventions.

Representation through arbitrary convention is heavily culture-bound, as conventions are mostly culture-specific. Representation through visual similarity is the least culture-bound strategy. Representation through semantic association falls somewhere between the two other representation strategies. This is because the reader must identify which characteristic of the element that is actually depicted relates to the underlying concept. One of the pictographs in our sample conveyed the concept “dry mouth” by depicting a cactus over a tongue (Figure 14). This pictograph can only be decoded correctly if the reader assumes that the relevant characteristic of the cactus is its ability to survive in dry weather. If the reader assumes that the relevant characteristic of the cactus is that it is prickly, the decoding would likely produce a very different meaning (e.g., “thrush”, “stinging in the tongue”).

5.3. Limitations and Future Directions

Although reasonably large, our sample may be biased in terms of concept prevalence. The most representative categories are not necessarily the most used. Certain categories of concepts may be representative due to facility of depiction rather than semantic prevalence. In this regard, future studies should consider a more purposive sample selection. That is, the sampling of pictographs could be guided by a preliminary selection of the most common terms used in the field and activities of the study.

To assess how representative our sample is in relation to the English language in general, we compared it to the Ogden’s Basic English Word List (Basic English Institute, 1996). This is a list of 850 words compiled by Ogden that, when used in combination, covers 90% of the concepts in the Oxford Pocket English Dictionary. Approximately 23% of the words in Ogden’s list are also present in our taxonomy. Table 3 shows the representativeness of our sample in each sub-category in Ogden’s list.

Table 3

Comparison of concepts in the taxonomy with Ogden’s Basic English Word List

There is a good chance that pictographs created using a combination of representation strategies may yield higher recognition rates. However, the possible combinations of two or more representation strategies in one pictograph are vast. At this point, we cannot anticipate with much confidence, which combinations would yield the best results.

In this study we did not analyze iconic representations related to clauses or sentences. Such analysis would require a different sampling method and different analytic procedures. Such a study is indeed valuable, as it would enable the proposition of principles for a pictographic grammar and text-to-pictograph conversion rules at the sentence level.

In this study we focused on pictographs related to the health care domain. The robustness of any taxonomy is highly task specific. Therefore, future validation studies should be conducted within a context that mimics the context in which the pictographs are supposed to be used. In future studies we also intend to examine the strengths and weaknesses of the proposed taxonomy on a group of end-users.

6. Conclusions

Our analyses of atomic pictographs identified much more variation in indirect representation strategies than had been previously reported in the literature. Further, our results indicate that the lexical and semantic attributes of a concept influence the choice of representation strategy. When combined with the results of empirical studies on pictograph recognition, our proposed taxonomy is expected to provide more robust metrics to assess pictograph recognition levels and consequently enable the generalization of the results of these studies.

​

Research Highlights

1. Icons can be created by visual similarity, semantic association, and by convention

2. Only concrete concepts (e.g., “car”) can be represented by visual similarity

3. Abstract concepts (e.g., “pain”) can only be represented by semantic association

4. Semantic association is the most diverse and the most used representation strategy

5. Icons created by convention are only understood if the reader already knows the code

Acknowledgments

This work is supported by the NIH grant R01LM009966. The authors would like to thank Dr. Charlene Weir and Dr. Andrew Chiarella.

Biographies

• 

Carlos Nakamura, Ph.D.

Carlos Nakamura is a research associate at the University of Utah, Department of Biomedical Informatics. His research focuses on the uses of technology to support learning, problem solving in decision making in health care, both from the consumers and the providers perspectives.

• 

Qing Zeng-treitler, Ph.D.

Qing Zeng-Treitler is an associate professor at the University of Utah, Department of Biomedical Informatics. Her research focuses on consumer health informatics and natural language processing of clinical notes.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

• Austin PE, Matlack R, Dunn KA, Kesler C, Brown CK. Discharge instructions: Do illustrations help our patients understand them? Annals of Emergency Medicine. 1995;25(3):317–320. [PubMed] [Google Scholar]
• Babbitt Kline TJ, Ghali LM, Kline DW, Brown S. Visibility distance of highway signs among young, middle-aged, and older observers: Icons are better than text. Human Factors. 1990;32(5):609–619. [PubMed] [Google Scholar]
• Bally C, Sechehaye A, editors. Course in general linguistics. New York: McGraw-Hill; 1966. [Google Scholar]
• Basic English Institute. Ogden’s Basic English Word List. 1996 Jan 1; Retrieved from http://ogden.basic-english.org/words.html.
• Blattner MA, Sumikawa DA, Greenberg RA. Earcons and icons: Their structure and common design principles. Human-Computer Interaction. 1989;4(1):11–40. [Google Scholar]
• Bocker M. A multiple index approach for the evaluation of pictograms and icons. Computer Standards & Interfaces. 1996;18:107–115. [Google Scholar]
• Bodenreider O. The unified medical language system (UMLS): Integrating biomedical terminology. Nucleic Acids Reseserch. 2004;32(Database issue):D267–270. [PMC free article] [PubMed] [Google Scholar]
• Bottero F. Writing on shell and bone in Shang China. In: Houston SD, editor. The first writing: Script invention as history and process. Cambridge: Cambridge University Press; 2004. pp. 250–261. [Google Scholar]
• Cowgill J, Bolek J. Symbol usage in health care settings for people with limited English proficiency: Part one: Evaluation of use o symbol graphics in medical settings. Scottsdale, AZ: Robert Wood Johnson Foundation; 2003. [Google Scholar]
• Delp C, Jones J. Communicating information to patients: The use of cartoon illustrations to improve comprehension of instructions. Academic Emergency Medicine. 1996;3(3):264–270. [PubMed] [Google Scholar]
• Dewar R. Design and evaluation of public information symbols. In: Zwaga HJG, Boersema T, Hoonhout HCM, editors. Visual information for everyday use: Design and research perspectives. London: Taylor & Francis; 1999. pp. 285–04. [Google Scholar]
• Discovery Education. Clip Art Gallery: Health and Safety. 2009 Oct 13; Retrieved from http://school.discoveryeducation.com/clipart/category/hlth0.html.
• Dowse R, Ehlers M. Pictograms for conveying medicine instructions: Comprehension in various South African language groups. South African Journal of Science. 2004;100:687–693. [Google Scholar]
• Dowse R, Ehlers MS. The evaluation of pharmaceutical pictograms in a low-literate South African population. Patient Education & Counseling. 2001;45(2):87–99. [PubMed] [Google Scholar]
• Familant ME, Detweiler MC. Iconic reference: Evolving perspectives and an organizing framework. International Journal of Man-Machine Studies. 1993;39:705–728. [Google Scholar]
• Garcia M, Badre AN, Stasko JT. Development and validation of icons varying in their abstractness. Interacting with Computers. 1994;6:191–211. [Google Scholar]
• Gaver W. Auditory icons: Using sound in computer interface. Human-Computer Interaction. 1986;2(2):167–177. [Google Scholar]
• Glaser BG, Strauss A. Discovery of grounded theory: Strategies for qualitative research. Mill Valley, CA: Sociology Press; 1967. [Google Scholar]
• Gurak LJ. Towards consistency in visual communication: Standardized icons based on task. Technical Communication. 1992;39(1):33–37. [Google Scholar]
• Hablamos Juntos: Language Policy and Practice in Health Care. Universal symbols for health care. 2009 Oct 13; Retrieved from http://hablamosjuntos.org/
• Hartshorne C, Weiss PE, Burks AW, editors. Collected papers of Charles Sanders Peirce. Cambridge, MA: Harvard University Press; 1931–1958. [Google Scholar]
• Heng KW, Tham KY, How KY, Foo JS, Lau YH, Li AY. Recall of discharge advice given to patients with minor head injury presenting to a Singapore emergency department. Singapore Medical Journal. 2007;48(12):1107–1110. [PubMed] [Google Scholar]
• Hirslanden. 2009 Oct 13; Picture file retrieved from http://www.hirslanden.ch/en/klinikenzentren/institute/klinik.cfm?ClinicID=3.
• Horton WK. The icon book: Visual symbols for computer systems and documentation. New York: John Wiley & Sons; 1994. [Google Scholar]
• Houts PS, Bachrach R, Witmer JT, Tringali CA, Bucher JA, Localio RA. Using pictographs to enhance recall of spoken medical instructions. Patient Education & Counseling. 1998;35(2):83–88. [PubMed] [Google Scholar]
• Huchingson RD. New horizons in human factors. New York: McGraw-Hill; 1981. [Google Scholar]
• Hwang SW, Tram CQ, Knarr N. The effect of illustrations on patient comprehension of medication instruction labels. BMC Family Practice. 2005;6(1):26. [PMC free article] [PubMed] [Google Scholar]
• Isherwood SJ, McDougall SJP, Curry MB. Icon identification in context: The changing role of icon characteristics with user experience. Human Factors. 2007;49(3):465–476. [PubMed] [Google Scholar]
• Janson HW, Janson AF. History of art: The Western tradition. Upper Saddle River, NJ: Pearson; 2004. [Google Scholar]
• Johansen JD. The distinction between icon, index, and symbol in the study of literature. In: Herzfeld M, Melazzo L, editors. Proceedings of the Third International Congress of the IASS: Semiotic Theory and Practice. Berlin: Walter de Gruyter; 1988. [Google Scholar]
• Jolly BT, Scott JL, Sanford SM. Simplification of emergency department discharge instructions improves patient comprehension. Annals of Emergency Medicine. 1995;26(4):443–446. [PubMed] [Google Scholar]
• Kim H, Nakamura C, Zeng-Treitler Q. Assessment of pictographs developed through a participatory design process using an online survey tool. Journal of Medical Internet Research. 2009;11(1):e5. [PMC free article] [PubMed] [Google Scholar]
• King AJ. On the possibility and impossibility of a universal iconic communication system. In: Yazdani M, Barker P, editors. Iconic Communication. Bristol, UK: Intellect; 2000. pp. 17–28. [Google Scholar]
• Lidwell W, Holden K, Butler J. Universal principles of design. Gloucester, MA: Rockport; 2003. [Google Scholar]
• Lindgaard G, Chessari J, Ihsen E. Icons in telecommunication: What makes pictorial information comprehensible to the users? Australian Telecommunication Research. 1987;21(2):17–29. [Google Scholar]
• Lodding KN. Iconic interfacing. IEEE Computer Graphics and Applications. 1983;3(2):11–20. [Google Scholar]
• Marcus A. Icon and symbol design issues for graphical user interfaces. In: del Galdo EM, Nielsen J, editors. International user interfaces. New York: John Wiley & Sons; 1996. pp. 257–270. [Google Scholar]
• McDougall SJP, Curry MB, Bruijn O. Measuring symbol and icon characteristics: Norms for concreteness, complexity, meaningfulness, familiarity, and semantic distance for 239 symbols. Behavior Research Methods, Instruments, & Computers. 1999;31(3):487–519. [PubMed] [Google Scholar]
• McDougall S, Isherwood S. What’s in a name? The role of graphics, functions, and their interrelationships in icon identification. Behavior Research Methods. 2009;41(2):325–336. [PubMed] [Google Scholar]
• Microsoft. Images and More: Category Healthcare. 2009 Oct 13; Retrieved from http://office.microsoft.com/enus/clipart/FX101321031033.ASPX?PID=CL100570201033.
• Moriyama M, Harnisch DL, Matsubara S. The development of graphic symbols for medical symptoms to facilitate communication between health care providers and receivers. Tohoku Journal of Experimental Medicine. 1994;174(4):387–398. [PubMed] [Google Scholar]
• Ngoh LN, Shepherd MD. Design, development, and evaluation of visual aids for communicating prescription drug instructions to nonliterate patients in rural Cameroon. Patient Education & Counseling. 1997;30(3):257–270. [PubMed] [Google Scholar]
• Picto Online. 2009 Oct 13; Picture file retrieved from http://pictoonline.pictogram.se/home.do.
• Pooviah R. Application of signage systems. 2000 Retrieved from http://designofsignage.com/application/symbol/hospital/index.html.
• Purchase H. Defining multimedia. IEEE Multimedia. 1998;5(1):8–15. [Google Scholar]
• RCJ Pictogram. 2009 Oct 13; Picture file retrieved from http://www.rad-ar.or.jp/02/08_pict/08_pict_index.html.
• Rogers Y. Icons at interface: Their usefulness. Interacting with Computers. 1989;1(1):105–117. [Google Scholar]
• Somerset Total Communication. Symbols Library. 2009 Oct 13; Retrieved from http://www.somerset.gov.uk/stcsymbols/search.asp.
• Spandorfer JM, Karras DJ, Hughes LA, Caputo C. Comprehension of discharge instructions by patients in an urban emergency department. Annals of Emergency Medicine. 1995;25(1):71–74. [PubMed] [Google Scholar]
• National Institute of Dental and Craniofacial Research. Three good reasons to see a dentist BEFORE cancer treatment. Geriatric Nursing. 1999;20(5):255–256. [PubMed] [Google Scholar]
• U.S. Pharmacopeia. 2009 Oct 13; Picture file retrieved from http://www.usp.org/audiences/consumers/pictograms/form.html.
• Webb JM, Sorenson PF, Lyons NP. An empirical approach to the evaluation of icons. SIGCHI Bulletin. 1989;21 (1):87–90. [Google Scholar]
• Yazdani M, Barker P, editors. Iconic Communication. Bristol, UK: Intellect; 2000. [Google Scholar]