A Social-Semiotic Approach to Color Analysis in Digital Media

Theory and Practice

Authors

DOI:

https://doi.org/10.17851/2237-2083.31.4.2009-2065

Keywords:

social semiotics, color theory, grammar of visual design

Abstract

This article aims to discuss the study of color from a social-semiotic perspective. The article begins with an overview of Gunther Kress and Theo van Leeuwen’s distinctive feature approach to color in their Grammar of Visual Design (2021) and concludes that the authors base their claims primarily on paintings and print media, which may skew the categories toward analog media. Drawing on color theory, mainly the contributions of Rhyne (2017), this article argues that social-semiotic analyzes can benefit from systematized and quantitative categories for color analysis, especially for digital media corpora. The article introduces several analytic categories, such as color harmony and the RGB color space, that are relevant to understanding how color works in digital spaces. The paper proposes the use of the free and open source software GIMP and ImageMagick for a qualitative and quantitative approach to the distinctive features of color and for a better understanding of the complexity of colors and their meaning potential. Based on these methodological procedures, the distinctive feature approach is revised with Rhyne’s contribution. Finally, the categories are applied to two stock images as a case study.

Author Biographies

  • André de Oliveira Matumoto, Universidade de São Paulo (USP), São Paulo, São Paulo / Brasil

    Mestrando em Letras pelo Programa de Pós-Graduação em Filologia e Língua Portuguesa da Universidade de São Paulo (PPG-FLP/USP). Bacharel em Letras (habilitação linguística/português) pela mesma universidade (2017-2022). Realizou a iniciação científica "A construção multimodal do Brasil nos videogames: diálogos entre a sociossemiótica e a ludologia" (2021-2022) sob orientação do Prof. Dr. Paulo Roberto Gonçalves Segundo, do Departamento de Letras Clássicas e Vernáculas (DLCV-USP), e com fomento da Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP) (processo nº 2020/13090-1). Tem como principais interesses de pesquisa os jogos eletrônicos, a multimodalidade e o discurso.

  • Paulo Roberto Gonçalves-Segundo, Universidade de São Paulo (USP), São Paulo, São Paulo / Brasil

    Doutor em Letras (2011) pelo programa de Filologia e Língua Portuguesa, do Departamento de Letras Clássicas e Vernáculas, da Faculdade de Filosofia, Letras e Ciências Humanas da Universidade de São Paulo (USP). Possui graduação em Letras pela mesma universidade (2005). Tem experiência na área de Linguística, com pesquisas orientadas à articulação entre linguagem, cognição e discurso. Atua, principalmente, no campo da Análise Crítica do Discurso, na interface com a Linguística Sistêmico-Funcional, com a Linguística Cognitiva e com a Teoria da Argumentação; e na área de Linguística Aplicada, pensando na articulação entre gramática e textualidade, por meio de um olhar amparado pelas perspectivas cognitivo-funcional e discursivo-textual.

References

BATEMAN, J.; WILDFEUER, J.; HIIPPALA, T. Multimodality: Foundations, Research and Analysis – A Problem-Oriented Introduction. Berlin: De Gruyter Mouton, 2017.

COLOUR vision deficiency (colour blindness). NHS, England, UK, 01 Apr. 2019. Available at: https://www.nhs.uk/conditions/colour-vision-deficiency/#:~:text=It’s%20a%20common%20problem%20that,to%20someone%20with%20normal%20vision. Visited on: 21 Jan. 2023.

GIMP DOCUMENTATION TEAM. GIMP Documentation. GIMP. Available at: https://www.gimp.org/docs/. Visited on: 21 Jan. 2023.

GIMP main page. GIMP, 2023. Available at: https://www.gimp.org/. Visited on: 21 Jan. 2023.

HALLIDAY, M.A.K. Language as Social Semiotic. London: Arnold, 1978.

HERON, P.; CRAMERI, F.; SHEPHARD, G. How rainbow colour maps can distort data and be misleading. The conversation, 06 Oct. 2021. Available at: https://theconversation.com/how-rainbow-colour-maps-can-distort-data-and-be-misleading-167159. Visited on: 21 Jan. 2023.

HODGE, R.; KRESS, G. Social Semiotics. Cambridge: Polity Press, 1988.

IMAGEMAGICK main page. ImageMagick Studio LLC, 2023. Available at: https://imagemagick.org/index.php. Visited on: 21 Jan. 2023.

INKSCAPE main page. Inkscape, 2023. Available at: https://inkscape.org/. Visited on: 21 Jan. 2023.

JAKOBSON, R.; HALLE, M. Fundamentals of Language. Mouton: The Hague, 1956.

JEWITT, C.; BEZEMER, J.; O’HALLORAN, K. Introducing Multimodality. New York: Routledge, 2016.

JOHANNESSEN, C. M.; TVEDE, M. L.; BOESEN, K. C.; HIIPPALA, T. A Corpus-Based Approach to Color, Shape, and Typography in Logos. In: PFLAEGING, J.; WILDFEUER, J.; BATEMAN, J. A. (eds.). Empirical Multimodality Research: Methods, Evaluations, Implications. Berlin/ Boston: De Gruyter, 2021. p. 159–185.

JOHNSON, J.; KARPATHY, A.; FEI-FEI, L. Densecap: Fully convolutional localization networks for dense captioning. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, Las Vegas, US. Proceedings […]. Las Vegas, USA: IEEE, p. 4565-4574, DOI: https://doi.org/10.1109/CVPR.2016.494.

JUNCTIONS (username). Optical grey squares orange brown. Wikimedia Commons, 10 Apr. 2007. Available at: https://commons.wikimedia.org/wiki/File:Optical_grey_squares_orange_brown.svg. Visited on: 21 Jan. 2023.

KANDINSKY, W. Concerning the Spiritual in Art. New York: Dover Publications, 1977.

KARPATHY, A.; FEI-FEI, L. Deep visual-semantic alignments for generating image descriptions. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, Boston, USA. Proceedings […]. Boston, US: IEEE, p. 3128-3137, DOI: 10.1109/CVPR.2015.7298932.

KINGDON Jonathan. The Kingdon Field Guide to African Mammals. Academic Press: San Diego, California, 1997.

KRESS, G.; VAN LEEUWEN, T. Colour as a semiotic mode: notes for a grammar of colour. Visual communication, [S.l], v. 1, n. 3, p. 343-368, 2002. DOI: https://doi.org/10.1177/147035720200100306. Available at: https://www.researchgate.net/publication/237623450_Colour_as_a_Semiotic_Mode_Notes_for_a_Grammar_of_Colour. Visited on: 21 Jan. 2023.

KRESS, G.; VAN LEEUWEN, T. Reading Images. Geelong, Australia: Deakin University Press, 1990.

KRESS, G.; VAN LEEUWEN, T. Reading Images: The Grammar of visual design. 1 ed. London/ New York: Routledge, 1996.

KRESS, G.; VAN LEEUWEN, T. Reading Images: The Grammar of Visual Design. 2 ed. London/ New York: Routledge, 2006.

KRESS, G.; VAN LEEUWEN, T. Reading Images: The Grammar of Visual Design. 3 ed. London/ New York: Routledge, 2021.

MATUMOTO, A. de O. A construção brasilidade em Street Fighter II: uma perspectiva sociossemiótica. In: GONÇALVES-SEGUNDO, P. R.; ARAES, C. R.; CASTANHEIRA, C.; ISOLA-LANZONI, G.; SILVA, L. P. da; MITSUNARI, N. A. S.; DOMINGUES, T. (eds.). Estudos do texto e do discurso: perspectivas contemporâneas. São Paulo, SP: FFLCH/USP, 2022a. p. 109–133. Available at: https://www.livrosabertos.sibi.usp.br/portaldelivrosUSP/catalog/book/935.

MATUMOTO, A. de O. Sobrevoando a Amazônia: a construção verbo-imagética do Brasil em três jogos shmup. Entrepalavras, Fortaleza, v. 12, n. 3, p. 92–122, 2022b. DOI: https://doi.org/10.22168/2237-6321-32512.

MATUMOTO, A. de O.; GONÇALVES-SEGUNDO, P. R. Towards a social-semiotic approach to visual analysis of two-dimensional games: a toolkit. Texto Livre, Belo Horizonte, vol. 15, no. 1, p. e39398, 2022a. DOI: https://doi.org/10.35699/1983-3652.2022.39398.

MATUMOTO, A. de O.; GONÇALVES-SEGUNDO, P. R. Uma Proposta Sociossemiótica para a Análise Visual de Jogos Bidimensionais: a construção de significado em Ducktales. Entreletras, [S.l.], v. 13, n. 1, p. 344–369, 2022b. DOI: https://doi.org/10.20873/uft2179-3948.2022v13n1p344-369.

PAVLIDIS, G. A Brief History of Colour Theory: Foundations of Colour Science. Cham, CH: Springer International Publishing, 2021.

POSSEBON, E. L. A teoria das cores de Goethe hoje. 2009. 168 f. Doutorado em Design e Arquitetura – Universidade de São Paulo, São Paulo, 2009. DOI 10.11606/T.16.2009.tde-10052010-144639. Available at: http://www.teses.usp.br/teses/disponiveis/16/16134/tde-10052010-144639/. Accessed on: 13 Jul. 2023.

PSIKYO. Strikers 1945 II. Kyoto, Japan: Psikyo, 1997.

RHYNE, T.-M. Applying color theory to digital media and visualization. Boca Raton: CRC Press, Taylor &Francis Group, 2017.

SARAVANAN, G.; YAMUNA, G.; NANDHINI, S. Real time implementation of RGB to HSV/HSI/HSL and its reverse color space models. In: International Conference on Communication and Signal Processing (ICCSP), 2016, Melmaruvathur, India. Proceedings […]: IEEE, p. 0462-0466, DOI: 10.1109/ICCSP.2016.7754179.

VAN LEEUWEN, T. Introducing Social Semiotics. London: Routledge, 2005.

VAN LEEUWEN, T. The Language of Colour: An Introduction. London: Routledge, 2011.

Video System. Aero Fighters 2 (Sonic Wings 2). Kyoto, Japan: SNK, 1994.

Video System. Sonic Wings Special. Kyoto, Japan: Media Quest/Phoenix, 1996.

VON GOETHE, J. W. Zur Farbenlehre. Tübingen, DE: Cotta, 1810. Available at: https://archive.org/details/gri_c00233125012873531. Accessed on: 13 Jul. 2023.

VON GOETHE, J. W. Goethe’s theory of colours. trans. by Charles Lock Eastlake. London, UK: John Murray, 1840. Available at: https://archive.org/details/goethestheoryco01goetgoog. Accessed on: 13 Jul. 2023.

Downloads

Published

2024-10-11

Issue

Vol. 31 No. 4 (2023): Multimodality through the lenses of Kress’ Social Semiotics

Section

Thematic issue 31:4 (2023): Multimodality through the lenses of Kress’ Social Semiotics

How to Cite

A Social-Semiotic Approach to Color Analysis in Digital Media: Theory and Practice. Revista de Estudos da Linguagem, [S. l.], v. 31, n. 4, p. 2009–2065, 2024. DOI: 10.17851/2237-2083.31.4.2009-2065. Disponível em: https://periodicos.ufmg.br/index.php/relin/article/view/55258. Acesso em: 26 dec. 2024.