Multimodal text: the essence, practical value and didactic potential

Размещено на http://www.allbest.ru/

Poltava State Medical University

Multimodal text: the essence, practical value and didactic potential

Leshchenko T.O., PhD in Philology, Associate Professor, Head of the Department of Ukrainian Studies and Humanitarian Training

Zhovnir M.M., PhD in Philology, Lecture at the Department of Ukrainian Studies and Humanitarian Training

Abstract

The paper focuses on the multimodal text as a linguistic and visual phenomenon and a didactic tool. Multimodality is considered as one of the ways of organizing the semiotic system for creating meanings and transmitting information. Didactic multimodal text was defined as a polyfunctional verbal-semiotic formation with a didactic features and goals. The expediency of using the term multimodal text is motivated by the fact that it corresponds to the Ukrainian tradition, accurately and transparently explains the specifics of non-homogeneous linguovisual complexes and can be associated with minimal losses as a modality zone. The main features of didactic multimodal texts were identified and described. The main purpose of the educational multimodal text was outlined. In addition, the rapid growth of the influence of the processes of globalization, digitalization and the strengthening on the reform of the education system in general was pointed out. The research has examined the issue of transforming modern gadgets and smartphones into integral multifunctional attributes of all spheres of modern life, personal and educational. The didactic potential of the considered texts was described and analyzed, in particular, the provision of a concise presentation of significant amounts of information and its perception due to the rapid activation of various cognitive mechanisms that complement and diversify the verbal channel of information delivery; better memorization and deeper understanding of the material through visual, rational and emotional perception. Their current role in learning the Ukrainian language as a foreign language was pointed out. It was proven that the considered educational multimodal texts encourage students to study the topic independently and in detail, contribute to the systematic presentation of the information block, turning into an element of entertainment.

Key words: multimodality, didactic multimodal text, foreign student, Ukrainian as a foreign language.

Анотація

МУЛЬТИМОДАЛЬНИЙ ТЕКСТ: СУТЬ, ПРАКТИЧНА ЦІННІСТЬ І ДИДАКТИЧНИЙ ПОТЕНЦІАЛ

У праці зосереджено увагу на мультимодальному тексті як лінгвовізуальному феномені й дидактичному засобі. Мультимодальність авторками дослідження розглянуто як один зі способів організації семіотичної системи для створення смислів і передачі інформації. Навчальний мультимодальний текст у розвідці дефіновано специфічним поліфункціональним вербально-семіотичним утворенням, якому притаманна дидактична спрямованість. Умотивовано доцільність послуговуватися терміноодиницею мультимодальний текст - як такою, що відповідає українській традиції, точно та прозоро експлікує специфіку немогогенних лінгвовізуальних комплексів. Вирізнено й описано основні ознаки дидактичних мультимодальних текстів. Окреслено провідну мету навчального мультимодального тексту. Крім цього, у статті зауважено про стрімке зростання впливу процесів глобалізації та діджиталізації на реформування системи освіти загалом. У розвідці також розглянуто питання перетворення сучасних гаджетів та смартфонів на невід'ємні мультифункціональні атрибути усіх сфер сучасного життя, зосібна й освітньої. Зауважено про їхню посутню роль у вивченні української мови як іноземної. Описано й проаналізовано дидактичний потенціал розглядуваних текстів, зокрема забезпечення стислого представлення значних обсягів інформації та її сприймання за рахунок швидкої активізації різних пізнавальних механізмів, які доповнюють й урізноманітнюють вербальний канал подачі інформації; краще запам'ятовування й глибше осмислення матеріалу через сприйняття візуальне, раціональне й емоційне. Доведено, що розглядувані навчальні мультимо- дальні тексти, зокрема й представлені за допомогою сучасних смартфонів, заохочують студентів до самостійного й детального вивчення теми, сприяють системній подача інформаційного блоку, перетворюючись на елемент едьютейменту.

Ключові слова: мультимодальність, навчальний мультимодальний текст, іноземний здобувач освіти, українська мова як іноземна.

Problem statement

Increasing globalization has caused visible changes in all spheres of peoples' modern life, particularly it has influenced on the social, cultural and economic relations. The advance information and communication technologies, as well as intercultural and interlanguage communication have been fostered. In addition to this, traditional education system was greatly influenced and modified.

Mobile devices have rapidly integrated into all fields and spheres of human activities, including education. Therefore, new digital reality became an irreplaceable part of every person's life within Ukraine and all over the world. Nowadays young people cannot imagine their everyday life without advanced gadgets and opportunities that they normally provide. More and more students consider accessing their learning material from the mobile devices essential.

By frequent utilizing existing gadgets without wires students are able to access to different information resources, develop their vocabulary skills and more complicated vocabulary through simulated games, intellectual and experiential learning, learning in pragmatic and real-life situations with ease.

Active use of the educational potential of cutting- edge mobile devices provides both teacher's and student's benefits to drive positive and visible change in learning and teaching. Mobile learning also provides obtaining information, exchanging training material in group mail, saving a record in an informal and continuous of this process. In this case, we can confidently assume that the mobile device makes the necessary ground for the learners and their teachers to interact socially and informally.

Under the influence of digitalization traditional ideas about the methods and outlined educational goals are being transformed, because modern students more often refuse from academic, partly formalized and outdated, training providing according to the program of a particular educational discipline.

Successful implementation of didactic tasks largely depends on methods and tools applied by teachers. However, in our opinion, the result and effectiveness of the classroom / extra-curricular activities are determined by the motivation of students to learn and memorize new information. Teaching process is effective only if it is based on methods and techniques that activate the learner' activity, and is aimed primarily at finding new ways to actualize each student's inner potential, developing skills independent cognitive search, students' mastering the ability to organize and regulate speech activity, developing the ability to systematically master communication skills, because continuous personal development of all students is a key component of our corporate (teacher - student) success.

In order to ensure external motivation to study the Ukrainian language in a medical higher education institution and the Ukrainian language, which is foreign to the English-speaking contingent of applicants for higher medical education in Ukraine, tutors must clearly and carefully outline their main educational goals, present the material in an accessible form adapted to the students' level of Ukrainian. It also does not seem to have the last chapter on teachers' ability to demonstrate own knowledge and professional qualifications. Modern tutor has to do the best to create emotionally favorable atmosphere for the assimilation of given facts and formation of basic skills, as well as activation of critical, imaginative thinking and cognitive activity of all students.

Modern educator is experimenting with general means and traditional algorithm of presenting educational information, appealing to innovative didactic methods and techniques, adapting to visible internal and crucial external changes of students, their needs and updated goals. We consider the thought of the researcher T. Leshchenko to be quite convincing: «Rational implementation of various information and communication technologies via extensive paradigm of visualization tools in the university educational process should contribute to ensuring the proper level of presentation and monitoring of educational infor- mation» [6, p. 79]. multimodal text didactic semiotic

We are convinced that the rational structuring and presentation of educational material with the help of multimodal text interspersions provides a concise presentation of significant amounts of information and its perception due to the rapid activation of cognitive mechanisms that complement and diversify the verbal channel of presenting information.

Analysis of recent research and publications

The main controversial issues, very desperate problems, which directly or indirectly related to visualization technology in education, are considered in the scientific researches of P. Anokhin, O. Asmolov, F. Bartlett, O. Betsko, O. Bigich, H. Burmenska, A. Verbytsky, I. Volodarska, V. Davydova, Z. Kalmykova, etc. The theoretical and methodological basis for the further thorough analysis and description of the educational space visualization issues were presented in numerous writings (L. Bilousova, L. Bilyavska, D. Bezugliy, Butenko, N. Zhityenova, O. Mansurov, M. Manko, O. Mileyko, V. Osadchyi, T. Soroka, B. Starychenko, M. Tsymbalyuk, V. Shvyrka, S. Shushkevich, V. Shcherbatiuk, etc.).

The didactic potential of multimodal texts was considered in the studies of domestic and foreign scientists. The increasing number of research has appeared that highlights the possible and appropriate ways in which multimodal texts may be used in language teaching and learning. Several ofthe researches are dedicated to the investigation of the challenges and benefits of mobile technology and its prospects for foreign languages learning, its peculiarities and capability (L. Baidak, N. Volynets, Z. Batrynchuk, O. Beshlei, I. Grigorenko, M. Kozdra, O. Krekoten, M. Kuznetsova, T. Kravchinska, M. Skrypnyk, M. Shibeko, A. Ferrari, R. Hobbs, A. Serafini, L. Shanahan, etc.).

The aim of the study. In this paper, we focused on the advantages and possibilities to use didactic multimodal texts in the Ukrainian language teaching process in the medical higher education institutions. The paper attempts to address the following research questions:

What is the essence of the term didactic multimodal text?

What are the main didactic features of didactic multimodal texts and their capabilities?

What are the benefits and challenges the foreign learners face while learning Ukrainian with the usage of didactic multimodal texts?

The research was performed within the anthropocentric scientific paradigm. General linguistic method of scientific description was used as basic. The paper also appeals to the generalization, systematization, classification, interpretation of linguistic scientific fact. The descriptive-analytical method was used to analyze and systematize the features of multimodal texts implementation into the educational process. multimodal text didactic semiotic

Presentation of the main research material

More recently new terms, in particular multimodal text and multisensory teaching and learning, came into common use. Both approaches refer to utilizing more than one sense to force and boost learning process. In addition to the traditional senses we can add other senses such as touch, smell, taste, and movement. Consequently, vision, auditory, tactile, olfactory, gustation, kinetic could be easily deployed in teaching. In this paper we concentrate on the multimodality connected with sight and hearing.

Researchers consider multimodality as one of the ways to organize semiotic system in order to create meanings and convey information. This traditional interpretation in scientific discourse functions in parallel with the idea of multimodality as a new genre that combines text with graphic images, audio or video visual effects. It is the combination of modalities with the latest digital tools that are actively used in the current learning environment.

In the context of multimodal linguistics, any text can be interpreted as multimodal / CTeolized text. It is worth mentioning, that multimodal discourse was originally defined and described in the research of Kress and T van Leeuwet as «a text that contains different semiotic systems» [5, p. 5]. Later the considered phenomenon attracted the attention of many scientists (O. Anisimova, I. Bilyuk, V. Berezia, Ya. Bystrov, N. Valgina, N. Volynets, L. Golovina, M. Ishchuk, A. Kybrik, O. Korda, T. Kravchinska, Miroshnychenko, E. Nezhura, O. Selivanova, Yu. Sorokin, I. Uschapovska, O. Fedorova, etc.). They mostly tried to clarify the definition and outline the specifics of the new linguistic phenomenon.

Despite the fact that in the linguistic tradition there is no unified approach for investigation of this issue, its theoretical understanding, formulation of the definition and constitutive features inventory of multimodal text formations, numerous definitions of the above-mentioned concepts have been shown and clarified in the research of domestic and foreign linguists. The following are examples of some of them: «A creolized text is a complex text formation in which verbal and iconic elements form a visual, structural, semantic and functional integrity, tainted by a complex pragmatic impact on the addressee» [1, p. 17]; «A polycode text is a combination of a natural language (linguistic) code with the code of another semiotic system» [3, p. 105]; «Paragraph means also include realized texts in which some of the graphic elements are iconic (illustrations, graphics, diagrams, images)» [2, p. 81]; «Multimodal written (printed) texts are texts created by involving several different modes or semiotic resources, the totality of which forms the overall appearance of the text and produces its meaning» [4, p. 100]. Consequently, cre- olized or multimodal text is a text consisting of two different parts, which are commonly known as verbal and non-verbal. These parts cannot exist and be used separately from each other. Due to the involvement of verbal and non-verbal phenomena in the multimodal text an additional semantic work is involved as certain background knowledge is used, which students refresh when reading the text.

Multimodal text in our view is the most flexible and appropriate of the existed terms that are used to denote multimodal textual phenomenon, in particular multimodal text, creolized text, video-verbal text, polycode text, linguovisual complex, semantically complicated text, isoverbal complex (isoverb), heterogeneous (non-heterogeneous text), «text of a new sense», etc. And, just as importantly, that our chosen term corresponds to authentic Ukrainian tradition accurately and transparently explains all specific features of non-homogeneous linguovi- sual complexes and therefore can be associated with the modality zone with minimal losses and incorrectness.

We propose to interpret multimodal text as a complex integral polysemiotic formation with a visible pragmatic effect. This conclusion can be drawn based on observations and relying on the researchers' opinions. The analyzed type of text is formed from two inhomogeneous parts: verbal (linguistic) and non-verbal (iconic) materializing with the help special symbolic paradigms. Multi-coded, verbal and iconic components demonstrate the integrity and coherence of the multimodal text and strengthen its communicative effect. Such types of texts may be partially and totally creolized. Partial text cre- olization includes coherence of verbal and visual components. Removing an image in such texts won't change or misrepresent the content, because the verbal component is considered as more important and then the non-verbal one. It is irreplaceable component. Integrating verbal and non-verbal elements fully depend on the visual element; therefore, its removing may caused misconception, wrong perception or total incomprehension of the content. Among the main characteristics of creolized texts the following are worth mentioning: message, comprehensibility, informativeness, pithiness, image correspondence with the text content, and visual correctness.

It is common known that the main purpose of educational multimodal text is didactic. Teacher creates new texts using a combination of verbal and non-verbal codes, as well as uses authentic ones so as to achieve the outlined didactic goal and effective pragmatic influence on the a student. It also must be adapted to the age, professional, acquired and knowledge, abilities and skills of the students. In addition, didactic multimodal text should contribute to the development of basic competences, learning by perception, understanding and understanding of information, its interpretation and reproduction.

Multimodal projects, involving home-made texts, can broaden a diversity of assessment techniques as they allow for complex estimation of gained language skills and competences. With the help of thematic multimodal texts students have the opportunity to understand main role of non-verbal and paralinguistic phenomena in communication. They became aware with its rules and features.

Multimodal text primarily performs the necessary educational information. In addition, it is normally created to provide students with the opportunity to distinguish, interpret and synthesize facts presented both verbally and graphically. Via using the considered texts the result, such as better perception, deep understanding and memorization of information, can be achieved through a rational combination of visual, rational and emotional perception.

Multimodal texts that are actively used in the educational process include posters, diagrams, tables, thematic pictures, cards, maps, illustrative fragments, network memes, comics, collages, motivators / demotivators, booklets, posters, logos, book trailers, scrapbooking, «doodles», samples of educational infographics, etc.

In each of the samples mentioned above, verbal and non-verbal components function in a single semantic continuum, forming a coherent multimodal text, a didactic product that, under the condition of rational use during training, activates the his / her memory, thinking activity, and imagination. It has a positive effect on motivation, independence and the possible research perspective of each student in general.

We consider the combination of several types of multimodal texts containing educational content to be completely justified and didactically motivated, because it is necessary to convey the content with the help of visual representations, realistic and associative images, color splashes, graphic schemes, drawings, slides, photos or graphs, as well as a carefully selected verbal sighs, because it is much easier to describe and interpret the essence and specifics of the topic under consideration. At the same time, the presented didactic multimodal texts should not duplicate the educational material, but, on the contrary, supplement and expand its content. Multimodal educational texts are not only a convenient form of information transmission, but also a new way of learning it.

Using didactic multimodal texts in Ukrainian language teaching helps to create a positive motivation to learn Ukrainian language; organize teaching process with the usage of basis of the individual academic patterns; enhance students' self-assessment of knowledge, skills and abilities; optimize the learning process and manage time.

Consider for example the process of fostering their Ukrainian language skills related to vocabulary via using didactic multimodal texts. In the algorithm we can highlight few essential core stages such as preparatory phase - introduction to the topic of discussion, activation students' background knowledge and updating their life experience, as well as studying the characteristics of the given multimodal text; content phase - acquaintance with the basic distinguish features and semantic interpretation of material; comparative analysis - comparison with having knowledge in this field or topic; repetition and memorization. Except the mentioned, project activity can be add as a way to develop or boost conceptual and visual thinking and creativity.

Our several-year practice of distance education of foreign students, future specialists in the medical field, testifies to the effectiveness of using mobile variants of didactic multimodal texts when learning Ukrainian as a foreign language. Foreign students who live and study in Ukraine have to learn Ukrainian Language as a Foreign, because it is used as educational and official languages. The progress of students in Ukrainian language education depends on many rates, particularly their social and cultural context, educator employment, curriculum, resources and facilities, the existence of a medium for free and unhindered communication, etc. Ukrainian pronunciation and some grammar rules are very hard to understand, especially the students from India and Pakistan.

Presenting different visualization tools, particularly didactic multimodal texts via smartphones, provides interaction between learners with a complete organized knowledge database activity is best suitable for learning and sharpening language skills. It is easy to choose the appropriate one for educational purposes - from the formation of basic, phonetic, lexical, grammatical, skills, development of skills, speaking, reading, listening, writing, to translating words and even text fragments, correcting pronunciation, writing, etc.

Using didactic multimodal texts involving advanced gadgets offer numerous considerable advantages, but also some drawbacks. Most of them were revealed by students and relate to some technical problem like internet signal, out of date phone, etc.

Conclusions

Multimodal text is a complex integral polysemiotic formation with a visible pragmatic effect. It consists of two different parts, which are commonly known as verbal and non-verbal. The main purpose of didactic multimodal text is didactic. Teacher creates new texts using a combination of verbal and non-verbal codes to achieve the outlined didactic goal and effective pragmatic influence on a student. In order to realize the intentions of the addressee the multimodal text must be endowed with didactic potential, fully subordinate to the purpose, contain language material in the verbal part, broadcast relevant linguistic and sociocultural information.

Since smartphones nave became sophisticated, and multifunctional, there are opportunities for learners to engage in more meaningful Ukrainian language learning. Using didactic multimodal texts in teaching Ukrainian, particularly online classroom, is the best way toward enhancing students motivation, diversifying offline or online learning process, and developing learning language skills. Presenting didactic multimodal texts via smartphones enhance free independent learning and add unexpected element of entertainment to the learning experience.

Via using the considered texts the result, such as better perception, deep understanding and memorization of information, can be achieved through a rational combination of visual, rational and emotional perception. Most of the drawbacks are related to technical problem.

Bibliography

1. Анісімова О. Є. Лінгвістика тексту та міжкультурна комунікація (на матеріалі креолізованного текстів). М.: Видавничий центр «Академія», 2003. 128 с.

2. Батринчук З. Р Креолізований текст як параграфемний елемент у сучасному англомовному епістолярному дискурсі. Закарпатські філологічні студії. Ужгород : Видавничий дім «Гельветика», 2018. Вип. 3. Т. 1 С. 80-84.

3. Ейгер Г.В. К построению типологии текстов. Лингвистика текста: материалы научной конференции при МГПИИЯ им. М. Тореза. Ч. І. М., 1974. С. 103-110.

4. Макарук Л.Л. Паралингвистические средства как композиционные составляющие современных англоязычных мультимодальных текстов. Актуальні питання іноземної філології. 2016. Вип. 4. С. 100-104.

5. Kress G. R. The multimodal landscape of communication. Medien Journal. 2002. V. 4. P. 4-19.

6. Leshchenko T. Visualization tools: using word clouds in teaching «Ukrainian as a Foreign Language». Engineering and Educational Technologies. 2020. Вип. 8 (2). Р 79-91.

Размещено на Allbest.ru