Developing test tasks to assess the components of prospective philologists’ translation competence

кандидат педагогічних наук, доцент кафедри української, іноземних мов та перекладу Вищого навчального закладу Укоопспілки «Полтавський університет економіки і торгівлі» (Полтава, Україна)

Розроблення тестових завдань для оцінювання сформованості у майбутніх філологів окремих компонентів перекладацької компетентності

З огляду на складну й комплексну природу перекладацької компетентності, як поточний, так і підсумковий контроль сформованості окремих її компонентів передбачає розроблення й застосування різноманітних засобів і процедур. Із цієї причини статтю присвячено дослідженню перспективних шляхів та потенційних можливостей для здійснення об'єктивного тестового контролю рівня сформованості у майбутніх філологів перекладацької компетентності загалом та певних її компонентів зокрема. Прагнення мінімізувати суб'єктивність тестового контролю є, вочевидь, пріоритетним як для навчання іноземнихмов, так і викладання перекладу. На основі сучасних досягнень лінгводидактики в галузі тестування рецептивних і продуктивних мовленнєвих умінь автор намагається екстраполювати та адаптувати відповідні ідеї до потреб контролю сформованості у студентів перекладацької компетентності. При цьому ключове протиріччя полягає у необхідності одночасно отримувати спонтанні та невимушені зразки реального виконання студентами перекладацької діяльності та об'єктивно оцінювати їх, що майже практично неможливо у разі використання тестових завдань із вільноконструйованою відповіддю через низку чинників. Тоді як значно об'єктивніші з погляд оцінювання тестові завдання альтернативного, множинного й перехресного вибору обмежують можливості студентів продемонструвати реальну здатність і готовність до виконання перекладацької діяльності. Реалізовано спробу розроблення різноманітних тестових завдань для відстеження засвоєння майбутніми філологами як декларативних, так і процедурних перекладацьких знань, а також для організації й спрямування ефективного процесу перекладу на різних етапах навчального процесу. Наведено й прокоментовано приклади тестових завдань, покликаних задовольнити різні потреби в процесі навчання перекладу. Визначено їхню позитивну роль як допоміжного засобу контролю, спрямованого на підвищення об'єктивності оцінювання та посилення формувального впливу у загальній системі контролю сформованості у студентів перекладацької компетентності, що має бути емпірично перевірена та доведена у ході подальших досліджень.

Ключові слова: контроль, об'єктивний та суб'єктивний тестовий контроль, тип і вид тестового завдання, перекладацька компетентність, декларативні перекладацькі знання, процедурні перекладацькі знання, майбутні філологи.

Main part

Introduction. In order to respond growing demand for philologists skilled enough to provide competitive translation and interpretation services in different domains foreign and domestic scholars carry out a great number of theoretical and practical researches directed to the enhancement of prospective translators' university training. Due to the intricate and congruent nature of translation competence, its efficient assessment requires even more attention. Performing both formative and summative functions assessment procedures should consistently provide valid and reliable data on students' translation knowledge and skill acquisition evaluating objectively their learning outcomes from different perspectives and encouraging them with motivational and constructive feedback. Taking into account the idea of translation triangulation (cross-assessment) (Hansen, 2008) and up-to-date trinary assessment trends (Casanova, 1997) lots of varied assessment techniques and procedures to gauge concurrently translation process, product and service at the relevant translation training stages (Kockaert & Segers, 2017) should be suggested and developed. Being combined with the terms of modern translation training in Ukrainian universities such as in and out of class activity distribution andparallelacquisitionofforeignlanguageand translation competence by the students (Chernovaty, 2013) the development and application of various testing techniques appears to be quite promising and efficient. These assessment tools are widely used in modern linguodidactics but still have quite limited employment in translation pedagogy. Most of the scientific sources insist on the impossibility of objective translation performance testing and prove it with the help of relevant empirical data (Ahmadi, 2011), while subjective open - ended translation tests graded with the help of versatile procedures and scales are still common. That is why the idea of the development of objective translation tests to complement the results of subjective ones and provide tools for productive self - and peer assessment appears to be quite topical in modern translation pedagogy.

So, the aim of the given paper is to study and realize testing potential for the improvement of translation competence assessment efficiency in prospective philologists' university instruction solving the following tasks: a) to analyse and adopt relevant testing concepts and ideas from modern linguodidactics; b) to develop and present the examples of the test tasks aimed at the promotion and regulation of the source text analysis procedures; c) to determine the opportunities of objective testing techniques for the assessment of particular declarative and procedural translation knowledge; d) to sum up further testing implications for the assessment of other translation competence components.

Theoretical backgrounds. Testing belongs to one of the most contradictory and still widely used assessment tools in modern linguodidactics. Its different theoretical and practical aspects were thoroughly investigated in the context of the development of testing tools for the assessment of the foreign language communicative competence acquisition in general (D. Douglas, 2000; O. P Petraschuk, 2000; F. Davidson, 2002; M. Milanovic, 2002; L.F. Bachman, 2004; J. Ch. Alderson & M. Cseresznyes, 2005; N. Yu. Hutareva, 2005; J. Boyle, 2007; R. Kopriva, 2008; O.H. Kvasova, 2009; A. V Drozdova, 2010; G. Fulcher, 2009; L. Lou, 2019; E.S. Tkachenko & I.A. Bredikhina, 2019; Sh. White, 2019). These works contributed greatly to the determination of testing aims, objectives and functions in the training process; suggested testing technique classification; described the procedures of the test task development and verification; determined communicative competence testing constructs and objects.

Speaking about testing competences in receptive speech activities, works devoted to testing listening-comprehension skills (O.V. Kmit, 2000; Ya. A. Krapchatova, 2014; P. Abeywickrama, 2018) and reading ones (J. Ch. Alderson, 2000; N.O. Brazhnyk, 2005; T I. Kushnariova, 2006; J. Sh. Caldwell, 2008; L.A. Kozhevnikova, 2009; T H. Korol, 2012; F. Arung, 2015) should be mentioned. Initially, testing students' competences in productive speech activities is considered to be more challenging than receptive ones. Therefore, the number of works devoted to speaking competence testing (O.O. Molokovych, 2001; O.O. Ukraiinska, 2009; Ye. V. Zarutskaia, 2012; F. Fulcher, 2014; M. Pawlak & E. Waniek-Klimczak, 2014; E. Szymanska-Czaplak, 2015; V.L. Vasilevskaia, 2016; A. Ugiljon, 2018) and writing (D. Yu. Teleshev, 2008; M.N. Melissourgou, 2015) is of comparatively small account. For fairness' sake it should be stressed that testing speaking and interpreting as well as testing writing and translating from the viewpoints of both communicative and activity-based approaches have lots in common. Just imagine the given source text as the translation or interpretation assessment means accompanied with the rubric in the form of translation brief. Translation product reflecting its performance is assessed on the basis of the rating scales by the rater /raters. All this corresponds to the typical integrative open-ended test in either writing or speaking. The assessment procedures are also similar. The only difference lies in the strict determination of the translation product properties by the features of the source text. Compared to speaking or writing ones, when they should be freely produced according to the test task, in case of translation or interpretation they should be reproduced according to the translation brief. Consequently, the received translation product evaluation involves the examination of the quality of this reproduction. It can be conducted with the help of either holistic (M. Garant, 2009; M. Williams, 2013), analytic (C. Han, 2017; A. Mendoza, 2018), error analysis in particular (C. Waddington, 2003; G. Hansen, 2009; G.D. Dewi, 2017, L. Chernovaty, 2013), and combined scales (G. Ghalib, 2015; M. Amini, 2018). The main problem with such assessment method is its high subjectivity and, as a result, insufficient reliability.

No wonder, that a lot of modern inquiries in the field of both linguodidactics and translation pedagogy are directed towards the search of the solution to this problem. Keeping in mind the objectives of this paper to present the perspectives of testing tool application in translation training, let us consider possible ways of testing techniques use in teaching translation to prospective philologists.

Findings. The ideas mentioned above can be easily adapted for the needs of translation competence testing, e.g. for the arrangement of either formative self - peer - or hetero assessment at the stage of the source text analysis in the process of translation, which is often neglected or even skipped by the students, as our teaching practice reveals. However, it is a well-known fact that adequate source text information processing and interpretation are essential to further target text formulation and editing, especially in case of specialized translation performance (Korol, 2019: 449). When students face a lot of problems caused by the lack of relevant background knowledge and difficulties connected with the processing of terminological units. So, at the initial stages of specialized translation training we suggest to apply teacher developed test tasks aimed at the monitoring of the source text comprehension correctness and fidelity by the students and partial elimination of the source text interpretation problems: alternative choice test tasks (true / false / not mentioned); multiple-choice gap-filling and answering the questions; different matching and rearrangement test tasks; open - ended gap-filling and answering the questions.

These test tasks can be formulated and presented in either the source or target languages. In both cases they will ensure students' individual performance and academic integrity, prevent trivial cheating (It should be noted here that due to the lack of in-class time at university translation training in Ukraine the majority of translation practice activities have to be performed by the students at home and submitted to the teacher according to the set deadlines) and direct students' translation activity providing some kind of guidelines and half way feedback at the stage of the source text analysis. Moreover, in the first case the given test task will be targeted to the reading-comprehension assessment only, while in the second one it will involve some elementary translation activity, on the one hand, being a bit more complicated, requiring from the students language switching skill application, and facilitating target text formulation stage providing some readymade translation solutions, on the other. The received testing results will make it easier to find, explain and overcome in the future the reasons of the mistakes and errors in the target text caused by the failures at this stage of the translation process. In our opinion, such test tasks can also promote the formation of students' selfcontrol and self-assessment skills. At first they can be checked up by the teacher or automatically by computer in case of Moodle platform use or other didactic testing software, for instance. Later students can be provided with the keys accompanied with some comments and explanations for self-assessment and, finally, their peers can check each other tests on the basis of the given keys or even their own judgement.

For example, in the context of translating English texts for the financial sector which are saturated with some precise and extralingual information usually combined with specific terminology, we suggest to use open-ended gap-filling based on the calculations carried out on the information given in the source text or chart / table filling directed to its restructuring and summarizing. In this case, students perform information decoding operations based on psycholinguistic grounds similar to translation ones. To our mind, such activity helps them understand the text material better transforming and decoding it for other activity performance and look at usually `hostile' to them figures and specialised terms from different point of view. The example of such source text based reading-comprehension test task is given below.

For example, You are going to translate the English text presenting company balance sheet into Ukrainian, read and analyse it properly, after that fill in the given table with the appropriate figures:

It should be noted here that this test task requires not only eliciting explicitly given in the source text figures and matching them with the balance sheet items but also doing some maths on the basis of some implicitly presented information in the text. Analytical operations required to perform this test task provide deeper insight into the source text content, sense and even author's intentions.

Preliminary trial of such test tasks use proved their positive effect on the quality of the target texts produced by the students compared to the translations performed by the students who were not asked to perform such tasks at the source text analysis stage of the translation process. Being based on the keys with clearly determined correct answer /answers such test tasks are definitely objective, informative to both students and teachers, appropriate for formative assessment procedures in the translation training process as well as for summative control as a supplementary tool applied after target text submission by the student. In this case, testing results provide the rater with some extra information which helps achieve higher objectivity of translation product quality evaluation giving insights of the nature of the students' mistakes and promoting their interpretation.

As it was mentioned before, testing objectivity depends on the type of the expected response (open - ended or with given options) and rating method. In this case all open-ended, in other words, productive or even reproductive and transformative by nature translation and interpretation tasks belong to subjective tests. Since they are not rated on the basis of the beforehand determined objective keys but depend on the rater's subjective judgement even being based on some guidelines in the form of the suggested scales. Does it mean that such subjective tests are not used in high-stake assessment cases? Vice versa, just have a look at the China Accreditation Test for Translators and Interpreters review as an example (Zhao & Xiandong, 2016).

Different attempts were taken to study the opportunity to replace subjective translation testing with objective one. However, no significant correlation between students' performance on translation production and different multiple-choice translation tests was fixed (Ahmadi, 2011).

In such a way, there is no real opportunity to substitute subjective translation performance test for objective key-based one yet. The main explanation lies in the difference of the constructs they gauge. The first one provides information on the students' overall translation performance in terms of a particular text. In spite of the subjective nature of the raters' judgement on the efficiency and quality of its completion this assessment tool elicits the testee's behaviour patterns indicating their ability to translate. While the key-based test usually covers only particular aspects of translation performance such as the ability to evaluate the translation version adequacy and equivalence, to spot and even correct the mistakes in the target text closely connected with the revision and editing skills. As a result, they can be efficiently applied as supplementary assessment tools in the process of prospective philologists' training rather than the major ones.

The process of translation competence acquisition is long and complicated, where each stage should be thoroughly monitored and assessed. In this case, objective test tasks can be of great use and help since they work pretty well on small testing objects. For example, they seem to be an ideal and economic tool for the assessment of students' theoretical knowledge acquisition (Tsygan, 2010) as the important prerequisite for the development of proper translation sub-skills and skills necessary for the efficient translation performance.

Different types of key-based test tasks can be developed and applied with this aim for the arrangement of either self - peer - or hetero assessment. For instance, test task in matching split sentences representing translation theory aspects based on students' mnemonic processes, reproduction, vocabulary unit combination and grammar structure anticipation skills. The last ones are considered to be an important prerequisite for the successful translation and interpretation performance.

For example, Match the beginning of each sentence (1-11) with its ending (A-K) to provide the key information on the translation equivalence concept.

The next test task in open-gap filling is also objective by its nature but still requires higher degree of mental and cognitive activity from the students. It involves not only theoretical knowledge check-up but also simple translation solution on their basis as well as trains students' anticipation skills extremely important for both translation and interpretation performance. Its level of difficulty can be reduced with the help of the bank of patterns to be used for filling.

For example, Fill in the gaps with proper information (either a word or word combination or whole definition).

The term `(1)' refers to a local word or phrase rendering (2) that doesn't translate because there is no obvious equivalent in the(3). An

(4) would be the fjords in the Scandinavian languages. In general, there are(5) of them for handling realia while translating. The first one we apply to translate the word `fjord' into Ukrainian, namely(6)….

Taking into account the results of previously carried out research (Muratova, 2006), we consider objective test tasks to be an efficient supplementary assessment tool. Their concurrent uses with the subjective translation production test tasks can significantly lower assessment subjectivity. That is why, we decided to supplement our translation - and interpretation-based tasks for the exam in Translation for the Financial Sector with the set of objective multiple-choice test tasks. The developed tasks were based on the authentic text fragments on financial and banking issues. According to their testing objects they were subdivided into the following groups:

1) test tasks aimed at the assessment of polysemantic term recognition and processing based on the proper context analysis. For example, Choose the sentence (a-d) where the word `interest' means the extra money that you must pay back when you borrow money;

a) How much are the monthly interest payments?

b) My parents encouraged my interest in banking.

c) The regulations were introduced to safeguard the interests of local banks.

d) We've got to balance economic interests and environmental interests.

(Here and further in the text the correct answer is underlined).

2) test tasks aimed at the assessment of the skill to apply particular translation transformation based on the relevant procedural knowledge acquisition measured on the separate word combinations. This test task completion also checks up term acquisition in the financial domain by the students, requires good working memory and attention span. Since all the equivalents and the ways of the rendition should be kept in student's working memory and these cognitive operations should be coordinated. For example, Choose the word list (a-d) in which all the financial terms and expressions are translated into Ukrainian with the help of calque:

a) the Board of Directors, Annual General Meeting, current account, savings account;

b) deposit account, to go into the red, run the bank; bank branch;

c) commodity money, legal tender, certified check, issue notes;

d) national currency, accept a bill, rate of interest, standing order.

3) test tasks aimed at the assessment of the skill to apply a particular translation transformation measured on the sentence level. For example, Choose the sentence (a-d) which traditionally should be translated with the help of omission:

a) Shares and stocks are the units of investment in individual companies.

b) Generally, the words `check' and `cheques' are used interchangeably.

c) Which job in the bank offers the best opportunities to exercise one's own judgement?

d) The Board may distribute dividends to the shareholders out of the profits once or twice a year.

In this case the testee should retrieve this term definition from the long-term memory, spot relevant translation problems in the given source text, produce in the working memory their possible translation versions and analyse them.

4) test tasks aimed at the assessment of the parallel skill to apply several translation transformations based on the ability to solve translation problems with high mental load. For example, How many times can omission be applied for the adequate translation into Ukrainian of the given English sentence: The places where the stocks and shares of listed or quoted companies are bought and sold are called stock markets or stock exchanges.

a) three; b) two; c) one; d) none.

The mechanism of this test task successful completion is overcomplicated due to additional task to count the number of all possible cases of the given translation transformation application.

Such test tasks can be presented to students both on paper and on the computer screen. To our mind, besides assessing declarative and procedural translation knowledge and some skills to apply them such test tasks can help to develop their concentration span, short-term and working memory, provide with the experience of taking translation decisions under time pressure. These psychophysiological properties are extremely important for future interpretation performance. It should be noted here that the development of such test tasks is time and effort consuming. However, they help save time at the grading stage due to key-based or even computer-assisted procedures.

Conclusions and prospects for further research. The carried out investigation has shown the prospects for key-based test task use as an objective tool for the assessment of specific components of prospective philologists' translation competence as supplementary assessment tool in terms of Ukrainian universities. They can be applied in different ways at translation training process. Test tasks in reading should be utilized for guiding and monitoring source text analysis stage of translation performance. Separate test tasks should be used for the assessment of translation theoretical knowledge acquisition. Finally, multiple choice test tasks aimed at the assessment of procedural translation knowledge acquisition and skills to apply them were suggested and developed on the basis of English financial texts. They combine objective grading procedures and promote latent translation phase encouraging students to translate with the use of particular translation transformations in order to answer the item correctly. They can differ according to the linguistic domain specific material applied, volume of the source text to be processed by the testee (set of word combinations or sentences), translation knowledge content and their use and cognitive load determined by the number of operations involved. Their reliability and practicality should be verified. As well as further investigation should concern their role and functions in the general system of prospective philologists' translation competence assessment.

References

test control philologist

1. Muratova I.A. Soderzhanie i format sertifikacionnogo jekzamena sudebnyh perevodchikov kak metodicheskaja problema. Avtoref. diss. kand. ped. nauk [Content and Format of the Certification Examination for Court Interpreters as a Methodological Problem]. Extended abstract of PhD dissertation, 2006, Moscow, Russia [in Russian].

2. Tsygan L.V. Eksperymental'na perevirka efektyvnosti metodyky kontrolyu rivnya sformovanosti usnoyi perekladacz'koyi kompetenciyi u majbutnix filologiv. [Verification of the Assessment Methodology of Future Philologists' Interpretation Competence Level]. Visnyk Kyyivs'kogo nacional'nogo lingvistychnogo universytetu. Seriya Pedagogika ta psykhologiya. - Visnyk KNLU. Series «Pedagogy and Psychology», 2010, Nr 18, pp. 130-139 [in Ukrainian].

3. Korol T.H. Perevodcheskij analiz teksta v sovremennoj praktike i didaktike perevoda. [Translation-oriented Text Analysis in modern Translation Practice and Didactics] Inostrannye jazyki: innovacii, perspektivy issledovanija i prepodavanija [elektronnyj resurs]: materialy ІІ Mezhdunar. nauch.-prakt. konf., Minsk, 21-22 March 2019. URL: http://elib.bsu.by/handle/123456789/218897. [in Russian].

4. Chernovatyy L.M. Metodyka vykladannya perekladu. [Methods of teaching translation]. Vinnytsya: Nova Knyha, 2013. 376 p. [in Ukrainian].

5. Ahmadi A. On the Validity of a Multiple-choice Translation Test as a Substitute for an Open-ended Translation Test in the Iranian University Entrance Examination. Perspectives: Studies in Translatology, 2011, Nr. 19 (4), pp. 307-314. https://doi.org/10.1080/0907676X.2011.556254

6. Casanova M.A. Manual de Evaluacion Educativa. Madrid: Editorial La Muralla, 1997. 254 p. [in Spanish].

7. Hansen G. Triangulierung im Unterricht: Ubersetzungsprozesse, Eigenkorrektur und Fremdkrrektur. Informatologia, 2008, Nr. 41 (2), pp. 105-115.

8. Kockaert H.J., Segers W. Evaluation of legal translations: PIE method (Preselected Items Evaluation). Journal of Specialised Translation, 2017, Nr. 27, pp. 148-163.

9. Zhao H., Xiangdong G. China Accreditation Test for Translators and Interpreters (CATTI): Test review based on the language pairing of English and Chinese. Language Testing, 2016, Nr. 33 (3), pp. 439-444.

Література

1. Муратова И.А. Содержание и формат сертификационного экзамена судебных переводчиков как методическая проблема: автореф. дисс…. канд. пед. н.: 13.00.02. Москва, 2006. 21 с.

2. Циган Л.В. Експериментальна перевірка ефективності методики контролю рівня сформованості усної перекладацької компетенції у майбутніх філологів. Вісник Київського національного лінгвістичного університету. Серія «Педагогіка та психологія». 2010. Вип. 18. С. 130-139.

3. Король Т.Г. Переводческий анализ текста в современной практике и дидактике перевода. Иностранные языки: инновации, перспективы исследования и преподавания: материалы ІІ Междунар. науч.-практ. конф., г. Минск, 21-22 марта 2019 г. Минск, 2019. С. 447-451. URL: http://elib.bsu.by/handle/123456789/218897.

4. Черноватий Л.М. Методика викладання перекладу. Вінниця: Нова Книга, 2013. 376 с.

5. Ahmadi A. On the Validity of a Multiple-choice Translation Test as a Substitute for an Open-ended Translation Test in the Iranian University Entrance Examination. Perspectives: Studies in Translatology. 2011. Vol. 19:4. P 307-314.

6. Casanova M.A. Manual de Evaluacion Educativa. Madrid: Editorial La Muralla, 1997. 254 p.

7. Hansen G. Triangulierung im Unterricht: Ubersetzungsprozesse, Eigenkorrektur und Fremdkrrektur. Informatologia. 2008. №41 (2). S. 105-115.

8. Kockaert H.J., Segers W. Evaluation of legal translations: PIE method (Preselected Items Evaluation). Journal of Specialised Translation. 2017. Vol. 27. P 148-163.

9. Zhao H., Xiangdong G. China Accreditation Test for Translators and Interpreters (CATTI): Test review based on the language pairing of English and Chinese. Language Testing. 2016. Vol. 33 (3). P 439-444.

Размещено на Allbest.ru