Leveraging AI tools for enhancing project team dynamics: impact on self-efficacy and student engagement

Размещено на http://allbest.ru

Leveraging AI tools for enhancing project team dynamics: impact on self-efficacy and student engagement

Olena Kuzminska Olena Kuzminska Doctor of Pedagogical Sciences, Professor, Professor at Information Systems and Technologies Department, National University of Life and Environmental Sciences of Ukraine, Kyiv, Ukraine , Denys Pohrebniak Denys Pohrebniak Master's student in the educational programme "Software for Information Systems", National University of Life and Environmental Sciences of Ukraine, Kyiv, Ukraine ,

Mariia Mazorchuk Mariia Mazorchuk PhD in Technical Sciences, Methodist of Research and Analytics Department, Ukrainian Centre for Education Quality Assessment, Kyiv, Ukraine , Viacheslav Osadchyi Viacheslav Osadchyi Doctor of Pedagogical Sciences, Professor, Borys Grinchenko Kyiv Metropolitan University, Kyiv, Ukraine Leading Researcher, Institute for Digitalisation of Education of the National Academy of Educational Sciences of Ukraine, Kyiv, Ukraine

Abstract

The issue of maintaining group dynamics in flexible remote teams is currently in the focal point of attention not only of commercial companies and government agencies as a response to the changing conditions of work organisation, including those associated with the effects of the COVID - 19 pandemic, but also of higher education institutions that train future professionals.

Since intragroup dynamics is a key factor affecting team performance, this paper is the result of the study conducted as part of the Group Dynamics and Communications course at National University of Life and Environmental Sciences of Ukraine (NULES) to determine the potential of using AI tools as tools to support the group dynamics of self-organised teams during the implementation of educational projects.

Based on the results of the free choice of project and team management tools, we categorised three groups among the 56 participants of the experiment according to the use or non-use of AI tools to enhance group dynamics, followed by self-assessment of the effectiveness of the selected tools in increasing their levels of self-efficacy and social engagement.

The survey was based on a Likert scale measurement, and non -parametric methods of analysis and statistical hypothesis testing were used to process the results. The self-assessment confirmed that the incorporation of AI tools in the implementation of a team -based learning project did not influence the improvement of students' social engagement and the dynamics of self-efficacy for both effective and ineffective students.

The authors identified the positive impact of using AI tools on its development for students with the average level of self-efficacy. We also presented an example of the use of Notion AI for enhancing group dynamics at each stage of team development according to B. Tuckman.

The authors suggest that the impact of using AI tools in the process of implementing a group learning project depends more on personal characteristics of team members than on the choice of a specific AI model. Determination of application specifics of individual AI tools in the process of training future specialists in the implementation of various types of educational activities, particularly ethical considerations is recognised as an area for further research.

Keywords: group dynamics; educational project; AI tools; case study; self-assessment; higher education.

Аннотация

Использование инструментов искусственного интеллекта для улучшения динамики проектной команды: влияние на самоэффективность и вовлеченность студентов

Проблема поддержания групповой динамики в гибких удаленных командах в настоящее время находится в центре внимания не только коммерческих компаний и государственных учреждений в ответ на меняющиеся условия организации труда, в том числе связанные с последствиями пандемии COVID-19, но и высших учебных заведений, готовящих будущих специалистов.

Поскольку внутригрупповая динамика является ключевым фактором, влияющим на эффективность работы команды, эта статья является результатом исследования, проведенного в рамках курса групповой динамики и коммуникаций в Национальном университете биоресурсов и природопользования Украины (NULES) с целью определения потенциала использования инструментов искусственного интеллекта в качестве инструментов поддержки групповой динамики саморазвития организовывал команды во время реализации образовательных проектов.

Основываясь на результатах свободного выбора инструментов управления проектами и командами, мы распределили 56 участников эксперимента на три группы в зависимости от использования или неиспользования инструментов искусственного интеллекта для улучшения групповой динамики, после чего провели самооценку эффективности выбранных инструментов в повышении уровня их самооценки- эффективность и социальная вовлеченность.

Опрос был основан на измерении по шкале Лайкерта, а для обработки результатов использовались непараметрические методы анализа и проверки статистических гипотез. Самооценка подтвердила, что внедрение инструментов искусственного интеллекта при реализации проекта командного обучения не повлияло на повышение социальной вовлеченности учащихся и динамику самоэффективности как эффективных, так и неэффективных учащихся.

Выявлено положительное влияние использования средств ИИ по его развитию для студентов со средним уровнем самоэффективности. Мы также дан пример использования понятия ИИ для улучшения групповой динамики на каждом этапе развития команда по данным Б. Тукмана.

Авторы предполагают, что эффективность использования инструментов искусственного интеллекта в процессе реализации проекта группового обучения в большей степени зависит от личностных характеристик членов команды, чем от выбора конкретной модели искусственного интеллекта. Определение специфики применения отдельных инструментов ИИ в процессе подготовки будущих специалистов к реализации различных видов образовательной деятельности, в частности, с учетом этических соображений, признано областью дальнейших исследований.

Ключевые слова: групповая динамика; образовательный проект; инструменты ИИ; кейс-стади; самооценка; высшее образование.

Introduction

Statement problem. Although it is natural for people to unite in thematic communities and groups for professional and social interaction, the effects of the COVID-19 pandemic (global context) and the war in Ukraine have led to the need for remote work and teamwork among employees in various fields, and, as a result, the digital transformation of relevant institutions and commercial companies [1]. Moreover, change management, as one of the components of the digital transformation of any system or institutional environment, involves staff support and training [2]. It should be noted that, in contrast to individual training, group training implies that participants learn not only from their own experience, but also from the experience of their fellow team members through the processes of social, cognitive and interpersonal interaction. Furthermore, dynamic situations necessitate the ongoing development of team members, specifically aligning individual capabilities with team tasks and acquiring and mastering the necessary resources [ 3].

The above trends are also present in educational systems, where collaboration is usually used not only to share and generate new knowledge, but also to develop students' soft skills (e.g., communication, problem-solving, self-reflection, and self-regulation). In particular, the research area of computer-supported collaborative learning (CSCL) explores how to effectively combine the rapid development of information technology with collaborative learning and development in small groups [4]. Thus, the freedom of choice and technological saturation of the educational environment [2], on the one hand, has a positive impact on students' academic performance, and on the other hand, personal characteristics of small group participants, or rather their incompatibility, often lead to significant difficulties not only in the implementation of educational projects but also to negative experiences of group interaction in general. That is why the need to enhance the group dynamics of academic groups and student project teams is becoming more relevant.

Analysis of the recent research and publications. Optimisation of group dynamics in the process of implementing case-based learning is the subject of research by M. Kochis et al. In particular, the authors emphasise the stimulating role of project-based learning in small groups not only for the development of student competences, but also for improving small group dynamics and ensuring psychological safety among peers. It should be noted that it is important to support students' responsibility for their own learning and team performance and to initiate discussions on ways to enhance group dynamics.

The positive impact of project-based learning on the training of engineering students is also reflected in the study conducted by a team of authors led by K. R. Ccama-Mamani [6].

We agree with the authors' opinion that despite the fact that basic engineering education often considers laboratory experiments as a practical component, these activities may not contribute to the development of soft skills; the pedagogical design and implementation of interdisciplinary projects are effective tools for developing professional and soft skills in students.

The effectiveness of the use of problem-based learning (PBL) and team-based learning (TBL) as universal for student development is also confirmed by the study of American researchers D. Kim and O. F. Iwuchukwu [7]. In particular, the authors used the approach of students' self-selection (self-organised groups) to form groups, and group development was studied using B. Tuckman's model.

In terms of tool support for group interaction, the most commonly used project management tool in the work of professional software development teams is Jira [8], and for educational projects it is Trello [9]. At the same time, and this is confirmed by a previous study by the authors of [9], the use of the Notion software (https://www.notion.so/product) as a tool useful for note-taking, project and task management, all in one workspace, is becoming increasingly common. In 2023, for the first time (based on the results of a wide vote of 2022 respondents), this software tool was ranked among the 100 best tools for training (https://toptools4learning.com/) future professionals at higher education institutions [6], [10], in particular, for advising students on the effective management of remote project teams using online tools and technologies [11]. It should be noted that intragroup dynamics stands out as the primary factor influencing team performance.

The results of the research by A. Mohan et al. [12] show that there is no strict algorithm or rules for enhancing group dynamics, as this process depends on the context (task complexity, team maturity, selection of tools, etc.). Another aspect that influences group dynamics is the role of the leader, both at the level of team formation and registration of new members [13] but also in providing support throughout the project implementation [14]. At the same time, the spread of artificial intelligence technologies and AI potential for supporting business processes at higher education institutions [15] can be used to enhance group dynamics of self-organised groups of students. It is this trend that is under-researched against the background of an increase, starting from 2021, in the number of studies on the application of AI in higher education. For the most part, researchers distinguish the following directions [16]: Assessment/Evaluation (1), Predicting (2), AI Assistant (3), Intelligent Tutoring System (4), Managing Student Learning (5). In the context of this study, attention should be focused on determining the impact of the use of AI tools on various aspects of learning and teaching in the context of higher education in general and in a specific university in particular [17]. The communicative aspect of the application of generative AI, mainly language models, which is implemented as a model of human-machine interaction, emphasizes the role of AI in personal development of students [18]. Since today's students already use and are going to use AI tools for communication and learning, the task of university teachers and researchers in the field of education is to find effective pedagogical technologies and approaches that will allow students to optimally use these tools.

The purpose of this study is to determine the potential of using generative AI tools as a tool for enhancing group dynamics of self-organised teams in the process of implementing educational projects.

Reinforcing the importance of self-organised, agile, and self-developing teams [1], as well as relying on the proven relationship between self-efficacy and student engagement in learning and academic performance [19], we set the following objectives to achieve the purpose of our study:

1. To explore the potential of AI-powered applications in enhancing group dynamics of self-organised student teams working on learning projects.

2. To empirically assess the influence of these AI tools on enhancing students' self-efficacy during the implementation of learning projects.

Research methodology

To achieve the objectives, we relied on the methodological foundations of the literature review process as a research method. For instance, this approach was applied in analysing methods and technologies for supporting group dynamics, as well as in exploring application of artificial intelligence tools in education [20]. Additionally, case study and survey methods [21] were employed to meet the first and second objectives of the study, respectively, involving student participants.

This research was conducted as part of the study of the Group Dynamics and Communication discipline by 4th year students of National University of Life and Environmental Sciences of Ukraine (Software Engineering educational programme). team artificial intelligence university

To address the first objective, we offered students a case study of an educational project with the Veterinarian Handbook working title (Fig. 1).

Fig. 1. The example of case implementation: the main page of the reference book with logo and background cover demonstration

Task statement: to disseminate information on the prevention of diseases of certain breeds of animals among veterinary workers, farmers and a wide range of stakeholders, including students of specialised educational programmes, there is a necessity to develop an IT solution (chatbot) based on the materials provided in the format of a text document to promptly inform (provide information in two languages: Ukrainian and English) these categories of potential users. The proposed solution should not "require" additional technological or financial investments for its implementation and maintenance, as well as the need for additional training for potential users to use it.

The conditions for the implementation of the training project were:

- the team building method: we carried out the project in small groups (3-4 people), students joined groups according to their own free will (self-organised teams [7]);

- the teams implemented the educational project over five weeks in accordance with the educational programme of the Group Dynamics and Communications discipline;

- students of each group chose the group interaction environment for team and project management according to the concept of group dynamics independently, however, the environment (in accordance with the tasks of the discipline, including the analysis of successful practices of professional IT teams) should include (and justify the choice) a task manager [9], data warehouse, tools for collaborative work with documents, software development tools, services for supporting team communication (instant messengers, video conferencing services, time trackers, etc.);

- the use of artificial intelligence tools [21] was not part of the requirements for the implementation of the educational project, nor of the restrictions - students independently determined what it was necessary and they selected necessary tools;

- group dynamics was monitored according to the team development model by B. Tuckman [23]; during the five stages of group development (forming, storming, norming, performing, adjourning) students kept observation journals and generated reports (self-reporting).

Since the results of the analyzed empirical studies prove the effectiveness of students' use of educational assistance based on AI tools for the development of self-regulation [24], a process by which students actively control and direct their learning activities using various cognitive and behavioural techniques (correlated with the concept of group dynamics), the first objective of our study was to identify the potential use of AI tools to enhance group dynamics of self-organized teams in the process of implementing educational projects.

Given that self-efficacy [25] and engagement [26] are, on the one hand, indicators of students' self-regulation [24], and, on the other hand, affect their academic performance [27], the survey method was used to achieve the second objective of our study, and in-depth interviews were additionally conducted.

In this study, we used pre- and post-tests to determine the level of self-efficacy and student engagement in the process of implementing the learning project (see Appendix).

The self-assessment and social engagement questionnaires were borrowed for use in the study from [25], [28] as they had been tested and had a margin of error of no more than 2%.

The self-efficacy scale, which uses a 5-point Likert rating system (determining the attitude to the proposed statements on a 5-point scale: 4 - strongly agree, 0 - strongly disagree) was developed based on the generalized self-efficacy scale [25] and translated for Ukrainian students.

After the translation into Ukrainian, a reliability analysis was conducted, which yielded a result of 0.72, indicating its sufficient reliability. Since self-efficacy is defined in [24] as a student's belief in their ability to mobilise motivation, cognitive knowledge, and action plans to meet the specific requirements of a given situation, we used this scale (pre- and posttests) to determine the impact of using AI tools on students' self-efficacy in the course of a learning project.

The engagement scale was borrowed from the engagement scales in mathematics and science proposed by Wang et al. [28] and was used in this study to determine the homogeneity of groups on the basis of group interaction (Social engagement). Although the study [24, Table 6] found that the use of AI tools had the least impact on the development of Social engagement, this characteristic is important for determining the homogeneity of groups by team interaction. The scale is a five-point Likert scale and, as proven in previous studies, has a fairly high reliability and validity.

We translated 4 statements from the Social engagement category into Ukrainian. In translating the questionnaire questions, we took into account the content component, which might differ from the language of the original questionnaires, so the error in the assessment using this tool should not exceed 5%. The reliability of the modified questionnaire was also determined. As a result, the reliability was 0.68, which is considered to be a sufficiently acceptable reliability.

The participants of the experiment, as mentioned above, were students who independently formed teams of 3-4 people, selected project and team management tools according to the concept of group dynamics, including AI tools.

As a result, three independent samples were formed out of 56 participants (the number was sufficient to draw conclusions about certain areas of assessment, the representativeness error does not exceed 5%):

- Gr1 (13 students): did not use AI tools or used them fragmentarily;

- Gr2 (29 students): used ChatGPT to help solve problems during their educational project; in general, the use of AI tools was unsystematic;

- Gr3 (14 students): used the built-in AI Notion as an assistant at different stages of the project (https://www.notion.so/product/ai).

The main hypothesis was that the self-efficacy of students who used AI tools systematically or partially would be higher at the end of the project than those who did not use AI. To test the main hypothesis, the following partial hypotheses were formed:

H1: The utilization of AI tools during the implementation of a team-based learning project does not significantly enhance students' social engagement.

H2: The impact of employing AI tools in an educational project implementation is contingent upon personality traits; specifically, for students with a high (or low) level of selfefficacy, the utilization of AI tools to facilitate group dynamics will be negligible.

H3: For students with an average level of self-efficacy, the influence of using AI tools is notable, yet it remains independent of the tool selection. That is, irrespective of whether students employ the integrated AI assistant in Notion or ChatGPT to address challenges at various project stages, their self-efficacy scores will improve. To determine the benefits of using a single project management tool with a built-in Notion AI module containing templates and tips in accordance with project and team management processes to increase students' self-efficacy, we conducted additional in-depth interviews with students who used AI tools in the course of implementing a training project (Gr2, Gr3) and showed high self-efficacy results - these are Gr3 students. In the course of data analysis, we used a set of methods and models to calculate all descriptive statistics.

The choice of certain indicators and criteria for evaluation was determined by the type of data, the evaluation scale, and the limitations of the methods. The SPSS statistical data processing software toolkit was used for calculations [29].

Results of the research

Exploring the potential of artificial intelligence to enhance group dynamics in self-organized teams. Since, according to the results of the survey [9] students with some professional experience chose Notion for project management, we offer a description of the use of this particular service to enhance group dynamics at each stage of team development during the implementation of the Veterinarian Handbook project as a kind of compensation for the lack of research on the use of Notion to support educational projects.

In addition, among the various combinations of groupware services chosen by the teams to implement the educational project (the most common services were Microsoft 365 cloud office services and integration of Trello functionality with Google Workspace), the use of Notion was the most appropriate for this educational project (determined by the results of the requirements analysis and formalisation of project tasks); it allowed increasing team productivity with just one application (announced in the Notion help guide); provided users with support at different stages of project implementation by Restrictions on the free use of this service, namely the limit on the number of participants in one team (5 people) and the number of samples of the built-in AI module, also favoured the use of Notion for small groups in the framework of educational projects. It should be noted that since Notion was chosen for project and team management by students (Gr3) who had a high level of self-efficacy and engagement and demonstrated the highest quality products as a result of the educational project (this parameter was not taken into account in this study), we suggested considering the materials of one of these teams as an example (Fig. 1).

Creating a chatbot as a companion (used to navigate reference information) to the information materials provided by the customer in the form of a text document was the team decision (the team implemented a project called "Chatbot").

Rationale: Notion integration with Telegram allowed not only to accomplish the task but also to improve group dynamics, and the design of reference information in the form of a Notion reference book (Fig. 2) enabled to structure data for the "bot work" and to provide the ability to edit Notion wiki pages in the process of interacting with customers or developing the information base.

Like other project management systems, Notion functionality allows users to create a team workspace (Fig. 3a). Since a user (student) can have several projects (Fig. 3b), it is advisable to develop different logos and covers for project recognition to facilitate orientation. For the same reason, different covers were chosen for different pages of the workspace to add visuals that make it easier to navigate within the project.

Fig. 2. Organisation of reference information in Notion: structure of the Wiki - a reference guide for bot navigation

Fig. 3. Organisation of Notion user workspaces: a) the main page of the Chatbot workspace; b) the list of workspaces for an individual user

To support the implementation of the project, the main page of the Chatbot space (Fig. 3 a) contains:

- a page with a general description of the project (1), which includes the project topic, task description, links to useful resources, as well as a list of project team members and their contact details;

- a kanban board (2) for assigning tasks and managing the process of their implementation by team members;

- a page with all useful/used/created project documents (3);

- a separate section of resources provided by the client (4), which contains a page with the terms of reference; in our case, the terms of reference were described in a Google document, the link to which was left on this page, which allows users to track the history of changes.

At certain stages, in accordance with the project objectives AI tools integrated into Notion were used to help plan and detail task descriptions; generate different types of texts (Fig. 4), including brainstorming, creating task lists, meeting agendas; translating and improving texts created by team members.

Fig. 4. Notion AI templates of thematic texts

In order to track group dynamics (understood as a system of behavioural and psychological processes that occur within a small group), in accordance with B. Tuckman's structure, freeze-frames were created during the course of the training project. In the case of self-organised teams, students used Notion and the built-in AI module to effectively interact at each stage of team development. A summary report of one of the teams is presented in Table 1.

Table 1.