PhD, Associate Professor O.B. Dmitriev
Udmurt State University, Izhevsk

Keywords: e-testing, test questions, test mission, Moodle E-Test system, test design.

Backgroundю Since 1990ies the national educational system modernization process driven by new information technologies have given a rise to a variety of advanced e-learning models that, as mentioned by V.S. Avanesov [1], may be basically classified into the e-learning as such and the remote education models with the relevant correspondence/ full-time formats and special digital contents and tools for the courses and progress tests. The e-testing methods may be viewed as both a critical part of the modern e-learning systems and a separate education service, with tests applied to rate the individual progress and correct the education process when necessary [2]. In opinion of some education experts including V.S. Avanesov [1], A.N. Mayorov [5], А.М. Bershadskiy [2] and N.P. Puchkov [6], efforts to develop e-testing tools for different academic disciplines are both challenging and important for the science of testology in general and for the electronic, remote and combined education models in particular.

Objective of the study was to develop and offer specific recommendations on how the Moodle E-Test system may be applied in the academic physical education curricula.

Methods and structure of the study. The study was designed to analyze, among other things, the practical e-testing experience accumulated by the Physical Education and Sports Institute of USU in the academic Information Science; Natural Scientific Basics of Physical Education; Biomechanics; and Sport Metrology [3] disciplines using the Moodle E-Test system.

The Moodle education platform may be described as the universal framework for specific e-learning systems equipped with sophisticated mathematical and software tools for the problem-specific e-test system designs. An overview of the e-test system design process basics, its algorithm and practical test samples is given in work [4]. The Moodle system offers a library of standard test questions (question bank) and the test customizing and management tools. In our academic practice we have successfully applied the Moodle system to collect a library of questions and specific tests for different academic disciplines.

Study findings and discussion. The e-testing methods shall be generally designed as recommended by the general testology and then customized to requirements of a specific discipline, with the academic Physical Education (PE) discipline being no exclusion. Knowing the specific requirements and based on our practical e-testing experience, we would offer the following key design recommendations:

• Each test question and task shall fit into the required difficulty range, with the difficulty considered the key parameter of the test system [1, 5, 6]; and with question quality rated by experts and the difficulty level by the percentage of the right responses in the test system adjustment and initial practical tests.
• Each test question shall be formatted as required by the standard task forms [1, 5, 6] which are offered by the Moodle system as templates: see Table 1 hereunder. For the task difficulty management purposes, we recommend to combine the elementary tasks. Given on Figure 1 is a test question with the multiple optional responses, with five numerical verisimilar distracters.

Table 1. Test question and task templates applied in the study

• Since the thinking patters of modern students are dominated by the so called ‘clipped consciousness’ i.e. thinking stereotypes better fit for graphical rather than textual/ contemplative data [7], we recommend offering them the test questions in a variety of multimedia formats [2, 3]. Thus Figure 1 illustrates a test question by the staged movement video-schematics to facilitate understanding.
• Since the academic physical education and sport disciplines are always limited in time and coverage, the test questions and tasks shall be diverse enough within this range, with parallel facet questions applied for diversity. Facet means herein the possible parallel task versions [1]. Given on Figure 2(3), for example, are such parallel tasks: ‘based on the same set of input data, we offer a few standard tasks that are to find movement kinematics for a few points (m4, m6, m8) on a body model’.

Figure 1. Test question with multiple response options and numerical distracters

Figure 2. Hierarchically structured question bank: (1) question bank; (2) highlighted category; (3) parallel facet questions under the category

• A question bank with multiple questions is recommended to be designed hierarchically [3] (see Figure 2), with questions dropping down in each category/ subcategory. The categories are recommended to be different in the following aspects (Figure 2): (1) questions shall be split up into the theoretical (TQ) and practical (PQ) ones; (2) questions shall be further classified into those for the full-time and correspondence course students; (3) questions may be further classified into the open/ closed types or by the templates mentioned in Table 1: multiple optional responses, matching, numerical, essays; and (4) the categories may be classified by the study topics.
• Having designed the problem-focused component of the test question, we shall format it [4]. The Moodle system settings may be used to set requirements to the test design and requirements for the automatic test modeling purposes – so that the system will produce totally new test for every test session within the present frame, with questions randomly chosen from the question bank [3].
• Opponents of the e-testing models complain that they give no room for expression of the student’s creativity and individuality in responses, plus the right responses are guessable. However, the system offers a textual format of responses for convenience of the student, with no optional questions whatsoever, with such responses scored by the examiner by 1 point maximum – as the case for the other test questions, i.e. the test score varies within the range of 0 to 1. The score will be saved by the system and factored in the total test score. It should be mentioned that the textual response format has proved popular and beneficial in the practical tests.
• For practical purposes we recommend every test to be formed anew with a preset design and random choice of questions in every category [3]. Many teachers recommend the test time being limited by 1 hour [1, 3, 5, 6], with the number of questions per test depending on the difficulty level and timeframe. It is the test system adjustment stage when these correlations shall be set. The test system adjustment (pre-commissioning) tests are recommended to include every test question and task from the question bank without any classification, with the test system checked and adjusted on an experimental group. When the question bank is large enough, a few system adjustment tests may be run to analyze the quality and difficulty levels of the questions. The larger is the experimental group the higher is the accuracy of the above parameters. Given on Figure 3 is the consolidated score matrix applied in the adjustment and practical test sessions for a question bank including three categories. Highlighted in the score matrix are the question column; categories; and the total score column; and it shows the following: (1) difficulty level (5.63) of the question 2; (2) difficulty level 6.22) of category 2; (3) mean arithmetic value of all the questions (5.06; or 60.77 on a 100-point scale).
• We recommend one of the following test options in actual practice on discretion of the examiner:

1. Tests allow any browsing through the question bank, with any set of questions chosen for the test [3] – since this format is most mentally comfortable for the students albeit the probability of cheating/ copying is higher in the case;
2. Every next question in the test process is accessible only upon response to the prior one, with neither back browsing to the prior questions nor corrections to the responses allowed, and with the test format close in this case to the verbal examination, so that the probability of cheating/ copying from one another is minimized.

Conclusion. Based on the practical experience of the comprehensive e-testing model, we give recommendations on how the Moodle E-Test system may be applied to design, adjust and implement a progress test system in the academic physical education curricula. To secure high validity and dependability of the test data, the test system shall offer adequate questions and tasks in reasonably convenient formats – to remove the stress factors associated with the examiner. Practical recommendations based on the study data and analyses give the means to reduce errors in the e-test design for the other academic disciplines and improve the academic progress tests on the whole and thereby the physical education quality in particular.

References

1. Avanesov V.S. Primenenie testovykh form v e-learning s provedeniem distraktornogo analiza [Use of e-testing with distractor analysis]. Obrazovatelnye tekhnologii, 2013, # 3, pp. 125-135.
2. Bershadskiy A.M., Belov A.A., Vergazov R.I. et al. Aktualnye problemy kompyuternogo kontrolya znaniy [Topical issues of e-testing]. Vestnik kompyuternykh i informatsionnykh tekhnologiy, 2005, no. 1, pp. 40-48.
3. Dmitriev O.B. Osobennosti postroeniya elektronnogo uchebno-metodicheskogo kompleksa po distsipline ‘Sportivnaya metrologiya’ v sisteme MOODUS [Electronic education guiding system application in academic ‘Sport Metrology’ discipline under MOODES system]. Teoriya i praktika fiz. kultury, 2017, no. 1, pp. 9-11.
4. Dmitriev O.B. Problemno-strukturnoe proektirovanie elektronnykh kontrolnykh testov dlya uchebnykh distsiplin [Problem-structural design of electronic control tests for educational disciplines]. Sbornik nauch. trudov IV Mezhdunar. nauch.-prakt. konf. 'Elektronnoe obuchenie v nepreryvnom obrazovanii – 2017' [Proc. IV Intern. res.-practical conf. 'E-learning in continuing education - 2017'], Ulyanovsk, April 12 – 14 2017. Ulyanovsk: UlSTU publ., 2017, pp. 89 – 95.
5. Mayorov A.N. Teoriya i praktika sozdaniya testov dlya sistemy obrazovaniya (Kak vybirat, sozdavat i ispolzovat testy dlya tseley obrazovaniya) [Theory and practice of test design for educational system (How to choose, create and use tests for educational purposes)]. Moscow Intellekt-tsentr, 2001, 296 p.
6. Puchkov N.P., Bryankin K.V., Maystrenko N.V. Razrabotka bankov testovykh zadaniy [Development of test banks]. Teaching aid. Tambov: TSTU publ., 2008. 64 p.
7. Chirkova T.I. Problema preodoleniya klipovogo soznaniya molodezhi v professionalnoy podgotovke psikhologov na urovne bakalavriata [Problem of overcoming mosaic thinking in young people in Bachelor training of psychologists] Psikhologicheskaya nauka i obrazovanie psyedu.ru, 2016, v. 8, no. 1, pp. 45–61.

Abstract

Objective of the study was to develop and offer specific recommendations on how the Moodle E-Test system may be applied in the academic physical education curricula. The study was designed to analyze, among other things, the practical e-testing experience accumulated by the Physical Education and Sports Institute of USU in the following academic disciplines: Information Science; Natural Scientific Basics of Physical Education; Biomechanics; and Sport Metrology. The questions, tests technologies and contents were developed using the problems-focused design method, with the Moodle E-Test system applied for the electronic progress/ final tests. The tests included combinations of classified questions, parallel facet questions, multimedia survey tools and open essays. Practical e-tests in the above disciplines showed a high validity and dependability of the test data. Practical recommendations based on the study data and analyses give the means to reduce errors in the e-test design for other academic disciplines and improve the academic progress tests on the whole and thereby the physical education quality in particular.