This research aims to develop an assessment model of the online lecture process that is valid, effective, and practical. Research Location in the Mathematics Education Study Program, Ahmad Dahlan Islamic University. This study uses a Research and Development (R&D) methodology with an ADDIE (Analysis, Design, Development, Implementation, Evaluation) approach. The implementation stage is carried out by applying assessment instruments that have been developed in several online classes at Ahmad Dahlan Islamic University. Data collection was carried out through observations, questionnaires, and interviews to assess the effectiveness of the assessment model. The results of the study show that this assessment instrument is able to increase the active participation and quality of student interaction in online lectures. The assessment instrument used has high validity and reliability with a content validity coefficient of 0.95. Conclusion The application of this assessment instrument also has a positive impact on the quality of learning, where students show an increase in the use of technology and digital literacy. Lecturers feel helped by the systematic and easy-to-use assessment format. Overall, this study concludes that the assessment model developed is effective in assessing student activities during online lectures and can be adopted more widely to improve the quality of online learning in various universities.
Higher Education has undergone a significant transformation in line with the development of information technology and the demands of the 4.0 technology era towards society 5.0. This created the concept of distance learning in response to this change [1]. E-learning is not only a complement but also a replacement for traditional lecture activities.
In the 5.0 era that applies the principles of the Internet of Things, Artificial Intelligence, machine learning, and big data, the ability of High Order Thinking Skills is very important, especially in the context of mathematics lectures [2].
Flipped learning, as one of the e-learning lectures models, changes the traditional lecture paradigm by utilizing online and offline platforms. Lecturers use videos to convey material asynchronously, which is then discussed synchronously in real or virtual space [3][4].
The birth of online lectures has raised new challenges related to assessment in the lecture process in schools and universities. Online lecture models such as flipped learning and blended learning, which are integrated into the mathematics lecture approach, highlight the need to develop appropriate online lecture process assessments. How to measure and assess the process of online learning activities is an important question, especially in the midst of the emergence of lecture models that combine online lectures with face-to-face (face-to-face).
Assessment is the process of collecting, synthesizing, and interpreting information to make decisions [5]. Assessment is not only assessing students but is very functional to assess the lecture system itself [6], [7]. Lecturers must answer the questions, "How should I teach?" and "what grade should I give to students?". From the two questions, lecturers must have an assessment concept in the online lecture process that is carried out [8]. To measure students' abilities, teachers can see; (1) placing instructional appropriately to correct student shortcomings; (2) skills or knowledge that students have mastered should be avoided instructionally.
An in-depth understanding of the assessment of the lecture process is the key to success in providing effective teaching [9]. Good lecturers are needed to facilitate online lectures. However, this process can be complex due to a number of obstacles faced, such as the unpreparedness of some students in attending online lectures due to technical obstacles or lack of focus during the learning process [10]. This provides an in-depth context on the importance of developing an online lecture process assessment.
Massive changes in the world of education will occur so rapidly and will have an impact on the urgency of the assessments needed in the world of education. What will the world do in 2100 or even 2200 to educational policies and practices in 2010? This is a difficult job with various challenges of the current era. The challenge for assessment in the 21st century is an emerging educational priority.
Assessment procedures are necessary to support the preparation of the next generation with the skills and values they need to manage emerging global challenges [11]. Lecturers who are able to conduct tests well will be more effective lecturers [8]. Assessment and lectures are inseparable. Assessment in Indonesian Language means assessment. Assessment or assessment is a process to make decisions using information obtained through the measurement of learning outcomes, either by using test instruments or non-tests [12].
Learning must be meaningful and students become more progressive so that they not only develop their cognitive shutter but can improve the affective and psychomotor shutter of students through the process of assimilation and accommodation developed by Piaget and Vigotsky so that the assessment process is not only at the end of the lecture but also needed in the lecture process [13]. A good assessment is the right assessment to measure indicators of achievement and in what way lectures are conducted. For example, problem-based lectures require appropriate assessment methods. For example, how to solve problems is a learning process that is very important to monitor.
For this reason, assessment techniques are needed in the form of observations, performance checks, and the like. If an objective test is chosen as the assessment method, for example, a multiple-choice technique, then the quality of the problem-solving process is difficult to monitor objectively. Therefore, assessment must be understood as an effort to make the teaching and learning process effective in order to achieve the goals of the lecture itself [13].
Assessment is an activity that is carried out on an ongoing basis to collect information on student performance. Assessment of children's development and learning has an important value. Not only measuring children's progress as a form of program evaluation, the assessment is also useful for identifying staff development needs and planning for lectures ahead.
Current perspectives in classroom assessment encourage the use of a variety of assessment strategies, tools, and formats, providing ample opportunities for students to demonstrate their lectures, utilize formative feedback in a timely and regular manner, and involve students in the assessment process [14].
The challenges and effectiveness of student assessment practices in online lectures, it was found that online lecture assessment involves various types of methods, such as written assignments, online discussions, fieldwork, quizzes and exams, and presentations. The challenges identified in the study create an urgent need to improve the concept and rubric of assessment in the context of the online lecture process [15].
The birth of online lectures also raises assessment problems in the lecture process, both at school and in college. The question of how to assess and measure the process of online learning activities is a question mark that requires a solution. Thus, the development and implementation of online lecture process assessment is not only a practical necessity but also the key to improving the quality and relevance of mathematics education in this modern era. This conclusion provides a solid basis for further research and development in this field.
This research uses the type of Research and Development research.
Procedure
The research and development procedure is carried out using the ADDIE (Analyze, Design, Implement, and Evaluate) model research and development procedure. The ADDIE model is a commonly used framework in instructional design and learning program development.
Each stage in this model is interrelated and contributes to the overall quality and success of the learning program. By following the steps in the ADDIE model, instructional designers can ensure that the programs developed not only meet the learning needs, but also achieve the expected outcomes in an efficient and effective manner.
Data Analysis
In the analysis stage, the researcher starts by conducting a preliminary study to understand the context and specific needs of the assessment of the online lecture process to be developed. This preliminary study includes data collection on student conditions, available facilities, and challenges that may be faced in online learning.
After conducting the needs analysis process, the researcher proceeds to the product design stage. The design of this product is based on various relevant theories, including learning theory, lecture theory, and assessment theory
The development process is a stage that is carried out by developing a design that has been designed at the beginning of the research by adding or subtracting assessment steps based on what happens and what is needed in the online lecture process. At this stage, the researcher also conducts the process of instrument validation and experimental validation to find out that the product that has been developed can be used properly.
At the implementation stage, the researcher conducted a trial of the use of assessment in the Mathematics Study Program of Ahmad Dahlan Islamic University, Sinjai Regency. This trial includes online lectures and the use of lecture assessments that have been developed as a product of the research process.
Instrument assessment uses the validation of the Gregory method, Content Validity Coefficient = .
Validity Data Analysis
Aspects | Indicator |
Quality of content and purpose |
|
Effectiveness Data Analysis
Aspects | Indicator |
Instructional quality |
|
Practicality Data Analysis
Aspects | Indicator |
Technical quality |
|
Criteria for Assessing the Implementation of Research Instruments
No | Interval | criterion | |
1 | 3.5 ≤ x ≤ 4 | Highly Active | |
2 | 2.5 ≤ x < 3.5 | Active | |
3 | 1.5 ≤ x < 2.5 | Less Active | |
4 | 1 ≤ x < 1.5 | Inactive |
Each meeting will be assessed from the accumulated score of each student divided by the number of students in the class. It is outlined in the following formula:
Dimensional effectiveness =
Effectiveness of student assessment in 1 meeting =
Effectiveness of meeting assessments =
Effectiveness of student assessment in 1 semester =
The result of the research developed are an online lecture prosess assessment model illustrated in the fallowing figure:
Recapitulation of The Observation Result
No | Subjek | Observation Indicators | ||||
Practicality | Effectiveness | |||||
|
|
|
|
| ||
1 | First Lecturer | 2.8 | 2.9 | 3.0 | 2.9 | 2.8 |
2 | Second Lecturer | 2.5 | 2.8 | 2.9 | 2.8 | 2.6 |
3 | Third Lecturer | 3.1 | 3.3 | 3.5 | 3.3 | 3.0 |
Value | 2.8 | 3.0 | 3.1 | 3.0 | 2.8 |
No | Subject | Observation Indicators | ||||
Practicality | Effectiveness | |||||
|
|
|
|
| ||
1 | First Lecturer | 2.8 | 2.9 | 3.0 | 2.9 | 2.8 |
2 | Second Lecturer | 2.5 | 2.8 | 2.9 | 2.8 | 2.6 |
3 | Third Lecturer | 3.1 | 3.3 | 3.5 | 3.3 | 3.0 |
Value | 2.8 | 3.0 | 3.1 | 3.0 | 2.8 | |
Achievement Category
| 2.9 (Practical) | 2.9 (Effective) |
Finally, the results of this assessment illustrate that most of the subjects were able to meet the established achievement criteria, with some subjects showing excellent performance. This reflects the effectiveness of the assessment model used and provides a positive picture of the quality of learning that has been implemented. The results of this study are in accordance with the theory of Piaget and Vigotsky who said that the assessment process is not only at the end of the lecture but is also needed in the lecture process. Both argue that a good assessment is the right assessment to measure achievement indicators and in what way lectures are carried out, assessments are needed in the form of observations, performance checks, and the like. Anderson (2003) said that it is important for students to know or be aware that lecturers are conducting assessments, including in the lecture process. Students should be told to do their best or reminded of the importance of demonstrating their beliefs and feelings in an affective assessment related to effort.
M. Fukuyama, “Society 5.0: Aiming for a New Human-Centered Society,” p. 4, 2016.
P. Mytra, A. Wardawaty, and R. Kusnadi, “Society 5.0 in Education: Higher Order Thinking Skills,” in BIS-HSS 2020: Proceedings of the 2nd Borobudur International Symposium on Humanities and Social Sciences, BIS-HSS 2020, 18 November 2020, Magelang, Central Java, Indonesia, European Alliance for Innovation, 2021, p. 242.
Y. El Miedany, “Flipped learning,” in Rheumatology Teaching, Springer, 2019, pp. 285–303.
A. Karabulut-Ilgu, N. Jaramillo Cherrez, and C. T. Jahren, “A systematic review of research on the flipped learning method in engineering education,” British Journal of Educational Technology, vol. 49, no. 3, pp. 398–411, 2018.
C. DeLuca, T. Chavez, and C. Cao, “Establishing a foundation for valid teacher judgement on student learning: The role of pre-service assessment education,” Assessment in Education: Principles, Policy & Practice, vol. 20, no. 1, pp. 107–126, 2013.
N. Khuzaini, “EVALUASI HASIL DAN PROSES PEMBELAJARAN MATEMATIKA,” p. 157.
N. Khuzaini and Nuryadi, “EVALUASI HASIL DAN PROSES PEMBELAJARAN MATEMATIKA”.
W. J. Popham, Classroom assessment: what teachers need to know, Eighth edition. Boston: Pearson Education, 2017.
J. E. Brophy and T. L. Good, Handbook of research on teaching. New York: Macmilan: M.C Witrtrock (Ed), 1986.
L. W. Anderson, Classroom Assessment: Enhancing the Quality of Teacher Decision Making, 0 ed. Routledge, 2003. doi: 10.4324/9781410607140.
C. Wyatt-Smith and J. J. Cumming, Eds., Educational Assessment in the 21st Century. Dordrecht: Springer Netherlands, 2009. doi: 10.1007/978-1-4020-9964-9.
N. R. Sesanti and R. D. Ferdiani, Assesment Pembelajaran Matematika. in Pertama. Jl. Karangduren Gang 10 (Perum Citra Graha Residence Blok B7) Pakisaji Kab. Malang: Yayasan Edelweis, 2017.
A. L. F. Tilaar, Assesmen Pembelajaran Matematika. Percetakan Unima Press, 2018.
J.-A. Baird, T. Hopfenbeck, P. Newton, G. Stobart, and A. T. Steen-Utheim, “Oxford University Centre for Educational Assessment.” Oxford, UK, 2014.
L. R. Kearns, “Student Assessment in Online Learning: Challenges and Effective Practices,” vol. 8, no. 3, 2012.