 |
| Cognitive Domain of Bloom's Taxonomy |
In modern age of
technology, assessment has been done using automated mechanisms. Consequently, technology and assessment create
new domain known as e-assessment. It is
an important component of e-learning and has become one of the most common
forms of technology enhanced assessment since the 1990s (Bruyn et. al., 2011). E-assessment
is an assessment carried out using various technology based tools, methods and
devices/services. Assessment is about evaluating learning outcomes (Stodberg, 2012).
It is a tool for monitoring student’s
activity using certain specified criteria.
There are several purposes of assessment as defined by Mehrens and
Lehmann (cited
in Cannon, 1995). These are; judging
mastery, measuring improvement over time, diagnosing student hitches, assessing
the teaching methods, calculating the effectiveness of the course and motivating
students to study etc. Use of BT may
help to design assessment for appropriate focus of e-assessments.
Many common issues
of e-assessment related to technology and devices were discussed by (Sangi, 2008). Some of such issues specially associated to
technology and devices have been researched over time. However, work on e-assessment of STQs still
needs more development. Because, it is a
most used and considered more valid process to assess depth of the student’s
performance at higher level education. Use
of e-assessment is quick and with development of e-assessment of STQs it may be
more valuable for institutions and faculty.
Additionally, use of BT will serve the base purpose of fair and valid
e-assessment perhaps. It may also provide flexibility, advance level
questions capacity which can reduce the load of admin, provide more storage
efficiency in systems (Nikolova, 2012).
Furthermore, E-assessment has
four basic forms i) Diagnostic ii) Formative iii) Summative and iv) Integrative (Crisp, 2011).
While, formative assessment is used during the learning process for the assessment of learner. Furthermore, it also used to
monitor learner’s ongoing progress and offer them feedback. In each type of e-assessment design
different formats of questions may be used. Currently many e-assessment techniques include
the multiple formats of questions. These
questions formats are broadly categorized into following:
Figure
1: Formats of Questions
However, objective type
questions (OTQs) are difficult to construct but easy to assess automatically while
subjective type questions (STQs) are easy to construct but difficult to assess
automatically. Therefore, STQs are often
evaluated by tutors, teaching assistants or instructors hence these are time
consuming to grade (Cannon, 1995). Additionally, issues are associated
with maintaining fairness, objectivity in grading and often maintaining language
sensitivity, which is influenced by teacher and learner’s ability to communicate.
The issues of variety in answer by
different students make STQs difficult to understand by the machine and thus
not practical in automated evaluations. It can be assumed that introducing systematic
process of BT in formatting the questions may help to overcome of some
difficulties mentioned above.
Additionally, the study of (Sangi, 2008 and Bruyn et. al., 2011) mentiond for incorporation
of traditional methods of assessment i.e. written tests, oral
examinations, assignments, portfolios, projects, presentations etc in
e-assessment. While, (Crisp, 2011) explained
that it can either be part of Learning
Management System (LMS) or a standalone application and also identify various
other modern techniques of assessment i.e. animation, blog, cloze, discussion,
free text, hot spot, matrix, MCQ, ordering, pull down, role-play, self or peer
review, simulation, virtual world and wiki. Furthermore, the study of (Nikolova, 2012 & Alruwais et. al., 2018) reveals
that e-assessment offers richer assessment experience, which increased the
flexibility, provides instant feedback, reduces the management burden, offer
greater storage and enhanced questions styles/formats. Hence, it evaluates the learner’s performance
electronically i.e. by the use of Information and Communication Technologies
(ICT).
Assessment is a
procedure for measuring the knowledge, attitudes and skills of the learner’s and useful
for teaching and learning environment, which promote learning, measure or
certify results of the learner’s (Clements et.
al., 2013). Whereas, Scottish
Qualifications Authority (SQA) publication define assessment i.e. “Assessment
is the process of evaluating an individual’s attainment of Cognitive, Affective
and Psychomotor. It has many uses but
these can be divided into two major categories: uses for individual candidates,
and uses for external organizations” (SQA, 2017). Hence, it is the most important and useful component
and tool to measure or evaluate the quality of teaching and learning
environments. Several assessment
techniques include the type of questions to be assessed by the examiner or the
assessment tool in case of automated assessment. Moreover, as per planning framework table of
BT the quiz and test are the most common methods used for the evaluation of the
learner’s (“Kurwongbss,” n.d). These questions are mostly categorized into
two major formats i.e. OTQs and STQs discussed in Figure 1.
The OTQs are also known as selected-response items require their answers to be picked form a list of choices while STQs are constructed-responses items require their answers in written form (Chalkboard, 2018). On the other hand, OTQs are difficult to construct but easy to assess automatically while STQs are easy to construct but difficult to assess automatically (Alrehily et. al., 2018). Moreover, OTQs are Forced-choice questions not effective to assess the skills competencies and depth of knowledge (Sangi, 2008). While, the SQA in its publications discusses the question and states as following under the heading “Non-objective questions”:
“The
downside of computer marking is that sometimes the assessment engine can be
less accommodating than human markers. One question type available with some
assessment engines that is not usually marked automatically is the free-text
extended response or essay questions. The responses entered by candidates to
these questions need to be scrutinized by a human. Candidates type their answer in a text box
and submit them in the normal ways that are then assessed by teacher” (SQA, 2005).
Various research
studies revealed that most of the work in e-assessment is done on assessing OTQs
(Sangi, 2008; Praveen et. al., 2014; Sankar, 2015;
Agrawal et. al., 2017; Alrehily et. al., 2018). However, cognitive domain of Bloom's Taxonomy was implemented
with MCQs format (Bruyn et. al., 2011 & Lajis et. al., 2018) to assess the medical knowledge and programming
skills of the learner's. Hence,
to measure the higher order thinking level i.e. cognitive, affective and
psychomotor skills of the learner which cannot readily be assessed using OTQs formats
and more authentic e-assessments are being proposed (Kuh et. al., 2014). However,
it is a challenge to test higher levels and promote deep learning in
e-assessment as identify by (Bruyn et. al., 2011). Consequently, the study of (Agrawal et. al., 2017) recommended automatic
grading of STQs because it will be very helpful for educational institutions if
the process of evaluation of descriptive answers be evaluate using
e-assessment.
As a result of all
this work, different commercial/academic tools and software have been developed
some of this software such as Moodle (Moodle,
2018), Blackboard (FinanceOnline, 2018),
e-Rater (ETS, 2018), ConductExame (ConductExam, 2018), Zoho (Praveen et. al., 2014) etc have electronic
assessment modules incorporated in the package. Although there are numerous
tools / software available as defined by (Sankar,
2015) but e-Rater and AutoMark are only well known tools which can
automatic grading STQs for English literature to check writing style,
grammatical and syntactical structure of sentences through Natural Language
Processing (NLP) parser. The online
tests available at different websites also include e-assessment of students
using OTQs. Even if there is some space
for answering lengthy i.e. STQs, their marking is done manually (Praveen et. al., 2014) and no mechanism is
provided for automatic grading of the long answers.
Furthermore, there is a little
work done in assessing the STQs electronically using BT domains due to
limitations involved in automated grading of subjective contents intended to
measure or verified abilities of the learner.
One possible mechanism is to electronically assess the in-depth
knowledge, attitudes and skills of the learner’s abilities etc organized in the
form of subjective contents may be an assessment model developed based on BT
domains and the semantic analysis of sentences. Hence, so far many Artificial intelligence (AI)
techniques and algorithms have been developed/devised to mark STQs. These involve Computational Linguistics,
Natural Language Processing (NLP), Statistics, Information Extraction (IE), Pattern
Matching (Dube et. al., 2014). While (Alrehily et. al., 2018) propose
another technique i.e Syemantic similarity and document simalarity to find matching
the responses of the learner.
Some of these techniques and algorithms are combined in existing marking
engines to achieve better accuracy.
Whereas, there is a little
work done in assessing the STQs electronically using BT domains due to
limitations involved in automated grading of subjective contents intended to
measure or verified abilities of the learner.
One possible mechanism is to electronically assess the in-depth
knowledge, attitudes and skills of the learner’s abilities etc organized in the
form of subjective contents may be an assessment model developed based on BT
domains and the semantic analysis of sentences. Hence, so far many Artificial intelligence (AI)
techniques and algorithms have been developed/devised to mark STQs. These involve Computational Linguistics,
Natural Language Processing (NLP), Statistics, Information Extraction (IE), Pattern
Matching (Dube et. al., 2014). While (Alrehily et. al., 2018) propose
another technique i.e Syemantic similarity and document simalarity to find matching
the responses of the learner.
Some of these techniques and algorithms are combined in existing marking
engines to achieve better accuracy.
Hence, there is a need of
time for implementation of Bloom's Taxonomy in electronic assessment especially for systematic
(and more objective) evaluation of subjective/divergent type questions.
As these questions are regarded as the most valid and reliable tool to evaluate the learner’s deep knowledge, understanding and skills. Hence, BT can be applied in
e-assessment for STQs as well. This
would provide the theoretical base for development of a new model of
e-assessment using systematic and well appreciated approach of BT. Which may further pave way for development of
Artificial Intelligence (AI) / Machine Learning (ML) based techniques for
evaluation of STQs in future. Additionally, it will not only help presentation of different question types but also ensure that higher cognitive skills can also be covered, thus making such systems pedagogically valuable.
Finally, it will be a fast,
accurate, cost-effective, reliable and valid assessment method. Additionally,
it will replace manual grading system which is costly, time consuming, lacks of
consistency and slow in feedback. Furthermore,
the ultimate beneficiary will be teachers, learners and Institutions leading to
system development. It will provide instant
feedback, faster evaluation, diagnosing learner’s hitches, evaluating the
teaching methods, save precious time of learners, evaluators and ensure
authentic assessment.
References:
Alruwais, N., Wills, G., & Wald, M. (2018). Advantages
and Challenges of Using E-assessment. International Journal of Information and
Education Technology, 8(1), 34-37.
Alrehily, A. D., Siddiqui, M. A., & Buhari, S. M. (2018)
INTELLIGENT ELECTRONIC ASSESSMENT FOR SUBJECTIVE EXAMS. Computer Science
& Information Technology, 47.
Anderson, L. W., Krathwohl, D. R., Airasian, P. W.,
Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., Raths, J & Wittrock, M.
C. (2001). A taxonomy for learning, teaching, and assessing: A revision of
Bloom’s taxonomy of educational objectives, abridged edition. White Plains, NY: Longman.
Agrawal,
L., Mhalunge, V., Pitale, B., & Bahir, K. (2017). Subjective Answer Evaluation
System. IJSRSET Volume 3, Issue 1, 514-517.
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H.,
& Krathwohl, D. R. (1984). Taxonomy
of educational objectives: Handbook 1: Cognitive domain. Longman
Publishing Group.
Cannon, Newble, D. & Robert. (1995). A handbook for teachers in universities
& colleges. London: Kogan Page.
Chalkboard, D. (2018). Selected-Response
vs. Constructed-Response Assessment Methods, Accessed on:
October 26, Retrieved from: https://www.mydigitalchalkboard.org/portal/default/Content/Viewer/Content?action=2&scId=505706&sciId=15387.
Clements, M. D., & Cord, B. A. 2013. "Assessment
guiding learning: developing graduate qualities in an experiential learning
programme. ." Assessment &
Evaluation in Higher Education, 38(1), 114-124.
Crisp, G. (2011). Teacher's
Handbook on e-Assessment. Sydney: Australian Learning and Teaching
Council Fellowship.
ConductExam. (2018). Subjective Type Questions. Accessed on:
October 08, Retrieved from: https://blog.conductexam.com/subjective-type-questions.
Dube, T., & Ma, M. (2014). Marking short free text
responses in e-assessment. In Workshop Chair (p. 16).
De Bruyn, E., Mostert, E., & Van Schoor, A. (2011,
September). Computer-based testing-the ideal tool to assess on the different
levels of Bloom's taxonomy. In Interactive
Collaborative Learning (ICL), 2011 14th International Conference on (pp.
444-449). IEEE.
ETS. (2018). How the
e-rater® Engine Works, Accessed on:
October 05, Retrieved from:
https://www.ets.org/erater/how/.
FinanceOnline. (2018). Compare Moodle vs Blackboard, Accessed
on: October 05, Retrieved from: https://comparisons.financesonline.com/moodle-vs-blackboard-learn.
Isaacs, G. (1996). Bloom's
taxonomy of educational objectives. Brisbane: The University of Queensland.
Lajis, A., Nasir, H. M., & Aziz, N. A. (2018, February).
Proposed Assessment Framework Based on Bloom Taxonomy Cognitive Competency:
Introduction to Programming. In Proceedings of the 2018 7th International
Conference on Software and Computer Applications (pp. 97-101). ACM.
Kuh, G. D., Jankowski, N., Ikenberry, S. O., &Kinzie, J.
L. (2014). Knowing what students
know and can do: The current state of student learning outcomes assessment in
US colleges and universities. Champaign, IL: National Institute for
Learning Outcomes Assessment.
Kaur, N., & Jyoti, K. (2012). Automated assessment of
short one-line free-text responses in computer science. International
Journal of Computer Science and Informatics, 2(1), 105-109.
Melovitz Vasan, C. A., DeFouw, D. O., Holland, B. K., &
Vasan, N. S. (2018). Analysis of testing with multiple choice versus open‐ended
questions: Outcome‐based observations in an anatomy course. Anatomical sciences
education, 11(3), 254-261.
Moodle. (2018). Moodle,
http://moodle.org/, Accessed on: October 05, Retrieved from:
http://moodle.org/.
Nikolova, M. (2012). "Characteristics and Forms of the
Electronic Assessment of the Knowledge ." Scientific works of the University of Rousse 94.
Praveen, S., & Khan, A. H. J. (2014). An Approach to
Evaluate Subjective Questions for Online Examination System. Published in International Journal of
Innovative Research in Computer and Communication Engineering, 2(11).
Sangi, N. A. (2008). Electronic assessment issues and
practices in Pakistan: a case study. Learning, Media and Technology, 33(3), 191-206.
Sankar, A. (2015). Towards an automated system for
short-answer assessment using ontology mapping. Int. Arab J. e_technology, 4(Guha), 1989.
Kurwongbss. (n.d). Planning Sheet, Accessed on: October 19, Retrieved from:
https://www.niu.edu/facdev/_doc/blooms_planningsheet.doc.
SQA. (2005). Guidelines
on e-assessment for Schools . Glasgow: Scottish Qualifications
Authority.
Stodberg, U. (2012). "A research review of e-assessment
." Assessment & Evaluation in
Higher Education 591-604.
No comments:
Post a Comment