Mendive. Revista de Educación, enero-marzo 2020; 18(1): 92-104

A look at the evaluation by rubrics through ICT

Una mirada a la evaluación por rúbricas a través de las TIC

Um olhar sobre a avaliação por rubricas através das TIC

Jasiel Félix Ferreiro Concepción¹, Carlos Rafael Fernández Medina¹ 

¹Agrarian University of Havana. Cuba. ORCID: https://orcid.org/0000-0002-0831-0375, https://orcid.org/0000-0002-9599-2625. Email: jasiel@unah.edu.cu, cmedina@unah.edu.cu

Received: April 8^th, 2019
Accepted: January 23^rd, 2020

ABSTRACT

With the objective of evaluating the exhibition of seminars, research papers, courses, work practices or other evaluative activity of the students of the Faculty of Social and Humanistic Sciences of the Agrarian University of Havana with a non-traditional methodology that is through ICT use, specifically with the online evaluation rubrics; An online evaluation rubric was created through the site: corubric.com where students and professors could have greater interactivity in the evaluation process, since the former will be able to evaluate and evaluate themselves through Information Technology and Communications as performed in an exhibition act; and the latter may also evaluate them and have the knowledge of knowing how students individually assessed their peers and themselves, to assess the reliability of the assessments, in case the subjectivity comes before the objective, that is, Since there are bonds of friendship or affinity among those evaluated, they evaluate depending on that and not on what they actually did in the exhibition. For this, it is clarified that the teacher's qualification has a greater weight and that this can veto the evaluation of a student if they perceive a non-objective manifestation in the notes to their classmates or themselves. Data collection and presentation of results will be performed and provided by the site.

Keywords: online evaluation rubric; Information and Communication Technologies; Evaluating exhibitions.

RESUMEN

Con el objetivo de evaluar la exposición de seminarios, trabajos investigativos, de cursos, prácticas laborales u otra actividad evaluativa de los estudiantes de la Facultad de Ciencias Sociales y Humanísticas de la Universidad Agraria de La Habana con una metodología no tradicional (a través del empleo de las Tecnologías de la Información y las Comunicaciones, específicamente con las rúbricas de evaluación online), se creó una rúbrica de evaluación online a través del sitio: corubric.com donde los estudiantes y profesores pudieran tener mayor interactividad en el proceso evaluativo. Los primeros podrán evaluar y evaluarse a sí mismos a través de las Tecnologías de la Información y las Comunicaciones según lo realizado en un acto expositivo; y los segundos podrán también evaluarlos y saber la forma en que los estudiantes, individualmente, evaluaron a sus compañeros y a ellos mismos, para valorar la fiabilidad de las evaluaciones, en caso de que la subjetividad se anteponga a lo objetivo, o sea que, al existir lazos de amistad o afinidad entre los evaluados estos valoren en dependencia de eso y no de lo que realmente hicieron en la exposición. Para ello se les aclara que la calificación del profesor tiene mayor peso y que este puede vetar la evaluación de algún estudiante si percibe una manifestación no objetiva en las notas a sus compañeros o a ellos mismos. La recolección de datos y exposición de resultados serán realizada y proporcionada por el sitio.

Palabras clave: rúbrica de evaluación online; Tecnologías de la Información y las Comunicaciones; evaluar exposiciones.

RESUMO

Palavras-chave: rubrica de avaliação on-line; Tecnologias da Informação e Comunicação; avaliação de exposições.

INTRODUCTION

Today the world is marked by the accelerated changes that the use of Information and Communication Technologies (ICT) entails, before which the educational context cannot remain indifferent. The appearance of these technologies are already part of daily life, in productive and leisure activities (Karsenti, as cited in Tapia, Navarro & Serna, 2017). Additionally, ICTs are permeating all the actors of the university, in particular the professor, whose academic practice will be conditioned, above all, by the technological knowledge he possesses.

The literature review in the field of the introduction of ICT in the university (Lara, Zatarain & Cárdenas, 2013; Sunkel , Trucco & Espejo 2014; Nolasco & Ojeda 2016 and Vinueza & Simbaña , 2017), also point out the benefits of the integration of ICT in the academic practices of teachers, with the purpose of generating new learning environments mediated by technologies. University teachers must understand the educational impact of social changes, know how to anticipate them and continuously generate new ideas that benefit the teaching and learning processes. In addition, they must make intelligent decisions about the adoption of technologies, connecting tools with the application of effective pedagogies.

Each time they intend to incorporate in the Learning Teaching Processes (PEA) more activities that encourage active learning and promote problem solving. Therefore, the responsibilities of teachers should change, they should be more facilitators of learning experiences in other fields, perhaps even other teachers, encouraging students to develop better research habits and ask deeper questions.

On the other hand, the diversity of the students and the educational situations that may occur, advises that the trainers take advantage of the multiple resources available on the Internet and around them to personalize the teaching action, and work in collaboration with other colleagues maintaining a research attitude in the classrooms, sharing resources, observing and reflecting on the didactic action itself and progressively seeking improvements in the actions, according to the circumstances.

The use of the Internet and ICT has made questioned the traditional educational model. At present, it has already been assumed that these are indispensable tools in higher education to respond to new learning paradigms (Hernández, Sánchez, Zarate, Medina, Loli & Arévalo, 2019). ICTs are providing opportunities and new challenges to seek more effective ways of teaching and thus be able to offer greater flexibility, more personalized learning and, ultimately, a more satisfactory training experience (Cabero & Llorente, 2015; Islands, 2017).

From the institutional perspective, the regular implementation of ICT in the teaching-learning processes in Higher Education is signifying a remarkable transformation in the design of new methodologies. But, despite the fact that ICTs are the engine of this transformation, they alone do not provide any added value if they do not influence methodological changes that, little by little, can transform the entire organizational structure of the university.

In addition, with ICT, new and remodeled evaluation and didactic methods can be proposed, in which it involves students with a more active role in their own evaluations and self-assessments in a critical, self-critical and conscious manner. Regarding this, it is stated that:

The task of evaluating must be considered in its qualitative and formative character, integrate it into the pedagogical process, that is, carry it out permanently during learning activities using non-traditional forms of evaluation and, in addition, letting students know what the criteria are which are used to assess their performance, so that this helps them review what they are doing and develop their capacity for self-evaluation, their critical and self-critical spirit (Ministerio de Educación Superior, 2016, p. 16).

Hence, greater synchronization is sought in the evaluation processes, using ICT support, leaving behind traditional old methods of evaluation, making use of one of the Marxist laws: "the Law of Denial of Denial" (Intriago Macías & Intriago, 2017, p.152) where the old is not denied but the fundamental is taken to contribute to the conformation and improvement of the new, giving a more innovative fruit and according to the new times and demands of change that they are living; and for this the use of ICT plays a fundamental role.

With these new methodologies for evaluation, review, elaboration and creation in a synchronous way through digital technologies, that is, at the same time and moment, no matter where they are, educational processes have achieved new levels of satisfaction, precision and coordination. For example, since the review of a work does not have to be that the student delivers it and the teacher reviews it on his part and then (sometimes days later) brings the results, comments and impressions about it by returning the student to take it, fix it and bring it back to start the cycle again, and so it is with almost everything say tutorial reviews, and so on. Now, with digital channels, such as Google Drive , the student and the teacher can review, comment and fix the document at the same time, even if they are not in the same place; Thus there is greater student-teacher interactivity, while the processes become more dynamic and agile.

The present work brings this methodology of evaluation in a digital way that is already being used in the world with some steps in Cuba and the Agrarian University of Havana where in one of its faculties, specifically in the one of Social and Humanistic Sciences, it is tried to implement the evaluation by digital rubrics. For this purpose, the corubric.com site is used and the work expositions of seminars, courses and current research as well as diploma theses, events, work practices, among others, are evaluated.

MATERIALS AND METHODS

With the objective of evaluating the exposure of seminars, research papers, courses, work practices or other evaluative activity of the students of the Faculty of Social and Humanistic Sciences of the Agrarian University of Havana, with a non-traditional methodology through employment of ICT, specifically with the online evaluation rubrics, a first approach is made. The group of 1st year of the law career of this faculty is selected, with enrollment of 25 students, who receive at the time of the experiment the subject of Computing, which teaches one of the authors of this work. Therefore, an intentional selection will guide the proposal in all the evaluation systems mentioned above in the faculty.

For this purpose, an online evaluation rubric was created through the corubric.com site , where students and teachers could have greater interactivity in the evaluation process; since the former will be able to evaluate and evaluate themselves through Information Technology and Communications as performed in an exhibition act and the latter may also evaluate them and know the way in which the students, individually assessed their peers and themselves, to assess the reliability of the evaluations in case the subjectivity comes before the objective, that is, when there are bonds of friendship or affinity among the evaluated, they value depending on that and not on what they actually did in the exhibition.

For this, it is clarified that the teacher's qualification has a greater weight and that this can veto the evaluation of a student if they perceive a non-objective manifestation in the notes to their classmates or themselves. Data collection and presentation of results will be performed and provided by the site.

A non-traditional methodology is chosen because it is intended that students also take part in the conformation of their evaluations and are able to evaluate and self-evaluate in a critical, self-critical and conscious manner. Since "The education that this era demands requires a radical transformation of the objectives, methods and contents of our plans and programs, of our classes, of our role in the teaching process to place the student in the spotlight (... ) " (Alarcón, 2015, p.3).

In addition, with the support of digital and educational technologies, greater interactivity, dynamics and rapid projection of the results are possible, while taking into account that it is more striking to carry out this exercise for students because it is something new and more participatory

This is expressed by the Ministry of Higher Education (2007) , in its resolution No.210 / 2007 that norms and regulates the teaching-methodological work of higher education, in its chapter IV related to the evaluation of learning, reflecting it as a process that It must have a systemic and bidirectional character. This bidirectional character is expressed in the feedback that is established between the teacher and the teacher, in which the teacher perfects the teaching-learning process, investigating the development of student learning to solve problems in the profession.

The students help them to create study habits to favor the increase of their cognitive activity, develop the responsibility for the study, the ability to self-assess their achievements and difficulties in the learning process. All this is a dynamic process in which not only the teacher evaluates, but also encourages student participation through group evaluation and self-evaluation.

Developing these principles is a challenge for the teacher in terms of continuing education, dedication to the preparation, projection, adaptation to the curriculum and curricula, for the successful completion of evaluation activities. This implies greater commitment to the process, getting out of your comfort zone (Marina, Pellicer & Manso, 2015; Roig, 2016), in addition to replace the evaluation through merely reproductive and memorial tasks, with a significant one that enhances creative and developer thinking.

For this, according to Fernández (2017) "It is important to investigate the potential of these tools (digital rubrics) in the context of the transformations of higher education in Cuba, based on the new teaching and learning models that arise, they suppose a high component of the use of the TIC (…) which allow a greater collaborative work of students and professors" (p.3)

These transformations demand a solid preparation of the teachers, which was raised during a working session with the technical mission of UNESCO for education projects in Cuba: "The Cuban classroom requires digital skills in advance from its university teachers. At the national level, the development of technical-professional skills in the efficient use of ICTs is translated as a necessity, while educating teachers capable of producing educational content with these resources "(UNESCO Technical Mission, 2015, p. 3).

RESULTS

To bring both professors and students of the Faculty of Social and Humanistic Sciences closer to the subject, we will begin by evaluating the exposure of a common research work in a subject to, according to the results, bring this system to the other processes and evaluative exercises before mentioned. A rubric is created to evaluate exhibitions on that site and they are sent to students by email.

The students, once all the registration procedures have been done on the site, review the rubric and present their criteria in this regard as to whether it is understood or not, if the questions are consistent with what they are intended to evaluate and if they understand the form to evaluate; besides that they already know the aspects that will be measured and thus they will know consciously, as they expose, if they are arriving or they arrived at what they were asked for.

For the experimental test of this exercise, the 1st year group of the Law degree of the aforementioned faculty was chosen, where at first they were carried out an evaluation exercise in which they had to expose a certain content and be evaluated by this online technique of the rubrics, with one made and previously agreed between the teacher and the students. Here the results were:

• The averages of the marks awarded by the students as a whole with that of the teacher are entirely different from those provided by the teacher (Table 1).

Table 1 - Average of rubric evaluations

Source: taken from the corubric.com site.

• Of a total of 26 evaluators and 25 evaluations that should exist, counting the teacher, only between 8 and 10 could evaluate and 13 be evaluated, since the majority had not yet done all the registration procedures, alluding to problems with the technologies and time on the machine (PC / Computer).
• All the students who evaluated were provided and provided their peers with the highest grade, only some did not because they missed marking one of the headings of the rubric.
• All the students who evaluate assessed their classmates' exposure in the same way without taking into account that not all of them did the same or managed to reach the same levels they were asked to reach.

From these results, it was perceived that it is necessary to reinforce the work with the students regarding the awareness of critically and self-critically evaluating themselves and their classmates. However, taking into account these details for future evaluations through this methodology, once the evaluations and assessments in this regard were completed, another rubric was made, but with the objective of knowing the level of satisfaction with this new evaluation procedure for them .

From the results of the latter, it would be determined if they saw relevance, whether they liked it or not, that they were evaluated in this way, if they preferred this mode of evaluation, whether they wanted to be an active part or not of their evaluation, among others.

The results of the satisfaction rubric were:

• 100 % of the students who evaluated and the respondents subsequently prefer this method of evaluation.
• Everyone perceives this new way more dynamic and participatory, where they have a more active role in their evaluation.

Thus, it was determined to subsequently carry out a formal evaluation to the students on a subject of the subject Computing, since it is one of those taught by the authors of this work; In addition, it allows fostering and strengthening new digital skills in students. Here they would have to present the results regarding their research on that subject.

Previously they were reminded and clarified that they would be evaluated through an online rubric, which had already been analyzed with them the previous time and that as a whole teacher / student would evaluate their own exposure and that of their peers on the subject that corresponded to them.

The rubric takes into account the following aspects: language, coherence, state of the presentation used to exhibit, use of technologies as support for the exhibition and creativity, where in each one it can be indicated from the lowest level in this regard to the highest level they could reach (Ferreiro, 2017). For example, in terms of coherence, they may select, depending on their own or other classmates' exposure, if they had (in other words) a bad, regular, good or excellent coherence in their speech; thus, all the others have the same evaluative logic.

Once the exercise was carried out and the online rubric was evaluated, the results obtained and the statistics thrown in this regard were assessed by the site. The results projected by the site can be seen in it, referenced above; but, analyzing them in more detail, it can be said that:

• The averages of the marks awarded by the students, together with that of the teacher, are mostly far from those provided by the teacher, but not as much as the previous time (Table 2).

Table 2 - Average of the rubric evaluations

Source: taken from the site corubric.com

• Of a total of 26 evaluators and 25 evaluations that there should be, counting the teacher, only 22 could evaluate, in some cases 21, 20; 19 or 18, as shown in the previous table, and 23 could be evaluated, since some students did not attend the meeting and others simply did not evaluate or allowed them to evaluate them by referring to problems with technologies and time in the machine.
• On the other hand, four did not evaluate all their partners, only some, using the aforementioned problems.
• The majority of students were provided and provided their peers with the highest grade or a good grade.
• A large part of the students valued their classmates' exposure in the same way, without taking into account that not all of them did the same or managed to reach the same levels they were asked to reach.

With the results of this first approach, it was considered that this way of evaluating through the online rubric should not be with such large groups, all being evaluated at the same time, since the detailed tabulation of the results becomes very cumbersome in order to determine and assess the evaluation of each one individually and then provide them with a note, according to the aforementioned requirements of critical, self-critical and conscious sense.

It could be evaluated by looking for a way for team representatives to evaluate others and vice versa, or as intended in the future with work practices, where it is only the court who evaluates and the student self-assesses and collectively obtain the Final note. In addition, the need to continue strengthening the work with students regarding the awareness of critically and self-critical evaluation of themselves and their peers is still perceived.

However, they were reapplied with the rubric of satisfaction and the results obtained were the same as the previous time, so it was deduced that the students, projecting in favor of them being carried out more frequently and in other subjects, accepted this evaluation procedure.

Hence, taking into account the aforementioned, it is proposed to maintain and extend these evaluative procedures not only in expositions of research papers, but for exposures of results of work practices, course work, events and even diploma thesis in all the years of the law career and, gradually, involve the other careers of the university; Not only evaluate the exhibitions but also other activities such as the review of works, projects, among others.

Because of this and with knowledge of these results, the faculty will begin to take the first steps in this regard, choosing the group of 1st year of the Socio-Cultural Management for Development, applying the same methodology of exposure evaluation. In future research, a comparison of the results and the degree of satisfaction of both groups will be carried out to determine if it is possible or not to incorporate it into the faculty evaluations. For this, teachers and students will be prepared on the subject and in the manipulation of the rubric on the site.

DISCUSSION

In relation to the rubrics and their implementation in the evaluation processes, there is a vast bibliography; Some authors define it as an evaluation instrument (Panadero & Alonso, 2013; Picón, 2013; Cano, 2015) and others in a broader sense as a methodology, technique and management instrument of the evaluation itself (Cebrián & Bergman, 2014).

On the other hand, Villalustre & Moral (2010) state that "The evaluation rubrics help to assign different weights to each task or sub-task performed in each activity. This helps students to estimate their own qualification of both practices, individual activities and of the group collaborative activity, and its acquired level of skills "(p.98).

Marín, Cabero & Barroso (2012) refers to these as a "scoring guide to evaluate the quality of the answers offered by the students, and also by the teachers, before a specific activity, which can range from a written composition, a multimedia production, a production or research work, or a portfolio "(p.355).

In this case, the evaluation rubric is used in a context of training university students, through the elaboration and interaction in a learning environment, these being very useful for self-training actions, based on the knowledge of the criteria that they will be valued during the activity.

In his research, Fernández (2017) conducts a study of the process of self-evaluation and self-regulation of learning with rubrics, posing as part of the results that these help the student to self-evaluate their work based on the understanding of the evaluation criteria, which reduces their anxiety level, generally improving academic results. In addition, it includes in its perception the role of the teacher, in terms of the contribution of the rubrics to clarify their own evaluation criteria. He recommends that, in order to achieve a more effective use of the same, these must be elaborated or agreed with the students in order to achieve their greater understanding.

Cebrián & Bergman (2014) broadens the conceptual spectrum, placing the rubrics "within a conception of formative evaluation, constituting an evaluation methodology as well as a technique and instrument for managing the evaluation itself" (p.15). For these authors, the rubric has its incidence, both in students and teachers, stating "the rubric seeks to better understand and internalize the evaluation criteria and quality standards, while teachers understand the best procedures that have caused really the success of learning "(Cebrián & Bergman, 2014, p. 17).

In all cases, the authors agree that, with the rubrics used in a formative evaluation focused on learning, it is intended to ensure that students achieve a greater understanding of learning goals and evaluation criteria, serving as guides to reflect the "processes and content deemed important" (Gatica-Lara & Uribarren-Barrueta, 2013), which allows them to "have elements of judgment to evaluate their own performance and establish improvement measures" (Valverde & Ciudad , 2014) in a self-regulation process, as well as in a feedback system.

It is necessary to reinforce the work with the students regarding the awareness that they critically and self-critically evaluate their classmates and themselves, since they still perceive affinity among the students when evaluating their classmates and do not do so with all the critical sense that this evaluation process should have. This evaluation process implies a high degree of critical sense, self-criticism and awareness of what is being done to obtain a reliable result and as close as possible to reality. The evaluation by online rubrics is projected as a more dynamic and participatory methodology for students.

BIBLIOGRAPHIC REFERENCES

Alarcón, R. (2015). The education sciences in an integrated and innovative university. Recovered from http://www.uh.cu/node/2671

Cabero, J., & Llorente, M del C. (2015). Information and Communication Technologies (ICT): formative scenarios and learning theories. Lasallian Research Magazine, 12 (2), 186-193. Recovered from http://www.redalyc.org/articulo.oa?id=69542291019

Cano, E. (2015). Evaluation by competences in higher education. Madrid: La Muralla, 224.

Cebrián, M., & Bergman, M. (2014). Formative evaluation with e-rubric: approach to the state of the art. REDU. University Teaching Magazine, 12(1), pp. 1522.

Fernández Medina, C.R. (2017). Development of digital skills of teachers and students in formative evaluation processes with ICT. Atlante Magazine: Notebooks of Education and Development. Retrieved from http://www.eumed.net/rev/atlante/2017/08/competencias-digitales-tic.html ISSN: 1989-4155.

Ferreiro, J. (2017). Rubric to evaluate an exposure. Heading prepared by the authors on the site corubric.com, Retrieved from https://corubric.com/index.php?r=rubric%2Fregister&id=3724&k=zcnvhjf1vy7p71jyzl7p5s93zs54pm

Gatica-Lara, F., & Uribarren-Barrueta, T. del NJ (2013). How to make a rubric? Research in Medical Education, 2(1), 61-65. https://doi.org/10.1016/S2007-5057(13)72684-X

Hernández, RM, Sanchez, I., Zarate, J., Medina, D., Loli, T., & Arévalo, G. (2019). Information and Communication Technology (ICT) and its practice in educational evaluation. Purposes and Representations, 7(2), 1-10. http://dx.doi.org/10.20511/pyr2019.v7n2.328

Intriago Macías, C., & Intriago Zambrano, E. (2017). Science, technology and society seen from higher education in Ecuador. San Gregorio Magazine, 19, 146-153. Recovered from https://dialnet.unirioja.es/descarga/articulo/6265804.pdf

Islas, C. (2017). The involvement of ICTs in education: Scope, Limitations and Prospective. Iberoamerican Magazine for Educational Research and Development, 8(15).http://doi:10.23913/ride.v8i15.324

Lara, J., Zatarain, C., & Cárdenas, A. (2013). Process of appropriation of ICT by teachers. In SAT Velandia and J. de J. Lara (Coords.), Uses and appropriation of ICT: experiences in the educational process. 17-52. Recovered from http://sistemanodalsinaloa.gob.mx/archivoscomprobatorios/_13_libro/63.pdf .

Marín, V., Cabero, J., & Barroso, J. (2012). The evaluation rubric in the university teacher training process. The proposal of the DIPRO 2.0 project. Educate, 48, 347-364.

Marina, J., Pellicer, C., & Manso, J. (2015). White paper of the teaching profession and its school environment.

Ministry of Higher Education of Cuba. (2007). RESOLUTION No. 210/2007. Havana: MONTH

Ministry of Higher Education of Cuba. (2016). Base Document for the design of the "E." curricula Cuba: MONTH.

MINREX (2015). UNESCO technical mission advances education projects in Cuba. Havana: MINREX. Recovered from http://www.minrex.gob.cu/es/mision-tecnica-de-la-unesco-avanza-proyectos-de-educacion-en-cuba.

Nolasco, P., & Ojeda, MM (2016). The evaluation of the integration of ICT in higher education: foundation for a methodology. RED-Distance Education Magazine, 4, 124. http://doi:10.6018/red/48/9

Panadero, E., & Alonso, J. (2013). Review on educational self-evaluation: empirical evidence of its implementation through self-assessment without evaluation criteria, rubrics and scripts. Journal of Research in Education, 11(2), 172197.

Picón Jacome, É. (2013). The rubric and justice in the evaluation. Íkala, Journal of Language and Culture. 18(3). Available at: https://www.redalyc.org/comocitar.oa?id=255030038006

Roig-Vila, Rl (2016). Education and Technology Proposals from research and educational innovation. Barcelona: Octahedron, ISBN 978-84-9921-847-2, 570 p.

Sunkel, G., Trucco, D. & Espejo, A. (2014). The integration of digital technologies in schools in Latin America and the Caribbean A multidimensional view. Chile: Economic Commission for Latin America and the Caribbean.

Tapia, C., Navarro, Y. & Serna, A. (2017). The use of ICT in the academic practices of the professors of the Benemérita Autonomous University of Puebla. Electronic Journal of Educational Research, 19(3), 115-125.

Valverde, J. & Ciudad, A. (2014). The use of e-rubrics for the evaluation of competencies in university students. Study on instrument reliability. Redu: University Teaching Magazine,12(1), pp. 4979

Villalustre, L., & Moral, ME (2010). E-PORTFOLIOS Y RÚBRICAS DE EVALUACIÓN IN RURALNET. Pixel-Bit Media and Education Magazine, (37), 93-105. Recovered from http://www.redalyc.org/articulo.oa?id=36815118008

Vinueza, SF, & Simbaña, VP (2017). Impact of ICT in Higher Education in Ecuador. Publicando Magazine, 4(11), 355-368.

Conflict of interests:

The authors declare no conflicts of interest.

Authors' contribution:

The authors have participated in the writing of the work and analysis of the documents.

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright (c) Jasiel Félix Ferreiro Concepción, Carlos Rafael Fernández Medina