Mendive. Revista de Educación, oct.-december 2019; 17(4): 497-511

Translated from the original in Spanish

Las notas de la Enseñanza Media como predictor del desempeño estudiantil en la Educación Superior Técnico Profesional

Claudia Patricia Ovalle Ramirez

Centro de Justicia Educacional Pontificia Universidad Católica de Chile. Chile. E-mail: covallera@gmail.com

Received: April 1^st, 2019.
Accepted: September 10^th, 2019.

ABSTRACT

In the current context, which seeks to establish the mechanisms of admission and selection in higher education in Chile, academic measures have been proposed for selection purposes. The objective of this study is to identify if middle school grades (NEM) are related to the academic performance of students in the first year of technical higher education. Bivariate correlations and least squares regression models (OLS) were used, controlling for variables of the individual and the school to establish the impact of the NEM on the results of the subjects of the first year of technical higher education. Student data from a professional institution (IP) of higher education in Chile was used with an enrollment of approximately 101,000 students, which is one of the largest providers of this type of training in Chile. The sample included students who enrolled in 2018. The results indicated that NEM is a variable that has a high correlation with the higher technical education grades and the regression models confirm a positive and significant relationship. However, the magnitude of the coefficients may be indicative that NEM is not a strong predictor of future performance in all technical careers.

Keywords: predictive validity; middle education notes NEM; professional technical education.

RESUMEN

En el contexto actual donde se busca establecer mecanismos de admisión y selección a la educación superior en Chile se ha propuesto que las medidas de tipo académico puedan servir el propósito de seleccionar estudiantes para los estudios técnicos superiores. El objetivo de este estudio fue identificar si las notas de la Enseñanza Media (NEM) están relacionadas con el rendimiento académico de los estudiantes en el primer año de Educación Superior Técnica. Se emplearon correlaciones bivariadas y modelos de regresión por mínimos cuadrados (OLS) controlando por variables del individuo y de la escuela para establecer el impacto de las NEM en los resultados de las asignaturas del primer año de Educación Superior Técnica. Se usaron los datos de estudiantes de una institución profesional (IP) de educación técnica superior de Chile con una matrícula de aproximadamente 101.000 estudiantes, la cual es una de las mayores proveedoras de este tipo de formación en Chile. La muestra incluyó estudiantes que se matricularon en el año 2018. Los resultados mostraron que las NEM son una variable que tiene una alta correlación con notas de educación superior técnica y los modelos de regresión confirman una relación positiva y significativa. Sin embargo, la magnitud de los coeficientes puede ser indicativa de que las NEM no son un predictor fuerte del desempeño futuro en todas las carreras técnicas.

Palabras clave: validez predictiva; notas de enseñanza media NEM; educación superior técnica.

INTRODUCTION

One of the important issues currently of Chilean higher professional and technical education is the legal mandate to establish a system of admission and selection for aspiring of technical studies in h8igher education (Ministry of Education, 2017). Among the possible selection systems, it has been suggested to include a measure of previous school performance that is not related to socio - demographic variables to make the selection of students more fair (Sevilla, 2015). However, medium technical education form skills and abilities that exceed academic teaching and that may not be reflected with measures of academic school performance such as grades. This implies that the selection of technical students to enter higher level may be biased and be affected by the use of measurement to establish their pre-admission to higher education skills.

Previous studies in the Chilean context have shown that the selection of students for higher education with standardized performance tests is not fair in terms of the bias presented by the items, favoring students who come from technical colleges (Ovalle-Ramírez & Alvares, 2019). In the context of higher - level technical training it is necessary to produce evidence on the appropriateness of using a measure of academic performance, like the notes of teaching media for selection to higher technical education. This evidence should demonstrate whether school performance scores have predictive value of future performance in the context of higher education. This measurement can be affected by variables of the individual, family and institutions of secondary and higher education.

In the present study, we try to control these characteristics (of the individual and of the institutions) and use a first-year sample of students enrolled in an IP (Professional Institute) to establish evidence of the predictive validity of middle school grades.

Literature Review

Secondary education in Chile is the last 4 years of schooling and grades obtained s in these courses are considered, by many institutions of the higher education , as a criterion for admission.

Literature has referred to the NEM (notes high school) in terms of predictive ability of these results have on the future performance of the student. While, some studies indicate that the NEM have great predictive power (Betts & Morell, 1999; Advisory Council of Rectors of Chilean Universities-CRUCH Technical Committee, 2006; Geiser & Studley, 2001) or the authors sign that the contribution of the NEM academic achievement in higher education is limited (Medina, Aguirre & Luengo, 2014).

The evidence of the predictive capacity of the average grades in higher education, comes from the literature and studies on institutions or universities Bastías, Villaroel, Zuñig , Marshall, Velasco & Beltrán (2000) , in a performance prediction model  the first year in the medicine career of 724 students , found that the N EM significantly predict the weighted grade point average in the third year , but usually , it is reported on a scale of 1.0 (minimum) to 7, 0 (maximum). According to the study, approximately 27 points of difference in the NEM score produce a tenth of a difference in the weighted average of grades at the end of 3 years of study. To obtain a similar result in the weighted grade point average, a difference of 53 points is required in the Biology Knowledge Test score for entry into the career. It is concluded that the NEM is a better predictor than the tests selection in Biology, verbal aptitude, or Mathematic aptitude.

The Technical Advisory Committee of the Council of Rectors of the Chilean Universities-CRUCH (2006) reports that NEMs are a factor that did not undergo changes between 2006 and 2015. However, it is striking that in a context where the notes maintain a stable predictive capacity (typically in the range 0.15 to 0.30), in one of the selective universities included in the study, this selection factor does have almost zero predictive validity in all the years studied.

Other studies that support the predictive validity of the NEM include that of Reyes Elgueta & Torres Pavez (2009) who conclude that the high school grades (NEM) and the University Selection Tests (PSU), have the greatest weight in the prediction of academic performance , even above variables and environment (r egion of origin, type of payment of education and gender). In their study the NEM , with both screening tests ( PSU Mathematics and PSU Language) , shows  a lower ratio with respect to variables such as the probability to finish the career, compared with the notes themselves , which have an impact on the probability of finishing the career almost one 5, 7%.

In the literature, several studies indicate that NEMs are not a good predictor of future performance. Perez, Ortiz & Parra (2011) with a sample of 117 students of Medicine related the NEM and proof of income to the university, PSU, cognitive and affective variables that are associated with academic success (self - efficacy, self - esteem, styles of learning and value profile). The results indicate that the PSU score of mathematics is negatively related to the other PSU scores and the NEM. This, according to the authors, would be explained because the NEM does not predict future performance in itself, but rather evaluates other aspects that impact performance and with which it relates significantly. Other aspects hits the learning style methodical study (rho = 0.311, p <0.001), a valor profile which emphasizes kindness (r (108) = 0.303; p <0.01) and the universality (r (108) = 0.326; p <0.01).

Medina , Abu & Luengo (2014) determined the predictive ability of the notes of secondary education comparing them according  to dependence of  school for a sample of 551 students undergraduate in Dentistry whose NEM average reaches  6.59 (0, 20 SD) , on a scale of 1 to 7 . It is concluded that, on average, the percentage of contribution of high school grades to the explanation of university academic performance corresponded to 10.8%. The highest contribution percentage was for the NEMs of private schools, 15.0%; followed by the high school grades of the municipalized and subsidized establishments with 9.6% and 8.6 % respectively. It is concluded that the predictive capacity of the NEMs was limited and that there is a prediction bias that disadvantages municipalized and subsidized schools.

Betts & Morell (1999) observed in a sample of university students that an increase of one point in school NEM translates into an increase in university GPA (grade point average) of only 0.53 points in regression models that account for 10% of the variation of the GPA. These models include the average grades, but also socio-demographic and resource variables of the school (eg educational level of teachers). The research emphasizes the importance of the variability that comes from the differences between schools, which can affect the response variable and in the way socio-demographic variables affect the GPA at the university level (students with lower economic resources tend to to have worse results).

Since there are no studies in the literature that relate NEMs to performance in technical higher education , the present study provides evidence of the predictive validity of middle school grades (NEM) in performance in professional technical training Top level S and used performance data of students who completed their first year in technical programs (duration of 2 years and a half and leading to the title of Senior Technician) and professional without BSc (duration of 4 years) of professional  institution  (IP) with largest enrollment in Chile (around 101,000 students by 2018). This quantitative study tries to contribute with empirical evidence on the potential of the NEM for selection and classification of students entering higher technical studies in Chile.

MATERIALS AND METHODS

Data

Middle School Notes (NEM) .Middle Teaching notes are the average GPA for each year (1 ° to 4 °) of the Middle Education , approximate to the second decimal . They have a scale of 1 to 7 points. The average of the NEM is transformed to a standard score, by means of conversion tables, thus constituting the NEM score , one of the selection factors for the entrance to the selective universities or Universities of the Council of Rectors - CRUCH- and those private attached to the System. Its minimum is 150 points and its maximum is 850.

Total Average Notes. It is the average of the marks of the Career subjects and the marks of the School subjects, which a student obtains when completing the first year of studies in a technical career or a professional career without a license. The Career grades correspond to the signature of each curriculum and the School subjects correspond to the subjects that are shared between several curriculum or careers of the same technical school (Health, Tourism, Administration, Computer Science, Design, Communications, Design, Resources Natural, Engineering).

Universe and Sample

The universe of this study includes students of the professional technical higher education in Chile, around 510,000 students (43% of total enrollment in the higher education). In the present study, data from 40,550 students of the higher technical education are included, selected for convenience of students shows enrolled in the first year in 2018. These students were distributed in 9 technical schools and 79 technical and professional careers and they studied 1 ° and 2 ° s semester of college and P professionals without BA in 2018.

Procedure

Bivariate correlations were calculated between the students' NEM scores and their first-year professional technical training notes. For each correlation value, standardized values ​​(conversions to Z scores) were established to compare the coefficients between schools and between careers. Z scores are deviations are walk that can take negative values and positive and that allow property has a comparisons between the correlations obtained for different schools and technical careers.

Likewise, regressions were developed for each of the technical and professional careers without a bachelor's degree , controlling for sex, age, mother's education, income quintile, year of graduation from secondary education, school dependence (municipal, private subsidized private) and type of license the high school student (technical or scientific humanist) . For the regressions, three models are presented in the table in Annex 2. The first regression model is given by the equation 1:

Equation 1

Equation 1 indicates that the average grade of technical higher education can be predicted from the middle school grades and an error term. This model is the null model that will serve for comparison coefficient with coefficient for the NEM in the models two and three, which also is p resent in the table in Annex 2. The equations two and three define the regression models employed:

Equation 2

The equation 2 includes a feature vector of the individual (mother’s education, entry quintile , age, gender) . Equation 3 includes the same vector of characteristics of the individual and introduces a vector of variables of the educational establishment (dependence, modality, year of graduation from secondary education).

Equation 3

RESULTS

Table 1 presents the results of the correlations between the high school grades with the first year grades of the students considering the school of technical and professional studies without a bachelor's degree. These correlations indicated that there are positive and significant relationships between the variables. However, there is variability in the predictive capacity of middle school grades per school as shown by the Z transformation scores of the correlations that allow comparisons between schools in Table 1.

The results show that while the correlation coefficient between the NEM and notes performance was higher and significant for the School of Natural Resources, Informatics and communications, the coefficient was lower for health and Business Administration.

Tabla 1- Correlations between the NEM and the School

Source: Prepared by the authors based on data obtained from an IP (Professional Institute) with the highest enrollment in Chile. **p<0.001, *p<0.005

The correlations between the NEMs and first year notes of the technical career by curriculum program are shown in Table 2. These show that the association between the NEM grades and the results in the first year of higher technical training were positive and in the majority of the programs were statistically significant. The relationship between variables had a range between 0.09 (correlation coefficient between the NEM and the notes of the first year for the career of Electrical Technician installations and projects) and one 0.65 (correlation coefficient for the career of Geomatics Technician) Careers in which the NEM has a lower coefficient  correlation with the notes of the career that  is on programs of Technical Radiology and Radiotherapy (0.102) and Nutrition and Dietetics (0.175). The coefficient was negative only in the case of the Heritage Restoration program (-0.278).

Tabla 2 - Correlations between the NEM and the Career

Note: **p<0.001, *p<0.005

Tables 3 and 4 show the correlations between the NEM and the notes of the first year of the higher technical education, considering the dependency school and technical secondary mode of the student. In the disaggregation of the correlation by dependence of Table 2, the coefficient was always positive and statistically significant. However, the NEMs were not associated with the results in the Health school when the student comes from paid private schools. However, this result should be interpreted as the number of students in a coach program avalanche that comes from this school sector is smaller than the number of students from municipal and subsidized private schools.

Tabla 3 - Correlations school by school considering dependence

Note: **p<0.001, *p<0.005

As for school mode, which the student comes, the relationship between the NEM and the performance in higher technical education is positive and significant for all schools but, its magnitude is low (<0.35). (See table 3)

Tabla 4 - Correlations by school considering the school modality from which the student comes

Note: **p<0.001, *p<0.005

Through OLS regressions (Ordinary Minimum Squares), the variables of the individual and the school institution were controlled in order to estimate the impact of middle school grades on performance in first year grades in professional technical higher education. Among the Variable control the quintiles were included, the educative modality (technical or humanistic), school unit (municipal, particular subsidized particular), the age, and the sex. The results in table 5 show the coefficients of the NEM variable for each of the 3 regression models developed (equations 1 to 3). Likewise, it includes the variance explained R 2 for each of the models.

In the first model (column 1 Table 5) , which includes only NEM varying , it was observed that the variability explained does not exceed 28 % to except racing Technical Graphic Design (67%) and technician in Renewable Energy (58%) , and for all the careers the NEM coefficient was positive except for the Patrimony Restoration career (-0.997 ) . The NEM increases performance by more than one point in some careers of 3 schools - Administration, Construction and Tourism - and for the careers of: Engineering in Administration, Engineering in Foreign Trade, Digital Animation, Graphic Design, Engineering and Technician in Electricity and Automation, Surveyor Technician and Risk Prevention Engineering, Agricultural Technician, Adventure Tourism and Tourism with an emphasis on Aero-Commercial Services. The other races have an NEM coefficient below one point.

To confirm these results, two additional regression models were developed in which was controlled by varying socio-demographic variables and the Institution school (column 2 and 3, Table 5). The second model indicates that the NEM will increase student results by 1 point only for the Foreign Trade, Audiovisual Technician, Digital Animation, Audiovisual Technician, Graphic Design Technician, Electricity and Automation Technician, Surveyor Technician, Engineering in Risk Prevention, Agricultural Technician, Adventure Tourism, and Tourism with an emphasis on AER. The second model shows that the NEM increased by less than 0.3 tenths the average of grades in technical higher education in acting careers, Agricultural Quality and Safety Technician , Chemistry and Pharmacy Technician.

The third regression model that includes the variables is socio-demographic and those of the school institution from which the student comes , shows that the NEM increase in more than one point the results in the careers of Engineering in Foreign Trade, Audiovisual Technician , Digital Animation, Elect Technician . And Automation, Surveyor Technician, Risk Prevention Engineering, Agricultural Technician, Physiotherapy, Adventure Tourism. However, the NEM did not produce higher increases to 0.3 in the average grade in racing Performance, Technical and n Quality and Safety Livestock, Tourism and Hospitality. Only Patrimonial Restoration has a negative coefficient for the NEM in this model.

Tabla 5 -MCO Regression Models

^a Modelo 1. Nem ^bModelo 2. Nem, educación de la madre, quintiles, edad, sexo ^cModelo 3. Nem, educ madre, quintil, dependencia, modalidad, edad, sexo, egreso media 2017. Nota: **p<0.001, *p<0.005

DISCUSSION

Depending on the type of program, there may be differences in the predictive potential of the performance that the NEM variable may have. The high school grades are a predictor of the future performance of students entering technical and professional programs in some careers, increasing a little more than 1 unit in the average of first-year grades. Although this gain is statistically significant, for some technical programs there may be other factors and s related to performance in higher technical education explain a greater proportion of the variance (NEM explains only a maximum of 30 % of the variability of GPA in 42 of the programs analyzed with model 3 ) . This is supported by the coefficients of the OLS regressions (models 2 and 3), in which by controlling for factors of the individual and the institution it is found that the NEM does not produce increases greater than 0.3 in the average of grades in some Careers such as Performance, Technician in Agricultural Quality and Safety , Tourism and Hospitality.

Studies in higher education indicates that the NEM has a predictive value restricted in the case of the academic results university level (Medina, Aguirre & Luengo , 2014 ; Perez, Ortiz & Parra , 2011) . The present study shows that the NEM varied in its contribution to the prediction of the results in the higher education technique depending on the techmician program, but in the majority of the cases refers to only marginal increases in average grades of technical education superior. In this regard, the use of NEM as a predictor of future performance, for example, for admission and selection of senior technical students requires the limited potential of NEM for some programs is considered.

The evidence of this study is consistent with studies in the higher education indicate that the notes of high school are not the best predictor of future performance in higher education (Betts & Morell, 1999 ; Medina, Aguir re & Luengo , 2014 ; Pérez , Ortiz & Parra, 2011) . In a scenario where free education allows more people enter higher education, an alternative to university education route, using the notes of the high school is not valid for student selection system technical higher education. Other alternatives to measures of academic performance may be the recognition of previous learning, the assessment of the competences of students who come from secondary education and the agreement between the branch or modality of studies in secondary education and higher education.

Although in a context where public resources are restricted, the consequences for economic growth are greater when there is no human capital formation at an adequate level of competition in developing countries (Hanushek, 2019). In this sense, excluding students for their previous academic performance from higher education can affect the life and formative projects of the students.

Future studies can address the predictive value of NEMs by extending this analysis to more institutions of higher technical training or IP and other technical programs not included in the analysis. Similarly, it may include the results in other variables dependent, as is progress or advancement Curricular in terms of credits completed by the student.

BIBLIOGRAPHIC REFERENCES

Bastías, G., Villarroel, L., Zuñiga, D., Marshall, G., Velasco, N., & Beltrán, M. (2000). Desempeño académico de los estudiantes de medicina: ¿Un resultado predecible? Revista médica de Chile, 128(6), 671-678. https://doi.org/10.4067/S0034-98872000000600015

Betts, J. R., & Morell, D. (1999). The Determinants of Undergraduate Grade Point Average: The Relative Importance of Family Background, High School Resources, and Peer Group Effects. Journal of Human Resources, 34(2), 268-293.

Geiser, S., & Studley, R. (2001). UC and the SAT: Predictive Validity and Differential Impact of the SAT I and SAT II at the University of California. Educational Assessment, 8(1), 1-26. https://doi.org/10.1207/S15326977EA0801_01

Medina Moreno, A. del P., Abu Zaid, M. K. S., & Luengo Machuca, L. (2014). Predictibilidad de las notas de enseñanza media según establecimiento de origen sobre el rendimiento académico en estudiantes de Odontología. Educación Médica Superior, 28(1), 65-73.

Ovalle-Ramírez, C., & Alvares, D. (2019). Coloquio Psicología 'Funcionamiento Diferencial en los Ítems de la Prueba de Selección Universitaria PSU: Detección del Sesgo Condicional a la Modalidad de Estudios. Presentado en Coloquio de Postgrados de Psicología, Pontificia Universidad Católica de Chile. Recuperado a partir de http://exestudiantespsicologia.uc.cl/evento/81-coloquio-psicologia-deteccion-del-sesgo -condicional-a-la-modalidad-de-estudios

Pérez, C., Ortiz, L., & Parra, P. (2011). Prueba de Selección Universitaria, rendimiento en enseñanza media y variables cognitivo-actitudinales de estudiantes de Medicina. Revista Educación Ciencias Salud, 8(2), 120-127.

Reyes Elgueta, A. A., & Torres Pavez, M. D. (2009). La PSU y otros Factores de Rendimiento y Éxito Académico Universitario: el caso de la Pontificia Universidad Católica de Valparaíso (Tesis). Pontificia Universidad Católica de Valparaíso, Chile. Recuperado a partir de http://www.pucv.cl/uuaa/site/artic/20190619/asocfile/20190619162259/memoria_2009_reyes_y_torres.pdf

Ministerio de Educación. (2017). Ley No 21.091/2017. Sobre Educación Superior. IIPE, UNESCO, Oficina para América Latina: Buenos Aires. Recuperado 8 de octubre de 2019, de http://www.siteal.iipe.unesco.org/bdnp/950/ley-210912017-educacion-superior

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
Copyright (c) Claudia Patricia Ovalle Ramirez