La validez y confiabilidad de los instrumentos de recolección predicen la efectividad de los programas de enseñanza-aprendizaje reportados en las tesis doctorales sustentadas en la Escuela de Posgrado de la Universidad Nacional de Educación 2005-2015
No Thumbnail Available
Date
2021-10-04
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Universidad Nacional de Educación Enrique Guzmán y Valle
Abstract
Para hacer aportes que contribuyan a expandir el conocimiento, la investigación en
ciencias de la educación debe emplear un método de investigación riguroso. Ello supone
usar procedimientos robustos e instrumentos de recolección de datos válidos y confiables.
Sin embargo, en el caso de las tesis doctorales en educación este supuesto no siempre se
cumple por la presión que tienen las instituciones académicas por elevar su producción
científica. El propósito de este estudio es determinar qué relación existe entre el tamaño
del efecto y la validez y confiabilidad de los instrumentos de recolección, así como con el
tamaño de muestra empleada, en aquellas tesis doctorales enfocadas en los programas de
enseñanza-aprendizaje orientados a mejorar el logro de aprendizaje de los estudiantes de
educación superior. Para ello se examinaron 54 tesis con diseño (cuasi) experimental que
se sustentaron entre los años 2005 y 2015 en la Escuela de Posgrado de la UNE. Dichas
tesis reportaron un total de 153 efectos. La ficha de recojo de datos permitió registrar
información referida a la validez y confiabilidad de los instrumentos empleados, así como
datos sobre los grupos experimental y control (e.g., media, desviación estándar y número
de participantes por cada grupo). El tamaño del efecto promedio y para cada intervención
se calculó usando la d de Cohen y la g de Hedges, incluyendo su p-valor e intervalo de
confianza al 95%. Para la meta-regresión se usaron como covariadas las variables tipo de
validez, magnitud del coeficiente de confiabilidad y tamaño de cada grupo. Los resultados
mostraron magnitudes de tamaño del efecto inusuales, que no guardan correspondencia
con lo reportado en la literatura académica. Para explicar este hallazgo se planteó la
hipótesis de inflación del tamaño del efecto, producto de cómo se diseñaron los
instrumentos de recolección de datos. Al final se presentan algunas recomendaciones para
diseñar instrumentos de recolección de datos válidos y confiables.
To contribute to the expansion of knowledge, educational research must use a rigorous research method. This goal requires using robust procedures, and valid and reliable data collection tools. However, regarding doctoral dissertations in education, this premise is not always fulfilled because academic institutions face a huge pressure for increase their scientific production. The purpose of this investigation is to determine what is the relationship between the effect size and the validity and reliability of data collection tools, as well as the used sample size, in those doctoral dissertations focused on learning teaching programs targeted to improve learning outcomes of higher education students. For that reason, 54 dissertations with a (quasi) experimental design and that were defended at the Postgraduate School of the Universidad Nacional de Educación, between the years 2005 and 2015 were analyzed. Those dissertations reported 153 interventions. The data collection form enabled the registration of information referred to the validity and reliability of the data collection tools, as well as data on experimental and control groups (e.g., mean, standard deviation, and the number of participants for each group). The average and specific effect size were computed using the Cohen’s d and Hedge’s g, including its p-value and confidence interval at 95%. For the meta-regression, the type of validity, the magnitude of the reliability indicator and size of each group were used as covariates. Results showed unusual magnitudes for the effect sizes, in sharp contrast with the effects reported in the academic literature. To explain this finding it was proposed the hypothesis of effect size inflation, a consequence of how were designed these data collection tools. Finally, are detailed some recommendations to design valid and reliable data collection tools.
To contribute to the expansion of knowledge, educational research must use a rigorous research method. This goal requires using robust procedures, and valid and reliable data collection tools. However, regarding doctoral dissertations in education, this premise is not always fulfilled because academic institutions face a huge pressure for increase their scientific production. The purpose of this investigation is to determine what is the relationship between the effect size and the validity and reliability of data collection tools, as well as the used sample size, in those doctoral dissertations focused on learning teaching programs targeted to improve learning outcomes of higher education students. For that reason, 54 dissertations with a (quasi) experimental design and that were defended at the Postgraduate School of the Universidad Nacional de Educación, between the years 2005 and 2015 were analyzed. Those dissertations reported 153 interventions. The data collection form enabled the registration of information referred to the validity and reliability of the data collection tools, as well as data on experimental and control groups (e.g., mean, standard deviation, and the number of participants for each group). The average and specific effect size were computed using the Cohen’s d and Hedge’s g, including its p-value and confidence interval at 95%. For the meta-regression, the type of validity, the magnitude of the reliability indicator and size of each group were used as covariates. Results showed unusual magnitudes for the effect sizes, in sharp contrast with the effects reported in the academic literature. To explain this finding it was proposed the hypothesis of effect size inflation, a consequence of how were designed these data collection tools. Finally, are detailed some recommendations to design valid and reliable data collection tools.
Description
Keywords
Validez,, Confiabilidad, Tamaño del efecto
Citation
Vilchez Roman, C. M. (2021). La validez y confiabilidad de los instrumentos de recolección predicen la efectividad de los programas de enseñanza-aprendizaje reportados en las tesis doctorales sustentadas en la Escuela de Posgrado de la Universidad Nacional de Educación 2005-2015 (Tesis de doctorado). Universidad Nacional de Educación Enrique Guzmán y Valle, Lima, Perú.