Student Evaluations of Teaching Are Mostly Awfully Wrong
Noel Otu
The University of Texas Rio Grande Valley, Brownsville, Texas, USA
Ntiense E. Otu
South Texas College, McAllen, Texas, USA
ABSTRACT
Student evaluations of teaching (SETs) have been used, researched, and debated for many decades. It is a common practice in higher education institutions, with the supposed purpose of improving course quality and effectiveness, but with unintended consequences of encouraging and motivating poor teaching and causing grade inflation. There is strong evidence that SET “effectiveness” does not measure teaching effectiveness. This paper reviews empirical research examining common concerns about the usefulness (positive and negative) and accuracy of SETs. The findings reveal that student satisfaction relates to their anticipated/expected grades in their courses; hence, they want to get good grades and their instructors want to get a good rating of SET, and this results in grade inflation. The key points are that SETs (1) allow students to speak their “mind”, (2) have no compelling correlation between quality of teaching and learning effectiveness, (3) reward easy, less demanding, and lazy teachers with a positive rating, (4) are biased against gender, attractiveness, ethnicity, race, etc., (5) are weaponized against “some” faculty members, and (6) are like asking convicts awaiting sentencing to evaluate the judge or jurors who convicted them.
KEYWORDS: Student evaluation of teaching, tenure and promotion, retaliation, biases
HOW TO CITE:
Otu, N., & Otu, N. E. (2023). Student Evaluations of Teaching Are Mostly Awfully Wrong. Universal Journal of Educational Research, 2(2), 168-183.
BIBLIOGRAPHY
Allen, W. R., Epps, E. G., Guillory, E. A., Suh, S. A., & Bonous-Hammarth, M. (2000). The Black Academic: Faculty Status Among African Americans in U.S. Higher Education. Journal of Negro Education, 69(1-2), 112–127. Retrieved from https://psycnet.apa.org/record/2002-10183-007
Accapadi, M.M. (2007) When White Women Cry: How White Women’s Tears Oppress Women of Color, The College Student Affair Journal 26 (2) p 208-215 (Google Scholar)
Armstrong and Wildmen (2014) Working across Racial Lines in a Not-So-Post- Racial World, In Maritza Ryes, (Ed) Presumed Incompetent – The Intersections of Race and Class for Race and Class for Women in Academia Symposium, Plenary Panel, Retrieved from ttps://commons.law.famu.edu/cgi/viewcontent.cgi?article=1122&context=faculty-research
Abel, R.L. (1990) Evaluating Evaluations: How should Law Schools Judge Teaching, 40J. Legal EDUC. 407-437. HEINOLIN Retrieved from https://heinonline.org/HOL/LandingPage?handle=hein.journals/jled40&div=40&id=&page=
Aleamoni, L.M (1999) Student Rating Myths Versus Research Facts from 1924 to 1998 Journal of Personnel Evaluation in Education 13 no 2 p 153-166
Anderson K.J. and Smith, G. (2005) Students’ Preconceptions of Professors: Benefit and Barriers According to Ethnicity and Gender, Hispanic Journal Behavioral Science, Vol. 27, issue 2.
Boring, A (2017) Gender biases in student evaluations of teaching, Journal of Public Economics 145, 27-41 Retrieval from http://www.sciencedirect.com/science/article/pii/S0047272716301591 https://www.sciencedirect.com/science/article/abs/pii/S0047272716301591
Boring, A., (2015). Gender Bias in Student Evaluations of Teachers, OFCE-PRESAGE- Sciences-Po Working Paper, Retrieved from http://www.ofce.sciences-po.fr/pdf/dtravail/WP2015-13.pdf
Baker, P. and Copp, M., (1997) Gender Matters Most : The Interaction od Gendered Expectations, Feminist Course Content, And Pregnancy in Student Course Evaluations, Teaching Sociology , Vol 25, 29-43., Retrieved from https://www.jstor.org/stable/1319109?seq=1
Benton, S.L.D, Dacon, D., and Pallett, W.H., (2013) Validity of Student Self- Reported Ratings of Learning “Assessment & Evaluation in Higher Education 3894 p. 377-388
Beam, K. S. (1989) The Gender Gap in Law School Classroom Beyond Survival, 14 VT REV 23 29
Borch, I., Sandvoilr and Risor, (2020) Discrepancies in Purposes of Student Course evaluations: What does it mean to be “satisfied”. Educational Assessment Evaluation and Accountability 3283-102 Retrieved from https://eric.ed.gov/?id=EJ1247575
Bavistic, A., Madera, J.M., and Hebl, M.R. (2010) The Effect of Professor Ethnicity and Gender on Students Evaluations: Judged Before Met, Journal of Diversity in Higher Education 3, no 4: p 245-256
Brooks, R.L., (1987) Anti-Minority Mindset in the Law School Personnel Process Towards an Understanding of Racial Mindset, 5 (1) Law & Inequality 1 Retrieved from https://scholarship.law.umn.edu/lawineq/vol5/iss1/1/
Boex, L. (2000) Attributes of Effective Economics Instructors: An Analysis of Student Evaluations. The Journal of Economic Education, Vol. 32, No. 3, 211-227
Burke, M.C., Hughey, A. W. and Cannonier, C., (2019) The Impact of Consumer and Academic Entitlement Attitudes on Student Course and Faculty Evaluations: A Call for Change to Academic Leaders, International Journal of Leadership and Change Vol 7 Issue I
Czekanski, K., Hoerst, B. J., & Kurz, J. (2018). Instituting evidence-based changes to improve first-time NCLEX-RN® pass rates. Journal of Nursing Regulation, 9(1), 11-18. Retrieved from: https://doi.org/10.1016/S2155-8256(18)30049-8
Chowning, K., and Campbell, N. J. (2009) Development and Validation of a Measure of Academic Entitlement: Individual Differences in Students' Externalized Responsibility and Entitled Expectations. Journal of Educational Psychology, 101 p 982-997
Caron, P., (2018) Weaponizing Student Evaluation, Tax Prof Blog Blue Journal Retrieved from https://taxprof.typepad.com/taxprof_blog/2018/09/weaponizing-student-evaluations.html
Clayson, D., (2009) Student Evaluations of Teaching: Are They Related to What Students Learn? Journal of Marketing Education 31(1) p16-30
Cox, S., Rickard, M. K., and Lowery, C., (2021) The Student Evaluation of Teaching: Let’s be Honest – Who is Telling the Truth? Marketing Education Review. Vol. 32, Issue 1.
Caldwell, K.J., (2018) Racial Bias and Student Ratings of Instructors, TEACHING SURVEY, Pitt: University Center for Teaching and Learning Retrieved from https://teaching.pitt.edu/wp-content/uploads/2018/12/OMET-Racial_bias_and_student_ratings_of_instructors.pdf
Clayson, D. E., (2021), A Comprehensive Critique of Student Evaluation of Teaching: Critical Perspective on Validity, Reliability and Impartiality, Routledge, Taylor & Francis Group, eBook.
Contra, J. (1993) Reflective Faculty Evaluation: Enhancing Teaching and Determining Faculty Effectiveness, (Jossey Bass higher and adult education series), San Francisco: Jossey Bass Inc Pub.
Chavez, K., and Mitchell M.W. K (2019) Exploring Bias in Student Evaluations: Gender, Race, and Ethnicity, Cambridge University Press.
Drowling, W. (2000) Why We Should Abolish Teaching Evaluations. The Daily Targum, Retrieved from www.wcdrutgers.net/targeval.htm
Denzin, N., and Lincoln, Y. (2011) The Sage Handbook of Qualitative Research Association, Retrieved from: https://scholar.google.com/scholar?q=Denzin,+N.,+and+Lincoln,+Y+(2011)+The+sage+handbook+of+qualitative+research+Association&hl=en&as_sdt=0&as_vis=1&oi=scholart
Denman, C. L., & Cohn, T. M. (2022). Use of Standardized Testing to Predict NCLEX-RN Success for Associate Degree Nursing Students in a Concept-Based Curriculum. Teaching and Learning in Nursing, 17(4), 378-382. Retrieved from: https://doi.org/10.1016/j.teln.2022.05.001
Dabler-Larsen, P. (2009) Learning Oriented Educational Evaluation in Contemporary Society. In Ryan K. E.,(Ed), The Sage International Handbook of Educational Evaluations 307-322 Los Angeles, (Calif)
Dano, T. and Stensaker, B., (2007) Still Balancing Improvement and Accountability! Development in External Quality Assurance in the Nordic Countries 1996-2006. Quality in Higher Education 13(1) 88-93.
Dovidio, J.F., Gaertner, S.E., Kawakami, K. and Hodson, G (2002) Why Can’t We Just Get Along? Interpersonal Biases and Interracial Distrust. Cultural Diversity and Ethnic Minority Psychology Journal 8(2) Retrieved from https://psycnet.apa.org/record/2002-02415-002
Edmunson, M. (1997) On the Use of Liberal Education as Lite Entitlement for Bored College Students, Harper's Magazine, 295. p 39-49. Retrieved from https://www.ljhammond.com/essay.htm
Esarey. J and Valdes, N (2020) Unbiased, Reliable and Valid Student evaluations Can Still Be Unfair. Assessment & Evaluation in Higher Education, Vol 45, Issue 8, Retrieved from https://www.tandfonline.com/doi/full/10.1080/02602938.2020.1724875
Feagin, J.R., and McKinney, K.D. (2003) The Many Cost of Racism, Rowman & Littlefield, Lanham, Md.
Face, G.M.F (June 23, 2019) The Customer is Always Right is Awfully Wrong. The Canberra Times, Retrieved from https://www.canberratimes.com.au/story/6219583/heres-why-the-customer-is-always-right-is-awfully-wrong/
Freishtat, R. L., (2016) Expert Report on Student Evaluations of Teaching, Ryerson Faculty Association and the Ontario Confederation of University Faculty, Retrieval from https://ocufa.on.ca/assets/RFA.v.Ryerson_Freishtat.Expert.Supplemental.Reports_2016.2018.pdf
Fook, J., Ryan, M. and Hawking, L. (2000) Professional Expertise: Practice, Theory and Education for working in Uncertainty, London: Writing and Birch
Frazier K.N. (2011) Academic Bullying: A Barrier to Tenure and Promotion for African A merican Faculty. Florida Journal of Educational Administration Policy, Fall Vol 5 Issue 1, p. 1-13
Galbaith, C., Merrill, G., and Kline, D. (2012) Are Student Evaluations of Teaching Effectiveness Valid for Measuring Student Learning Outcomes in Business Related Classes? A Neutral Network and Bayesian Analyses. Research in Higher Education, Vol. 53(3) 353-374
Hobson, S. M. and Donna, M.T. (2001) Understanding Student Evaluations: What All Faculty Should Know, College Teaching, Vol. 49, no 1. p 26-31
Heath J.K., Weissman, G.E., Clancy, C.B., Shou, H., Farrar, J.T., & Dine, C.J. (2019) Assessment of Gender-Based Linguistic Differences in Physician Trainee Evaluations of Medical Faculty Using Automated Text Mining. JAMA Network Open, 2(5)
Higgs, K., (2022) A Brief History of Consumer Culture, The MIT Press.
Hefferman T. (2022) Abusive Comments in Student Evaluations of Courses and Teaching: The Attacks Women and Marginalized Academics Endure, High Education, Retrieved from https://link.springer.com/article/10.1007/s10734-022-00831-x
Hamermesh D.S. and Parker, A (2005) Beauty in the Classroom Instructor’s Pulchritude and Putative Pedagogical Productivity, Economics of Education Review 24 (4) 369- 376.
Himmeibach, V, (March 19,2019) Course Evaluation: What’s Wrong and How to Fix it., Retrieved from https://tophat.com/blog/course-evaluation/
Hollis L D (2022) In the Room, But No Seat at the Table: Mixed Methods Analysis of HBCU Women Faculty and Workplace Bullying. Journal of education, Retrieved from https://journals.sagepub.com/doi/abs/10.1177/00220574221102329
Huemer, M., (2001) Student Evaluation a Critical Review. PhilPaper, Retrieved from http://www.owl232.net/papers/sef.htm
Islahi, F., and Nasreen, N. (2013) Who Make Effective Teachers, Men or Women? An Indian Perspective, Universal Journal of Educational Research, 1(4) p 285-293
Johnson, V.E., (2003) Grade Inflation: A Crisis in College Education, New York, Springer
Jan, T (April 7, 2010) Getting Students to Make Their Mark: Fewer Bothering to grade Professors as System Go Online. The Boston Globe, Boston.com.
Kupenda in Reyes, M., (2014) Reflections on Presumed Incompetent The Intersections of Race and Class for Women in Academia. Symposium. The Plenary Panel, Scholary Commons @ FAMU from htttps://link.springer.com/article/10.1023/A:1026447822732
Kopinak, J., (1999) The Use of Triangulation in a Study of Refugee Well Being, Quality and Quantity, International Journal of Methodology Springer, Vol. 33 (2) 169-183.
Kafedric, E., Malec, D., and Niksic, E., (2018) Differences Between Male and Female Secondary School Students in Assessing Their Physical and Health Education Teachers’ Competence, Sport Science 11(1) p. 64-70
Kreitzer, R. J., and Sweet-Cushman, J., (2022) Evaluating Student Evaluation of Teaching: A Review of Measurement and Equity Bias in SETS and Recommendations for Ethical Reform, Journal of Academic Ethics 20 (1) p. 73-84
Lippmann, S., Bulanda, R.E, and Wagenaar, T. C. (2009) Student, Entitlement Question Issues and Strategies for Confronting Entitlement in the Classroom and Beyond. College Teaching, 57 p. 197-204.
Li, D; Benton, S.L., Brown, R., Sullivan, P., and Ryalls, K. R;(2016) Analysis of idea Student Ratings of Instruction system 2015 Pilot Data. IDEA Technical Report No19 Assessed 1 October 2019, ERIC, Retrieved from https://eric.ed.gov/?id=ED573674
Lim, M. H., Kwan, V.S.Y., Cheung, A., and Fiske, S.T. (2005) Stereotype Content Model Explains Prejudice for an Envied Outgroup: Scale Psychology Bulletin 31, no. 1, p 34-47
Lawrence, J. W. (2021) Student Evaluations of teaching are not valid. American Association of University Professors. Retrieved from https://www.aaup.org/article/student-evaluations-teaching-are-not-valid#.Y_zgcXbMK70
Laube, H., Massoni, K., Sprague, J., and Ferber, A. (2007) The Impact of Gender on the Evaluation of Teaching: What we know and what we can do. NWSA Journal 19(3), p 87-104
M.A Martinez., Chang, A., and Welton, A.B. (2017) Assistant Professor of Color Confront the Inequitable Terrain of Academia? A Community Cultural Wealth Perspective, Race, Ethnicity and Education 20(5) p 696-710
March, H. W., (1987) Student Evaluations of University Teaching: Research Findings, Methodological Issues and Directions for future research, International Journal of Educational Research, 11, 253-369.
Mason A. D., (2010) Police Culture and Ethics Toward an Understanding and Expansion of Police Culture and Ethical Research, Graduate Theses and Dissertation paper 11482 Retrieved from https://dr.lib.iastate.edu/entities/publication/242a0143-5814-4e98-95e2-1a1df074de5d
Merritt, D.J (2008) Bias, the Brain and Student Evaluations of Teaching, John, S’s Law Review, Volume 82 #1
Mengel, F., Sauermann, J.,and Zolitz, U., (2019) Gender Bias in Teaching Evaluations, Journal of the European Economic Association, 17(2) p 535-566
Mitchell, M.W., and Martin, J., (2018) Gender Bias in Student Evaluations: Political Science and Politics 51 (3)1-5
Norris, D. (2019). Assessing the Learned Learner When Using a Concept Curriculum in Nursing Education. Research Issues in Contemporary Education, 4(n1) 35-44
N/A #1 (2022) Course Feedback as a Measure of Teaching Effectiveness, Stanford Evaluation & Research, Retrieved from https://evals.stanford.edu/evaluating-teaching/course-feedback-measure-teaching-effectiveness
Ortega - Luston, R., & Rodriguez - Soto, 1. (2014) Challenges, Choices and Decisions of Women in Higher Education: A Discourse on the future of Hispanic, Black, and Asian Members of the Professoriate, Journal of Hispanic Higher Education, 13 (4) p285-302
Onueghergie, A., Daniel, E., and Collins, K. (2009) A Meta-Validation Model for Assessing the Score-Validity of Student-Teaching Evaluations. Quality and Quantity, 43(2), p. 197-209
Olabi, R. M., (2021) The Pregnancy Drop: How Teaching Evaluations Penalize Pregnant Faculty, Humantities and Social Sciences Communications, Article number 253.
Okoye, K., Arrona–Palarios, A., Camacho-Zuniga, C., Hammout N., Luttmann, E., Nakamura, J. E., and Hosseini, S., (2020) Impact of Students Evaluation of Teaching: A Text Analysis of the Teachers Qualities of Gender, International Journal of Educational Technology in Higher Education 17(49).
Pounder, J. S., (2007) Is Student Evaluation of Teaching Worthwhile? An Analytical Framework for Answering the Question. Quality Assessment in Education Vol 15 No 2.
Patitu, C., and Henton, K., (2003) The Experience of African American Women Faculty and Administrators in Higher Education: Has Anything Changed? New Direction for Student Services, 104. Wiley Periodical Online.
Pope, D. C. (2001). “Doing School”: How We are Creating a Generation of Stressed Out, Materialistic, and Miseducated Students. New Haven, CT: Yale University Press
Rodriguez, J., (2019) The Weaponization of Student Evaluations of Teaching: Bully and the Undermining of Academic Freedom, Journal of Academic Freedom, Vol 10
Raban, C., (2007) Assurance “Versus” Enhancement: Less is More Journal of Further and Higher Education 31(1) 77-85. Retrieved from https://www.semanticscholar.org/paper/Assurance-versus-enhancement%3A-less-is-more-Raban/0ceb8a92d16aa073633b5aca4314227916a3a098
Rempel, R. J., (2018) The Validity and Utility of Student Evaluation, Perspective on Social Work, Volume 13 no.1
Reid, I.D., (2010) The Role of Perceived Race and Gender in the Evaluation of College Teaching on RateMyProfessor.com Journal of Diversity in Higher Education 3, no 3. P 137-274
Reyes, M., Kupenda, A. M,, Onwuachi-Willig, A., Wildman, S. M., and Wing, A. K, (2014) Reflection on Presumed Incompetent: The Intersection of Race and Class for Women in Academia Symposium – The Plenary Panel, Scholarly Commons @ FAMU Law. Retrieved from http://digitalcommons.law.scu.edu/facpubs
Radhakristnan, P., Frederickson, M., and Toh, S.M (2021) How Student Evaluations of teaching are Biased Against Women and Ethnic Minority Professors. ResearchGate. Retrieved from https://www.researchgate.net/publication/348152682_How_Student_Evaluations_of_Teaching_are_Biased_against_Women_and_Ethnic_Minority_Professors
Ray, V (February 9, 2018) Is Gender Bias an Intended of Feature of Teaching Evaluation? Inside Higher ED.
Rivera, L.A. and Tilesik, A. (2019) Scaling Down Inequality: Rating Scales, Gender Bias and the Architecture of Evaluation. American Sociological Review 84 (2) p 248-274
Remmars, H. H. and Brandenburg, C. C. (1927) Experimental data on the Purdue Rating Scale for Instruction. Educational Administration and Supervision, 13, 519-527
Remmer, H. H. (1933) Learning, Effort, and Attitudes as Affected by three methods of Instruction in Elementary Psychology, Purdue University Studies in Higher Education (Monograph No 21)
Shulman, L (1988) Disciplines of Inquiry in Education: An Overview. In R. M. Jaeger (Ed) Complimentary Methods for Research in Education (p3-7) American Educational Research Association, Retrieved from https://www.aera.net/Publications/Books/Complementary-Methods-for-Research-in-Education
Spooren, P., Brock, B. and Mortelman, D;(2013) On the Validity of Students Evaluation of Teaching: The state of the Art. Reviews of Educational Research, 83(4) p.598-642
Seldin, P., (1984) Changing Practices in Faculty Evaluation, Jossey Bars, San Francisco, CA
Seldin, P (1993) The Use and Abuse of Student Rating of Professors, The Chronicle of Higher Education, Vol 39 No 46. A40
Searcy Y. D. (June 5, 2017) Let Me Speak to the Manager: Higher Education Should View the Public Not the Student as the Customer, INSIDE HIGHER EDUCATION
Smith P.J. (1999) Teaching the Retrenchment Generation: When Sapphire Meet Sacrates at the Intersection of Race, Gender and Authority, 6 WM + Mary J. Women + L53, 162-163.
Sieverding M, Eib. C., Neubauer A. B. + Stahl T., (2018) Can lifestyle preferences help explain the persistent gender gap in academia? The “mother work less” hypothesis supported for Germany but not for US early career researchers: PLOS ONE 13 (B): e0202728.
Swann J.M. (2022) Academic Bullying and Diversity: Challenges and Solutions, The Scientist Forum, FEBS Letters/Volume 596, Issue 221p 2855-2858
Spooren, P. Brockx, B., and Mortelmans, D. (2013), On the Validity of Student Evaluation of Teaching: The State of the Art. Review of Educational Research, 83(4), 598-642
Sloman, S.A., (2002) Two Systems of Reasoning: in Gilovich, T and Kahnerman, D., (Ed) Heuristics and Biases: The Psychology of Intuitive Judgment, Camdridge University Press 279
Stroebe, W, (2020) Student Evaluation of Teaching Encourages Poor Teaching and Contribute to Grade Inflation: A theoretical and Empirical Analysis, Basic and Applied Social Psychology Vol. 42Issue 4
Stroebe, W, (2016) Why Good Teaching Evaluations May Reward Bad Teaching: On Grade Inflation and Other Unintended Consequences of Student Evaluation, Perspectives on Psychological Science, Vol 11(6) 800-816
Sidanius, J and Crane, M (1989) Job Evaluation and Gender: The Case of University Faculty: Journal of Applied Social Psychology 19, no 2 p 174-197
Subtirelu, N.C. (2015) She Does Have an Accent but – Race and Language Ideology in Student’s Evaluations of Mathematics Instructors on RateMyProfessors.com. Language in Society 44 no 1. p 35-62
Smith, B.P. (2007) Student Ratings of Teaching Effectiveness: An Analysis of End of Course Faculty Evaluation, College Student Journal, 41, p 788-800
Searcy Y.D. (2017) Let me speak to the manager! Higher Ed should redefine customer service, and craft policy with the public not the student most in mind, Inside Higher Ed Careers, Retrieved from https://www.insidehighered.com/views/2017/06/05/higher-ed-should-view-public-not-student-customer-essay
Smith, W.A., Allen, W.R. and Danley, L.L., (2007) “Assume the position --- You fit the description, American Behavioral Scientist 51(4).
Sabbe, E., + Aelterman, A (2007) Gender in Teaching: A literature Services. Teachers and Teaching, 13(5) p 521-538
Stewart, K (2009) Lessons For Teaching Millennials, College Teaching 57p 111-118
Theall M., and Franklin, J., (2000) Creating Responsive Student Rating System to Improve Evaluation Practice, New Directions for teaching and Learning 83, 95-107
Tillman, L.C., (2001) Mentoring African American Faculty in Predominantly While Institutions, Research in Higher Education, 42(3) p295-325.
Trower, C. A., and Chait, R. P., (2002) Faculty Diversity: Too Little for Too Long, Harvard Magazine, Retrieved from https://www.harvardmagazine.com/2002/03/faculty-diversity.html
Titus, J. (2008) Student Ratings in Consumerist Academy: Leveraging Pedagogical Control and Authority. Sociological Perspectives 5/(2): 397-422.
Uttl, B, and Smibert, D (May 9, 2017) Student Evaluation of Teaching: Teaching Quantitative Courses Can be Hazardous to One’s Career. Peer Journal, e3299.Retrieved from https://pubmed.ncbi.nlm.nih.gov/28503380/
Williams, J., (2016) Quality Assurance and Quality Enhancement: is there a Relationship? Quality in Higher Education 22(2) 97-102.
Ward D (2021) An overview of research on use of student surveys to evaluate teaching. University of Kansas,
Wing, A. K (October 29, 2014) Lesson from a Portrait: Keep Calm and Carry On, in Reyes, Maritza I., Angela Mae Kupenda, Angela Onwuachi-Willig, Stephanie M. Wildman, Adrien Katherine Wing, Eds. Reflections on Presumed Incompetent: The Intersections of Race and Class for Women in Academia Symposium - The Plenary Panel, 29 Berkeley J. Gender L. & Just. 195 (2014)
Wang, L and Gonzalez, J. A. (2020) Racial/Ethnic and National Origin Biases in SET. International Journal of Organizational Analysis. Vol 28 No 4
Plagiarism scanned by
UJER is open access and always free
