(how) are student evaluations connected to student learning?

Many have written about problems with student evaluations in terms of, essentially, replicating standard hierarchies of sex, gender, and racial inequalities. But a study was released today providing critical (even devastating) news on the lack of connection between student evaluations and actual student learning. Here’s a snippet from the Inside Higher Ed article reviewing this study:

A new study suggests that past analyses linking student achievement to high student teaching evaluation ratings are flawed, a mere “artifact of small sample sized studies and publication bias.”

“Whereas the small sample sized studies showed large and moderate correlation, the large sample sized studies showed no or only minimal correlation between [student evaluations of teaching, or SET] ratings and learning,” reads the study, in press with Studies in Educational Evaluation. “Our up-to-date meta-analysis of all multisection studies revealed no significant correlations between [evaluation] ratings and learning.”
These findings “suggest that institutions focused on student learning and career success may want to abandon SET ratings as a measure of faculty’s teaching effectiveness,” the study says.

Hmmm. And good to know!

https://www.insidehighered.com/news/2016/09/21/new-study-could-be-another-nail-coffin-validity-student-evaluations-teaching#.V-KgR7Y__XE.twitter

 

Interpreting biased surveys on teaching quality (i.e. student evaluations)

Student evaluations are an important part of most faculty members’ careers. Collectively they have the potential to make or break hiring, promotion, and tenure decisions. The quantitative and qualitative evaluations that students give can also feel extremely personal.

Choose excellent

At the basis of the power of teaching evaluations is the belief that they reflect reality. Yet there is often a serious disconnect between statistical/scientific standards for drawing generalizations from data, and the seeming fetishization by some administrators of student evaluations.

Ideally, the moment of teacher evaluation allows students to freely express the truths of their experience in the classroom; this then serves to both empower the student, and provide the instructor constructive feedback on what worked and what could be improved.  But in practice the ritual of writing and receiving evaluations has many pitfalls.

  • For students, the teaching evaluation moment may seem futile (why bother to provide feedback since the course is already over?).
  • For faculty members, the moment of opening and reading one’s teaching evaluations may be filled with anxiety, dread, and even trauma. This may be especially true for women, faculty of color, queer and trans faculty, and other minoritized faculty members.

The bottom line is that student evaluations as currently institutionalized are full of flaws. Because these flaws impact the livihoods and careers of faculty members, we need to do a serious overall of how to both evaluate and to interpret student evaluation data. In the words of  this Inside Higher Ed author:

By now, many of us know about the research that shows that college students’ ratings of their professors are influenced by expectations associated with professors’ gender, class, race and age. Because these ratings influence hiring, promotions, raises and opportunities for awards, we cannot simply dismiss them; instead, we must deal with them head-on.

https://www.insidehighered.com/advice/2016/06/17/removing-bias-student-evaluations-faculty-members-essay

Existential crisis in the classroom — this time also for the Prof.

“When college is more about career than ideas, when paycheck matters more than wisdom, the role of professors changes. We may be 50-year-olds at the front of the room with decades of reading, writing, travel, archives or labs under our belts, with 80 courses taught, but students don’t lie in bed mulling over what we said. They have no urge to become disciples.” (Bauerleinmay, NYT, May 9, 2015)

A recent New York Times op-ed struck a chord for me. What’s the point of a professor? the author, MARK BAUERLEINMAY, a professor at Emory University, asks. Bauerleinmay argues that contemporary university students (at least in the US) are increasingly seeing professors in utilitarian terms. They are grade providers, rather than mentors, advisors, and role models. I think that this is an overgeneralization but I do worry about this trend for my own students (and myself) as well. How can I attempt to make a difference in the lives of my students –and how can I find a meaningful life as a professor — when at the end of the quarter the only thing that matters is the grade?

Furthermore, this quarter I am teaching three courses; it will be a miracle if at the end I can rattle off everyone’s name without pause. But more than the problem of name/face recognition, the issue of face time — meaningful one-on-one face time to discuss and challenge ideas and writing — is one that I continue to struggle with. Especially in an era where hybrid and online classes are becoming pushed for many reasons (some of which which benefit students, and some which don’t).

IMG_6806

Bauerleinmay offers one solution to the model of making professors like any other standard service worker: to require regular one on one meetings with students to challenge and guide them on their work. But how to do this in the midst of all the other obligations of tenure track faculty life? Especially when one has so many students that one can hardly remember their names? I am working on this. In the meanwhile, here is the text of the Bauerleinmay’s Op-Ed:

——–

What’s the Point of a Professor?
By MARK BAUERLEINMAY 9, 2015

New York Times 

ATLANTA — IN the coming weeks, two million Americans will earn a bachelor’s degree and either join the work force or head to graduate school. They will be joyous that day, and they will remember fondly the schools they attended. But as this unique chapter of life closes and they reflect on campus events, one primary part of higher education will fall low on the ladder of meaningful contacts: the professors.

That’s what students say. Oh, they’re quite content with their teachers; after all, most students receive sure approval. In 1960, only 15 percent of grades were in the “A” range, but now the rate is 43 percent, making “A” the most common grade by far.

Faculty members’ attitudes are kindly, too. In one national survey, 61 percent of students said that professors frequently treated them “like a colleague/peer,” while only 8 percent heard frequent “negative feedback about their academic work.” More than half leave the graduation ceremony believing that they are “well prepared” in speaking, writing, critical thinking and decision-making.

But while they’re content with teachers, students aren’t much interested in them as thinkers and mentors. They enroll in courses and complete assignments, but further engagement is minimal.

One measure of interest in what professors believe, what wisdom they possess apart from the content of the course, is interaction outside of class. It’s often during incidental conversations held after the bell rings and away from the demands of the syllabus that the transfer of insight begins and a student’s emulation grows. Students email teachers all the time — why walk across campus when you can fire a note from your room? — but those queries are too curt for genuine mentoring. We need face time.

Here, though, are the meager numbers. For a majority of undergraduates, beyond the two and a half hours per week in class, contact ranges from negligible to nonexistent. In their first year, 33 percent of students report that they never talk with professors outside of class, while 42 percent do so only sometimes. Seniors lower that disengagement rate only a bit, with 25 percent never talking to professors, and 40 percent sometimes.

It hasn’t always been this way. “I revered many of my teachers,” Todd Gitlin said when we met at the New York Public Library last month. He’s a respected professor of journalism and sociology at Columbia, but in the 1960s he was a fiery working-class kid at Harvard before becoming president of Students for a Democratic Society.

I asked if student unrest back then included disregard of the faculty. Not at all, he said. Nobody targeted professors. Militants attacked the administration for betraying what the best professors embodied, the free inquisitive space of the Ivory Tower.

I saw the same thing in my time at the University of California, Los Angeles, in the early 1980s, when you couldn’t walk down the row of faculty offices without stepping over the outstretched legs of English majors lining up for consultations. First-year classes could be as large as 400, but by junior year you settled into a field and got to know a few professors well enough to chat with them regularly, and at length. We knew, and they knew, that these moments were the heart of liberal education.

In our hunger for guidance, we were ordinary. The American Freshman Survey, which has followed students since 1966, proves the point. One prompt in the questionnaire asks entering freshmen about “objectives considered to be essential or very important.” In 1967, 86 percent of respondents checked “developing a meaningful philosophy of life,” more than double the number who said “being very well off financially.”

Naturally, students looked to professors for moral and worldly understanding. Since then, though, finding meaning and making money have traded places. The first has plummeted to 45 percent; the second has soared to 82 percent.

I returned to U.C.L.A. on a mild afternoon in February and found the hallways quiet and dim. Dozens of 20-year-olds strolled and chattered on the quad outside, but in the English department, only one in eight doors was open, and barely a half dozen of the department’s 1,400 majors waited for a chance to speak.

When college is more about career than ideas, when paycheck matters more than wisdom, the role of professors changes. We may be 50-year-olds at the front of the room with decades of reading, writing, travel, archives or labs under our belts, with 80 courses taught, but students don’t lie in bed mulling over what we said. They have no urge to become disciples.

Sadly, professors pressed for research time don’t want them, either. As a result, most undergraduates never know that stage of development when a learned mind enthralled them and they progressed toward a fuller identity through admiration of and struggle with a role model.

Since the early 2000s, I have made students visit my office every other week with a rough draft of an essay. We appraise and revise the prose, sentence by sentence. I ask for a clearer idea or a better verb; I circle a misplaced modifier and wait as they make the fix.

As I wait, I sympathize: So many things distract them — the gym, text messages, rush week — and often campus culture treats them as customers, not pupils. Student evaluations and ratemyprofessor.com paint us as service providers. Years ago at Emory University, where I work, a campus-life dean addressed new students with a terrible message: Don’t go too far into coursework — there’s so much more to do here! And yet, I find, my writing sessions help diminish those distractions, and by the third meeting students have a new attitude. This is a teacher who rejects my worst and esteems my best thoughts and words, they say to themselves.

You can’t become a moral authority if you rarely challenge students in class and engage them beyond it. If we professors do not do that, the course is not an induction of eager minds into an enlarging vision. It is a requirement to fulfill. Only our assistance with assignments matters. When it comes to students, we shall have only one authority: the grades we give. We become not a fearsome mind or a moral light, a role model or inspiration. We become accreditors.

 

Original article can be found here: http://www.nytimes.com/2015/05/10/opinion/sunday/whats-the-point-of-a-professor.html?smprod=nytcore-iphone&smid=nytcore-iphone-share&_r=1

Touching words

I’ve never been able to bring myself to read a book online. Articles and reports? Yes, all the time. For things that are easily skim-able — online is awesome. But for things that need to be savored, I need to not just see the words but feel them. Sometimes I also even like to smell them. (You know that smell of a brand new book? Or a really old book? It just enhances the overall experience).

Turns out, that my preference for touching words that are most touching to me is also a good learning tactic.

A new research study by Anne Mangen and her colleagues at Stavanger University in Norway found that Kindle users were significantly less likely to remember details of a story than were individuals who read the story in a traditional hard copy book. This study builds on other evidence of the value of kinetic experiences in learning and the ways that our brains function differently when processing information on a screen versus on paper — including the skill of reading long sentences, paragraphs, and chapters without the distraction of hyperlinks.

touching.wordsGiven my own movement toward more and more digital/online teaching, as well as the tendency in the social/natural sciences to assign articles (not books), and non-fiction (not fiction) … these findings are inspiring me to ensure that I always have *at least* one full book assigned per class. (For a few years, I moved away from books altogether, in part as a way to increase convenience and savings for my students. But recently I have returned to bringing back hard copy books as much as possible; since the books are always available on reserve, this is also not necessarily more expensive for students).

Mangen’s study can also be seen as adding to the evidence of the benefits of reading fiction. So this study is also making me even more committed to making time for more novels and creative non-fiction.

Two stories about this Kindle study can be found here:

And here’s a story about the value of reading fiction:

 

 

The importance of liking your work

A recent op-ed by Psychologist Paul O’Keefe provides empirical evidence for what most of us already know from our daily lives: it’s easier to work on something if we gain intrinsic joy and/or meaning from it. Scholars describe this process as being in “the zone” — it’s a feeling of deeply joyengaged focus, so much so that hours may seem to pass in a blink. This is important to remember for our lives in general, but specifically here also critical for thinking about teaching and learning. O’Keefe writes:

“…interest matters more than we ever knew. It is crucial to keeping us motivated and effective without emptying our mental gas tank, and it can turn the mundane into something exciting. Teachers, managers and parents must play an instrumental role in fostering interest in their students, employees and children — interest that will help them achieve their most important goals.”

Below is a link to the full article:

http://mobile.nytimes.com/2014/09/07/opinion/sunday/go-with-the-flow.html?smid=fb-share&_r=0&referrer

Listening into learning

I am a sociologist, which means that I am trained to analyze cultures, organizations and the patterns of human behavior within them. Being a sociologist also often comes with a deep curiofile0001376876131sity and love of humanity. At least that’s the case for me. But my training in sociology did not bring automatically bring the ability to teach. Turns out, analyzing people and cultures is a different skill set from intentionally making a positive impact on them. Much less giving them a tangible set of academic tools.

My first solo teaching gig — just on the cusp of the turn of the 21st century — was at Seattle Central Community College. I was still a PhD student myself, and was really nervous. I had never before put together my own syllabus, much less attempted to teach a group of adult students something meaningful over the course of ten weeks. As part of my prep for my new gig I interviewed another adjunct (i.e. poverty wage) professor at SCCC. I don’t remember her name now but she was an artist, both in her subject and in her approach to life. She was generous with her time and advice, but I remember mostly one thing she said. On the first day of her classes she would ask her students:

Why are you here? Why are you sitting in this room, with me and these other people, on this day? As opposed to the infinite number of other places you could be right now. Yes, of course you are here to fulfill your academic requirements. But so what? in the big picture, why, really, are you here?

This simple yet existential question has stuck with me both in imagining the range of mechanisms by which my students might end up in my classes — but also, more importantly, in trying to closely listen to students to help make their time in my classes meaningful. I also keep coming back to this question also for myself. Why am *I*  teaching this class? What do I most hope for everyone to get out of this experience? Fortunately I usually have pretty good answers for myself (and the day I don’t, I will quit teaching).

So. The point of this blog is to reflect on — and expand — my own teaching practices in an open way that is accessible to my students and others interested in engaging in pedagogical conversations. And as is the case with all conversations, the best ones are when there is a healthy dose of curiosity — with lots of listening, observations, and also — hopefully — laughter.