ļ»æHeadagogy 10 - Peer Assessment Ep2
Thu, Oct 20, 2022 8:56AM ā€¢ 46:14
SUMMARY KEYWORDS
students, grade, peer assessment, learning, engraving, assessment, peer feedback, reality, standards, critical thinking, paper, class, important, discipline, power, writing, intellectual, valued, understand, educator
SPEAKERS
Steve Pearlman
Steve Pearlman 00:00
Welcome back to Headagogy and the second episode in the series about the value of peer assessment as a means of engraving. In the previous podcast, I made two central points that I think are worth noting again, as we move forward. The first was that just giving students the power to grade themselves in what would be one of the more extreme cases of engraving, I think, sends some bad messages to students, I don't think it necessarily treats those students fairly, who are those who are more critical on themselves. And at the same time, more than that, I think it's doing something very negative and very dangerous, which is that it's suggesting to students, if not confessing to ourselves within our disciplines, that we really can't assess students fairly. And as such, what we're also doing is then sacrificing to an extent our capacity to know how to teach our disciplines, because if we're not assessing students along the way, and I'm not suggesting that the only way to assess students is to give them a grade, but if we're not assessing students along the way, then we're also telling them that we are not capable of making important decisions about how they should learn what pedagogies we should invoke, and so forth. Of course, I want to make it clear again, that that was a more extreme example. And I want to make it clear, again, that any movement to one grading, whether it's exactly the one I would choose or not, is very important. I think it's exceptionally important and supported by a great amount of research, that the most traditional forms of assessment that are still often invoked in education, K through PhD, certainly are not the best ones when it comes to learning. Research tells us that very clearly. And we need to respect that. And so I think all of us on the engraving spectrum are essentially on the same team, some of us might go a little farther down the spectrum, some of us might do a little less, I don't think that matters as much as the importance of pushing back against traditional assessment. And instead of creating people who are passive receptors of authoritarian judgment, we must use engraving for the sake of our society and for the sake of people's well being and happiness to create people who are willing and capable of and see the value in engaging truths for themselves and engaging and challenging power structures. The second major point that I made in the previous podcast was the foundational idea that we have to involve students in assessment, because assessment is the locus of power in academia. And that grades and assessment come from places that students cannot access, because they're coming from how we view things in our disciplines, including conversations with colleagues and how we were graded and tacit practices to which students have absolutely no access, if we want to truly in culture ate them to our disciplines. And if grading and assessment is the most powerful thing that we do, then we have to use grading and assessment not as a barrier for students or just as a summative judgment for students. But as a formative, constructive, educational, pedagogical force to help them understand what it is we value, why we value it, and how we apply that value. As I had said, it's not that students just don't understand what our standards are, it's that students can't understand what our standards are under typical structures, because they simply have no access to all the factors that create meaning to those standards for us as members of the discipline when they are not yet members of that discipline. Peer assessment, however, is the way that we can help students understand what we value, involve them in that power structure, not necessarily as equals, but as participants under our guidance and tutelage, and help them move to places where they can appreciate how much we know where our expertise is why we look at things, including their own products of education the way that we do. And as a final note, I just want to remind you of King Arthur, and how he gave the sword over to Europeans to Knight him, and how in doing that, in sharing his power, he gained more power. That's an important metaphor for what we're doing. But I think that's enough of a recap. So let's move on to the second episode about peer assessment. I want to explain why your students do some of the things that they do and why you did the things that you did when you were a student to do the same things, of course that I did when I was a student, as students were put into an impossible position that I described, we want you to do things that the members of this culture and discourse community will respect, but you're not a member of that culture or discourse community. But even though you're not a member, and you don't really understand our culture, please do something that is culturally appropriate and insightful. Well, that's an impossible situation. So students know, on an intuitive level that they can't do it. So what do they focus on instead, if they can't do the thing of genuinely joining the discourse community of genuinely participating in our conversations as members of our field, then all they can do is focus on the representation of how well they have done that, than the representation, the representation of ultimate power of how they can do that is the grade the grade itself. Remember, they don't really know or understand what that grade actually represents, they can't really comprehend its currency. And that's opposing its currency is always valid, which I've already said that it is not. But even supposing that that currency was always valid, and meaningful and accurate, they can't understand what it's truly representing, because they're not part of that culture yet. So if I can't really do what it is that you're asking me to do, if I can't really understand the true meaning, I'll just focus on the representation of the meaning. And the representation of the meeting is the grade. I'm going to quote from my own dissertation here, which was actually on peer assessment, referencing some of that work here, quote, the problem with grades is that they mediate the student's ability to experience the discourse community, a discourse community, by the way, is really just a fancy term for members of a community or members of a particular discipline, who have a common language and a common understanding of certain terms and practices. Again, a community of people with a common language and understanding of meanings. Again, the problem with grades is that they mediate the student's ability to experience the discourse community, which does not mean that we should temporarily put grades aside, but rather that we should put an end to the mediation. instead of considering grades as a barrier to authentic learning, we should recognize that grades can function as the doorway to authentic learning. But that only happens when we get students to stop focusing on the grade itself. And I promise you that pure assessment, even though it sounds like it amplifies students focus on the grade, it actually de emphasizes it, we'll get into that. But again, because they have no access to the discourse community, that is our culture, they focus on the grade itself, and Brian Hughes to sum this up very wonderfully, he said, grades contextualize the evaluative moment, instead of focusing on the text, this kind of assessment focuses on student's ability to achieve a certain grade, which approximates an instructor's evaluation of their work, rather than encouraging students to develop their own assessments about what they are writing for students, then writing can become an elaborate game of getting the words right. And that goes back to what bartholomay had said, students are trying to mimic the language that we might use, they're trying to get the words, right, they're trying to say the things that we think they want them to say the way we think they want them to say it, so that they're able to get the grade that they want. But we've just extracted all of the authenticity from that learning experience that we could just in that moment, because they cannot access our true understanding of meaning within our field to student goes for the grade, and the entire educational process has been sacrificed. They're going for the representation of learning, not the actual learning. This focus on grades goes back to something that Kenneth Burke referred to as deterministic screen, things that we all have, as we look at the world, we can't not have Terminus stick screens. They are biases that we have when we look out into the world based on everything that we are our worldviews, our philosophies, our upbringing, our culture, where we live, our religion, and so on and so forth. And the thing about Terminus tick screens Burke road is that every Terminus tick screen is at once a selection of reality, a reflection of reality, and a deflection of reality. So it's a reflection of selection and a deflection, which is to say no screen ever represents the entire reality. We're always only seeing a piece of it because we're only looking at it through one lens. That lens is reflecting reality to a degree. It is certainly selecting reality to agree and it's deflecting reality to a degree, but grades become the dominant deterministic screen and they deflect Students from looking at the deeper reality. And I think this distinction is wonderful. And it comes from Susan Langer. And the distinction between a sign and a symbol. In her distinction, a symbol has no intrinsic connection to the thing that it represents. f i, r, e are symbols that we all decided make up a word that we can read to mean something that represents, you know, that burning stuff. But those four letters really have no intrinsic connection to fire. Other cultures have other languages that represent fire with entirely different symbols. And we could all decide to make up any other symbol that we wanted to represent fire. So that's a symbol fire, the word is a symbol of the thing, fire, a sign is different. A sign has an intrinsic connection to that thing that represents smoke is a sign of fire for where there is fire, there is smoke. And so if we are seeing smoke, there is a good chance that there is actually fire. We of course, can yell fire when there isn't, as Steve Martin, in his commentary on philosophy, put it philosophy you remember just enough to screw you up for the rest of your life.
11:18
And you study the important like ethical questions, is it okay to yell a movie in a crowded firehouse?
Steve Pearlman 11:24
We can write fire when there isn't. In fact, most of the times we write fire, it's not really because there's actual fire. But there generally isn't smoke unless there's fire, smoke is a sign of fire. Now, why is this important to our discussion of grades, it's so important to our discussion of grades because for students grades are a symbol of learning, not a sign of learning, meaning that often we know because grades are not necessarily valid, that they have no intrinsic connection to the thing that they're representing, literally. So we know that many faculty members think they're grading objectively when they're not, or think they have standards, but can articulate them and so on. So they give a grade that, therefore is only symbolic of learning, but no learning might have taken place. I did this in my course. 30 years ago, when I gave students grades on their papers that I thought signified learning when in fact, their papers were no better than before. I gave them symbols of learning. But there was no intrinsic connection to actual learning because there was no actual learning. But let's say the grades are valid from some objective standard. Do students therefore see them as signs? No, students can't see them as signs because the students don't know what fire is yet. Students don't know what the discipline is really valuing students aren't part of that culture. So they cannot see the grade as a sign. They might think they see the grade as a sign. But what they're really seeing is the grade is a symbol because they don't understand its connection to anything authentic. So students will focus on the symbol the grade instead of focusing on the learning, because it's symbolic. It goes back to why Hubert wrote that students focus on the elaborate game of getting the words right to get the symbol of learning, because that's all they can do. Now, that has explained the nuts and bolts of the construct of why authoritarian traditional assessment is so problematic. And it's not problematic because it's ill intended. It's problematic because despite our good intentions, and sometimes our good efforts, and sometimes our seemingly good practices, students ultimately cannot understand what it is our culture as members of a discipline as a discourse, community is truly valuing how we're really seeing the world, they can only try to mimic it. And because they know they can't really see it, they're instead naturally going to focus on that thing, which is achievable and at least symbolizes learning, and that's going to be the grade. So grade has become the barrier the intermediary between the students and being historians, the students and being good writers, the students and being good critical thinkers, and Enter the Dragon Bruce Lee admonishes a younger student about a finger pointing to the moon. It is like a finger pointing away to the moon.
14:17
Don't concentrate on the finger or you will miss all that heavenly
Steve Pearlman 14:22
glory. But the difference in this scenario is that the students can't really see the moon, they can only see the finger pointing to the moon and the finger pointing to the Moon is the grade. The moon is behind the clouds that are obfuscating the discourse community to which they cannot possibly belong yet, so they'll focus on looking at the finger. And that's why in 1992, Maryland Frankenstein Yes, her name is really Frankenstein. I think one of the most brilliant books ever written by the way, but Maryland Frankenstein row are in critical mathematics education, that quote, dominant language can distort people's ability to know reality critically. And she's referring to languages capacity to mediate experience. And I wrote if we embrace the preformed language other people use such as grades, and allow it to be deposited into us without critical inquiry that we do not interact with reality directly, but interact instead with other people's language about that reality. And the language about that reality is grades, the power of grades, the power that those hold is so strong, that it mediates the experience to the reality which is learning. And that is what engraving so nobly is trying to resolve. It's trying to say, Listen, we don't want grades to mediate the learning experience of the students, we don't want it to be a barrier to learning. We don't want the students to look at the finger pointing at the moon, we want them to look at the moon, and we want to know what they think about the moon, as we damn well should. But merely removing the grade doesn't mean we're necessarily helping them Look at the moon better, because we need to teach them how we look at the moon, how a discipline looks at the moon, how physicists look at the moon compared to philosophers. And so now we understand why we focus on grades why students can help but focus on the grades as symbols of learning not even signs of learning necessarily, because they don't have access to the reality that the grades are representing. And we understand why engraving as a movement is generally resisting that as it should. But before I talk about peer assessment, I want to talk about peer headed Goji. We'll resume in just a moment. But first, if you're a high school, college or graduate school educator, then I'd like to offer you a full free preview of my online level one critical thinking program for students. I actually developed this program because so many educators have asked me for a way to jumpstart their students critical thinking skills. This program, which is approximately a three hour student experience does the following. It teaches your students three essential mindsets for thinking critically, it teaches them a copyrighted neurobiological process for thinking critically about any subject in any discipline. And then it does something particularly distinct, it prompts students through a step by step process in which they actually compose a very short essay entirely driven by their own critical thinking. Students can complete this program outside of class with no impact on your class time, and you can see the final product when they're done. I think you'll find this to be an exceptional program for your students. But whether you assign it or not, I'm confident that it will be an asset to you in terms of infusing critical thinking in your own approach to teaching. So provided you're an educator, I'd be excited to grant you a free preview of this program, please just come to the critical thinking initiative.org/podcasts Sign up with a.edu email address. Or if you don't have a.edu email address, just email info at the critical thinking initiative.org. With confirmation that you're an educator, again, please just come to the critical thinking initiative.org/podcasts and sign up for a free preview of the entire program. Please make sure you either sign up with a.edu email address, or email me at info at the critical thinking initiative.org with other confirmation that you're an educator and I'd be excited to grant you free access to a program preview. And for everyone who's listening. Please remember to like and share pedagogy. Find the critical thinking initiative on Facebook and LinkedIn and follow me on Twitter at at Steve J. Perlman. That's at Steve J. Perlman, now back to headed Goji. But before I talk about peer assessment, I want to talk about peer review or formative peer feedback, where students are not empowered to grade one another or if empowered to put a grade for one another haven't really been part of a discussion with faculty member and practice about what those grades will mean. So peer feedback is give your paper to the student and then you can take the students advice and make some changes and so on. First off, there is the problem that that's obviously the blind leading the blind. Plenty of research on peer feedback makes it very clear that students don't trust one another to offer any kind of authoritative feedback on their work, even if it's coming from a student who they might think is performing in the class better than they are. And that's because, as I like to think of it, peer feedback is a game of Schrodinger his cat. Schrodinger his cat is again a physics postulate that talks about quantum realities you We have a cat in a box, and there's an isotope there that could trigger a poison. We don't know whether the poison was triggered or not. And therefore, we don't know whether the cat is alive or dead until we open the box. So in quantum terminology, in quantum thinking, the cat is effectually in a super state of alive and dead at the same time, until it is viewed as being alive or dead. Well, it's no mystery from the research on peer feedback. And from so many students experiences with it, that they think it's a bullshit exercise. Because they know that they don't really know what the teacher wants, they know that their comment could be wrong, could be undermined by the teacher could be misguided, even if well intentioned, they know their feedback to their peer could be that way. And they know their peers feedback to them could be that way. Because this feedback from those students is not constrained or in powered by an understanding of the discourse community to which the students have no access, as we talked about before, they don't really know where the grades coming from, and therefore they don't really know how to advise the student to get to the grade. And that Schroedinger is can you go into a peer feedback session, and your peer gives you a comment about what to do with your paper, and you take that suggestion, and you apply it to your paper, you don't know if that change you made based on what your peers said, is alive or dead? Until the educator says whether or not it's alive or dead? If the educator says, Yes, I'm glad you made this change and improved your paper, then that comment is a living comment. It's great. But if that comment was wrong, and the teacher says you should not have listened to this, well, then that comment just killed your paper. And your paper exists in that super state of alive and dead until the educator says which one, it really is, the professor is the one who gets to determine that the professor knows, but you don't know. And your peer didn't know until you went and did that. And in fact, there's plenty of evidence, as I said, that feedback doesn't work. You can look at Sadler, who said that peer feedback often doesn't lead to any meaningful change or improvement in student work, or Nelson and Carson, and 2006, that students prefer teacher feedback to peer feedback, or a number of studies that students don't trust feedback from their peers, because it's the teachers who gets to really determine the reality, the students have no capacity to access that reality, nor to affect that reality. They're not participants in the actual assessment process. That's part of a culture to which the students don't have access. Now, by the way, I'll also mention that teacher feedback on student work doesn't always result in positive changes to student work. That's a whole other discussion. And it's for a number of different reasons, some of which you can relate to what I've talked about here already, but nevertheless, if teacher feedback does not affect what students do on their papers, then peer feedback, where peers are no arbiters of the reality at all. Certainly, not only typically does not but I would argue should not affect what students do. Any student who listens to another student's advice about what to do with their paper, instead of going to the professor and asking what the professor thinks is crazy. I remember this. In fact, when I started teaching, it might have been that same first semester, where we did a peer feedback exercise in class, as I had done many times as I was a student, and was still doing as a graduate student. And a student had done that exercise in class, a very diligent student, and she came to my office, and she said, Well, I'd like to know what you think about this. What comments would you give me? This student said to do X? Should I do that or not? And at the time, I was utterly ill equipped to contend with her questions. Because if I said to her, No, I don't think you should do it, that students said, I would undermine the nature of peer feedback entirely. And if I said, Well, what do you think relative to what the students saying, then she would counter with, I don't know what to think that's why I'm coming to you. These are entirely fair questions that she was asking. I was just ill equipped with answers at the time. Before I got into peer assessment, I learned to answer those questions better, but never really satisfactorily. Now that I've taken peer feedback or peer review of the table, let me talk about peer assessment. And let me tell you very plainly what this involves. It simply involves showing our students how we go through the process of assessing something. Now that's obviously not needed when we're dealing with something like a multiple choice test, but since I'm not much in favor of multiple choice tests, but that aside, what I'm talking about is using peer assessment to encourage students to look at that man behind the curtain Behind the curtain, to encourage students to come across the veil and look at their work and their peers work the way that we in the discipline as experts in the field, as more seasoned intellectuals look at their work and what we're valuing in it and why we're valuing it. Now, I recommend, of course, that we do this with a rubric. And if you're thinking wait a second, you said before that rubrics don't work. I said, rubrics don't work. When we are not empowering those rubrics by revealing the culture around them and how they're interpreted and applied, we have to bring students into the process of that. And that means showing them how we look at it. So because so much research as in Chung in 2017, and Duric, in 2016, show that training students is so important with respect to their application of standards and their understanding of standards, let's take the time to show them how we grade something. So I might do this with a couple of sample papers. But I find it much more effective typically to work with students papers. And again, I do this from a writing perspective, I've done this also with presentations, you can do this with any modality where standards can be applied beyond a multiple choice test, of course, or something of the equivalent. And we will have our rubric. And my rubric, of course, is my critical thinking rubric, because students find critical thinking meaningful and important. And it doesn't mean that we're only looking at the thinking, obviously, they have to be thinking about something. So therefore, we're also measuring their engagement and thinking about subject matter. But we focus on our Critical Thinking rubric. And I like to use actual writing from students in the class. And I like to use it with the students name on it. And I'm going to tell you that that's not necessarily something everyone has to do. But it's my preference, because one of the things that I think it's so important for us to do is to demystify, and destigmatize the nature of grading. And if we play to the fears that students have about being graded, by letting them hide from it, then I don't think we're doing them the best service, nor are we doing them the best service and the fact that part of what we need to do in life, if we're going to do anything valuable, is be able to receive constructive criticism positively, as well as learn to offer it positively. And that's something that peer assessment does. And I'll talk about the research on that a little later on. It builds those soft skills that are so important that companies want so much that our world needs so much. And because I do this as a fail forward mastery construct, students have time to know that their early grades, which are typically very low, in fact, the first ones are almost always failing, but that's in my class and might not be in your class, that they have time to reiterate their efforts and improve their grades, and only the later grade counts. So their best grade effectively counts two months later from when we start this process, to go over a couple of ground rules for peer assessment about how to phrase comments as I statements, how to be constructive instead of negative, and so on and so forth. I'm not going to go into all of the literal practices. Right now, though, perhaps if there's interest, I'll do so in a future podcast. And then I tell the students something very, very important, something that I encourage you to tell the students, which is that at the outset of this experience, in our first initial grading sessions together that because I'm the one who understands the institutional culture, I understand what the institutional community wants them to do. I understand how I'm interpreting their work. And I'm the one who's ultimately responsible for the grade, no matter what else happens, I have to be the one to enter that grade at the end of the semester. And my goal here is not to have a democratic experience yet yet being very important, but not to have a democratic experience yet about what grade the paper should receive. My goal is to enculturated them to how I and the discourse community would read the paper that's in front of them, and to teach them to read it like I do. Now, if that sounds authoritarian, I would just amplify the use of the word yet. Nevertheless, we sit down, we look at the paper together, I asked the students to start interpreting the rubric giving their impression of what's going on. And typically, much of that is very misguided. They're trying their best. They're being very earnest. They're offering thoughtful comments in the sense that they are not just babbling, but they're not offering accurate comments. And then we talk about how I would see it instead of what they're seeing. So if our question is, do we see a thesis statement here, or do we see a strong moment of evaluation? The students might say, Yes, here's where we see it here. Here's why. And then I will say, Well, no, here's why I interpreting it differently. Here's what I'm seeing going on and what I'm looking at. And I'll say, as a footnote, that authors typically find this very constructive, even though they're often failing or getting very low grades on their early papers, they find it a very positive experience. If we couch the discussion successfully, which we can do, the students ultimately understand that my goal is to help them understand how I see the paper at first, and why the grade is emerging the way it is and what I see going on that they don't see going on. And in fact, they value the fact that I, as a more seasoned intellectual, can show them how a more seasoned intellectual looks at what they're doing. They don't begrudge that. They didn't come to college, to learn from somebody who's not more seasoned in that experience, and that field and that discipline and more skilled than they are. That's why they're there. They know they need to learn more. They're appreciative of learning more and appreciative of seeing how I see thing. And I need to speak for just a moment about how compassionate this is for the students. Refer back to what I said earlier that students have absolutely no idea where their grades come from, they do not and typically cannot truly understand what's being valued, why it's being valued, what the standards really are, how the standards are really being applied. They often feel as though and very often it's the case that they are being evaluated somewhat idiosyncratically that their grades are coming from each of their educators own individual predilections about how to assign a grade about what quote unquote good writing is, or what good student ship is, or what a good presentation is. And so they enter those evaluative moments with trepidation and concern, because they hold no opportunity, no power to truly make their work conform to what's being expected of them, and therefore to ensure for themselves recognition of their learning, if not an actual grade, and instead can only hope that what they're doing truly meets the educators standards and expectations. And many students entirely relinquished the idea, even before they get in your course, that those standards and expectations could even be fair, fairness, veracity, validity, doesn't even enter their minds as something that's necessarily feasible, because it's simply if not, never having been their experience has so seldom been their experience. And remember, the construct is one of such whereby, in most cases, they truly could not possibly really understand what's being asked and valued and why. So if you're thinking that peer assessment is stressful, then even if we concede for the moment that at the very outset of the experience, there could be some stress involved in that the cumulative effect of peer assessment is so compassionate for students, because by removing the mysteriousness of evaluation by in cultivating them into what's being valued, and why it's being valued, and how we, as the members of the discipline are examining their work, and helping them understand and become practiced in what they need to do in order to produce work that's valued within the academy. And within the discipline, we are at the same time relieving their stress, there is an essential difference between the kind of stress we might experience in wanting to attain a meaningful standard and having to work to do that, and the far worse stress of trying to meet a standard when we don't even know what that standard is or how it's going to be applied to us. And as I'll mention later, research on peer assessment and other alternative forms of assessment support the fact that students like it better, and they find it less stressful. And of course, it doesn't cease with an initial interaction. Because as the semester progresses, my control and my power and my interpretation of what's happening will recede. And their power and their interpretation of what's happening will expand because they become enculturated, to how the discipline sees that paper. And they're able to apply those standards very quickly. In fact, there's a lot of research on this, some dates back now 20 years or 30 years that shows that with minimal training much less than what I've already described. In fact, students can assess one another with a relative consistency of other teachers. I'm going to say that again. With relatively little training students can assess one and Another with similar competence and consistency to teachers, because it's much easier to look at somebody else's text than to improve our own. I'll talk about how it does improve their text. But if you're concerned that they can't pick this up, believe me that they can pick this up. And they will pick it up very quickly. In fact, by mid semester, or after a few iterations of this, I'm able to come into class and play a very silent role in what occurs, sometimes I'll just sit there and listen to the class will grade a student's paper, and they'll do so as effectively or at least as accurately as I would do so and then maybe we break from doing this as an entire class. And then we can break into peer assessment groups where I am not a participating individual, but where they are assessing one another from an authoritative standpoint, meaning that they understand how those criteria are going to be applied by me. But that's not the most important thing that happens, the most important thing that happens is that eventually, the students start to challenge my interpretation and my understanding of the papers, they start to challenge my assessment, and they start to improve my assessment, they will start to say things and gain insights and apply the standards in ways that I did not. And that doesn't happen all the time. It doesn't happen often. But it's great when it does. Because I can say to the students, wow, I didn't think of that. I didn't look at it that way, that is a sharper application of the standard than what I had. Or we can put it up to the class as a group of collective intellectuals who are all part of the community now, not entire equals, but colleagues. And I can say to them, what do you all think, between Susan's interpretation of this and my interpretation of this? And sometimes we get into some interesting debates and discussions about it. And that teases out all of our understandings about what's going on. And a number of things happen that are so important. First of all, any new interpretations or better interpretations than mine that emerged do so under certain constraints that Stanley Fischer spoke to in the book, is there a text in this class, he wrote, the unfolding of a new interpretation will thus proceed under two constraints, not only must what one says about a work be related to what has already been said, even if the relation is one of reversal, but as a consequence of saying it, the work must be shown to possess in a greater degree than had hitherto been recognized the qualities that properly belong to literary productions. In short, the new interpretation must not only claim to tell the truth about the work, but it must claim to make the work better. In other words, we don't have to worry once students are in collaborated to this, that a cockamamie grade for a text will emerge, that the students will advocate for some cockamamie crazy idea about how this text should be read. They will advocate for smart, better understandings of how the texts will be read than what I might have seen. And they will do that because they will have a language that's part of the cultural ready, and a lens. That's part of the culture already. They're part of the discussion. Now, they're not extricated from the discussion. They're not separated from the discussion, they are a part of the discussion, they are colleagues. I'll give you an example of that. That's wonderful. I was teaching at the Indiana University of Pennsylvania, it was my first semester there. And I was doing peer assessment, I had been doing peer assessment for a while already, my class was being observed by a senior faculty member. And the class was a very good class, we had excellent discussions and a very good relationship. We graded a number of papers very successfully and clearly together to great success and to great improvement in everyone's work. The evaluating Professor sad as part of our circle, as I always put the room in a circle, and we do this, and we were grading this paper, and we could not determine if the paper had a clear thesis or not. And I can replicate the reasons why that was confounding to us. But we couldn't quite tell. And so there are debates going back and forth. And if there were a functional thesis in the paper, it would be receiving relatively good grade, it wasn't going to be in a but it was going to be a relatively decent grade. And if there wasn't a functional thesis in the paper, then it would receive a relatively poor grade. So the stakes for this particular student were relatively high. And here I am thinking, Oh my God, the one time I'm observed, we can't come to a resolution as a class about what's going on. This faculty member observing me is going to think this is a bullshit exercise. And he's going to walk into the chairs office and say you gotta get this plucking guy out of this class because he doesn't know what the hell he's doing. So with a few minutes left in class, before I turn to the student who, whose paper it was because I only let them speak at the end of that process, I turned to the faculty member who is observing. And I said, What's your take on this? Do you feel as though there's a thesis or not? And I was opening myself up very vulnerably in front of the class, because I thought it was entirely possible that she would say, Well, of course, there's not a thesis. And here's why. Or, of course, there is a thesis, and here's why. And not only would I be undermined, but our entire process wouldn't be undermined, the intellectual sanctity of the students would be undermined, and so on and so forth. But instead, she shook her head, and she laughed. And she said, I don't know. She said to the students, not to me, I cannot believe the caliber of the discussion that I'm seeing about this piece of writing. She says, I've never heard students speak so proficiently about writing, as you all are in this discussion. But something else more important than that happens through peer assessment. Returning to our discussion of the importance of engraving, one of the criticisms I frequently get from faculty, when I tell them that I do this peer assessment is that they don't want students to focus more on the grades. But why do students focus on the grade, they focus on the grade because they don't know how to focus on the more meaningful work, they are not part of the discourse, community and the culture, they don't know where the grade comes from. So they focus on the grade because they can't really understand the true nature of the intellectual work that the grade is hopefully signifying, not symbolizing, the grade again, as I said, has become the intermediary between the actual work of the field and what they're doing. And because the grade is a wall, instead of a door, all they can do is focus on the wall, they can't see the moon, they can only therefore look at the finger pointing to the moon. But pure assessment shows them the moon, it shows them what we're really valuing, and why we're valuing it and how we're reading it. And it's giving them an opportunity to participate in constructing that understanding, they start to be able to co construct that reality with us not in an artificial way where we just give them power or voting rights, but in an intellectual way, where they can truly challenge our understandings of how we're reading the paper. In understanding what we're really seeking, and why we're seeking it, they come to see the grade as less material, they realize that the actual work is the thing that's important, it stops being the word fire, and it starts being smoke. And by understanding the smoke, we can understand the fire that creates it, and they start to focus on the work that needs to be done. I cannot tell you how often in student evaluations, I see the phrase, I've cared less about my grade in this class than in any other class. And let's not forget to talk about power, because with respect to power, something else of eminent importance occurs. The students learn not to be passive receptors of authoritarian judgments and information and truths. They learn instead, that they have the intellectual capacity to engage those supposedly truths, that they can participate, that they can be the intellectuals who can push back when necessary. They are no longer passive, they are active, they're no longer subjugated by but empowered by the act of grading. And isn't that what we want in the world? Doesn't that represent what our democracy needs? It needs people who are not passive receptors of what an authoritarian figure judges as being true, it needs people who can critically interrogate what an authoritative figure might be describing as truth, why they might be describing it as truth, and how to challenge that truth fairly and intellectually, how to value not the authoritarian judgment on them and the authoritarian Goldstar. That might make them feel good, superficially, but how to value intellectual work as its own reward, how to feel empowered, that they can engage authority successfully, how to understand that they can disagree with one another, and critique one another and do so constructively with one another and help one another in this service of one another. Isn't that what we want and need? Don't we want intellectuals who engage the world that way who don't take what's given but question it, who will stand up to it when they think it is wrong and be able to articulate in meaningful ways why they think so. and how they might have something better a better alternative, which doesn't mean they always do and don't we want them to respect that as well. As John Trimper wrote in critique of Mt consensus, he said, we need to distinguish between consensus as an acculturative practice that reproduces business as usual and consensus as an oppositional one that challenges the prevailing conditions of production. The point of collaborative learning is not simply to demystify the authority of knowledge by revealing its social character, but to transform the productive apparatus to change the social character of production. Peer assessment invites students to join us to change the social character of production itself. In other words, not just to generate an empty consensus, but rather to challenge norms and to build consensus around the idea of stronger intellectualism on the whole. So next time on pedagogy on the third and final part of the series about peer assessment, I'll talk about the research on how students feel about peer assessment and what kind of outcomes we see from it.
