Steve Pearlman (00:00):
On critical thinking and related pedagogies. TK was one of the participants there and he's among the faculty members at the University of Wyoming who's taken that workshop to heart and gone on to do amazing things with the critical thinking process that we developed and the pedagogies that are related to it. And the reason I want to conclude this series on peer assessment with this conversation is because I think it's so important that you hear from other instructors who have recently gone through the process of implementing it as well as from their students and what the student experience is like engaging this process. And it's not without a couple of bumps, but the bumps are actually what makes it so powerful. And while TK says a couple of nice things about me along the way, it's really TK who gets the credit because he's the one who changed.
Steve Pearlman (00:54):
He's the one who worked to implement this. He's the one who has forged phenomenal relationships with his students through the process in ways that are a reflection on him and his personality and his pedagogy, and the credit goes to the students for going through what was ultimately an exceptionally rewarding experience, as you'll hear, but not necessarily an easy one. And in certain respects, as I've been saying about peer assessment, what makes it so valuable is in certain ways the ways in which it isn't easy. But you're going to hear, as I talked about in the series of peer assessment, the power of giving Excalibur the way, the power and value of involving our students and being real with our students with respect to assessment and what's at stake and why it's at stake. And the value of course, of focusing on critical thinking as the centerpiece of the learning experience.
Steve Pearlman (01:50):
But you hear me blather on enough. So without further ado, here's tk Stout, Mattie, Amy, and James. Hey, thanks everyone for being here today. I'm really excited about this podcast because I think our listeners around the world could really use at this point after this long series about peer assessment, a perspective from people who've been through it, including other instructors who've done it, as well as certainly the student perspective on it. And not just peer assessment, but of course how peer assessment when specifically linked with critical thinking instruction can be a wonderful marriage of powers that forms a gestalt that's hopefully more powerful than either one individually. So let me just introduce who we have here or let them introduce themselves and we'll start with TK who is the instructor of the class. And then we'll just go around and let everyone give just a ten second introduction on who they are.
TK Stout (02:48):
Thank you Steve. Thanks for having us here. My name is TK Stout. I am a 32 year retired United States Air Force combat veteran who now teaches four classes here at the University of Wyoming. The class that we're going to uh, dive into today is a critical thinking course. It's called the Critical Thinking Initiative at the University of Wyoming. It was initially designed for just veterans cuz uh, I am a veteran and teach most of my courses to veterans, but it's become a very inclusive class. And as we have here, we have two traditional students and one non-traditional student, a veteran. I've been teaching this course for two years. This is my second semester at it. I think it needs to be said that I'm a disciple of Dr. Steve Pearlman and his critical Thinking initiative and went through a summer institute about four summers ago and really became a believer and have been experimenting and experiencing things like un grading and self-assessment. So I've been here since 2016 and, uh, hope to be here for a while ago.
Maddie Davis (03:44):
I'm Maddie Davis. I'm a first year student at University of Wyoming and I'm one of the traditional students that TK brought
James Roberts (03:51):
Up. Um, I am James Roberts, former active duty Air Force. Um, now I just continue on as a DSG in the Wyoming International Guard. Um, this was my first semester as a full-time college student and this was kind of my first take on an English course and a, uh, college setting as well.
TK Stout (04:11):
Yeah, and I think we can break down those acronyms for our civilians out there. Steve. Uh, DSG is a drill status Guardsman. So James was full-time Air Force and now he's in the guard reserve doing part-time things on the weekend. And, uh, two weeks annual training in the
Steve Pearlman (04:24):
Summer. Thank you both for your service to the country. It means a lot to everybody.
TK Stout (04:28):
We are proud to serve Steve.
James Roberts (04:30):
Yep. Thank you sir.
Amy Besam (04:32):
All right. Um, my name's Amy Baam, also one of the traditional students. This is my first year at the University of Wyoming, so yeah, it's really fun.
Steve Pearlman (04:40):
Great. So tk, why don't you give me a little background on the construct of the course, how you've designed it, what it's trying to teach, how it's trying to teach, what the course flow looks like. Just maybe a, a high level overview of what's going on with this thing. Well
TK Stout (04:56):
It is part of the uh, U S P uh, and that's the university studies program. So to graduate from here, there are some core classes you must tick off as you go through your program of study. That's a first year seminar course, A communications one, Communications two, communications three. So this particular course is a communications three course and it's is designed to be a critical thinking and composition course. My history of critical thinking goes back to 2004 and the Iraqi war, uh, when we were trying to develop a critical thinking sergeant or critical thinking soldier in the Pentagon. So, uh, we could never really come up with a good definition of critical thinking. Could we measure critical thinking? How could we teach it? So it's been in the back of my head since 2004. So when we got here, went to your summer institute and then put the course together to teach students critical thinking mean my, um, my first assumption is we all are critical thinking all the time every day as we kind of seen in our metalogues during the semester.
TK Stout (05:57):
But I really wanted to find labels for that. Learned a language and literacy for critical thinking. And then because of those communication two requirements, like how would I know people are critical thinking, like we always say we know it when we see it, but we wanted to put it in that writing format in pros so we could really show people that we are critically thinking, you know, uh, we have a saying in the military like you, you're from the show me state, I'm from Missouri, so you gotta show me that you're critical thinking and not just think that we're doing it. So we put together that course with your um, online third party website, the critical thinking level one that you've heard about on your podcast if you're a podcast listener. So we use that to uh, kind of just establish our foundation. And then we really focused on the five paragraph essay.
Steve Pearlman (06:39):
Let me just interrupt you for a second, TK, because I just wanna make sure we're not conflating terms here. When you're talking about the five paragraph essay, you mean not the typical five paragraph essay that most people are taught to write in at the end of high school or sometimes in freshman comp, but the five paragraph critical thinking essay, that's a different structure that I talk about in my book or in the program that's online. Is that correct?
TK Stout (07:07):
That is absolutely correct. And that five paragraph essay looks like a paragraph on analysis, one on a critical question, one on evaluation, another paragraph that speaks to what complicates things and the last paragraph for conclusions. But yet it's a very specific critical thinking initiative, five paragraph essay.
Steve Pearlman (07:26):
Great. So what's the rest of the course process then and how does peer assessment come involved in that?
TK Stout (07:33):
Well the rest of the course process was they looked at a question and then uh, went through that, those five components of the core process and then did a presentation like what was their problem that they wanted to look at. Uh, we did self-assessment from the very beginning and really experimented with self-assessment. And I think one of the lessons I learned, Steve, was they needed a little more guidance and leadership from the instructor. They would do the self-assessment and then asked like, am I right, Am I aligned with this? What should it look like? And my students so aptly coined the phrase, we need a punch in the face early and often to let us know what the grading criteria, what that rubric looks like and what would the instructor do with that kind of stealing from one of one of your books. I can't remember which one it is, but you've quoted Michael Tyson said Everyone has a plan till I get punched in the face. We just took that from your book and I think actually I think Robert was the one who said that like, Hey, uh, we need to be punched in the face earlier, which meant TK show us how to grade this so we can get in alignment with the rubric and what it should look like. So I think self assessment in my humble opinion was a little willy-nilly the beginning of the semester, but needed a little more guidance and nudging from the instructor as we got further into the course.
Steve Pearlman (08:44):
So let me clarify this for everybody listening by self assessment, you were having them apply the critical thinking rubric onto their own work and sort of give themselves a grade, but their grades were off, if I'm interpreting this correctly and their self assessments weren't working because the punch in the face that you're referring to, which I really advocated for peer assessment, is really showing them what an application of the final standard would look like against their initial work or early work in the semester, which really therefore typically means everyone's not doing very well in terms of their grades at first at the outset. Right? And that's the punch in the face. Is that an accurate summation of what you're describing?
TK Stout (09:23):
Affirmative. That's exactly what that punch in the face was for. And I, I might go to Robert on this because uh, Robert, I'm just gonna throw you under the bus immediately, like I think you got a 12 outta 25. We also mitigate this punch in the face cuz we have a resubmit to mastery. So as you can go back and work on that paper and resubmit it and James was really not happy with the, the punch in the face, but then he was the one who also came back when we did kind of a focus group on the class that said, needed that earlier and then I could recover, then I could mitigate. So James can you speak to that experience?
James Roberts (09:55):
I think exactly what he said, you know, I think it would've benefited me a little bit earlier in the course. I think also coming in off of uh, quite a few years out of, you know, any type of organized education, I think I kind of had like this pre perceived notion that uh, my writing style was good enough and um, I think I kind of had that rude awakening where I realized just how hollow a lot of my methods for writing had been prior to this class, you know, so really getting to see someone like tear everything down and kind of have that even tension within the classroom of me and TK talked about that. You know, there was even this little bit of uh, just um, abrasive atmosphere that we had with one another for maybe a week, two weeks before we kind of got it ironed out. I think I was really beneficial and exposing that earlier on would've definitely been beneficial for me. I think
TK Stout (10:46):
The primary reason I invited James onto the podcast was for this tension that we experienced over maybe a two or a three week period where James didn't want me looking over his shoulder and looking at his work and I could feel, you know, the vibe, like I changed the rules on him from a traditional composition class. But then when we talked about that, I guess I would appeal to your instructors, professors and faculty members, Steve, that there is some tension, but man, if you can't pivot into it, turn into the tension. There are some really big rewards at the other end. But uh, after we got through it again, he said, Hey, we deconstructed things, we broke it down and TKs asking us to do a different thing in this class. And now James is actually very good at that specific thing, critical thinking and writing in a different kind of way.
Steve Pearlman (11:28):
And the tension you're talking about to clarify for everybody is that when confronting students with this new critical thinking standard, which often is really the first authentic critical thinking standard they've ever encountered, that they're not meeting that standard and don't hold that capacity when the course starts off. So there's this standard by which they're being assessed that they're far away from and that creates some frustration and some tension in the classroom. Let's go to Amy and Mattie as well about what it's like running up against that new expectation for critical thinking. Did you experience some of that same tension or frustration or whatever words you might like to apply for it when you encountered the class? And what was that like for you?
Maddie Davis (12:13):
I never really experienced the critical thinking any of it. In high school. There was a little bit, but it was never introduced as critical thinking. In high school I was in AP and IB classes, which were the more difficult classes in high school. But um, as soon as I entered into t K's class, it was a whole different way of thinking and a way of grading and it kind of hit me in the face a little bit as well. Not as much as it hit James, cuz I was used to the like school aspect of everything, like right, coming right out of high school. But it definitely was a different grading style and seeing the like comments that TK made on papers. I never had that in high school. It was always just a grade, no comments on what I did wrong, what I needed to fix. But TK gave us that opportunity to, I guess, fix things that we did wrong and not make it like the final grade. And that was all that was entered into the grade book, I guess.
Steve Pearlman (13:06):
What was it like though suddenly being asked to do a kind of critical thinking that you hadn't done? Did you feel like that was out of your reach? Did you feel like it was in your reach but you weren't there yet? What was that expectation? What was it like confronting that all of a sudden?
Maddie Davis (13:22):
I think initially it was definitely like it was in my reach, but I wasn't there yet. But the more we went through everything and the more you explained it and we got into it and applied it more to our lives and to our work in school, it made a whole lot more sense and it was more in reach at that point.
Steve Pearlman (13:39):
Amy, you wanna jump in?
Amy Besam (13:41):
I had a really good English teacher and so I had that experience in high school a little bit, not to the same extent that we pushed for in this class, but I was lucky enough to be a little bit familiar with this writing style, pushing for your own thoughts and weighing other people's opinions and everything. And, and so it was a little bit easier for me to grasp and to understand, but it did, like Maddie said, having that application to schooling and to our lives and everything, I think that really pushed it even more.
TK Stout (14:11):
And I think Steve, the experience for both student, I think a student and teacher was trying to break the old paradigm of this class being transactional. James was really good at finding the instructor's words and then parroting those words back to the instructor and then I get my B plus or B and I'm happy. And oftentimes students don't even look at the feedback on they go with this more transactional environment. So after the challenge, after the tension, I believe there was more of a transformational environment in the class. Like some students transformed and they had some aha moments and uh, they had some learning moments that might not be reflected in the transactional grading process. You know, like, oh, I got an 82 on that, but man I had a really super impactful aha moment, right? Or now I can take this to application level.
TK Stout (14:58):
That was some of the things I saw in the metal logs and I was really happy when we went from, I mean they have to trust this, like maybe they haven't seen it, maybe they haven't experienced. Amy's really fortunate that she did have this experience in high, in high school, but the rest of us, the majority of us didn't have this come into the class, open the top of my head, pour some stuff in, gimme a 50 question, multiple choice test midterm and at the finals. And that's what academia looks like. But um, it can look much different as Steve is kind of laid out for us. And uh, and um, it's not easy get to that transformational environment, but I think we made it this semester and I'm really happy with the results.
Steve Pearlman (15:33):
Before we go forward anymore, let me just mention what the meta log is. I've talked about it in some other podcasts. Once students start learning the critical thinking system that we use, the meta log is a very powerful exercise that takes that system at least in its simple form and starts to apply it to decisions that students are making every day in their daily lives. So they keep sort of a log or a journal about a decision they made and then they break it down according to this critical thinking process and eventually that critical thinking process starts to affect the decisions that they're making instead of just be reflective about it. But let me turn back to you for a second, TK and then we'll go more broadly back to everyone in the class because we have so many educators who are listening to my rantings and blathering about the importance of taking on this critical thinking model, bringing it to the students. And because you've mentioned this factor of discomfort and I've mentioned this factor of discomfort and we working through that in, into what you're really terming as a transformational experience. Can you talk about what your apprehensions were or where your angst lay in the process of moving your students through that process and when it became more comfortable for you and what were the triggers?
TK Stout (16:55):
I think my main apprehension is the same as any other faculty member.
Steve Pearlman (16:59):
Pedagogy will resume in just a moment. But first, if you're a high school, college, or graduate school educator, then I'd like to offer you a full free preview of my online level one critical thinking program for students. I actually developed this program because so many educators have asked me for a way to jumpstart their students' critical thinking skills. This program, which is approximately a three hour student experience, does the following. It teaches your students three essential mindsets for thinking critically. It teaches them a copyrighted neurobiological process for thinking critically about any subject in any discipline. And then it does something particularly distinct. It prompts students through a step by step process in which they actually compose a very short essay, entirely driven by their own critical, critical thinking. Students can complete this program outside of class with no impact on your class time and you can see the final product when they're done.
Steve Pearlman (17:57):
I think you'll find this to be an exceptional program for your students, but whether you assign it or not, I'm confident that it'll be an asset to you in terms of infusing critical thinking in your own approach to teaching. So provided you're an educator, I'd be excited to grant you a free preview of this program. Please just come to the critical thinking initiative.org/podcasts. Sign up with a.edu email address. Or if you don't have a.edu email address, just email info at the critical thinking initiative.org with confirmation that you're an educator. Again, please just come to the critical thinking initiative.org/podcasts and sign up for a free preview of the entire program. Please make sure you either sign up with a.edu email address or email me at info at the critical thinking initiative.org with other confirmation that you're an educator. And I'd be excited to grant you free access to a program preview. And for everyone who's listening, please remember to liken share pedagogy, find the Critical Thinking initiative on Facebook and LinkedIn and follow me on Twitter at at Steve j Pearlman. That's at Steve j Pearlman. Now back to pedagogy.
TK Stout (19:16):
I think my main apprehension is the same as any other faculty member when doing something for the first time. And to trust your students to be generous, to pull that curtain back and have some transparency and say, Hey, you guys just gotta trust me on this. You know, I don't know where we're gonna land, but there's a safety net here. So when you get up in front of the classroom, again, breaking the paradigm for the faculty member, like I am not the sage on the stage. I might not even be the guide on the side, but we're gonna collaborate. This is gonna be messy, we're gonna get through this and hopefully there'll be some value and some rewards, a value proposition at the end. So I think my risk factors are the same as any other faculty member being an imposter, not being as smart or as comprehensively informed on critical thinking as I think I should be to stand up in front of the classroom.
TK Stout (20:01):
But I would say to mitigate that there is a great critical thinking community of faculty members here on campus. I would include Steve on that. He's been a lifeline. Like I could call Steve when I've had a victory in the classroom or when I've fallen flat on my tail in the classroom. So no faculty member is an island. And if I didn't have that support system, so when I fall down, make a mistake, get back up and and and ride again. And I would say I need to give a lot of credit to the students, to Amy, Maddie and students like James that are generous and allow the faculty member to be transparent and to still follow that teacher down that critical thinking road.
Steve Pearlman (20:39):
And so for the rest of you, there were moments in the class early on where you were communicating things back to TK about what the needs were that you had in the class. You had done some meta logs, you were doing some personal assessments of your own work. We haven't gotten into peer assessment yet, which we will in a minute, but something wasn't jiving, right? Everything wasn't clicking yet. Where did you give him feedback or what were you experiencing maybe that he picked up on that was meaningful in terms of prompting the shift that was required?
TK Stout (21:10):
Yeah, and before they answer Steve, I think I need to make a pedagogical note here that on certain days we just get in a circle, we call it a campfire circle, and we just open it up to talk about these things like, hey, what are we doing well, what do we, what do we need to do differently? What are the problems? So really provide and facilitate that environment so the student can have agency and have power and be able to give me feedback. So we really work hard to create that environment. And then Amy, Maddie and James do a great job of kind of stepping up and stepping into that. And, uh, what did you say? Like maintaining a very balanced discourse of the pedagogy that's going on in the classroom. So I'll go to Amy. Amy, what did you think of, of that feedback and what kind of feedback did you give me in particularly to, to try to pivot us towards a better learning environment?
Amy Besam (21:57):
Yeah, I think that a little campfire circles were kind of the turning point of the class because it does create that open environment. Like even, even just sitting in the circle and being able to face each other makes it so easy to say, Hey, I've been struggling with this, with this one part, or whatever it is. And it kind of opens up for, you know, that peer assessment and or for us to say, hey, like how can I get help from anybody in the class? Because, you know, we're all at different points in our learning and and it's not just asking you tk, what we need to do better, but it's taking from our classmates learning as well.
TK Stout (22:33):
Yeah, and Steve, I would say this is our ex caliber moment where it's nights at the round table, we sit in our campfire and you know, you can stand up in front of the classroom and say, I really want this to be a balanced classroom and give you agency and empower you, You know, but when you really give them the opportunity, they see that this teaching and learning dance that we do, there's equal responsibilities here from a student perspective and an instructor. So Maddie, what, what did you think of the campfire in the, the kind of feedback? I think it's really interesting that Amy said, Hey, I only only wanna know what TK thinks, but my peers, how do you do analysis really well? But Maddie, go ahead.
Maddie Davis (23:09):
I definitely agree with Amy about the campfires and like that made it an easy way to get our issues out. I guess it was just a very open space to be able to talk. And this isn't a typical 200 person lecture course. We actually are able to have normal conversations with you and like talk about what's wrong and you are very understanding about things that are wrong and you are willing to change that in the course, which is not a normal course in a college setting. But, um, having that structure of the class really opened it up for failure and for growth in the class itself.
Steve Pearlman (23:48):
I think that's a really great segue into talking a little bit about peer assessment and TK already touched on it because what I'm advocating for in this is the sharing of power, but not the sharing of power just in the sense of an open discussion, but rather truly in the sense of how are we critically thinking about what we're doing in the class? How are we critically thinking about how we're assessing one another, the standards that are at play, the methodologies that are at play. Because if we're gonna be truly valuing critical thinking, then we have to be willing to put everything up for critical thought or it's sort of a disingenuous move. So tk, maybe you can just start us out a little bit by telling us how you started to roll peer assessment into this course structure. How'd that emerge?
TK Stout (24:37):
Well, it certainly emerged from, uh, looking at your literature, Steve, and then talking to you and kind of brainstorming like how would we do this here at the University of Wyoming? I have a very fortunate position where Amy and Maddie were in my first year seminar course. Now they followed me to a comp two course. So we have a relationship, we've established some trust, but we all established that trust to trust each other and to do peer assessment. So I followed your prescription, Steve to do it, trusted the students to do it, and then they gave me feedback in that campfire environment and said, Hey tk, we need more guidance on this. We think that we know what we're doing as far as assessing each other's papers, but can you give us a smoke check where we called a vector check or whatever. And that was the punch in the face that we talked about. We talked about it and had that discourse in the campfire, but I think we needed to do more doing like roll our sleeves up. They swapped papers, gave each other comments on each other's papers, like in the word comment function. And then these brilliant students said, tk, what about meta grading? Right? Was it meta grading? Is that the term you all came up with? Which I've never heard before. They said, Could you not?
Steve Pearlman (25:40):
I'm stealing that. I'm so totally stealing it. .
TK Stout (25:43):
Yeah, no, uh, you maybe if you could put a citation in there, Steve, that would be cool. But, um, no meta grading. I said, Would you grade our grading? And that would even give us more insight and a better way to align with what we think you're looking
Steve Pearlman (25:56):
For. But before anyone else jumps in, let me just note that this is exactly what I'm talking about with respect to peer assessment. That it's not just either the falsity of telling students to give each other feedback where there is no reference to a grading standard and no authority and empowerment of the student, no agency there, but also not top down grading from the instructor. You were still there to guide that experience for them until they became more adept in it themselves. What was it like for you guys as the students, because I think there are two aspects to this that are really important. One was what was it like for you to have to grade one another and bear that responsibility? And secondly, how did it inform your understanding of the standards and what, what the expectations were for yourself? There's being confronted with the peer grading process and what's involved in that and the responsibility and the empowerment, but then there's also, what did you learn from it in terms of critical thinking and writing and how did it affect your own experience in that? So I guess you guys can jump on that however you like.
James Roberts (27:01):
Um, I think as we began with the course, you know, we had that first session where we did a little bit of peer review and I know we also used an example on the board and kind of did it as a class first. Um, and we just used a, I believe what previous paper, uh, tk, uh, from another semester and peer reviewed it. But I think the biggest thing was is that kind of helped give us a baseline as a classroom on what he expected us to do for the peer grading. Do I think that as a class we probably still met like the level of critique that needed to be met? No. Um, just because it was a new concept for most of us, but that's where we kind of brought into one of our dialogue sessions talking about the meta grading and everything and, and really giving the instructor the power to not do the grading for that peer assessment. Um, but really what they're doing is focusing on, well, how are my students looking at their peers and breaking their peers down? Um, and not in a bad way, but just in a, in a sense where, you know, are they capable students to actually do this peer assessment right now or they kind of pulling their punches per se. But TK if you'd like to elaborate on how you thought we did as far as that initial peer assessment, you know, after reviewing through our papers if we needed to, um, ramp up some of our methods,
TK Stout (28:18):
Well, you, y'all gave me the feedback. I think I needed to have more guidance and and and nudge you more towards that. And I think meta grading is a great solution or part of the solution set
Steve Pearlman (28:26):
To that. That concludes part one of this two part episode with tk, Mattie, Amy and James. Catch us next week for part two.
