Headagogy 3 - Stommel FINAL.mp3
January 12, 2022
1h 5m 19s
Steve Pearlman, Ph.D.
START OF TRANSCRIPT
Steve Pearlman, Ph.D.
For this episode, I've had the true pleasure of talking with Jesse Scammell about upgrading, which, as he characterized it, is and I
love this phrase. A furious eyebrow raise at the culture of assessment Jesse is a faculty member in the writing program at the
University of Denver. He is also a co-founder of the Digital Pedagogy Lab and Hybrid Pedagogy, The Journal of Critical Digital
Pedagogy. Two of his recent publications include Care is a Practice, Care Is Pedagogical and the Human Work of Higher Education
Pedagogy. In his words from his website, he's got a rascal pup, Emily, a clever cat, Loki and a badass daughter, Hazel. And in fact,
you might hear some of Emily and Hazel in the background of this podcast as one of the factors of COVID. Our podcast, where
people are at home instead of in their offices, we get into issues of standards and how standards actually relate to learning and
whether or not standards relate to learning. We get into grades as a technology. We get into the pressures that faculty feel with
respect to assigning grades that often don't coincide with faculty desires about teaching and learning. We get into discussions of
intrinsic versus extrinsic motivations and how assessments can aid or more often impede stronger learning outcomes in those
regards. And ultimately, we get into questions of politics and liberalism, government power structures and grading as a constructive
power in itself. You'll hear that I don't necessarily agree with Jesse on all of his positions, but we don't have to agree on everything,
and I see tremendous importance on what he's bringing forward. And I hope the conversation challenges everyone's conceptions
about grades and how to grade and what grades are for and what grades should do. So here's Jesse stumble on pedagogy.
Steve Pearlman, Ph.D.
So, Jesse, welcome to the podcast. Thanks so much for joining me today.
Hi, yeah, it's good to be with you.
I want to have a really frank conversation today about grades and academia and upgrading in academia, and there are some hard
questions that come with that. But I want you to know, and I want listeners to know that I actually dissertation on the premise of
upgrading in a respect. At least my dissertation was on the idea of grading with students and letting students grade one another in
class, and I think it's really important that we do that. I had a lot of reasons for it. One of which being that I think grades are certainly
the locus of power in the institution. And if we want students to be able to really be critical thinkers, then one of the things, perhaps the
foremost thing that we first need to get them to think about in academia is what is being valued in academia and why it's being
valued, and that's largely expressed through grades. I have a lot of other reasons for it, but I think that sort of gets the conversation
rolling at least, and I want you to know where I'm coming from. So I think I don't know if I agree with everything that you're going to
have to say today, but I'm in your camp. I think largely we're in the same camp here in terms of a general viewpoint about grading in
its traditional form, having a lot of problems coming with it. So let me turn over to you, and maybe you can start with an introduction to
our listeners about what upgrading mean. You don't literally mean there are no grades happening. So what is on grading?
Yeah. Well, for me, I'm grading is a sort of furious eyebrow raise at the cultures of assessment that have spread so rampantly in
education. And so it isn't about taking grades off the table. It's not. It's not about imagining a universe in which grades don't exist
because that is just not the world that we live in. Instead, it's about critical thinking about grades. And I think most importantly, on
grading is about teachers critically thinking about grades. But even more importantly, it's about teachers bringing students into a
critical conversation about grades and assessment and how they work and how they affect the learning that happens in a classroom.
And a lot of my work on grading is founded in some of the principles of critical pedagogy. I didn't wake up one day and decided I
wanted to be an expert in grades and assessment, but I have noticed over many years that grades are the thorn in the side of critical
pedagogy and the thorn in the side of so many of the different practices that I try and use in my classroom. And also, they seem to
frustrate a lot of the conversations that we have about education. And so I think about work like Pollo Ferriz work, and I think about
the idea of intentional education, which I can connect to some of the things that you talked about with relationship to your work that
ultimately at the core of this is finding a way to engage students as full agents in their own education.
And so to disrupt some of the power dynamics. And the truth is, we can't just imagine power dynamics don't exist and we can't
imagine hierarchies don't exist. They continue to exist despite our best efforts. And so the most important thing that we can do is to
look at them. Carefully and critically, and also to do whatever we can to turn them on their head and so that this idea of intentional
education from Paulo Freire is the idea that teachers are also students, students are also teachers and that we bring the complicated
movement between those roles into the work that we do in classrooms. I show up as a full student, as a full learner in the classroom,
and I invite students to be engaged in the pedagogical conversations that make the course go the pedagogical architectures that
make the course go. So that's sort of what I mean. I'm grading. I think it's actually really problematic to imagine that we can just the
grades can cease to exist, that there can be some kind of practice upgrading TM on trademark, on grading 10 things that you can do
tomorrow in order to remove grades from the learning environment. I think that that's problematic because they're there nonetheless.
Every classroom, the grade hovers above every classroom that I've ever been in, irregardless of whether or not I'm putting grades on
individual student works.
Steve Pearlman, Ph.D.
I'm sure what our listeners are wondering here is we're not getting rid of grades. They're not only here to stay, they're becoming
ratified and more entrenched in a lot of respects. But you write in your blog, How do one grade? I haven't put grades on student work
since I started teaching as an instructor of record in 2001? And at other places, you talk a number of times about how you allow
students to essentially write their own grade for the course. And so I have no doubt that there are a lot of listeners who are wondering
what kind of new agey bullshit this is, that it's all touchy feely, a kumbaya classroom and everyone gets to assign their own grades
and so forth. And what kind of liberal nonsense is this? And I don't think that's ultimately where you're going, but I think that that
perception or concern is certainly floating out there. So what does upgrading mean? How do grades emerge in your class?
Well, I think a lot of times people take an grading and put it at odds with rigor. I think that the word rigor is just as problematic as the
word on grading. Potentially both of them require scrutiny and require nuance and the idea that somehow, if you're not grading that
the class is new, agey is touchy feely. I do actually get perceived as touchy feely often or as not rigorous or I've even been accused
at one point of pandering to students. The word pandering in that phrase, I think, is very odd, and I could probably spend a whole
podcast just talking about that idea of pandering, because just the notion itself suggests a really, really distinct hierarchy between
students and teachers. I would say that that upgrading is for me going back to the idea the roots of critical pedagogy. It's about
dialogue, it's about a conversation. It isn't just as simple as letting students grade themselves. It's about actually having that be a
dialogue between me and the students between the students and each other, and that the final grade that the students get in the
course isn't as simple as a student puts a grade on a self-evaluation, and that's the grade that appears on the transcript. It's instead a
series of conversations that I have with students throughout the term that help them arrive at a grade.
By the end, I say in all of my syllabi that I reserve the right to change grades as appropriate. You've probably seen in my work that I've
raised grades more often than I've lowered. Grade students are incredibly hard on themselves. This culture of assessment is
something that pervades the work that we do as teachers, the way we structure institutions, but it also pervades the work that
students do themselves. And so ultimately, this conversation, this dialogue results in students proposing a grade for themselves and
me transferring that grade into the learning management system or into whatever technological system I have to use to do a drop
down and pick the grade that they give themselves. It's really interesting because that final moment of assessment, that final moment
where it is me, Jesse stumble, not the student who's picking the grade from that dropdown really shows the ways in which that
hierarchy has not been subverted. It is still me at the end picking that grade. Ideally, I'm doing that through a robust conversation that
I've had with students so that they are participants in that process. So assessment isn't something that's being done to them, but
assessment is itself a part of the learning that they're fully engaged with.
If that answers that, and the thing that I would say to the people who put on grading at odds with rigor is that ultimately self
assessment self-reflection is incredibly hard work, and it's an incredibly important work. People often say, Well, wouldn't you want
your doctor to be graded? And I say, No, I actually I want my doctor to be capable of self assessment. When they're holding a scalpel
above my body, I want them to be able to make critical decisions. About whether or not they're going to kill me on that table, and at
that moment, I need them to be able to be aware of what they're capable of not. I don't need proof from some outside person that
they got a degree and got in, got a grade. I need them to be able to make those critical decisions. So I think that ultimately self
assessment is probably the most rigorous kind of assessment that we can work towards. But I think because it's so difficult and it's so
difficult to arrive at it successfully that we rely on external and extrinsic crap in order to basically cover over the fact that the real hard
work is something that so many of our educational institutions have just washed their hands of,
Steve Pearlman, Ph.D.
Who are encouraging students. I know to do this through a lot. Sometimes I've written self reflections and you're dialoguing with them
effectively through that process and I suppose, challenging some of their assumptions and conclusions about themselves as well as
encouraging them. If we look at more traditional assessments, I think one of my main complaints and one of the challenges that the
research demonstrates about these is that they're often not necessarily really associated with something meaningful in any way. We
might be assessing what students have memorized, but it's something that they're typically go on to forget or in the case of papers,
which is where I spend a lot of time that grades emerge almost from an ether, where students have no idea where that grade is
coming from or what it really means. And if we put the same paper in front of multiple faculty members, we get multiple grades, which
suggests that there's no real anchor. There's no intellectual currency to that grade if it can be varied depending upon which
Professor stands in front of it. What does it really meaning if it can mean anything to anybody? And there have been plenty of studies
done where they'll take papers and put them in front of 20 faculty members, and grades will range from an a three and a half, so
students have every right. And I think if we want to be intellectually honest, we have to look at grading as problematic in many
regards. At the same time, though, what you're proposing, I think has a challenge if students are giving themselves the grade and it's
based on dialogue and self-reflection. Where is the dependent variable in terms of what they're necessarily learning in the class? On
the one sense, we can look at grades coming down from on high as perhaps lacking meaning and authenticity. But we could look at
your system and say there's an equal lack of authenticity there because students could effectively be rating themselves very well. But
what's really being learned and how do we know what's being learned? Is there any real assessment of what's being learned from
that kind of perspective?
I think that underlying your question is an assumption that authenticity or real assessment would somehow be associated with
objectivity, that objectivity is the necessary thing that we need in order to do that work. And I think learning is deeply subjective.
Different people learn at different times, in different ways, to different degrees and different teachers, as you pointed out, assess and
respond to student work or learning in different ways, at different times to different degrees. And there are so many variables that the
idea that we would arrive at one and that that one, we would be able to be 100 percent objective about, I actually think, do institutions
do do that? But the thing is, a new implicated this in your question that ultimately we measure the things that we're capable of
measuring with objectivity. And then we forget about all of the other stuff that's actually, in my mind, much more deeply important.
How do you objectively measure whether a student has had an epiphany, whether they've changed their mind about something,
whether they have sat uncomfortably in their not knowing? These are things that are at the heart of education and at the heart of
learning, and they're the sort of muck that we sit in when we're learning. And yet they're the things that evade our ability to
quantitatively measure them.
So I actually, I guess what I would say is that I don't think that there is there is necessarily something authentic or objective or real
about that grade that ends up on the student transcript at the end, no matter how we arrive at it, because I don't think that the actual
messy work of learning can be reduced to a quantitative measure. So another part of upgrading, I would say, is centering objective
quantitative measurement as the gold standard of did something happen? What occurred? What were the outcomes? And I think
ultimately the reason why I like self-reflection the grade at the end I used to do, I used to. When I asked students to self-reflect, I used
to ask them to give themselves a grade. And then the next question was talk about your grade and talk about your learning process
and how you arrived at that grade. And what I found is that the self reflection and self assessment that they did was all a defense of
the grade. And this happens so much for writing teachers or teachers assessing anything that. So much of our feedback that we give
to students is about defending the grade that we give.
Steve Pearlman, Ph.D.
I was going to say that's borne out by research on written responses to student work. There's no ambiguity to the fact that faculty
spend most of their time on papers, or at least too much of a percentage of it justifying the grade instead of speaking to other matters
about the student work. So that's literally true.
Yeah. Well, and then the research actually shows a couple of things. It shows that the feedback that teachers give isn't effective. It
doesn't actually have an impact on students. And that's because the feedback itself is really just a defense of a grade. And then also
that students are extraordinarily unlikely to read the feedback at all. I mean, think of all of the teachers, and I'm sure you've seen this
research, all of the research, all of the teachers copiously writing feedback in earnest and with goodwill and how much of it goes
unread. And the reason it goes unread is because ultimately our system is structured so that what matters at the end of the day is
that grade, not student improvement. And also so much of the feedback is ultimately ignorant of student improvement because
teachers are put in this precarious position where we're made to feel like we have to constantly defend the grades that we give. So if
there's ways that we can dissenter that measurement so that students can do that work of self-reflection, that work of writing process
letters, the work of metacognition without it being associated with the grade. And so how one thing I did in my self reflection process
is I asked the question about process, self-reflection, self assessment before I ask the question about grades, and that actually
fundamentally changed the things that students were writing because they weren't completely writing in order to defend a grade. The
other thing that I think we can do and I, you know, I make this recommendation often.
It isn't that one of the ways we can make feedback, the center of the work we do is separating feedback from grades so that the
moment that we're giving feedback is very separate from the moment that a grade is happening. And that way, as far apart as we
can put those things, the better. So if you are still giving grades on student papers, for example, make sure that we're not giving
students grades on every draft. They get a grade at the end. And then why bother giving feedback on that final draft if all the students
are going to do is read the grade. So you think about the formative work of giving feedback throughout the process and then just
letting the grade stand at the end because the feedback isn't useful at that point. And so thinking about that, I mean, that's in some
ways what my on grading practice does is it takes the grade and it has it be something that happens at the end. And then everything
that happens up until that point tries to dissenter that thing that's going to happen at the end. And sometimes it fails to, because when
that thing just happens at the end, the stakes of it feel high and mysterious. And so the key in my brain is talking to students really
frankly about that process so that it doesn't seem mysterious so that it's not an invisible goalpost. So it's not some strange thing that
affects their entire life that just happens at the end.
Steve Pearlman, Ph.D.
I want to step back here for a second with respect to your point about feedback from faculty members. Any listeners who want to can
go back to an earlier episode of the Critical Thinking Initiative podcast, where we spent a whole episode talking about not only why
students don't pay attention to feedback and generally the problems with it, but what the research also suggests in terms of what kind
of feedback is actually effective. It seems like a good time to mention that that exists. I agree with your point here about how we are
trying to move towards something where students are participants in this experience and forming and creating an understanding of
knowledge in that process. I use a critical thinking rubric, a process rubric that's sort of neurocognitive driven. And so part of my
process in this in my own grading is to make students familiar with what that means and what those standards are. Because I think
that's an authentic thing to assess and a meaningful thing for students to assess is whether or not they're thinking and to what
degree and so forth. All the power gets put out on the table. We're doing something that's meaningful to everybody, and then we're
also doing it democratically or at least collaboratively. Not at first. And I tell students at first, you don't really understand what these
Steve Pearlman, Ph.D.
I'm going to help you understand them. But as we all develop an understanding later, we're all challenging our understanding from
one another and I get challenged by the students and it's wonderful. But they're able to have ownership of it, and they're able to know
where their grade would be because they can effectively put their own grade on things at that point. But they're doing so against
some kind of a standard. I think then the question becomes for what you're suggesting, what's the response to the professor in any
given subject matter? But the professor who says, Well, you know, I'm teaching this political science class and frankly, the students
have to know some things about political science and be able to dissect political science structure in certain regard that I have to be
able to see them do or the. Is this professor who's saying, I don't care if students have an epiphany or not in my class? Doesn't matter
to me if they have an epiphany, I don't really care. I do care that they're able to know these things about business or act and function
in ways that would be functional in a business environment. What's the response to those concerns?
Well, I mean, I could talk from a I could talk from the perspective of just a pedagogical perspective. If you want your students to learn,
having epiphany epiphanies is a good thing. It doesn't matter what they're learning. Changing your mind about something is a good
thing. Having significant learning experience does that are unexpected, is a good thing, helps us remember, helps us retain, makes
us more invested in the work. So something like intrinsic motivation versus extrinsic motivation. If at the end of the day, I want my
students to learn x y z or I feel like they have to know this political science set of perspectives, ultimately, then what we want to do is
use the most effective route to get them their grades are proven pretty. Why in a lot of different research, not be that effective,
extrinsic motivation. Not that effective. It doesn't mean that we throw out extrinsic motivators altogether. Extrinsic motivation is still an
important lever. It still exists. It's useful, but for the most part, intrinsic motivation is more powerful and more useful, more
pedagogically useful. So ultimately, those things that I was talking about changing your mind about something, having an epiphany,
these are all things that I would locate in that intrinsic motivator camp. And so ultimately, I would say to the person, Well, if you want
why, here are the things that can potentially get you there.
Here are the things that would potentially lead you astray. And then you kind of build your pedagogical approach from there. I mean,
there's lots of research as much as people think, well, they won't do it unless I tell them they have to do it. There's lots of research
that just throws that out the door, out the window. That's actually just not true. They are more likely to do it if you don't tell them to do
it. And here's the five things you can do instead of doing that. And so I guess I would say that if you're if you're looking for an output,
here's the inputs that you can get. Get you there. I would also say that there's a lot of conversation about, well, in my discipline, there
are these sets of things that a student has to know in my discipline. There are these standards that we must hew to that doesn't
actually change the nature of human learning, which is again, different people learn at different times, in different ways, to different
degrees. That is just the sort of base nature of human learning. And so to imagine that we've cut up the world into one, oh one and
one and two, and you must learn these things and one on one before you get to 102, it doesn't actually align with how humans work,
which is that some people might not get it until 102 or 103.
I mean, it's it's the classic sort of Bloom's taxonomy that you must memorize these six things before you can then analyze them. I
don't remember anything unless I analyze unless I think critically, that's the only way I remember, and that isn't necessarily unique to
me. That's and it's not necessarily a truth of all people. It is that different people learn in different ways and that we have to create
environments where different students can thrive. And also the second that we imagine that there is a certain set of standards that
students must meet in a particular way, then we are we are doing a disservice to non-traditional students, doing a disservice to
disabled students, doing a disservice to neurodivergent students, doing a disservice to students who are struggling because of
identity issues or bias. Or ultimately, we're imagining a perfect student that would learn in a particular way. And then all of those
students who are struggling are imagined as somehow imperfect, rather than building a structure that's flexible and allows different
students to thrive in different ways.
Steve Pearlman, Ph.D.
Pedagogy will resume in just a moment. But first, if you're a high school, college or graduate school educator, then I'd like to offer you
a full free preview of my online level one critical thinking program for students. I actually developed this program because so many
educators have asked me for a way to jumpstart their students critical thinking skills. This program, which is approximately a three
hour student experience, does the following. It teaches your students three essential mindsets for thinking critically. It teaches them a
copyrighted neurobiological process for thinking critically about any subject in any discipline, and then it does something particularly
distinct. It prompts students through a step by step process in which they actually compose a very short essay entirely driven by their
own critical thinking. Students can complete this program outside of class with no impact on your class time, and you can see the
final product when they're done. I think you'll find this to be an exceptional program for your students, but whether you assign it. Or
not, I'm confident that it will be an asset to you in terms of infusing critical thinking in your own approach to teaching. So provided you
have a dot, edu or public school email address, I invite you to just come to the Critical Thinking Initiative dot org. Click on the podcast
page and sign up to see the online program in its entirety. Again, just come to the Critical Thinking Initiative Dawg and to your Dot
Edu or public school email address, and you'll receive a code for a free program experience. Now, back to pedagogy.
Steve Pearlman, Ph.D.
I totally agree with everything you're saying with respect to intrinsic motivation. The research on it's abundantly clear everyone learns
better with intrinsic motivation. Extrinsic motivators can actually be detriments to getting people to engage more deeply. Certainly, it
suppresses critical thinking. It suppresses students willingness to take risk intellectually in the classroom. It can find students to want
to know what the next frog they have to leap over is in order to get through the hoop that does the grade that they want to get for that
course. And it really narrows everyone's vision of what education is and sucks the life out of it, right? I mean, I think what you're
talking about, which I think we forget too often in academia more collectively so that these are human beings, we're human beings
dealing with human beings, and we should be dealing with all the messiness and fuzziness that that entails. And for people who look
outside of academia and think teaching is easy and simple, they're looking at it from that sort of Cartesian perspective. Somebody
says something. Therefore, the students will learn it, and that's good teaching, whereas anyone who's really doing it knows that it's a
deeply involved humanistic process. And I think an error that academia has made, which is what you're pointing to and what I agree
with and talk about a great deal is that we've sapped some of the humanity out of that, that it is this more antiseptic experience and
it's not dealing with the messiness. We're not dealing with the full student, their full mind, their full intellect.
Steve Pearlman, Ph.D.
At the same time, where is the other side of the coin for you? If we're saying to students, and I appreciate your point, look, maybe
you're not going to get everything this time around and certain students are going to learn at different places. Maybe there's no
objective standards that are really effective for what we want students to accomplish in a certain given class. And I certainly agree not
only that one oh one one oh two divisions are artificial and ineffective and contrived, but that disciplinary distinctions are contrived in
a lot of respect. The notion that history doesn't also have to do with the study of economics or the study of literature from that period
or sociology or everything, just as literature does and just as business does. All of our lives are intertwined in all these things. We
don't go through our lives with different subject matters during the day. But at the same time is whatever a student learns or doesn't
learn OK, as long as they're having an epiphany that for them is meaningful, but perhaps have learned very little about the subject
matter. There are some bar there, right, where maybe they're not engaging or fluent in that subject matter in any degree or to enough
degree, or is there not such a bar? And are you saying really, as long as the student is able to represent some meaningful reflection
or seeming growth from their perspective, they're very subjective perspective that it's OK.
I guess I mean, I would push on the word, OK, it's not. It's not a matter of whether it's OK or not. It's a matter of the fact that it just is.
And I would connect this to the idea. I've been hearing this phrase so much, particularly in K-12, but also even used in higher ED. To
some degree, the idea of learning loss, the idea that somehow in the last 18 months, learning just stopped happening. Well, that's
actually not true. And in fact, I think most people would observe if they just thought about it for a second. No, we probably learned a
lot more in the last 18 months than we did in any of the preceding sets of 18 months. We learned different things. We learned in
different ways, and the same is obviously true of our students. I think that that idea of learning loss is predicated on this notion that
there was the set of things that we were doing and then the set of things we were going to do next year. And somehow, if we can't
continue business as usual, that good things are not happening and whether it's good or not, it just, you know, students necessarily
learn they necessarily have learning experiences irregardless of what what we do.
I think that what I would say is that pedagogically, if I think, do I want to have an influence on what students learn when they learn,
how they learn? How can I best have an influence on that at that moment when a student is struggling and they maybe can't see the
degree to which they're struggling, I want to be able to step in and tell them, here's what I see happening, and here's where I think
you could push yourself. Here's the hard question that I feel like you need to hear it this moment. How do I get there for that moment
to happen and to? Happen effectively, the student has to trust me. The student has to be intrinsically motivated and I have to have
built a relationship with the student where that can even be heard at all. That isn't done through a set of standards. It isn't done
through a set of goalposts or hoops that they're jumping through. It happens through a different set of things and it happens through
that critical conversation and dialogue. So I guess I would say that if I want to be able to have an impact at that moment that you just
described, then there's work that I needed to do to get there.
And that work has nothing to do with standards. It has nothing to do with having high expectations. It has nothing to do with giving
grades or closely monitoring or policing student learning. It has to do with building a relationship, a relationship where two things can
happen. One, I can see it when it happens, and two, I'm able to intervene. And the other thing that I think that we can do is help
students be, as Paula Ferriss talks about bee readers of their world be critical readers of their world. And ultimately, that allows that
student ideally to also do that in their other classes where I'm not present. So if I can get students to be critical readers of grades and
assessments as a system, then when they go into their other classes where they're graded very differently from, they are in mind.
They retain that ability to continue doing that work elsewhere in their education, too, because I don't just want to intervene in my
class, I want to help students be readers of their world so that they can intervene in all of their classes.
Steve Pearlman, Ph.D.
Yeah, I think that's where we overlap a lot in that as I ask students to interrogate the critical thinking and they're able to sort of grade
their own papers and grade one another's papers, they're also able to examine how grading is happening anywhere that they go in
academia, who is holding the power for it ultimately, and how it's being wielded and through what mechanisms it's being wielded.
I guess I would say that I want it to be different in their other classes. I mean, at the beginning of this, and I don't think you you meant
this in the way that I'm going to describe right now. But you mentioned camps, the upgrading camp and the grading camp and the
rigor camp. To some degree, I don't necessarily think of them as camps. Again, learning happens in different places, in different
ways, and I want students to go out into the world and have lots of different kinds of learning experiences. So I don't want every
teacher to model their pedagogy on mine. I want them to find the pedagogy that works for them and their discipline with their
students. And I want students to have different experiences across their education. And then, I mean, we talk about things like that
standardized test that the nursing student has to take in order to get their licensure. That's a different set of learning experiences. And
yes, I want the things that I do to help prepare them for that moment where they have to do that standardized test in order to get their
licensure. And how do I do that? I don't do that by subjecting them to standardized tests for years, upon years, upon years. I do it by
helping them be critical readers and being critically engaged in their education.
Steve Pearlman, Ph.D.
I'm going to read something to you from my dissertation, and one of the things I like about this interview with you today is that it made
me go back and look at my dissertation again. And, you know, partially horrified. But in certain respects, it was interesting to revisit
some things, and I'm going to read a passage here where I'm talking about one of my motivations and rationales for why I think it's
imperative and moral and empowering for students to be involved in grade discussion in authentic ways, in ways that count not just
what do you think about your grades for this class, but really where they're effectively participating in that and affecting it? And so I'm
referencing Fienberg Peoples and Hart Davidson from nineteen ninety seven, and I write grading artifacts such as assignments and
rubrics quote are only half of the technology of grading, the other half being the network of cultural, institutional and personal values,
rules and decisions that make up the technical code of the grade seen in this light, the teacher's job of grading papers amounts to
participating in a number of arguments within and among the institutional frameworks in which she works and with any number of
specific subjects, students writing, programming administrators, other teachers, parents, et cetera, end quote. And my point is that
when we are putting a grade on a student work, there are all these other conversations and pressures and bases of knowledge and
conversations that are happening among faculty members. Some of them are very conscious and deliberate, perhaps with other
faculty members about how would they grade something or what do you grade on or what does this course need to cover? And some
of them are very subconscious and institutional and just assumptions that we all have or perspectives that we all have about what
academia is, what grades are or what they represent.
Steve Pearlman, Ph.D.
We might not even be self-aware about. So students can see this little grade come down onto their transcript or onto a paper or onto
a test. And all they can see is this sort of lifeless artifact from it, and they're kept behind the screen. They're never able to see and
understand, much less, really even participate in all those other things that these authors are talking about, all the conversations and
pressures and assumptions within the institution. So in my thinking, the problem is therefore that because it's this lifeless, refined
artifact that emerges from what might be a very rich process or culture, but when it reaches them, it's this lifeless thing that they really
can't fully appreciate and understand that students therefore have sort of no alternative in education. And I'm essentially using this a
bit too much. But they have very little alternative in education. But to just go for the grade because it's the only thing that at that point
for them can have real cultural value and meaning the learning could be nice too. And we hope at points that they get involved in the
learning and some certainly do. But on the whole, I think more broadly, they're just going after the grade because they're not able to
see all these other things that are going on. And so is conversations and assumptions that are much more interesting than just the
grade itself. I think I'm hearing in what you're saying some of that, but I'd like your perspective on that.
Yeah, I mean, I think that our process is in fact to go, maybe go even further. Our processes actually teach them to ignore that other
stuff because the second that we start associating a letter grade or a quantitative mark with an artifact, it suggests that the artifact is
the thing that matters and it gets it almost distracts the students and distracts us from all of those other things. And I mean, even even
if I think about something like portfolio grading, which the idea of portfolio grading is to somehow that your portfolio is meant to reflect
your process. Somehow the end of the day, most portfolio grading is just an extreme version of associate quantitative marks with
artifacts. You're just associating with a stack of artifacts, and the bad thing about that is then which artifact is that in there that led to
that grade? It just makes it even more inscrutable. It's already inscrutable to take a thousand words that a student writes and then put
a B-minus on them, and for the student to try and imagine what is it that I did in those thousand words that led to this being a minus?
And then the fruitless conversation that we have with them to try and explain it. Now, if you look at portfolio grading, it's really just the
most extreme version of that. Your portfolio gets a B-minus, and so you then it makes the whole conversation even more inscrutable.
Steve Pearlman, Ph.D.
I couldn't agree with you more. I'd like to buy you a beer sometime and talk about this because one of the things I really rail against is
the idea of and not the way you're doing it, because the way you're doing it, as you've mentioned it earlier in the podcast, that
ultimately the student is putting their own grade on something at the end of the experience. I think that the anti grading movement
that happened in comp for a long time with the idea that, well, we'll let students write drafts and write papers and we won't grade
them. We'll just give them feedback and then we'll maybe put a portfolio together. But at the end, we're still going to put a grade on it.
I think it's terribly disingenuous. I think it's a bait and switch because you're not really getting rid of the grade. You're just not telling
students what the grade is. You're reading something with that student. And effectively still, there's still a grade lingering and maybe
there's even a grade in mind, and maybe the student thinks they're doing great and they're doing terribly. But they don't know that
until the end of the process, when suddenly a grade appears. I think it's a very disingenuous way to approach students.
We talk about something like trauma informed pedagogy, which I think is has always been an important thing for us to think about,
but is especially important right now. What kinds of things lead to trauma and anxiety? Well, one thing that affects people who have
trauma or creates anxiety is when the stakes feel really high and what happens if you just delay all of that grading until the very end
and where you're doing this portfolio style grading is the actually the stakes go up at every point along the way. That grade at the end
is in fact looming even larger than it would if you were doing it more often throughout the process. The truth is, there's a sweet spot
how often you grade, if you're going to do hierarchical grading where the teacher is determining the grade, you can't do it all at the
end because then you've actually just defeated the purpose you've saved yourself as the teacher, the anxiety of having to put grades
at all of these different points. And you've also saved yourself from having to have hard conversations with students where you talk
about grades. I mean, I feel like that movement of the grade at the end is really all about saving the teacher anxiety, not saving the
student anxiety. Because if you just throw a grade at the end and it has huge stakes on the students, the turns over, you're not
responsible anymore for having the hard conversations with students about how that grade happened.
And as you were talking earlier, I was thinking also about just grades as currency because we can talk about grades as motivators,
but also grades our currency in our system, which means that grades have quite literally dollar values attached to them. If you think
about a system like Georgia where they have the Hope Scholarship. And students have to maintain a B average or better, and a
student comes to you and they say, I will lose my hope scholarship if I don't get a B in this course or better. And you say, well, the
grade is the great, the greatest, the great. You're actually taking money from students. And imagine that student is housing insecure
and they lose their scholarship and you've made them homeless. All of a sudden goes back to what you're talking about, this being a
moral issue. This is so much more complicated than note. The grade is the grade. I am just objectively evaluating your artifacts, your
stack of artifacts and whatever happens stance. And I have no moral responsibility for thinking about whether you're going to lose
your scholarship or whether your food or housing insecure. I feel like that's really irresponsible of teachers and again, is the sort of
idea of objectivity is really about saving teachers from having to actually grapple with the moral, much larger moral conversation that
we need to be having about assessment and grades and the sort of currencies that we traffic in in education.
Steve Pearlman, Ph.D.
Let me speak to that for a minute because I'm going to read something you wrote. And if there's a place where maybe you and I
diverge, this might be it. But I'd like to hear more about what your thinking is on it, of course. I'm sure it's something I need to attend to
more in my own thinking, and this is in your upgrading. Frequently Asked Questions Blog post, you write Not every student begins at
the same place, nor is it even reasonable to imagine every student can or should end up at the same place. Ultimately, critical
pedagogical practice has to acknowledge the background, context and embodied experience that both teachers and students bring
to the classroom. Any predetermined, standardized metric will almost necessarily fail at that. And so I think what you're saying and I
want to put words in your mouth, which is why I'm asking, I think what you're saying is students are going to come with different
preparations. Some are more ready, some are less ready. They're going to have personal lives that are going to be affecting what
they're doing. There might be tragedies during the semester or what have you. We've all all encountered students who have
encountered a tragedy along the way and things like this. And I think what you're saying is that our assessment of students has to
take into account where they began and who they are, rather than just whether or not they meet for lack of a better word. The
objective standard that we're trying to get them to is that sort of what you're suggesting?
Let me let me give you an example because I don't think this is about person. I mean, I think the way that I would describe it as, I
would say, their material circumstances and the complexity of their material circumstances and also their educational histories. So in
a number of my presentations lately and I think and I think I've shared it in some of my work as well, I talk about data around
suspension from preschool and that black students are much more likely to be suspended from preschool than white students. And I
talk about how that suspension from preschool, how that becomes a part of that student's educational history and when they come
into our freshman composition class. That experience of being suspended from preschool changes how they engage with their
education, and ultimately we have to take that into account. We have to take into account whether a student is food or housing
insecure. I do a little thought experiment in some of my workshops where I ask, OK, that student in Georgia, in Georgia has come to
you, and they say, If you give me lower than a B in this class, I will lose my scholarship. I'm food insecure. What grade do you give the
student? And I think that I have a moral obligation to give the student to be or better, and I don't actually care what they did.
I think that I have an ethical responsibility to recognize and take into account the full ramifications of how that grade impacts that
student and impacts their future learning, as well as reflects their current learning. And so ultimately, that's what I'm talking about is
that we we take everything into account, which means their educational histories, their current material circumstances, the impact
that any individual grade might have on their future learning. So like, if I give a grade to a student that results in them being expelled
from school because they're on academic probation, is that was that the right thing to do? It might be the right thing to do if I had a
conversation with them and they said, Really, I'm not. I'm only in school right now because my parents are making me be in school,
but I had to have had that conversation. I and I had to have taken that into account. I had to have listened to the student and listened
to their specific circumstances and allowed that to impact the assessment and the conversation I'm having with the student about
Steve Pearlman, Ph.D.
Yeah, I don't know. I'm struggling with it and I'm struggling with it in the sense that on the one hand, there is a morality to what you're
proposing here that I think is important that we consider and maybe we're not giving enough consideration to in academia as a whole.
And certainly you're offering a very extreme couple of examples for this, but those extreme examples can also exist in the world. And
so they're not on. Fair examples, but you are working at far end of the spectrum, and that's OK. I think that's legitimate, a legitimate
place from which to frame that argument. So if I'm teaching a class and my goal is to get students to think critically and write well and
we all understand roughly what that standard is, we can all apply it relatively equally to one another is writing and so forth. I don't
know if I can look at the student who hasn't done it as well and say, Well, that's OK. You're still going to earn the same grade as
somebody who has, even if that person's come from a more disadvantaged place coming into the class, I'm not making a
presumption about who they are and so forth, but maybe I know that or don't know that regardless. Because what I also feel is
important in ethical for me is that I'm equipping students to go out there and do what they need to do and representing that actively
and fairly to them. And I think that there's a fairness question to the student who comes and says, Well, wait a second now. Yeah, I'm
not going to lose my scholarship, but I want the higher grade as well.
Steve Pearlman, Ph.D.
I worked hard. Maybe I worked harder than this student did, and maybe I achieved more than the student did. Why should that
student get the better grade than I'm getting or an equivalent grade or just be pushed up because of their life circumstance? Now, I
think I'm in a slightly different context than most of the people listening, because grades for me are again a participatory process, and
I fully agree that we should. Without question, if we have a student who is in that kind of a circumstance or any circumstance, we
should be the humanistic people that we need to be and serve that student and work with that student toward their success. So if I
know a student is in a certain way struggling or coming from a disadvantaged position, I will break my back to work with that student
to help them rise up and get to that b. If that the B that they need right in order to maintain the scholarship. And anyone who wouldn't,
I think, should question their role as an educator. That's what we're committing to do is work with the whole person, model,
compassion and model goodness for them in those respects as well. But how do you respond to the student who's saying, Wait a
second, though? That's not fair to me because you're elevating that person's grade, because maybe they maybe they came from a
different place, but maybe I had some hardships too. Or what have you? And I work my butt off, and I think I did better than is being
represented or they're getting something that is equivalent to where I'm getting when they didn't really earn it.
I think honestly, how I respond to that student as I take that as that hard moment, hopefully I've built trust with that student so that I
can be really frank with them in that moment and say, Why are you worried about comparing yourself to what someone else is
doing? Let's talk about your work, and let's talk about your learning and your learning process. This isn't about what your peers are
doing and comparing yourself to your peers. So that idea of fairness is so often predicated on the sense in which the role of
education is to rank students against one another. And there are so many things within our system that that do that ranking and that
reflect that. That is, in fact what so many of our educational systems are built to do. Peter Elbow writes a piece called Ranking,
Evaluating and Liking sorting out three forms of judgment, and he talks about ranking, evaluating and liking, and he talks about the
benefits and the use of each and why each exists. He makes the argument that ranking is actually detrimental and that it is not the
role of grades, not the role of assessment, not the role of educational institutions to rank students against one another. And in fact,
the more ranking that we do and the more we focus on ranking, the worse we are at evaluating, which is the thing that he argues is
the thing that we are doing in education and as educators. And so ultimately, that moment, the second that I hear a student
comparing themselves to each other or I hear a teacher talking about fairness as being measured by students neatly compared to
one another, which is not what I heard you doing, then I immediately think we're failing at doing the much more critical work, which is
evaluation, which is a much more complex, engaged process.
And then there's that third bit which is liking which he talks about, which is allowing the work of assessment to retain subjectivity so
that it is actually about humans relating to one another, human seeing one another. Human's being surprised by one another.
Human's having hard conversation about what work looks like. But honestly, why did that student come to me and ask for a different
grade? What was the conversation that I was having with that student? Maybe I do just give that great student a different grade, but I
say the reason is because of something else. And let's talk about that. It's not because, like, why are you coming to me asking me
and comparing yourself to another suit? There's something deeper going on there. Let's talk about that. Let's not talk about
comparing yourself to other people. Probably it comes from them being very high performers and having learned throughout their
entire educational career that the way that you succeed is by being better than your peers and the way that you prove your learning
is by being better than your peers. And you know, that's the thing that I want to work with and talk talk through with that student.
Steve Pearlman, Ph.D.
I certainly agree, I think. Ranking is a problem, especially when we have those extreme examples of the professor who will say, you
know, I'll only give one a and then it trails off from there, there might be, I might allow to a minuses and so on. There's this curve
established where only one person can sort of win the class and certainly that creates a negative competitive atmosphere. And I
appreciate your point that, as I said, the antiseptic nature of assessment is certainly problematic and certainly detrimental. But aren't
students ultimately going to be judged externally by other metrics as they get out into the world competition for jobs or promotions or
what have you? And I don't like the idea of talking about education as a vocational prep. Education should have its own merit for
sure, and that should be primarily what we're focused on. But I don't think we could ignore the reality as well that grades carry that
currency out beyond the institution. And so what about the student who says, maybe I'm not interested in comparing myself per say to
the student, but this is going to have consequences. You know, my grades have consequences and I'm working for those grades and
they are taken as whether they are legitimately that or not. They are taken as a representation of my knowledge and ability outside of
this institution. You've undermined that by giving this other student a grade that for lack of a better phrase for us here, they didn't
earn. What's your response there to that?
My response is, I mean, it comes back to just a fundamental question that I think that we all need to be having about grades who are
grades four. There's lots of answers to that. Grades are actually within our current system. Therefore, a lot of people there for a lot of
different people, there are a lot of audiences for grades. I think that the one audience that I don't give a shit about is future employers,
graduate schools. My giving of grades is not a somehow a communication, a way of communicating to some future employer.
Whether or not a student is capable or able. That is, to me, the worst way to use grades. The best way is to use them as a way of
communicating with students. And so they need to be the primary audience for grades, in my view. And the ultimately, there are much
better ways of communicating to future employers or to future grad schools. And the truth is, when we actually look at it, most
employers are not making decisions about whether or not a student is the right person for a job based on their grade point average.
And so I want to get students to a place where they can be successful if I even do like if I see my role as being vocational, if I want to
prepare them for a job market, it's not by helping them pile their GPA up to high heaven. It's about getting them to be able to do the
things that will actually set them apart in a job market that is extraordinarily competitive and extraordinarily difficult. So in some ways,
the grade is a red herring at that point to another conversation that I and hopefully I already had that conversation with the students.
And so we're not even having this conversation about how my grade point average will affect whether I get employed at some future
Steve Pearlman, Ph.D.
I don't know if I can go that far, but I certainly I certainly agree with so much of what you're describing here and also the specificity
and the detail with which you can see this structure and operate within it with what you're doing. Certainly, the larger construct of
grading in academia, I think, is horrendous in so many respects. Again, mostly because it's not necessarily really connected to
authentic learning or meaningful learning, or, for my purposes, critical thinking and growth in those more humanistic areas and soft
skills. And what have you. Maybe I'm asking some hard questions. I ultimately really support, or at least find myself in a similar camp
to so much of what you're suggesting. What's the path for an educator? Maybe very persuaded or at least intrigued, and wants to
explore possibilities of easing up, at least on the grading? And we know that upgrading is a process. It's not a thing. So I'm going to
read the problems you list with traditional grading and then use that as a springboard to talk to the listeners about how can we start to
wade into the pool if nothing else? So you write, and this is in grades are dehumanizing. Upgrading is no simple solution. Well, first of
all, you're right, there's nothing ideologically neutral about grades, and I think that's maybe the heart of what we're talking about here
entirely. So you write, we pit students and teachers against one another. We rank students in fiercely competitive ways. We measure
output with little concern for the learning process. We demean student work by crudely quantifying it. We start from a place of deep
suspicion of students. We assess in ways that reinforce bias against marginalized students. I think that sets up a structure that has
some hard truths in it to varying degrees, perhaps for varying educators or varying institutions. How do we start to break out of it?
Well, so if I think about like, what motivates our first step? This is neat and tidy, and I don't mean it to be new and tidy, there's a lot of
nuance here, but imagine there's two ways that you could approach your first steps in thinking about how you approach assessment.
You could say I want grades to be more effective, more meaningful. I want them to do better work in the world, et cetera, et cetera. Or
you could decide I want to minimize the harm that they do. And if we talk about camps, I'm in the second camp. I want to minimize the
harm that they do. That is the most critical work that we need to do at the start of this work, of inspecting our processes at our
institutions. Start by minimizing harm. Then after we've minimized some of that harm, then maybe we get to a point where we talk
about how to make them effective. Then and then ultimately, if I if someone was wanting to begin to rethink how they approach
grading and assessment, this goes back to where this conversation began. I think the most important first step that you can take is to
start talking to students about grades and start having complex, deep conversations, even if it doesn't change how you approach
grades or you imagine it won't change how you approach grades. Because the truth is, the second that you start to have
conversations with students about grades, it will necessarily influence and change your process so it can start by having a
conversation with students about grades. And that can be as simple as assigning something like Alfie Coen's The Case against
grades and just having a forty five minute conversation with students about what grades are, how grades make meaning, who grades
are for, and that conversation by itself ripples out to all of the conversations and all of the ways that students are engaging in the
process of assessment. And it also will allow us to be better at the work that we do again. Understanding how grades affect students
and how grades make meaning for students changes the way that we use them.
Steve Pearlman, Ph.D.
For me, it's also that if we want students to think critically, then we can hold all the locus of power for ourselves. We set up these
grades as a barrier. There's almost a defense mechanism that they can't question our authority too much or challenge what we say
too much, which is exactly if we want them to think critically what we need them to be able to do. Maybe it's not something they're
always going to do or always is positive, but it's something that we want them to be able to do. And I think that we have to start to
break that down so that grades become something that can be interrogated and discussed and are out on the table as much as the
subject matter, because grades are how ultimately we're reviewing in academia the success of the discussion of the subject matter.
Well, how can we successfully engage or critically engage a subject matter if we cannot also critically engage how we're viewing and
judging the success of that discussion? So I think those things have to come together, and that's why I broke away from traditional
grading and involve students in the process. Before we wrap up and I got to say, I think we could go on for hours. I certainly could go
on for hours with you. So maybe we'll have you back in another time. And I hope there are a lot of listener questions. But one thing
that I do want to touch on because I think it would be an error not to do so in the current climate. What relationship, if any, do you see
between traditional grading as it's typically manifested in academia and what we see happening politically in this country? Because I
think there are connections, but I'm wondering if you see any or if you're willing to talk to them.
I mean, I think absolutely, and I was thinking back to your comments that you were making about students being engaged in the
process of their own education. And I think about that critical moment I was referring to earlier where when a student needs a hard
question asked of them or they need to be pushed in a particular way or challenged, or there's a particularly tricky kind of evaluative
move that would be useful to move their learning forward. At that moment, they're going to be most receptive to it, in my view, if they
also feel comfortable asking those hard questions of me asking those hard questions of themselves, asking those hard questions of
each other. So ultimately, if we think about the work of education as being about producing citizens, not necessarily producing
workers, not necessarily producing people who have grades attached to them, but we think about producing citizens. We go back to
this idea of Paulo Ferreira helping students become critical readers of their world. Why do we want students to be critical readers of
their world? It's not so that they can critically intervene in discussions about grading during their undergraduate career. It's so that
they become people who can critically intervene in their world throughout their lives. And so the goal is not just to get them to be
critical readers of grades as a system, it's about helping them be critical readers of systems, period, so that they do that throughout
their life. So ultimately, we talked about moral what is the moral duty associated with grades? The reason that there's a moral duty is
because education is not just about training students for jobs, it's about helping students be citizens.
Steve Pearlman, Ph.D.
And so would it be accurate to say that you think that the current structure? As they're typically existing are antithetical to that and to
extrapolate that further. Are you implying that too much of the populace, perhaps as a result of these structures, lacks the ability to
look critically at political structure at authoritative figures or what have you?
I mean, absolutely. I mean, it's so it's interesting because so often people point to education as this liberalizing force that essentially
educate our higher education system is training people to be good liberal citizens that are going to go out and elect all the Democrats
when. I mean, the fact is that I think that we can point to something like Facebook as part of the problem. Part of the reason why a
authoritarian was elected to the presidency of the United States. But we can also look to our own systems of education to explain why
people didn't ask the hard questions right at the moment that they needed to. So as much as Facebook is responsible for Donald
Trump, I think also our educational systems also have to take responsibility. That doesn't mean that all of our institutions that every
single one of them is should take the blame for electing Donald Trump. We should all take a really hard look at all of the shit that we
all were complicit in that led to an authoritarian dictator being elected to the presidency of the United States.
Steve Pearlman, Ph.D.
I've had the privilege of doing a number of interviews about vaccine hesitancy, about Trump, about so forth with respect to critical
thinking and education, and I make a similar point. There is an argument that academia is in certain respects, a bastion of liberalism
with respect to the humanities and what it discusses at certain points and broadening viewpoints. Right. If nothing else as a goal, but
structurally and certainly with grades is one of the greatest representations of that. It's the exact opposite. You know, we've trained
students to follow authority to accept the word of authority as truth, to seek the praise of authority, to fear the condemnation of
authority and so forth. Through all of these grading practices and all these structures, recreating people who are more optimistic than
they are individualistic than they are members of a society who want to reason their way through it. I agree with you. Education as a
whole has to take a hard look at itself, not in terms of the content that it's trying to teach, though always that, but we're always doing
that right. But more in terms of the structure of it. And I think upgrading, as you're describing, it certainly is one important step on that
path. So before we close out here, let me certainly open the floor to you and ask you if there is anything that you want to add anything
that I should have asked you that I didn't. Why did Steve forget to ask me the central question that everyone else asked me in every
interview? Turn it over to you before we wrap up here.
You know, the thing that I feel the most anxious about is I have a four and a half year old daughter. My four and a half year old
daughter is in preschool. I'm already hearing things about the expectations of what a student entering kindergarten would have, you
know, and here's the things that we're trying for. Here's the things that are expected when they enter kindergarten, and I am frankly
terrified. I am frankly terrified to watch my four and a half year old being pulled into this system that you just described. I believe
public education is absolutely essential, so I would never shed on public education. On the other hand, I don't think that we're doing
our job as well as we need to be doing. And I think one of the things that we can point to is we can look at the people who've been in
the educational systems. The longest people like you and I, people who did undergraduate degrees, who did K through 12
undergraduate degrees, graduate degrees PhDs and then now continue to work in education. And we can ask ourselves, what has
education made us into? What has education become? What is this? What have the systems of education increasingly become? And
I think that we need to take that hard. I mean, ultimately, the way that I solve the problem for my four and a half year old is taking a
long, hard look at my own institution and asking myself, What are the structures in this institution that actually help develop critical
thinkers? And what are the structures in my own institution, in your institution, in every institution of education that actually get in the
way? And I think we can be doing a better job.
And I think it's our I mean, I think it's constantly our job to look at these systems. Someone asked me about whether we're at a
moment where we're revising our systems and whether that's why I'm grading is becoming such a locus of conversation. And my
response was to say education in our educational systems have constantly be under revision. We can look back to John Dewey. We
can look to Henry David Thoreau, Ralph Waldo Emerson, we can look to Virginia Woolf, we can look to bell hooks, we can look to
Paulo Ferrari, we can look to Peter Elbow. We can see all of these moments throughout the history of education, where it was under
revision and what is our responsibility right now is to keep it under revision, keep it under constant revision. Otherwise, we'll end up
with the. Authoritarian state that a certain orange person wanted us to have.
Steve Pearlman, Ph.D.
Well, Jesse, it's been an absolute pleasure. I'm going to really strive to have you back at some future point and whenever, wherever
you are, I'm going to buy you a beer, that's for sure. Thanks so much for coming on the podcast. I really appreciate it, and I think that
our listeners are really
Steve Pearlman, Ph.D.
Going to benefit from this discussion. Thank you.
END OF TRANSCRIPT
Automated transcription by Sonix