On Tuesday we identified some key terms/concepts and talked about scoring guides. If you were there, could you please provide a quick update by (1) identifying what you think the most important assessment concept is and why; and (2) identify one reason you might use a scoring guide. If you were not with us on Tuesday, please identify a question you'd like us to consider today.In writing, please simply use the respond button; you don't need to create a new post.
Thanks!
One thing that I think is really important to consider in assessment is the difference between analytical scoring of a paper or portfolio and holistic scoring of a paper or portfolio. I think this gets lost in many models, particularly for folks who are not familiar with the rich history of portfolios and composition studies. Rubrics tend to be the default method by which all things are assessed in certain education circles, and that's fine, but discussing a holistic model rather than an analytic model (which is what we typically see in the literature and find as examples on the web when we look for them) is a worthwhile endeavor. I do prefer more holistic scoring guides for students to use as heuristics for their eportfolios and other large projects, or as revision guides for those projects. I don't like rubrics that appear to be prescriptive and confining -- and I don't like to give them out too early in the process (I like checklists that guide the process of planning and thinking about the prompt and project, instead, so that the students are not writing to the rubric and stifling their own creativity and idea generating strategies).
ReplyDeleteHaving co-taught with you, you know I share as similar perspective--especially using a scoring guide as heuristic for the ePortfolio. I have moved away from scoring guides for individual papers and projects. I have also moved away from many of the checklists this semester, fearing that they had become too prescriptive as well. I'm not sure my fear was entirely grounded, but it also hasn't seemed to negatively affect the students' work.
DeleteI love holistic portfolio assessment as I think it gives students who have always been in a "high stakes," graded writing classroom a chance to explore and use "trial and error" something we know from cognitive science is beneficial to student learning (Gee, "Good Video Games and Good Learning). And I agree, prescriptive is not beneficial in this activity: I just read some articles in graduate school that explains how prescriptive assignments don't align with systemic functional linguistics, and do prefer a functional grammar approach versus the traditional grammar approach. But contextualized, and given measurements that make sense in the rubric, I think they can be very useful. Opening rubrics up to student input could make this even better, I think.
DeleteMy feelings about scoring guides or rubrics are mixed, really. The term even - scoring guide - feels reductive and grade-oriented to me (even though that term accurately reflects its purpose). Perhaps I just want to harbor the illusion that I am giving formative feedback (that just might have a grade attached). I do think that rubrics can be used productively, especially when you include students in the formation of the guide. This has worked well for me when doing multimedia projects with students, and it helped to come up with a set of criteria that allowed for the fact that students might not be that skilled in the tech medium that they were using. I think it actually allayed some of their anxiety about not being able to produce a product that looked "professional" because we determined evaluative criteria that highlighted more rhetorical issues.
ReplyDeleteMichele, I'm interested in your self-assessment that you're "just want to harbor the illusion that [you are] giving formative feedback". Do you score papers or just offer feedback? I'd like to know more about your process.
DeleteMost of what I do is feedback but I will give students tentative grades on their emerging work. For example at an advanced point in the drafting process, I will tell a student that if this were the draft that I see at the end of semester, it would be in the C range or that it would not be passing. I think that they need this sense of clarity as to what my reading of the evaluation criteria is. Their idea of what constitutes an A essay or what makes an essay not passing is often quite different than mine. I used to avoid any grades but found I was really just using code words in place of the letter.
DeleteI have been thinking about validity in assessment lately. My concern is over whether I am assessing in a way that is helpful to student learning. In the classroom, I worry that I am providing guidance on student projects that is helpful without taking over the students' work. At the course level, I worry over what the final grade communicates about a student to others. At the program level, I worry whether our community assessment is providing us with an accurate picture of students' abilities.
ReplyDeleteBrenda, This is a concern for me as well. I've been asking students to reflect on their work (and then I evaluate that reflection), but I don't think the questions I'm asking really tap into anything except my students' desire to tell me what I want to hear. I am wondering whether I need to assess my assessment process, especially by asking students what they think of my responses. I'm also concerned that my program doesn't necessarily assess whether the program is achieveing what it is setting out to achieve. It feels like a lot of layers of uncertainty!
DeleteI would tend to think that classroom assessment would be valid if it is fair: i.e. measures what you have asked the students to do. One tool that might be useful to that end is having colleagues peer review the assignment or have them write to the assignment and then use your assessment tool to assess and evaluate those results. If another professor can't write to the prompt and be measured reasonably with the assessment tool, either might need to be revised.
DeleteThe term would be "rubric." My relations with rubrics began well enough; I found they were a useful tool. But, over the years, the "rubricization of education" has lead me to a distinct aversion of rubrics -- it seems to me so many rubrics are without purpose or sense, and leave me to wonder "Why would anyone rubric THAT?" But recent discussion and reading have lead me to believe my relationship with rubrics can be reawakened. I used to use a simple rubric on the cover sheet I would return to students with my comments, and later used some of those cover sheets for my students during peer review. They really like the rubric for peer review, so I think I need to explore that option again. However, this time I think we'll come up with a rubric together, as a class (perhaps in small groups first, and then synthesize and adapt), and perhaps different rubrics for different peer reviews, such as the proposal my 1001 class just peer reviewed. I'd also like to include a "personal goals" section on every rubric somehow -- I think that will help students to engage more deeply in the peer review and revision process. It's still in process...but I think I must put aside my rubricicidal tendencies and make peace with rubrics. If I do, I'll have another powerful tool in my instructor tool box for feedback and student engagement.
ReplyDeleteThat's funny that you mention coming up with a rubric with the class you teach, as I was just thinking about that yesterday. I thought it would be a great idea because it would get students (hopefully--I know some of these ideas are way too optimistic) to really think about what writing should do, what their writing should do. If you try out this activity with a class, let us know how it turns out and I will do the same.
DeleteOne of the ideas that has captured my attention regarding assessment is to have students play more of a role and to encourage them to see assessment as something to guide their learning as opposed to something that tells them what they have (or haven't) learned. I have tried to frame comments as guides rather than pronouncements, but I think that inviting students to take part in the assessment may help them feel more invested in the process.
ReplyDeleteAs for scoring guides, Kathleeen mentioned that the move from recitation to writing occurred so that those providing feedback would do so more fairly and evenly. When I'm on my 43rd paper, I know my "evenness" has gone out the window. A scoring guide, even rather rudimentary, helps keep me honest and at least generally effective in providing comparative responses (especially for summative assessment). I have used it for this purpose and plan to use it for portfolio assessment as well (I don't use portfolios as we've been talking about them here, but I plan to do so).
One reason I might use a scoring rubric is to make grading easier on myself by avoiding repetition in student comments. Often, in providing written comments, I find myself saying the same thing again and again, and through providing a rubric I can easily provide more general feedback on student work. In addition, if students have access to the scoring rubric they know what is expected of them and the scoring rubric can provide parameters that they can look to while completing and revising written assignments. For all of these reasons, scoring rubrics are a good idea.
ReplyDeleteYesterday, I enjoyed our discussion of assessment and learned many terms that I was not familiar with. What seems most important to me is the idea of authentic assessment--making sure that students are assessed on something that will actually be useful to their future. It is important that the assessment is "real", and if so, students are more likely to be engaged as they can understand the purpose of the assessment.
Hi Megan,
DeleteI agree with you that a well designed rubric can make grading of the final product easier. I like to share rubrics early in the process so students can provide feedback or suggestions for revising the rubric. I find it helpful to use the rubric to "grade" a sample assignment together as a class. By grading together, the students and I develop a shared vocabulary to talk about writing. I am able to explain what I am thinking as I grade their final product. I also like to use the rubric to have students evaluate their own writing. It helps me to see where they think their writing falls on the scale while they are in the process of drafting the paper. I use that information to provide more specific feedback to help them revise their paper.
Brenda
I think you make a very good point here Megan - one that I have not easily been able to come to terms with myself - teaching has to be manageable or life becomes burdensome. So, rubrics provide a safety net that I am willing to give myself, even if I don't sincerely believe it is in the best interest of all of my students. I cannot always, everyday, all the time, give everything I have to every student, rubrics protect students best interests where my own abilities fall short. Authentic is the priority - maintaining it is often like chasing butterflies - when you catch one it's really worth it!
DeleteYes--after our discussion yesterday I began thinking about rubrics in a way that I hadn't before. I don't use them now, but am considering creating one for the reasons I listed and also a reason that someone else gave--fairness.
DeleteI have another concern about assessment that came up when I did the pre-reading. So often grades and "points" are used as student motivation. I like the idea of providing more formative assessment that is not grade-based and am interested in the idea of even not providing "grades" until the end of the semester, but I am not sure how realistic this is. I find that students are motivated by grades.
Scoring, grading, bench-marking - I find it really difficult to come to a place where these terms coincide with my understanding of the creation of knowledge. And to think we can assess all student work in any systematic manner is more a commentary on our insight into learning than on students' abilities to learn. However distracted I might become with the "process" of assessment - its meaning, value, and impact - I am academic enough to know that assessment is a necessary evil of sorts. Evil in that it cannot capture the total picture (who/what suffers because of that) and necessary in that to forward learning we must have a foundation from which to proceed. So, for me the goal is to avoid prescriptive assessments that restrict students' willingness to venture into unknown learning ventures. A pie in the sky ideology at best.
ReplyDeleteSharon--I'm zeroing in on your comment about encouraging students to venture into unknown learning territory. I'd like to do so as well, and yet I'm still unsure about evaluation of students for attempts that might fail or assessing students who are just not going to achieve what others who start at a "further" point are achieving. In other words, do we assess growth or some identifiable goal, especially in regard to analytic thinking which seems to be set at very different locations for different students.
DeleteFor me, the student is the most important part of the assessment process. I really thought that the theory Kathy talked about yesterday had a lot of merit--that an assessment is only valid if it supports students learning. If you remove the ego, the desire to sustain one's livelihood, and all politics--the reason most of us teach writing is because we actually want our students to learn how to write better. They are our audience and if our lessons or assessments don't take them further down the path of writing awesomeness, then we haven't succeeded. So, the student trumps all. The scoring guide is there for three reasons, but it is not king: 1) to show the public that we have a plan for teaching writing and that the plan makes sense 2) for students, so then know what to do, but also so that they can begin to believe that they are becoming better writers simply because they won't believe they have gotten better at writing unless they can see checked off boxes 3) for the instructors, to help organize and determine what must be taught, what is measurable, and what reasonable success is. However, the problem we find with rubrics--or why they make us feel uneasy--is that the whole is always greater than the sum of its parts. We have all had a student score a grade either higher or lower than we know their paper merited. A real rubric, that can actually and accurately assess any piece of writing would be so thorough and complex that it would consume far to much time to be used practically--so our alternative is something that hits the big concepts, is semi-effective, yet doesn't take more than 15 minutes for a 5 page paper. If only we were eternal beings...
ReplyDeleteI get ya Joshua. I don't know if you have read "Responding to Student Writing" by Richard Straub, but that might be helpful to you in feeling like your comments are useful to your students. It includes samples from lots of big names in the field (only remember Peter Elbow's at the moment because it was so long.) But you might check it out if you aren't familiar.
DeleteLike many of the other respondents, I am very interested in student-generated scoring guides/rubrics. (Plus, I just learned the difference between the two yesterday!) This falls in line with other student-generated learning efforts I've employed in the past for teaching information literacy -- asking students to generate criteria on evaluating sources or discerning the differences between source types, for example. Why give them a criteria for this when students will learn more from generating their own? What I’d like to learn how to do now is HOW to employ this with respect to information literacy.
ReplyDeleteAnother question has to do more with assessing information literacy generally – particularly when course outcomes already include information literacy concepts. How, then, to assess these as a librarian (and information literacy program coordinator) alongside comp instructors and WPAs?
This is from Yvette Watson: I believe that the most important assessment concept is making sure that students are properly assessed based on what they have learned. They come into the classroom with basic knowledge of what they know, and instructors generally teach them concepts that are relatable and on a higher-order taxonomy. Successful scoring guides include rubrics of various sorts--whether content-specific or holistic. I use scoring guides mainly for larger writing projects, such as research essays and portfolios. I enjoy doing this because, not only does it help students to streamline what they are being evaluated on so that they can complete the criteria accordingly, but I also have a foundation that I can use when evaluating my students' completed work. Also, I can add more input to what I am evaluating and give students meaningful feedback that correlates to the content of and builds off of the scoring guide.
ReplyDeleteI've never been a fan of using rubrics as a means of assigning points for grades. I do, however, like the idea of using a rubric/scoring guide as a means to establish boundaries and expectations for students. Having students see on paper what I am looking for in their writing is good practice for students in terms of meta-writing awareness. I also like the thought of students becoming co-creators in the process of forming rubrics for their essays. This gives the class a sense of buy-in and awareness that they do know how to detect, evaluate, and create good writing. One of the last things we talked about yesterday centered on this idea of rubrics or course outcomes as being useful tools--but should not trump audience, and I think that is very important. That's why I like rubrics as a means of introducing to students writing expectations and boundaries--not necessarily as something that directs students, but as a way to guide students.
ReplyDeleteThere is a need for holistic grading. At the same time, there is a need for scoring guides if they are not used prescriptively. As guides, they can be helpful and informative to the writers, instructors, departments, colleges, etc. They can be an essential part of the learning and teaching process.
ReplyDeleteI use a primary trait assessment scoring rubrics as a way to provide a picture to students about where they are doing well on an assignment and where they need to focus there attention. I use them as both formative assessment and summative assessment as a way to guide them and also as a way to provide feedback and rationale for why they were graded as they were. I'm rethinking what I do with assessment based on yesterday's discussions because of the idea that assessments are only of value if they are tied to some way to student learning. How do I do that in all of my assessments, that's what I'm rethinking.
ReplyDeleteI have used rubrics it seems like forever ... :)....and keep revising and refining these in the process of applying, trial and error and then re-trying, applying and revising these together with my students. I also collect direct feedback from my students in an embedded way as I have them apply the respective rubrics to their own pieces before I grade and then grade the pieces. Afterwards, in a face-to-face feedback conference we sit down, discuss the writing and the rubric descriptors and clarify the rubric expectations/descriptors if needed. Having a "comparison" of our (my students and my own) views in a tangible way seems to provide a good platform for feedback discussions.
ReplyDeleteHowever, my concern and question is that I have not sought out (due to time constraints and a lack of access to a consistent professional community of scholars at UC) a way to refine my rubrics together with any colleagues or to even see what others may be using to assess genres common to the levels I usually teach (eg. ENG 1000 - "memoir" "narrative" etc.) . I am concerned that my rubrics may not be as effective in assessing my students as I think as I am --as I am creating and developing these rubrics in isolation from other fellow "professional" eyes.
Does anyone have any recommendations or practical suggestions for a way to somehow get honest feedback from the "professor" perspective to ensure quality?
I am also curious how everyone else sets up the assessment (do you grade individual pieces all along the semester and also the portfolio as a whole at the end or all at the end?). Also, how do you guide your students' compilation of portfolios? I am looking for ways to support student clarity and understanding of the overall purpose of a portfolio and student "buy in" in portfolios as a process-based assessment. I often get resistance from students who claim that a portfolio is "redundant"..etc.
Thanks. I look forward to hearing your thoughts.
Karin,
DeleteHave you ever tried peer reviewing rubrics or assignments with other instructors? Have you ever written to your own prompts and then use your rubric to evaluate? Just a couple of ideas.
Daniel
When introducing the first research project in my English composition class, the students asked me where to find the rubric. I explained to them that there would not be a rubric for the activity. They looked shocked and one student asked me, "But how will I know what you want from us?" And this is one issue I have with scoring guides used on individual activities--it forces students to take the stance that I am the supreme audience for their work. Even if attention to audience is part of the scoring guide, my reading and evaluation is inevitable.
ReplyDeleteOf course my evaluation is inevitable, but I have found using the scoring guide for individual papers/projects doesn't allow the students to encounter the writing as they will in the world outside of school or help create a situated learning experience. I do like using scoring guides for portfolios--especially ePortfolios, where students design their own space to showcase their work and/or learning. Gotta go.
Bob--One thing I'm always surprised at is how students don't see the assignment itself as a rubric. If I ask them to do a, b, and c, and to be sure to do so with (insert general essay requirements here), isn't that, essentially, a rubric? I'm not sure why students don't see it that way, but it makes me think I should make that a conversation in itself--how can students find a rubric in an assignment? This then could carry over to any assignment they are provided, even (gasp) when they go out on the job and are given a task to do.
DeleteBob,
DeleteOne thing that I realized this week, partly based on the Huot article and partly based on discussion Tuesday, is that the rubric the students use and the rubric the instructor uses doesn't have to be the same. Our evaluations may be more in-depth or focus in different areas. However, I do see some problems there -- it has to be fair to what the students constructed together for their rubric. I'm considering writing a rubric as a class during peer review for each project and then addressing the student's rubric when I respond. We'll see how it goes. I think we'll go through the same process for the portfolios.
I think reflection is the most important part of the assessment process. It is the best way to get writers to internalize what they are doing when they write. I also like it because it is a meta-writing activity that may provide discovery for students about their own approach to writing.
ReplyDeleteA successful blog post, what does it look like and how do I assess it? I assign either a word or paragraph length as a guide and provide a writing prompt. I ask students to think about the prompt and respond to it by explaining and giving examples of what they mean, as best they can. The assessment is tied to whether they followed prompt directions and demonstrated some explanation and example for their points. These are more informal writings, so I'm not looking so much at mechanical stuff. The rubric identifies whether or not they met the terms of the prompt and whether or not they made some attempt to move beyond simply making a statement/point and tried to explain it to others.
ReplyDeleteOK Folks: I lost my first entry so here it is in a nutshell: the concept that resonated with me was formative assessment, and the reason that I use scoring guides is to lend some consistency to my grading.
ReplyDelete