Revenge of the Rubrics

[Dintiradan]

So, people have been waiting for the contest judges to release their written comments for the last design contest. Don't worry; I'm still working on them, albeit slowly. I'd like to find out how CSR readers prefer their reviews written. At this point, I'm going to raise a topic near and dear to the hearts of many here:

 

Rubrics.

 

Wait. Put down those tomatoes. Let me explain.

 

Like everyone else here, I find it rather easy to assign ranks to scenarios. This one was lacking in a few areas, but still playable? Give it Substandard. That one was innovative, had slick combat, and a good story? Give it Best. Giving a subjective score is easy when you only have five scores.

 

But when judging the design contest, we had to do more than just say which were good and which were bad. We had to say which were better than others. And that gets tough when each entry excels in a different area.

 

Enter the rubric. It's tough to decide, say, which of Incorruptible or The Cradle were better. But it's easier say that Incorruptible had better graphics and The Cradle had more content. So what I did for the contest was come up with six categories, each of which I gave a score from {0,1,2,3,4}. Summing the categories gave a final score, which I used to rank the scenarios. As an added bonus, the final score is in [0,24]. So divide by 5 (always rounding down), and you get the [0,4] interval again, which can mapped to the five scores that the current CSR uses.

 

(I want to point out: the primary purpose I had in mind for the rubric when I was making it was to rank scenarios that were close in quality, not give them a final score. If the rubric would give a scenario a score I don't feel it deserves, I would consider that a failing of the rubric, not my feelings. That said, there were a couple cases where the rubric gives me a final score when I feel that the scenario sits somewhere between, say, Average and Good.)

 

"Okay, you used a homemade rubric to rank the scenarios in the contest. So what."

 

The question for you, dear reader, is whether you want my upcoming text reviews of scenarios to be based on this rubric or not. Most reviewers here just give a few paragraphs on their impressions of the scenario, followed by a final score. Only a few break down their review category by category (Nioca, Grimm, etc.). Which style of review do you prefer? Ordinarily I wouldn't ask, and instead just publish my review in whatever way I wanted. But most of this community has historically been against rubrics, and I'm wondering what the feelings about them, and category-based reviews in general, are now.

 

And finally, here are the categories I used when judging the contest:

- Content (how many encounters, amount of writing, etc.)

- Story (interesting characters, lack of plot holes, etc.)

- Pacing (how well tension is maintained, are players informed of what to do next, etc.)

- Combat (is combat challenging enough, are encounters interesting, etc.)

- Scripting (customisation made to scenario, lack of bugs, etc.)

- Aesthetics (intuitive and interesting town/dungeon/outdoors, custom graphics, etc.)

[Enraged Slith]

I think you should say as much as you think ought to be said. I'm not going to hold you to anything.

[BJ Back From the Beyond]

Personally, I feel that the more information about why you like a scenario or didn't like a scenario, the better. This makes reviews a little more difficult to write, but also makes them more useful to players and designers in general.

 

Also, I should really get some reviews for the contest scenarios up.

[keira]

The best scenario rating format is one that says what needs to be said. Getting OCD and categorizing is nice, but in the end I don't think it's a big deal exactly how the review is set up.