Jump to content

Analysis of Scenario Ratings


Niemand

Recommended Posts

Inspired by TM's Scenarios-by-Level chart, I've finally gotten around to doing something I'd meant to do a long time ago: making a chart comparing CSR ratings of scenarios.

 

Disclaimer: The categorizations of scenarios, while subjective, are my estimates of community opinions, not my own opinions. As described below, I attempted to organize categories only according to patterns I observed in the data.

 

The chart to which I will be referring is here .

 

When I initially made this chart, it was simply in alphabetical order. However, I noticed immediately several distinct groups of scenarios: those with a small spread in ratings and a high average, a small spread and a medium average, and a small spread with a low average. Then, there were scenarios with high, medium, and low averages whose spreads were much larger. Having seen this, I began to sort the scenarios into the categories in the current chart. After I'd sorted all of the ones that seemed by eye to obviously belong to each group, I wrote a set of rules describing each group. Using made up numbers, (because I don't remember all of the real inputs to the calculation) supposing that the lowest average of a good scenario was 8 and the highest average of a medium scenario was 7, I averaged these and declared that the dividing line was at an average of 7.5. This is the list of actual rules that I derived:

 

  • 'Good': Average >= 7.4
  • ---'Controversial': Spread >3.0
  • 'Medium': Average >= 5.7
  • ---'Controversial': Spread >2.2
  • 'Poor': Average >= 3.7
  • 'Bad': Average <3.7

Note that 'Controversial-Good' scenarios should not be considered poorer than 'Good' scenarios for which there is 'Consensus', there is merely a greater difference among people's opinions, but the averages lie in the same ranges. Also the distinction between 'Poor' and 'Bad' was mostly for aesthetic reasons, but the division was placed in the relatively large gap between 9Var and UV.

In conclusion: I have no idea what this tells us, but I think it looks pretty.

 

A few final notes:

The source code of the program I wrote to draw the chart is available. The data I used was yesterday's data on Aran\'s CSR summary page . The program is very bare-bones, it reads in a plain text file on each line of which is a scenario name in quotes, then separated by spaces the minimum, average, and maximum ratings the scenario received. I'm hoping to either re-write it to parse Aran's page (or get him to port it to php and have it run directly from his data :p ).

 

I also tested the chart in every web-browser I have; it renders correctly in Safari 3 and Firefox 2, slightly less well in Camino, Opera, etc. Hopefully at worst only the rulers get screwed up, so everyone can see it more or less correctly.

Link to comment
Share on other sites

  • 2 weeks later...
Quote:
Originally written by Restoration Action Figure:
ASR isn't controversial though.

Great chart. One comment: isn't 'poor' typically a worse gradation than 'bad'? I always remember excellent, good, fair, bad, poor from school (and Angband pseudo-ID).
Apparently descriptive linguistics does rot your brain. :p

I've always considered "poor" to be less bad than "bad". And the worst Angband pseudo-ID is "terrible"; as far as I'm aware, there's no "poor" pseudo-ID.
Link to comment
Share on other sites

You're right about Angband. I went to look for examples and I found... not much. Most of the rating scales or systems I can find through a quick google search that use bad or poor, do not use both. This is accomplished in various ways -- having only one negative rating, having a more clearly horrendous word for the worse option, "very bad", etc.

Link to comment
Share on other sites

I haven't seen "bad" in any grading system at school; the worst was always "poor" or "insufficient". I have seen it in a rubric for marking the condition of a library book, though - the scale went "New", "Excellent", "Fair", "Poor", "Bad". Apparently saying that a student is bad would be rude, but the library book doesn't mind.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...