Jump to content

Housekeeping: Revised Blades of Avernum Ratings System


Recommended Posts

  • Replies 385
  • Created
  • Last Reply

Top Posters In This Topic

Originally Posted By: Kelandon
Might be nice to move the information about how scenarios get marked "Top," "Worthwhile," etc., from the FAQ into the list page itself. That is, instead of:
"Top Scenarios

None yet!"
etc.
change to:
"Top Scenarios (at least 30% of the reviews have rated "Best")"
etc.


But what if a scenario has three reviews, and two say its terrible etc... ? I don't think a scenario can be the best if the majoriety thinks it stinks.
Link to comment
Share on other sites

Originally Posted By: Master Ackrovan
But what if a scenario has three reviews, and two say its terrible etc... ? I don't think a scenario can be the best if the majoriety thinks it stinks.


First, a scenario needs at least five reviews to be put in those lists.

Second, the system is designed to account for differences in personal taste. For instance, I thought BoE Za-Khazi Run was a wonderful, fun, gripping scenario, but most people shredded it in their reviews. I like exploration and adventure in exciting new places. Most of the designer base (which makes up most CSR reviewers) prefers plot-based scenarios. The point of putting the scenario in Best even if it's not thought that way by the majority is that enough people think it's a Best-quality scenario that there's a decent chance you might too. So you follow the link from the list and read the reviews, and decide for yourself.
Link to comment
Share on other sites

ADoS is correct. I chose five because that is the absolute lowest I would consider a sample to be statistically significant. Even that is probably too low, but we need to start somewhere.

 

Also, if 30% of our population thinks it is one of the best scenarios, there is a decent chance a generic player might like it too, even if the rest of the reviewers hated it. The point with this are to base recommendations upon how a significant number of people feel, not some averaging scheme. Now I agree the thresholds are arbitrary, but if you feel they should be something else, feel free to comment.

 

We may to a Review Contest when this thing gets off experimental mode. Good idea Kel! smile

Link to comment
Share on other sites

Will the BoE version be up in time for the rating contest?

 

It may have to be an extended rating contest, because school is starting soon for a lot of people, I believe. In the next week, I have to sort out my possessions for a yard sale on August 29th, and then school starts for me on September 9th.

Link to comment
Share on other sites

Quote:
I also was thinking that maybe we should put some popular scenarios that many would be familiar with, such as Bahssikava, Canopy, Mad Ambition, Frostbite...


I agree. I've been hoping the authors will contribute them to the testing.

Quote:
Will the BoE version be up in time for the rating contest?


That is the hope. It may be difficult from the sheer volume of BoE scenarios, however. I still need to think on how we could do this.
Link to comment
Share on other sites

Well, there are a lot of BoE scenarios, and I'm not trying to be rude, but are really complete garbage. My suggestion would be to hold off and put these scenarios up last. A lot of designers are not around anymore so I doubt they would mind their scenarios being up. I'll also contribute all of my scenarios to the new CSR.

Link to comment
Share on other sites

I'm having troubles coming up with keywords for Quarhag Pass. So far I've got "Avernum Universe" and "Multiple Endings." What else is it? Suggestions?

 

Anyways, you can put Rats Aplenty up there for now, so:

 

Rats Aplenty

Levels 1-5

Version 1.0.2

Avernum Universe, Humor, Dialogue Heavy, Beginner

Link to comment
Share on other sites

I did ask in general if you had any objections on AIM a while ago and you didn't say anything. Also, you did help me write the forum rules and did not seem to push that authorization is needed first. If you want it removed, I'll do that.

 

Right now I want to start adding all publicly available scenarios to the reviews as my time permits.

Link to comment
Share on other sites

...post 'em. Due to discrepancies with version numbers (readme files differing from .bas files), I've included the .bas file's version number, and the date of the most recently-published version.

 

Adrift

Levels 1-5

Version 1.1.0 (4-30-09)

Avernum Universe, Beginner, Short, Combat Heavy

 

Darkness

Levels 20-30

Version 1.1.0 (7-29-07)

Avernum Universe, Dark, Combat Heavy, Dungeon Crawl, Short, 24-Hour Scenario

 

Embers of Rebellion

Levels 10-15

Version 1.1.0 (4-30-09)

Avernum Universe, Branching Plot, Dialogue Heavy

 

Shades of Gray

Levels 20-25

Version 1.0.0 (4-30-09)

Avernum Universe, Multiple Endings, Cutscene Heavy, Short

 

In any case, the most recent version is always here. Feel free to include that wherever you wish.

Link to comment
Share on other sites

Just one more thing to all mods if you modify anything on the BoA Reviews forum. For everyone's benefit, the settings for posts/edits should be to exclude signatures but to include edit time stamps. The former is to keep things clean. The latter is so other mods know when the scores have been updated. Thanks! smile

Link to comment
Share on other sites

  • 2 weeks later...

Look, I'm gonna be honest here. In the case of Nephilim Mystery, it's difficult to pad a review out to 100 words. So far, methinks this is the only problem with it so far. I'm not concerned, personally.

 

Originally Posted By: Karl Projektorinski
And, I recall the 100 words being a guideline, not a hard rule?

 

I was under this impression as well.

Link to comment
Share on other sites

Looking at this from the outside, I would really suggest that you adopt some kind of guideline to avoid at least TM's review. TM's review contains an insult but no actual content. Reading that review would probably make me less likely to explore BoA at all, if I were new to it, just based on what it says about the community it comes from.

Link to comment
Share on other sites

Of course, 100 words plus is a good thing. Also, having people put something down is good as well. I'm willing to consider some remedy here in terms of recourse an author can take in the event of a review lacking in detail. The trouble is coming up with something where it is clear that a review was inadequate.

 

Perhaps a good guideline is this: at the request of the scenario author, a review may be not counted or stricken. This request is only honored if absolutely no detail is given as to why the ranking was assigned.

 

Examples of reviews that may be stricken:

 

"It's very bad."

"I did not enjoy this scenario."

 

Examples of reviews that cannot be stricken:

 

"The combat was tedious."

"There was no plot."

 

So in other words, if any attempt is made to describe some element of the scenario, the review will be counted no matter what. How does this sound?

Link to comment
Share on other sites

We can do whatever we want with TM's review; however, I want there to be a process in place that we can (mostly) agree upon for deciding which reviews are "good" and which ones are "bad". So yeah, we can strike this review or leave it without too much fuss except that it sets a precedent.

 

The real question I want to resolve via debate is what happens when (and it will happen) we have this where the scenario belongs to an active designer. What is the protocol we should follow? Remember, what we do sets a precedent for future decisions.

 

Thoughts?

Link to comment
Share on other sites

I don't understand why this is such an emotionally charged issue at the moment. I want to have this discussion now before we get into a situation that *is* emotionally charged.

 

So what happens if I (or another moderator) don't/doesn't like someone else's review? How do we decide whether to strike it or not? What are the standards we should employ? Clearly, just a moderator just arbitrarily deciding is not the way to go because we can get into a lot of trouble here. I'm not asking for a rigorous set of criteria, just some guidelines.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...