Jump to content

Housekeeping: Revised Blades of Avernum Ratings System


Recommended Posts

If you want to update your review that would be fine. Just make a new post and the old one can be made to go away. As long as we have moderators it should be fairly easy to update that. Certain members cannot post here directly, but I do support posting by proxy here because I think it is good for the community.

 

Really? Maybe we are talking to different groups of people. Most people I've spoken with have generally been supportive of moving it here. We get greater visibility and accessibility here, especially when it comes to people who don't know anything about Blades.

 

I would be supportive of a poll to find out what designers and established community members think. I'm not sure if a straight vote by majority is the right thing to base a decision from, but I do think it is good to at least get everyone's opinions and concerns aired and to try and address as many as possible.

 

EDIT: Just to clarify, I'm not sure just basing things off of 50% + 1 to be a good thing. If there is significant support (hard to put an exact number) and not a massive amount of vehement oppositon then we should move it here.

 

I am interested in hearing all of your concerns with moving it here. Hopefully we can address them or make a good effort to do so.

Link to post
Share on other sites
  • Replies 385
  • Created
  • Last Reply

Top Posters In This Topic

Okay, I have an idea. It would run alongside whatever numeric rating we have in place, and would basically be an interface for users to 'tag' content.

 

Reviewers would be given a small list of basic traits of a scenario (such as combat-heavy, dramatic atmosphere, etc.). The reviewer would select what traits they believes belong to that scenario, and that's pretty much all they have to do.

 

Later, when a person looking for a scenario went to this site, they would be choose what they were looking for. So, if 'Bob' wanted a large-sized plot heavy humorous scenario, he would select 'Plot Heavy', 'Humorous', and 'Large' on a form and the site would filter out unwanted scenarios and pull up a list for him (The more options selected, the less results someone would get).

 

Limits would be put in place, however. x% of reviewers would need to say a scenario had a certain trait before it would qualify as that trait and be listed in the 'search' function. Also, scenarios would be prevented from being listed in the search function until x people had submitted tags.

 

This would be hosted on a completly third-party site (This is a little heavy for UBB, obvoisly) and links could be put into each scenario thread that takes users to a page for that scenario that shows it's tags (and how many voted for them).

 

 

 

Critique?

Link to post
Share on other sites

I once thought this was a great idea a long while back. The problem is we tried them before with one of the contests a long while back. Issue is no one could agree on the definition of those terms whereas everyone can pretty much understand what the overall score is.

 

For example, your definition of "dramatic atmosphere" or "innovative" might differ from mine. This variance in terminological understanding makes things like this problematic. It's one thing when people disagree if a scenario has "quality X", it's quite another when no one can agree what exactly makes "quality X".

 

Also, simpler is better. Remember that real humans have to write reviews and real humans have to compile the results. The more complex the system, the greater the chance of error and the less chance of having people willing to actually do this. Also, having it span multiple sites just compounds the problem. I think we should keep things as central as possible.

 

Link to post
Share on other sites

I don't think the 100 word rule will be hard anymore. Nonetheless, it is highly encouraged.

 

I agree that porting things is particularly thorny. It would be nice just to start over, but such is life. I do think we can effectively "translate" old reviews over to the new system. It won't be perfect, but it will probably do. The limiting factor is the effort required, especially on the BoE side.

 

I agree with Ephesos. Let's do our best to make something robust and good for the long term. That's why I'm advocating simple so people a five years from now have an easy system. As much as I hate UBB at times, this is probably the most stable place we are going to have for this. I don't envision these forums folding until Spiderweb Software does.

Link to post
Share on other sites
Originally Posted By: *i
Looks good. I would suggest trying to add that reviews should try to shoot for being 100 words if possible. This gives reviewers an artificial bar about what the helpful length is.
Or you could just say "write at least one good-sized paragraph" if you don't want to give a minimum word count. That still keeps the guideline without assigning a precise number to it.
Link to post
Share on other sites

Avoiding rankings would do much to alleviate my original problems with CSR. I used it exactly as Thuryl described, and it led me nowhere. It was only when I started ignoring the numbers and clicking randomly through the reviews and ignoring certain people's reviews that I found anything, but it was so slap-dash and haphazard that I found myself getting frustrated and stopped.

Link to post
Share on other sites

Kel, I'm sorry, but I really don't know what we can do to make you happy unless we nuke it all from orbit and make everyone rewrite every review they've ever written.

 

And I would also posit that some of your complaints have a lot more to do with small sample sizes.

Link to post
Share on other sites
Originally Posted By: Kelandon
Avoiding rankings would do much to alleviate my original problems with CSR. I used it exactly as Thuryl described, and it led me nowhere. It was only when I started ignoring the numbers and clicking randomly through the reviews and ignoring certain people's reviews that I found anything, but it was so slap-dash and haphazard that I found myself getting frustrated and stopped.


This is, I suspect, the same problem that film critics face when reviewing films: they care a lot more about cinematography, innovation and all of that technical froofery than your typical cinemagoer, and they rate accordingly. On average, scenario designers (or at least, people involved in the design community) value different things in a scenario than ordinary players do, and scenario designers are also the people most likely to post lots of reviews. If the sort of people who write reviews simply don't agree with the sort of people who read them when it comes to what makes a good scenario, no rating system will solve that problem.
Link to post
Share on other sites

Seconded.

Speaking as a very ordinary player, I found the reviews largely unhelpful, since I have to wade through a lot of opinions I might or might not agree with. All I really want to know before I start a scenario is:

 

1. Does it work?

2. Is it a party-building, quest-and-reward type of scenario, or some kind of arty one-off?

 

Technical innovation and artistic merit are all very well, but they don't necessarily make an enjoyable scenario.

Link to post
Share on other sites

Eph: In case you hadn't noticed, I was endorsing part of Stareye's plan. Not that hard to make me happy. tongue

 

Turtle's point should be noted, as well. Rubrics suck, but something along the lines of ADoS's breaking out different categories might actually help a lot in addressing that players are looking for different things.

 

Then again, maybe what I'm really looking for is having a few of the bigger, HTML reviews that (I think) pre-dated CSR, in addition to the CSR reviews. Not sure. Eh. Not that important.

 

Another thing that I've regularly found frustrating, mostly as a designer but to some extent as a player as well, is that people tend to be rating the Beta or the 1.0.0 version. I've never left scenarios at 1.0.0; the HLPM has been through upwards of 30 versions at this point, and Bahs, LP, and Exodus were all dramatically modified in response to player comments. I don't know if there's anything we can do to address this, except maybe adding a rule that people need to indicate which version it is that they're rating, but I thought I might bring it up anyway in case someone else has a better idea.

Link to post
Share on other sites

I agree with Kelandon from the players perspective on the Beta reviews. It's very aggravating reading what the Beta was, and not the final product, which is what I will be playing. I don't want a *rule* to be in place, but I don't think many people will object to requesting common sense.

Link to post
Share on other sites
Originally Posted By: Kelandon
Then again, maybe what I'm really looking for is having a few of the bigger, HTML reviews that (I think) pre-dated CSR, in addition to the CSR reviews. Not sure. Eh. Not that important.


I don't think this would be a bad thing, but keep in mind that only a few scenarios ever had those reviews, and most that had them only had one.
Link to post
Share on other sites
Originally Posted By: Celtic Minstrel
Yes, we should probably require that you specify which version of the scenario is being reviewed.


...Um. I posit that this is unnecessary. A gentle reminder post from the author or a moderator that reads "please download the newest version" should suffice.

And I'm really starting to like the thumbs up/down idea more. Porting could be as simple as 5.0 and above is a thumbs up, below is down. Maybe leave a "meh" category for middle-of-the-road ratings.

I had kind of a weird reaction when I realized that I (quite recently) rated A Visit to the Madhouse higher than Valley of Dying Things. I mean, I really enjoyed both, and both are quite well-made. So that difference in the scores started to feel trivial.

...ah, forget it. I'm going to go back to actually working on a scenario now.
Link to post
Share on other sites
Originally Posted By: Ephesos
Originally Posted By: Celtic Minstrel
Yes, we should probably require that you specify which version of the scenario is being reviewed.


...Um. I posit that this is unnecessary. A gentle reminder post from the author or a moderator that reads "please download the newest version" should suffice.

Huh? How does reminding people that they should get the latest version help anything? After all, if I'm a player, and I'm reading a review from 8/12/09 but today happens to be 8/12/10, I may have the latest version but still can't tell for the life of me whether the review is for the version I've got or not unless the review specifies.
Link to post
Share on other sites

I s'pose that would be fine.

 

Though, as Eph has pointed out, most people DON'T pay attention to the version numbers and as a result keep old versions around. (I'm recalling a discussion we had that went something like, "What? You mean there's a version of the HLPM after v1.1?") So this wouldn't necessarily be a perfect fix, but I'm not sure that it's a big enough concern that we need a perfect fix.

Link to post
Share on other sites

All right. Let's move ahead with this because if we don't need anything nothing is going to happen.

 

Version numbers should be included and some post should be made by the moderator or author to inform people that the scenario has been updated.

 

We can create some categories as far as genre. I think we should leave it to the author to assign them. If reviewers disagree, they may direct complaints within the reviews and adjust scores accordingly.

 

I'm think a 1 to 5 integer scoring system is probably the best we can do. I do like thumbs up and thumbs down, but I do like having something that distinguishes between good and the best.

 

I'm looking to start the pilot program of this on a few scenarios this weekend. Please offer any ideas.

Link to post
Share on other sites

I have a thread for APF, one of my scenarios from half a decade ago. Please give some reviews to test this.

 

For those who asked me to use your scenario, thanks! Could you please get me:

 

Difficulty

Version number

Keywords

 

See my APF post for help coming up with these.

Link to post
Share on other sites
Originally Posted By: *i
For those who asked me to use your scenario, thanks! Could you please get me:

Difficulty
Version number
Keywords

I'm guessing Difficulty refers to the level recommendation, not the actual difficulty of the scenario. Either way:

Difficulty - Beginner
Version Number - 1.0.3
Keywords - Linear, Combat Heavy, Avernum Universe, Short
Link to post
Share on other sites

If you're too impatient to read a long review, then don't. Nobody's forcing you to do so. Your loss if the review says something insightful that would determine whether you'd play the scenario. "tl,dr" isn't a helpful comment and you know it.

 

Don't be discouraged, Nioca.

 

I'd volunteer my scenarios, but they're only for BoE. When you create the forum, though, I'll give the details.

Link to post
Share on other sites

If someone is commenting that a review is too long, aren't they saying it wasn't helpful? I thought Diki was giving open and honest feedback on a process which was in flux. Therefore, I don't really understand the compulsion to berate her for at least contributing something. My understanding is that this change is being designed to help a player find something which they would like playing. Shouldn't the requirements of reviews reflect that goal? I'll withhold judgment for the time being.

Link to post
Share on other sites

I should clarify that it's not the review's length in and of itself that's the problem. If the review was long but all of it was interesting, I wouldn't have said anything. But the review is only long because it's repetitive. The summary, the notable points, and the rest of it all cover basically the same thing, so everything gets said at least three times. That makes for a pretty boring review, especially since it's unlikely that the review will be radically different from the other reviews.

 

In short, make sure nothing is getting said over and over again and the review will be a lot more fun to read even if it is still long. (Well, I'll enjoy reading it a lot more, anyway.)

 

---

 

Originally Posted By: Salmon
Therefore, I don't really understand the compulsion to berate em for at least contributing something.

In all fairness, most of what I contribute is probably complaints. tongue

 

Also, Dikiyoba fixed your typo.

Link to post
Share on other sites

Some people find longer, in depth comments more helpful. Others don't. We're not going to find two players alike and it's impossible to please everyone. Rather, if you don't like the way a review is done, make your own. That way everybody wins. smile

 

One thing I do know to be counterproductive is criticizing people who are just trying to help. Sure, offer suggestions on how things can be improved. That's fine. However, it might be best to do so privately.

Link to post
Share on other sites

More scenarios are always welcome! smile

 

Just need the scenario name, recommended levels, version, and some keywords. Consult the thread in the Rankings subforum for assistance there.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...