Jump to content


In which I abuse my access to the database for the purposes of statistics

Forum Stats Graph!

  • Please log in to reply
88 replies to this topic
sylae sylae

Wisebottom

  • Global Moderator
    • Member ID: 5,482
  • 3,336 posts
  • LocationPortland, Oregon, United States

Posted 22 October 2012 - 11:02 PM #1 In which I abuse my access to the database for the purposes of statistics

What follows is a graph of the forum-wide daily postcount (posts per day) in the form of a 20-period moving average. Note that this graph does not take into account any posts that have been purged or otherwise removed.

Posted Image
In addition, here is a version with some of the game release dates noted, so as to correspond with the spikes (these release dates could be off, they're just based off a quick forum search). Also. here's a graph of the raw data, with the 20-perion MA in black

Also: The 20 most postalicious days in SW history:
Spoiler

Edited by Sylae, 22 October 2012 - 11:18 PM.
Further graphs of other stuff may or may not be revealed at a distant point in the future


Student of Trinity Student of Trinity

Interloper

  • Member
    • Member ID: 2,306
  • 6,622 posts
  • Locationthe big Hilbert space

Posted 23 October 2012 - 02:51 AM #2 In which I abuse my access to the database for the purposes of statistics

What happened in early 2004?
As long as entropy is doing what we want, we take it for granted, but as soon as it starts killing us, we go all whiny.

Randomizer Randomizer

Tormented Battle Mage

  • Global Moderator
    • Member ID: 4,341
  • 15,914 posts

Posted 23 October 2012 - 03:30 AM #3 In which I abuse my access to the database for the purposes of statistics

The period around 2006-7 was when Aran posted monthly numbers for posts by members with a top 10 list.  After a particular slow summer there were the spam games.
"I don't know how we can ever repay you!" - Rescued damsel in distress
"Oh, I'm sure we can think of something .... How much gold do you have?" - "Hero"

"We're all screwed if Cthulhu shows up. Try to enjoy life in the meantime." - Adam Rawnsley

Tyranicus Tyranicus

Dollop of Whipped Cream

  • Member
    • Member ID: 4,151
  • 4,537 posts
  • LocationShadowvale

Posted 23 October 2012 - 04:30 AM #4 In which I abuse my access to the database for the purposes of statistics

View PostStudent of Trinity, on 23 October 2012 - 02:51 AM, said:

What happened in early 2004?
I believe there was a forum purge in early 2004.  A lot of posts before that point were deleted.
"There are less fortunate people out there who can't sit in a chatroom and complain about privilege. You should check your privilege-checking privilege." —Dintiradan
BoA Scenario DatabaseSpiderweb Music DatabaseNethergate Graphics RewindAvernum Side Quest ListExile & Avernum for Intel Macs

Actaeon Actaeon

Easy Street

  • Member
    • Member ID: 3,558
  • 2,011 posts
  • LocationOut West

Posted 23 October 2012 - 05:38 AM #5 In which I abuse my access to the database for the purposes of statistics

This is lovely, Sylae! I am somewhat surprised I didn't see something of the sort from Slarty, though perhaps I just missed it.
Plaudite, amici, comedia finita est.

On Being a Conflict On Being a Conflict

Alorael

  • Administrator
    • Member ID: 276
  • 22,118 posts
  • LocationElsewhere

Posted 23 October 2012 - 05:55 AM #6 In which I abuse my access to the database for the purposes of statistics

This sort of data mining is far easier with IPB than it was with UBB.

—Alorael, who's sure it was possible with UBB. It just wasn't remotely convenient. In any case Aran would have been the statistician. Slarty is much more of a narrative historian.

Actaeon Actaeon

Easy Street

  • Member
    • Member ID: 3,558
  • 2,011 posts
  • LocationOut West

Posted 23 October 2012 - 06:25 AM #7 In which I abuse my access to the database for the purposes of statistics

In the long run, yes. But I haven't seen Aran post anything of the sort for a good while, while Slarty got quite in depth with his last big poll. In any case, I think it's safe to say that Sylae will be carrying that mantle for a while. That's fine; she wears it well.
Plaudite, amici, comedia finita est.

Triumph Triumph

Wrapped in Light

  • Global Moderator
    • Member ID: 6,248
  • 3,015 posts

Posted 23 October 2012 - 07:20 AM #8 In which I abuse my access to the database for the purposes of statistics

What was happening on 8/4 and 8/11 of 2011?  Were those Slarty's nifty PDN-based puzzle/trivia games?

"Dikiyoba didn't know what to say for your negatives, Triumph. Dikiyoba supposes you must just be that awesome." - Dikiyoba
"Triumph == Santa Claus? This explains... nothing." - Dintiradan
"Per usual, Triumph said it better." - Slarty


Tyranicus Tyranicus

Dollop of Whipped Cream

  • Member
    • Member ID: 4,151
  • 4,537 posts
  • LocationShadowvale

Posted 23 October 2012 - 07:46 AM #9 In which I abuse my access to the database for the purposes of statistics

View PostTriumph, on 23 October 2012 - 07:20 AM, said:

What was happening on 8/4 and 8/11 of 2011?  Were those Slarty's nifty PDN-based puzzle/trivia games?
That was when I ported BoA scenario reviews from the CSR creating hundreds of new posts in the process.
"There are less fortunate people out there who can't sit in a chatroom and complain about privilege. You should check your privilege-checking privilege." —Dintiradan
BoA Scenario DatabaseSpiderweb Music DatabaseNethergate Graphics RewindAvernum Side Quest ListExile & Avernum for Intel Macs

sylae sylae

Wisebottom

  • Global Moderator
    • Member ID: 5,482
  • 3,336 posts
  • LocationPortland, Oregon, United States

Posted 23 October 2012 - 08:22 AM #10 In which I abuse my access to the database for the purposes of statistics

View PostMomentous Inertia, on 23 October 2012 - 05:55 AM, said:

This sort of data mining is far easier with IPB than it was with UBB.

—Alorael, who's sure it was possible with UBB. It just wasn't remotely convenient. In any case Aran would have been the statistician. Slarty is much more of a narrative historian.
It's about the same (I'm just using the SQL directly, it's a lot easier than the builtin stats tools), and might even be a touch easier in UBB, since the database schema appears to be a lot simpler than with IPB.

The top ten posters if board purges were taken into account:
Spoiler


The Loquacious Lord Grimm The Loquacious Lord Grimm

Deranged Subterfuge and Key Lime Pie

  • Member
    • Member ID: 4,290
  • 1,003 posts
  • LocationA top-secret bunker hidden somewhere in New Jersey.

Posted 23 October 2012 - 08:30 AM #11 In which I abuse my access to the database for the purposes of statistics

.

View PostRandomizer, on 23 October 2012 - 03:30 AM, said:

The period around 2006-7 was when Aran posted monthly numbers for posts by members with a top 10 list.  After a particular slow summer there were the spam games.
Alas, the great Spamfest strikes only the fourth-highest peak. [/nostalgia]

I can't remember offhand, when did Spiderweb start on Steam? Late 2010/early 2011? Would explain the rise in the base posting quantity shortly thereafter.


The Silent Assassin wants you all to know that the Great War against All Things Puce is proceeding on schedule. Be very sure which side you're on.
-Lenar Labs
What's your destiny?

Soon.

Also, fanfic.

sylae sylae

Wisebottom

  • Global Moderator
    • Member ID: 5,482
  • 3,336 posts
  • LocationPortland, Oregon, United States

Posted 23 October 2012 - 10:05 AM #12 In which I abuse my access to the database for the purposes of statistics

View PostLenar Labs, on 23 October 2012 - 08:30 AM, said:

I can't remember offhand, when did Spiderweb start on Steam? Late 2010/early 2011? Would explain the rise in the base posting quantity shortly thereafter.
Avadon got on Steam 7/2011, I believe.

Number of posts made by the top spammer each day

Percentage of the day's posts that were by top spammer - Early data is not shown, as the only remaining posts from that period are by SW and thus a wall of purple is all that's there.

Actaeon Actaeon

Easy Street

  • Member
    • Member ID: 3,558
  • 2,011 posts
  • LocationOut West

Posted 23 October 2012 - 10:20 AM #13 In which I abuse my access to the database for the purposes of statistics

... How does one exceed 50%? I mean, who would you be talking to?
Plaudite, amici, comedia finita est.

Tyranicus Tyranicus

Dollop of Whipped Cream

  • Member
    • Member ID: 4,151
  • 4,537 posts
  • LocationShadowvale

Posted 23 October 2012 - 10:38 AM #14 In which I abuse my access to the database for the purposes of statistics

View PostLenar Labs, on 23 October 2012 - 08:30 AM, said:

Alas, the great Spamfest strikes only the fourth-highest peak. [/nostalgia]
General was pruned in 2009, so many of the posts from that era are not part of the equation.
"There are less fortunate people out there who can't sit in a chatroom and complain about privilege. You should check your privilege-checking privilege." —Dintiradan
BoA Scenario DatabaseSpiderweb Music DatabaseNethergate Graphics RewindAvernum Side Quest ListExile & Avernum for Intel Macs

Callie Callie

SuperCalliefragilisticexpialidocious

  • Member
    • Member ID: 4,904
  • 4,168 posts
  • LocationNevada, USA

Posted 23 October 2012 - 11:05 AM #15 In which I abuse my access to the database for the purposes of statistics

View PostActaeon, on 23 October 2012 - 10:20 AM, said:

... How does one exceed 50%? I mean, who would you be talking to?
When Iffy first joined practically every other post was by him.
"I'm so glad you mentioned [goat heads] were weeds. I had never heard of them before, and thought maybe Excalibur lived in an area strewn with the heads of slaughtered goats. : p " - Tyranicus
Restless Souls - My first BoA scenario

Dikiyoba Dikiyoba

Heart of Avadon

  • Member
    • Member ID: 4,222
  • 9,001 posts

Posted 23 October 2012 - 11:18 AM #16 In which I abuse my access to the database for the purposes of statistics

Yes, but even every other post would be only 50%. The graph is sometimes showing 70% of the posts were by one person, so something is very wrong with the graph. (Alternately, Sylae was somehow able to recover all of Alorael's deleted posts in the Mod Board. :p )

Dikiyoba.

The Almighty Doer of Stuff The Almighty Doer of Stuff

Heart of Avadon

  • Member
    • Member ID: 72
  • 4,415 posts
  • LocationMassachusetts

Posted 23 October 2012 - 12:58 PM #17 In which I abuse my access to the database for the purposes of statistics

It's easy. There are very few posts on a particular day, and one person posts in numerous threads. This doesn't have to be a single back-and-forth conversation.

saunders saunders

Shadowwalker

  • Moderator
    • Member ID: 3,570
  • 465 posts
  • LocationPruned

Posted 23 October 2012 - 01:15 PM #18 In which I abuse my access to the database for the purposes of statistics

There were days when Drakefyre managed to have the last post in every visible thread.
Damn, I posted again.
Move along, there is nothing here to see.
Deep down, you know you should have voted for Alcritas!

Arancaytar Arancaytar

Archivist

  • Global Moderator
    • Member ID: 2,001
  • 10,474 posts
  • LocationThe Empire of Ermarian

Posted 23 October 2012 - 01:44 PM #19 In which I abuse my access to the database for the purposes of statistics

My successor has shown to be a worthy pony indeed. :p

Might be interesting to see the stats split by General / Game fora, to identify the impact of game releases more clearly.

Side note: I'm unable to back up the post table due to memory issues. I really hope that doesn't mean I'll have to rewrite piperbot again...

sylae sylae

Wisebottom

  • Global Moderator
    • Member ID: 5,482
  • 3,336 posts
  • LocationPortland, Oregon, United States

Posted 23 October 2012 - 03:32 PM #20 In which I abuse my access to the database for the purposes of statistics

View PostAʀᴀɴ, on 23 October 2012 - 01:44 PM, said:

Side note: I'm unable to back up the post table due to memory issues. I really hope that doesn't mean I'll have to rewrite piperbot again...
I haven't tried selecting the post body yet, but tossing in a limit 0,50000 and so on helps to make getting data easier. Of course, then you have to deal with the HTML table...thing...and it may or may not screw with the output, I haven't checked very much into it :p

Edited by Sylae, 23 October 2012 - 03:33 PM.
Also I saw a mention or two of an XML interface to the forums. A rewrite might be rather easy...


Randomizer Randomizer

Tormented Battle Mage

  • Global Moderator
    • Member ID: 4,341
  • 15,914 posts

Posted 23 October 2012 - 03:41 PM #21 In which I abuse my access to the database for the purposes of statistics

Iffy made over 50 posts on his first day to get a new title.  :(
"I don't know how we can ever repay you!" - Rescued damsel in distress
"Oh, I'm sure we can think of something .... How much gold do you have?" - "Hero"

"We're all screwed if Cthulhu shows up. Try to enjoy life in the meantime." - Adam Rawnsley

On Being a Conflict On Being a Conflict

Alorael

  • Administrator
    • Member ID: 276
  • 22,118 posts
  • LocationElsewhere

Posted 23 October 2012 - 04:24 PM #22 In which I abuse my access to the database for the purposes of statistics

The easiest way to have a huge percentage of posts is to do something like reply to or start a topic, have someone else respond, respond to the response, and then have no further conversation. That right there gets you to 67%. A few posts with no responses (100% of the activity in the thread) can hit 70%. Not that that's happened; I think missing posts are a much more plausible explanation. Maybe those 70% are all that remains unpurged.

—Alorael, who still fondly remembers working with Arctic Wolf to provide any possible response to all of Morgan's posts so he could be the second to hit 1000 on the UBB long, long ago. And there weren't even titles for it back then.

Arancaytar Arancaytar

Archivist

  • Global Moderator
    • Member ID: 2,001
  • 10,474 posts
  • LocationThe Empire of Ermarian

Posted 23 October 2012 - 08:37 PM #23 In which I abuse my access to the database for the purposes of statistics

Quote

XML interface

*nerdgasm*

Iffy Iffy

The Dough is Rising

  • Member
    • Member ID: 4,897
  • 2,629 posts
  • Locationinside my comfortable box

Posted 26 October 2012 - 03:52 PM #24 In which I abuse my access to the database for the purposes of statistics

View PostRandomizer, on 23 October 2012 - 03:41 PM, said:

Iffy made over 50 posts on his first day to get a new title.  :(
I'm fairly certain the number was 30.
(19:01:38) Tyran: There is a time and a place for unfunny idiocy?
(19:01:56) nikki: Yes, Tyran: http://spiderwebforu...orum/3-general/

sylae sylae

Wisebottom

  • Global Moderator
    • Member ID: 5,482
  • 3,336 posts
  • LocationPortland, Oregon, United States

Posted 27 October 2012 - 10:19 PM #25 In which I abuse my access to the database for the purposes of statistics

Posted Image
all posts by hour.

BONUS: SW versus CalRef

Edited by Sylae, 27 October 2012 - 10:27 PM.
I also considered doing it versus the calref chat messages, but that's only a month or two of data since it purges itself


Arancaytar Arancaytar

Archivist

  • Global Moderator
    • Member ID: 2,001
  • 10,474 posts
  • LocationThe Empire of Ermarian

Posted 28 October 2012 - 02:44 AM #26 In which I abuse my access to the database for the purposes of statistics

I'm surprised at the CalRef graph having three modes. I've seen many 24h activity curves, and it seems that with sufficient data they all settle down into a single global maximum. Though a small community with a few very active members could probably get a curve that still reflected those members' individual curves.

The Loquacious Lord Grimm The Loquacious Lord Grimm

Deranged Subterfuge and Key Lime Pie

  • Member
    • Member ID: 4,290
  • 1,003 posts
  • LocationA top-secret bunker hidden somewhere in New Jersey.

Posted 28 October 2012 - 09:03 AM #27 In which I abuse my access to the database for the purposes of statistics

View PostSylae, on 27 October 2012 - 10:19 PM, said:

all posts by hour.
Dumb question: what time zone?


The Silent Assassin's Explosive Kittens of Doom have been put into a temporary production delay due to the anticipation of Hurricane Sandy. Most of you who ordered any should have already received notification and a complementary pie, the rest of you now know, and will be further warned, if you receive a package labeled "Cherry Pie", please send it back immediately without opening it. That package was mislabeled and actually contains some of the explosive components.
-Lenar Labs
What's your destiny?

Soon.

Also, fanfic.

Actaeon Actaeon

Easy Street

  • Member
    • Member ID: 3,558
  • 2,011 posts
  • LocationOut West

Posted 28 October 2012 - 09:11 AM #28 In which I abuse my access to the database for the purposes of statistics

Mountain Daylight Time. It's at the bottom.
Plaudite, amici, comedia finita est.

sylae sylae

Wisebottom

  • Global Moderator
    • Member ID: 5,482
  • 3,336 posts
  • LocationPortland, Oregon, United States

Posted 28 October 2012 - 09:13 AM #29 In which I abuse my access to the database for the purposes of statistics

Yeah, plus the memberbase has changed a bit since 2008, as well as the times when people are active. I mean one of those spikes has got to be due to Vergil, and he hasn't forumed in a while. Really, the forums themselves don't get much love since everyone is in the chat.

Tyranicus Tyranicus

Dollop of Whipped Cream

  • Member
    • Member ID: 4,151
  • 4,537 posts
  • LocationShadowvale

Posted 28 October 2012 - 09:16 AM #30 In which I abuse my access to the database for the purposes of statistics

View PostLenar Labs, on 28 October 2012 - 09:03 AM, said:


Dumb question: what time zone?
It says MDT at the bottom.  This stands for Mountain Daylight time or GMT-6 (EDT-2).
"There are less fortunate people out there who can't sit in a chatroom and complain about privilege. You should check your privilege-checking privilege." —Dintiradan
BoA Scenario DatabaseSpiderweb Music DatabaseNethergate Graphics RewindAvernum Side Quest ListExile & Avernum for Intel Macs

sylae sylae

Wisebottom

  • Global Moderator
    • Member ID: 5,482
  • 3,336 posts
  • LocationPortland, Oregon, United States

Posted 28 October 2012 - 09:21 AM #31 In which I abuse my access to the database for the purposes of statistics

Which is changing on November 4th at 2:00 AM! Don't forget to update the one or two devices you have remaining that don't automatically do this!

Edited by Sylae, 28 October 2012 - 09:23 AM.
my high school's bell system was from the 70s and couldn't handle daylight savings. they had to manually change all the bell times. it meant every spring and fall there was a week-long period where the bell system was so ridiculously off.


Arancaytar Arancaytar

Archivist

  • Global Moderator
    • Member ID: 2,001
  • 10,474 posts
  • LocationThe Empire of Ermarian

Posted 28 October 2012 - 09:24 AM #32 In which I abuse my access to the database for the purposes of statistics

View PostSylae, on 28 October 2012 - 09:13 AM, said:

Yeah, plus the memberbase has changed a bit since 2008, as well as the times when people are active. I mean one of those spikes has got to be due to Vergil, and he hasn't forumed in a while. Really, the forums themselves don't get much love since everyone is in the chat.

Ever since SV stagnated and it was CalRef's turn to become the foremost (and now only) satellite, I've felt a little twinge that I didn't get in on it earlier. Now there's all this history and I'd feel like an outsider. :p

The Loquacious Lord Grimm The Loquacious Lord Grimm

Deranged Subterfuge and Key Lime Pie

  • Member
    • Member ID: 4,290
  • 1,003 posts
  • LocationA top-secret bunker hidden somewhere in New Jersey.

Posted 28 October 2012 - 09:29 AM #33 In which I abuse my access to the database for the purposes of statistics

View PostTyranicus, on 28 October 2012 - 09:16 AM, said:

It says MDT at the bottom.  
I have no idea how I missed that. Shame on me. :(  

The Silent Assassin wants me to remind you that the last hurricane to come through our area got us trapped in the bunker for two weeks after a false alarm and a security mishap. In the event that you need to contact him and he doesn't know, please be patient: he's packing a supply of plastic spoons this time, just to be safe.
-Lenar Labs
What's your destiny?

Soon.

Also, fanfic.

Karan S'jet Karan S'jet

seiðmaðr

  • Member
    • Member ID: 3,440
  • 4,255 posts
  • LocationInvisible Cities

Posted 28 October 2012 - 09:34 AM #34 In which I abuse my access to the database for the purposes of statistics

View PostAʀᴀɴ, on 28 October 2012 - 09:24 AM, said:

Ever since SV stagnated and it was CalRef's turn to become the foremost (and now only) satellite, I've felt a little twinge that I didn't get in on it earlier. Now there's all this history and I'd feel like an outsider. :p

Welcome to how Refugi felt about SW. :p
The mutilation of the self is the only path towards real personal growth.
WHAT DO YOU SEE?
I MUST KNOW WHAT YOU SEE
TELL ME. . .
WHAT AM I?
I CANNOT SEE

sylae sylae

Wisebottom

  • Global Moderator
    • Member ID: 5,482
  • 3,336 posts
  • LocationPortland, Oregon, United States

Posted 28 October 2012 - 09:42 AM #35 In which I abuse my access to the database for the purposes of statistics

View PostAʀᴀɴ, on 28 October 2012 - 09:24 AM, said:

Ever since SV stagnated and it was CalRef's turn to become the foremost (and now only) satellite, I've felt a little twinge that I didn't get in on it earlier. Now there's all this history and I'd feel like an outsider. :p
I would like to respectfully disagree on the feel like an outsider part. Plenty of folk have no idea of the awesomeness of PS, or the delicate political subtones of *whip* *meow*, or had to put up with Vergil being Vergil, and yet they are just as much Refugi as the old guard.

But don't let me preach to you about it, stop by the chat and see for yourself :p

Edited by Sylae, 28 October 2012 - 09:47 AM.
if it makes your inner nerd feel better, i've been working on a XMPP <--> CalRef chat bridge. sockets are fun!





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users