New Scaratings

How good is a particular trivia site?

There are many answers to that question, all, to some degree, correct. Here are three:

1) A trivia site cannot be described by statistics; while average scores and medal count matter, they're imperfect metrics. A trivia site is its people, including the players, the servers, the other employees, and the other patrons. The quality of the people, in the context of the physical plant, is the quality of the site. Reviews are the best metric.

2) A trivia site is as good as its reputation in Premium Games, especially Showdown. Once a day you get a chance to show your stuff without repeat questions; once a week you get to show your stuff with most of the best trivia players in the nation playing simultaneously. Good scores in games full of repeat questions mean jack. Good scores in Premium Games define excellence.

3) Buzztime provides site rankings. Read them. They're an excellent metric, the very best we have.

What I was hoping to find was a little different, though. I wanted a metric that would tell me, if I walked into a site where there were Buzztime players and I joined them, what would the caliber of play be?

Reviews are good for gauging the fun of a site, but they're irregular at assessing how good the quality of play is. The crowds that arrive at sites for Showdown are not necessarily reflective of the crowd that most often plays at those sites. Candidly, Showdown and other Premium Games favor densely populated areas that can attract excellent teams across sites and across good players' differing personal networks. Those great Showdown teams aren't usually the guys and gals you meet if you hit the bar four hours earlier or later. Lastly, the Buzztime metrics reflect both quality of play and frequency of play: a horde of monkeys with Playmakers trained to hit a number when a question pops up will rack up a few Lunchtime Trivia and Buzztime Trivia perfect scores, and a horde of dim-witted permanent bar denizens with over 25 million P+Points each might even do better than the monkeys. The Buzztime metrics show six-month aggregation of gold medals. They don't reveal true quality of play.

So I scratched my head and I said to myself, "WB, what really defines excellence at Buzztime?" After much contemplation, I came up with two different ideas:

1) Gold Medals Are Not The Best Metric. The Sum Of Silver And Bronze Medals Is Better.

That's counterintuitive, but I'm convinced that, at a site level, it's true. Here are two big reasons why:

- It's tough to get a Silver or Bronze Medal by cheating. It's easy to get several Gold Medals; it's kinda tough to collect second and third place by cheating.

- Easy games don't have Silver and Bronze Medals. Easy games have three or more perfect score Gold Medals. Tough games have Silver and Bronze Medals.

So let's look at the top sites when there ranked by aggregate total of Silver and Bronze Medals:

307 86th Street Pub
210 DeMoris
144 Bargos
125 Danny Ks Cafe and Billiards
119 Cadillac Jacks
116 Buffalo Wild Wings Ironton
110 Steel Pit Sports Grill
106 Heroes Wichita
99 The Concert Pub
96 East Side Marios Lakewood

Hmmm...no big surprises. Three West Coast sites move up significantly...it seems that there's greater opportunity for Silver Medals and Bronze Medals in the games following the Premium Games than in the games before the Premium Games, but Danny Ks, Steel Pit, and East Side Marios Lakewood are solid sites, nonetheless. BWW Ironton takes a bit of a drop; Diamond Strike and TGI Fridays Dayton, both Lunchtime Trivia sites, take a drop; Mitch's Corner, barely off the list, takes a big drop.

But I was looking for a different metric. I don't want to know aggregate medals; I want to know average quality of play by site, if anybody happens to be playing. So I came up with this:

2) The Quotient Of Site Total Score Divided By The Sum Of Silver And Bronze Medals Gives The Best Metric Of Average Player Quality For A Site.

Think about it...it's a metric that discounts medals earned by cheating, and it also discounts medals earned simply by having several boxes in play for every game, just in case. It's a good metric of average quality of play by site.

Looking at the top 200 sites only, which rise to the top?

Here's my list:

219,935 Bob's Place
607,161 Buffalo Wild Wings Dayton
634,844 Mezzanine Lounge
700,823 86th Street Pub
731,609 Indigo Joes Wildwood
733,046 DeMoris
769,627 Steel Pit Sports Grill
796,420 Buffalo Wild Wings Bridgeport
881,062 East Side Marios Lakewood
894,519 Linksters Taproom Sarasota
937,329 Buffalo Wild Wings Albuquerque 2
961,655 Cadillac Jacks
986,661 Danny Ks Cafe and Billiards
1,012,219 McKennas Seafood Inc
1,044,179 The Concert Pub
1,058,488 Elsas Cantina
1,104,560 Mr Bs Restaurant Sports Lounge
1,150,388 Bargos

I took the list down to Bargos, one of my favorite sites. Bargos is still a top fifteen site, but it's not number three any more: Bargos has lots of friendly players who are good, but not a truly elite crew. It's just that they play a lot...walk into Bargo's without warning and you'll find a very competent crowd of really, really great people. But it seems they're not quite as good as the players at the top sites.

Four sites were colored red above, indicating my concern with their site ranking. Bob's Place is an outlier because of the way that they play. This has been discussed before; the stars are great people and very good players, but the way that they play invalidates their outlier by this metric. The other three red sites' scores reflect the scores of players registered there who barely ever play there, with enough medals in each case to distort the results.

A few things stand out:

1) Mezzanine Lounge nudges out 86th Street Pub for first place. Wow...I love 86th Street and I've never visited Mezzanine Lounge, but the reputation of the Mezzanine Lounge suggests that this might not be a fluke.

2) DeMoris comes very close to 86th Street Pub for third place.

3) Steel Pit rises near the top...kudos.

4) TIEFLY at BWW Bridgeport emerges as a force with whom to be reckoned. For those of us who know him, this makes sense.

5) I double-checked the stats of BWW Albuquerque II, a little-known site for those of us out here in the East. Wow...these guys have trouble getting Gold Medals, but their averages are VERY competitive. Unless I'm missing something, this ranking isn't a fluke...again, WOW!

***

This isn't the be-all and end-all of metrics. I just wanted to try to assess average quality of play. I hope that you all found the post trivia-related and interesting.

Not a terrible way to look at it but does have a glaring problem in that it discounts both golds earned without cheating and silvers/bronzes earned with cheating. I do like the idea of a medal efficiency metric compared to overall score, and agree there is no easy way to smooth out either cheating or multiboxing (no matter how capable a single player or even team may be, they are not five times more capable simply because they care enough and have fast enough fingers to play five boxes per player) and it's not like I have any better answer, but I don't think you can flat out exclude golds, especially in the era of 10Q games being the norm where perfection is less elite. I get clean 15k CD perfects a couple times a year - 3 if I am lucky. I average 4-5 or so a month 10k perfects, and the difference in game count is not that great. It's just not all THAT hard for a good soloist to ace a BT, let alone a good team bar. Cheaters too do not simply abuse gold counts. In Lexitopia for example, the suspiciously inflated scores rarely reach the 10k which is almost always required for gold, but hover in mid to high 9s where silver and bronze sometimes reside. Not sure if that's the lag between bars rarely being a full question, or Lex's scoring algorithm, or laptop/data entry speed for Borg cheats rather than lag cheats. Lag cheaters do better on non-timed questions such as pyramid and final SD, or quick-fire games like Speed Freaks where getting a question or more ahead is likely, and only the very smoothest Borger can do it fast enough to ace a 10Q timing 10 times in a row.

_________________
Please forgive any strange typos or grammar errors. I am typically using voice recognition software to enter text, and sometimes editing works differently from how I expect.

You could include gold medals that had only a solo winner in BT, or golds that had less than 5 in CD, and I'd think those would primarily be representative of "directionally correct" scores.

Thanks to the get-togethers and such, I've been lucky to play with some of the best. Lots of good players out there.

But many don't play in a team mannner. DTHDLR can get on a good solo roll, and there is the beast, MEGUMI. I have played with both, and would do it again any time. However, Jim doesn't play the 5-6 boxes, while Bill does. Does it matter? I guess it could where Bill, due to a box or two issue, could finish 1, 2, & 3 (& 4 & 5...). There, the silver and bronze medals would figure into the count. Same with ERIC and his plethora of handles.

I know that tiefly/BOTFLY have finshed 2nd & 3rd in CD games, so both get medals for the same game. This can skewer the medal count/site quality somewhat, I would guess.

A recent site that LUREED/NICO have been playing at, Breaking Traditions, has really moved up the medal count. Why? Because those two have finally found a home site where Donna doesn't have to drive 50 minutes away...each way, to play. They have found a local site where they have gotten a few people involved and played very well...and often. Makes me envious enough that I'm planning a road trip down there. Only thing keeping them from more golds is me not being down there to answer the fish and fishing questions for Steve!!!

As Dave has found out, now that he is turning to the dark side (Team play), it only takes teaming with one person who has sacrificed their time to watching 'reality tv' to know the winner of 2005's Amazing Race and say "4" to get a 10k game instead of the 9k game without the one assist.

My only hope is the off-chance that there is a question regarding one of BOTFLY's many vices.

After all, I think I know where the strong teams are, who the strong players are, who some of the cheats are (but not all, nor all of the borg sites). I know the folks I like playing with...or against. And of the places I've played, I have my favorites. And I have my list of places that I want to play before BT makes all questions for the youngins. And some of the folks with whom that I want to play BT, whether they would tolerate me for long or not.

But as someone who gets lots of 2nds and 3rds, thanks for noticing and the shout out.

BWW Albuquerque isn't a fluke. That's one of 2 alternate sites that the Coaches crew gathers at, especially on weekends. BLZBUB, MENTAT, and a few other very good players.

Yes, Kenny is a good player. He and another NM player (DUFF?) were in Cincy for the Tollofafuckup or whatever it's called.

In fact, Ken was on the old Backyard Grill team in NOVA when I first found out the concept of 'team' play.

Hey there Mr. Fly! I only play on the Dark Side on Tuesdays and Wed Six, plus a very occasional game or two when the Tuesday guys show up other days. Very few BTs etc (as can be seen by my pedestrian, unimproved averages I am ashamed to say). I'm only an Episode II Anakin That said yes it is amazing what even a fairly minor separation of Venn circles will do to scoring range. Both Dragon and I have considerably improved BB averages together while having significant overlap on strengths - and nary a reality show fact between us sadly. The team on Tue has little movie/music strength even though that's a smallish segment for the day. We have a pretty good pure science guy too and some savvy gap-fillers. I think we're missing one other strong all-rounder with a pop bias to make the team quite capable of hanging in there. Still not enough to turn me into Episode III though.

_________________
Please forgive any strange typos or grammar errors. I am typically using voice recognition software to enter text, and sometimes editing works differently from how I expect.

All great points, of course.

Rhino, offer a better way of minimizing inclusion of cheating results than eliminating consideration of gold medals. You're a brilliant man: in a community of very smart women and men, you stand out. You regard my humble method as having a "glaring problem;" what would be better?

Great ideas! How could we data mine that from published results at Buzztime?

TIEFLY, a very common outcome of a tough game is for an excellent team play site to finish 1-2-3. Your boxes' finishing 2-3 on occasion isn't giving you a big break in the greater scheme of things.

Your scores do stand out in this metric. It's just one metric, a rate metric by site that tries to minimize effects of cheating. (Many sites around Mesa, AZ seemed to drop substantially from their Official Buzztime Site Rankings by this metric, for example.) But it puts your site near the top, and, as a guy who plays against you, MEGUMI, PMAN, DTHDLR, and others you mention quite frequently, I see that as deserved.

Of course, that's just MY opinion...

Well, unction notwithstanding, the main problem is that you can'ty really have a valid measure of how well bars play trivia that ignores completely them playing it as well as possible - coming in first. Silvers and bronzes may be more unusual than golds in these days of easily attainable perfects, but golds are still reflective of better play. I don't see how golds are any more tainted by cheating than any other rarefied achievement we could use to rank bars, be that lesser medals or gaudy averages. The exception here may be Appeteasers, where cheating needs nothing more technical and demanding than having a compliant friend in a more easterly time zone - more on that later. Now yes if you want to diminish the influence of the concentrated multiboxing medal-multiplication you do have to apply a ratio as you did. Overall score is a good one. Number of games played may work at least as well. The difference beteween 10000 and 9994 on mulitbox scores has next to no impact on using a score numerator, but an extra medal-less game has a bit more simply because we are dividing lower numbers.

If you really want to demote the impact of multibox golds what you could think of doing is getting a good sample of the ratio between silvers and golds among top bars and then limiting the golds to within, say, a couple of standard deviations away from that "norm" to cap outliers while still recognizing wins.

Personally I am intrigued by the idea of medals per top 100 game. Even the best teams and multiboxers are likely to split on a hedged answer or be a second too slow on box 3 when they had to struggle for the right choice a chunk of the time (86th St gobbles up golds like nobody's business, but even they don't routinely average 10k/15k as a site game after game). Anybody worth measuring is very likely to keep that third box in the top 100 though so they will keep cranking that denominator up, lowering their effective rank. They will get more medals than non-multiboxers of course, but they'll also divide them into a greater number of top 100 games.

Now no method is perfect but if you use games as the denominator, exclude Appeteasers (not only for the easier cheating but for the easier perfect on only 7 Qs plus the lesser competition), cap the ratio of golds/medals to within a limited variance from the central tendency, and apply a correction factor for medal/top 100 to smooth out the multiboxing impact, it should be very useful.

One thought by the way is that multiboxing IS harder and does not deserve to be eliminated from results, merely limited. Spotes is a far better player than I am for example, but I doubt even he would claim he is twice or three times as good because he plays 2 0r 3 (or however many - it varies I believe) boxes - so we do have to limit that. But in a hypothetical game where I and his three boxes all get 10k golds, he has still shown himself to be better by being able to score 3 perfects. To do that you need more immediate recall, to avoid the "oh yeah" last second lightbulb that just catches the 1000 before it counts down where hitting 3 would be harder. You have to be much more certain -no lucky 50/50 correct choices - as multiboxers are likely to split. And of course you can't be distracted from the physical box management. Yes getting three golds IS tougher than getting one - it's just not 3 times tougher.

_________________
Please forgive any strange typos or grammar errors. I am typically using voice recognition software to enter text, and sometimes editing works differently from how I expect.

Heck, we also need to add in some factor for alcohol consumption. If after 6 stiff drinks I manage a gold against Spotes, I have done something that is at least 3 times as hard as playing stone sober.

/Spotes loses all medals because he doesn't drink, under this metric!

BWW Dayton is up there because of JAGER. He is a great player, but I don't know if he ever plays at that site. Add his count to Cadillac Jacks and see where they stand after that.

If you do that, then I'm WAY better than he is. Heck I'm probably better than the whole 86th Street crew. I don't want to live in a world where that may be true.

_________________
Please forgive any strange typos or grammar errors. I am typically using voice recognition software to enter text, and sometimes editing works differently from how I expect.

I don't know, Dave.

Steve can down some beer, and the bat rastard that plays as OPNFLY can suck down some alcohol, too.

Remember: there are those who play BT and don't drink. There are some who play BT and drink. And there's SPIEGS group; folks who drink and play BT.

And then, there is...BOTFLY!!!

Well on the bright side the WV ranking would go way up under the logarithmic booze to medal ruling too. There should be a special platinum medal for perfect games achieved while under the power of gravity.

_________________
Please forgive any strange typos or grammar errors. I am typically using voice recognition software to enter text, and sometimes editing works differently from how I expect.

yeah that durn OPNFLY, when it's happy OPNFLY time, look out! tho no one can beat STEVE and TURBO when it comes to dedicated booze AND trivia. Whereas I drink but rarely, and SPOTES goes for the refreshing 23 flavors of Dr. Pepper

gavv

As you guys saw when I was there, Gavv, I'm a Dewaholic. Slamming those down like a fish.

Should've had a Irish Car Bomb there. LOL

_________________
Mitch (MTCHWF/WLFFLY)

Co-Home Bar: Buffalo Wild Wings Edwardsville, IL and Time Out in St Louis, MO

No.

Before replying at a later date to the multiboxing issue, no.

On an individual game level, a gold medal certainly reflects better play in that game. On that we may concur.

On a site level, the sum of silver and bronze medals can most certainly be more indicative of site quality than the sum of gold medals. Reread my original post:



Now, if we could somehow segregate easy golds from harder golds, then count only harder golds, that would be superior still...but we cannot easily do that. Buzztime doesn't print those stats in easy-to-use format. One can get game details by site the next day, but not game details by nation (excepting Premium Games). We can, however, be sure that silver and bronze medals reflect excellence in games too tough to tie for multiple golds. Unless one believes that on a site level, some sites dominate so convincingly that they earn fewer silvers and bronzes because their site (not just one player) is stacked on golds in such challenging games, then the sum of silvers and bronzes IS a better metric of site performance at more challenging games.

Likewise if we could just exclude cheating sites, that would be great, but the tests that would catch a site piling golds and nothing else might not so easily catch sites that supplement already-good play with sudden strings of great play distinguishable from their normal play by either casual observation (if screaming at repeated leaderboards at other bars) or statistical analysis (if checking the signature significance of a given set of answers' correctness relative to the site's/player's norms). I've actually looked at your idea of looking at the issue by standard deviation of the gold / (silver + bronze) ratio; my take is that it didn't work as well as just ignoring gold medals because the ratio depends upon the nature of the games and times at which a site excels.

I understand the gut desire to include gold medals in any measure of excellence. With the information readily available from Buzztime, I strongly suggest that the sum of silver and bronze medals, not the sum of gold medals, is a better six-month metric of site, not individual, quality of play.

I read it - I disagree with it. It is no harder to get a silver or bronze by cheating than a gold - it just takes a slight reduction in lag or reaction time. It's not 'gut feel" to include golds - it's basic sense given how easily perfect scores are attained on 2/3 of all games played by good soloists or reasonable and above teams without any cheating at all. If you want to exclude multiple golds it's easy to eliminate outliers by using a ratio. without throwing the baby out with the bathwater (except in this case there is more baby than bathwater).

_________________
Please forgive any strange typos or grammar errors. I am typically using voice recognition software to enter text, and sometimes editing works differently from how I expect.

One other issue with this metric is that, while discounting gold medals from the equation it still leaves their value in the numerator for determining overall site score.
By doing as well as possible for those golds, we've actually lowered our site score in this metric.
I see no easy way to determine the value of gold medals to provide an accurate site score that does not include them.
Besides, who's worried about gold medal cheaters?
I think most of us on here are informed enough to know which bars have the goods and which ones pray to the igods.

While I agree that most of us posting on Scaratings know that, I'm not at all sure that all of the group reading Scaratings know that.

In an amusing coincidence, without naming names or sites, I was playing away from home Saturday night, and I was defending you and your teammates at 86th Street from accusations of cheating.

Now, see, if they'd checked my new metrics, they'd know better.

It is amazing to me how many people suspect other bars and players of cheating when they are getting beaten.

_________________
"Truth Like Football. Gets kicked around much, before reaching goal." - Charlie Chan

" Don't look back, The bastards might be gaining on you." - Satchel Paige

Frank

We had some jackass on Facebook saying The Concert Pub was cheating during The Season.

Thanks for defending my honor, which is more than I've ever done.
Those kind of allegations don't bother me anymore. One tends to get used to that sort of thing.
If anything, I've actually gotten to like it when folks suggest stuff like that. It makes me feel like I could accidentally be doing something right...
Of course, if it wasn't for us, they might double their gold medal total to 30. And that's when the party starts...