Thursday, 18 October 2012

The Penyahtewan Assembly

The games first "epic" came and went in the last week.  We had five feats plus a "make up" feat after we failed the third feat of the assembly.  I wanted to offer a quick review of the feats in the assembly and say a few things about what can be learned for future feats.

First of all, a couple of metrics for feats.  The first is the percentage of the total that was completed by the top 26 contributors - that is, if you add up the contributions of the top 26, what is that as a percentage of the total that was accomplished.  This gives us an idea how much the success of the feat was dependent on "hardcore" players or people who really grinded to get the top spot.

The second is the percentage of the total that was completed by the top 1009 contributors.  The feat page only lists the contribution of the top 1009 so it seems like another benchmark.

Finally, we'll take a look at how many glitchen participated.  Since we know how much was contributed in total, and how much was contributed by the top 1009, we know how much was contributed by everyone else.  We also know the maximum contribution of every out of the top 1009.  That means we can calculate an absolute minimum number of participants.  Of course the minimum number of participants doesn't give us a great picture of how many people participated.  For example, on Call to Feast the lowest contribution in the top 1009 was 1 invite, which means that we know for a fact all other participants put in exactly 1 invite.  By contrast, the 1009th place glitch contributed 1830 sparkly for Sparkla's Shine on the Giants.  It's possible that most of the people past the 1009th person contributed over 1000 sparkly or it could be that thousands of people only put in one.  I'm open to suggestions for how to get a decent estimate of how many people participated, but none of the methods I've come up with are satisfactory to me, so I'm not venturing a guess. Still, I'll discuss it a little bit for each one.

Hispikulus's Amazing Ramble
Contribution by top 26 - 11.7%
Contribution by top 1009 - 78.9%
Minimum participants - 1955

Collecting Qurazy Quoins was an okay idea for a feat, but this really went wrong when they changed it part way through so that only Qurazy Quoins out in the world counted.  First of all, changing it partway through is always going to be a bad idea because even if it isn't working the way you wanted it to, some people have already benefited or been harmed by the way it works.  Second, the change itself was a very bad one.  People who had already explored the world were never going to be able to win the feat, but we at least could have made a meaningful contribution if we were allowed to contribute 50-ish quoins a day through tickets and reminiscences.  In the end it made very little difference.  The super mega goal was 503k and the world achieved only 138k.

As for the number of participants, the minimum contributed was 31.  That means there is a lot of room for people past 1009th place to contribute less.  If people below 1009th place averaged half of as many as 1009th place then the participants would be 2841.  Because it would have been hard to play the game and not contribute anything, I suspect the participation for this feat was quite high.

Because of the change in the middle, the exclusion of those who had already explored the world and the ludicrously unreachable super mega number, I'm giving this feat a D.

Deaths and Reminiscences
Contribution by top 26 - 33.8%
Contribution by top 1009 - 99.1%
Minimum participants - 1180

In some ways this was my favourite feat of the lot.  When it went up, I saw posts in the forums that said that because there was a reward for mourning but none of dying this feat was a bit of a problem.  People flocked to Cebarkul only to find that most of the people there wanted to rack up points but had little interest in feeding points to others.

But that's actually why I liked it.  People who took a selfish approach of trying to up their own numbers by hoping someone else would die did poorly.  People who approached the feat cooperatively, finding a partner who they could trade deaths with, did very well and topped the leaderboards.  Those people at the top with over a thousand mourns also contributed over a thousand deaths to the collective effort to reach the world's goals.

Another great thing about this feat was that we made the super mega with just 15 minutes to go.  Most of the people I talked to were really pushing it in the last couple of hours trying to make sure the feat succeeded.  I personally went and suicided in Cebarkul over 120 times, adding nothing to my own score but upping the world's score as I went.  Succeeding in the super mega goal was a great feeling, especially after the hard work that was put into it.

That being said this clearly was not a terribly popular feat.  The lowest contribution in the top 1009 was 3 so in addition to the minimum participants shown above, we know the maximum participants was 1522.  And this was for a feat that took place on a weekend day.  This is perhaps in part because a glitch playing casually would be very unlikely to stumble across the feat by chance.  You can also see that a very disproportionate amount of the work was done by a few hardcore grinders.  Basically this feat was ideal for people who love to grind things out, not so much for other people.

Though I love the fact that the feat inherently rewarded cooperation and left those who couldn't really think of the cooperative option behind, the lack of interest from the general glitch public means it wasn't a roaring success.  Because someone simply playing the game probably wouldn't have encountered the feat, I can't bring myself to give the feat more than a B, even though I want to.

The Call to Feast
Contribution by top 26 - 9.7%
Contribution by top 1009 - 98.3%
Exact number of participants - 1062

There are lengthy discussions about this feat in the forums and I don't feel the need to rehash them all here.  People were upset because they felt like they were being asked to spam friends and family, and I think stoot gave a pretty good explanation of what the Tiny Speck staff were thinking when they made this one.  I think it's fair to say this feat was a poor choice on the part of the developers.  But that's not what I want to talk about.

In a lot of ways the developers of an MMO have to walk a fine line in creating challenges for the players.  If the challenges are too easy then they aren't fun.  If they are impossible then they are even less fun - it is not interesting to lose a game against an omnipotent god.

So when we collectively fail dismally at a feat, achieving only 29% of the total, it means there was a huge misread on the part of the developers.  In this case the negative community reaction certainly played a part - because the 1009th person contributed only one invite we know the precise number of contributors, 1062, and it is lower than the minimum possible contributors to all other feats.  But if instead three times as many people had contributed we still wouldn't have made it.  It's not like hardcore players could have grinded out more invites in order to help the world reach the goal.  One can only imagine what the bonus and super mega goals on this one were.  Even if there had been no negative community reaction, I think the developers wildly misguessed how much people would be able to contribute to this one.  Success was never an option.

I'm sure this will surprise no one, but it is impossible to not give this feat an F.

Sparkla's Shine on the Giants
Contribution by top 26 - 19.1%
Contribution by top 1009 - 89.7%
Minimum participants - 1540

In the early hours of this feat there was an illusion that we were going to easily beat the goals, but it really was just an illusion.  Many people emptied out their prodigious stockpiles of sparkly, either donating them or selling them off at outrageous prices to people who really wanted first place.

Comparing this to Deaths and Reminiscences I think it comes up a little bit lacking.  First of all, instead of cooperation being the watchword of the feat, the feat was more like a get-rick-quick scheme for those who didn't care to put themselves on the leaderboards.  Secondly, the numbers on the goals were way off.  Sure the initial cashing in of existing stock made a big dent, but we were never going to get to the bonus goal, let alone make a decent try for the super mega.

For Deaths and Reminiscences a dedicated partnership could get a couple of mourns a minute each.  By contrast, mining sparkly you can get around 67 per minute.  That means that with similar effort, and ignoring stockpiles, the super mega goal from Deaths and Reminiscences would translate to under two million sparkly donated.

The fact that the world managed to donate over nine million sparkly in the day was quite impressive, and really shows the popularity of this feat.  In fact, the 1009th place glitch donated 1830 sparkly, so if many glitchen donated just a few hundred then the number of participants in this feat may have four or five thousand.

It was never remotely possible for use to fail the minimum goal and never really possible for us to make the bonus.  The result is that our efforts towards the larger goal didn't matter that much.  Add to that the fact that many people were selling rather than contributing and this feat seemed excessively focused around jockeying for position on the leaderboard rather than around cooperation.  On the flip side of that, though, this feat was a great opportunity to meet other glitchen.  On home streets and out in the world glitchen were saying hello and selecting "Help Mine."

As a result of all of these observations I find this feat very hard to grade.  In the end I think I am going to have to give it a reserved B.  The miscalculation of the goals makes me feel it shouldn't get more than a C, but the level of participation and the opportunity it provided to experience the sharing environment of the game pushed my over the edge.

Questing and Contesting
Contribution by top 26 - 24.0%
Contribution by top 1009 - 80.3%
Minimum participants - 1988

I'm sure this feat had a very high level of participation.  The 1009th place glitch provided 10 quests to the effort, and I'd wager heavily that the vast majority of glitchen who were in Ur at all during this feat contributed at least one.  If the number who contributed only one or two was high then it's possible that four or five thousand glitchen worked on this feat together.

This feat was a bit of a train wreck.  I'd imagine almost everyone who reads this blog is aware of how this worked, but just in case, I'll explain that going to visit Gwendolyn in Jethimadh tower triggers a series of three quests, so each red-green triangle key could be converted into three quests towards the total over about two minutes.

I'm sure this was not the intention.  I can't imagine that the developers thought to themselves that we'd all be standing around in Corridor Five running time trials on the Gwendolyn quests trying to get the optimal route and loading times.  I very much appreciate the fact that they did not change this midway through as they did with the first feat - after all, that would have just given the people who figured it out first an insurmountable lead.

But if they didn't expect us to use this method to complete quests then the goals they set for the feat were transparently ridiculous.  I think it is fairly safe to say that anyone who contributed more than about 40 quests must have used the Gwendolyn method, and that many of those people would have been able to contribute less than 20 had they not done so.  At that rate even with the very, very high participation in this feat we would have fallen drastically short of the bonus goal, let alone the super mega.  As it was we made a reasonable run towards absolute victory, but the method to contribute significant numbers was too arcane, and the keys were selling for 10k or more by the end of the day.

Because of the get-rich-quick scheme, the oversight in testing that allowed Gwendolyn to slip through, and the outrageous goals I'd giving this feat a D - even though it gave me a large rock.

The Incantations of Absurdity
Contribution by top 26 - 10.5%
Contribution by top 1009 - 55.2%
Minimum Participants - 5572

I had to double check that minimum participants figure, but it's right.  At the very least 5572 glitchen did this one, and I wouldn't be surprised if it was a fair bit more than that.

The idea that we wouldn't have seen this feat had we not failed the Calling to Feast is a little bit mindblowing.  This feat did a better job of getting people to travel the world than Hispikulus's Amazing Ramble, it encouraged cooperation more than Deaths and Reminisences, and it gave more opportunity to meet new glitchen and interact with them in a fun way than Sparkla's Shine.  It was free of errors and exploits, and it allowed us all to come together to reach a staggering goal.  This feat also gave no advantage to new or old players, to players with skills or without - everyone had equal change to contribute.

We hit the super mega with just nine minutes left to go, and the level of participation surely warrants our level of success.  I think the goal levels were perfectly set on this one.

This was an amazing way to finish off the epic.  We all did our best and we made it just in as a result.  I give this feat an unreserved A though I recognize that the components that put it together would be hard to reproduce.

The Epic as a Whole
The epic was not terribly well done.  The goal numbers seemed poorly thought out, a couple of feats had problems with changing rules and unexpected routes to success, and the epic was badly marred by the third feat.  All that being said as a first outing I don't think I can be too down on the whole thing.

I'd say there is lots of room for improvement in future feats and future epics.  I do hope, however, that epics are something that happen very rarely - a couple of times a year at most.  It was a great week to play Glitch, but if it happened against next month the shine would start to wear off quickly.


  1. Hmmmm...

    A lot of the arguments for the goals being poorly thought out seem to be less strong if you assume equal amounts of participation in each feat, and that super mega should actually be really hard to get, and not some sort of "real goal" hiding behind two others.

    What I mean is, if we'd had "Incantations"-like participation in Questing & Contesting, then the levels would have made more sense-- even if super-mega would still have been hard, it wouldn't have been impossible w/o Gwen. Same with Sparkla's, although less convincingly (since we may well have had high participation, it was just masked by the funnelling through the richest contributors).

    I wonder if the real issue is that they're having a hard time gauging what participation levels they're going to get? And given that I had way more fun doing Quests than Incantations and wasn't bothered at all by spamming my friends for this game, I can totally believe that it's really hard to figure out what the people of Ur are going to love.

    Another example-- they may well (and, per stoot, arguably did) have thought "Everyone will want to invite people-- it's so easy to just type e-mail addresses in boxes and hit send, so we'd better set those levels pretty high".

    Anyway, thanks for crunching the numbers. I'm always excited when there's a new post here.


  2. I do think there wasn't a lot of good thinking about the goals. As I noted, the deaths one we know for a fact that we had relatively low participation - no more than 1522 people mourned a grave. Yet we made the super mega. On sparkly I think it's fairly safe to guess we had at least 2500 participants but didn't really come close to the bonus. Maybe we didn't deserve the super mega on deaths, but either way I don't think that the numbers are well done. A big part of that is learning, but some of it is just not properly thinking things through. In the whole world there are only 261 sparkly rocks, home streets have another couple thousand at most. Given that a sparkly rock regenerates every 8 minutes and contains an average of about 78 chunks their bonus goal requires harvesting about a third of all of the sparkly in the entire game over 24 hours.

    As for the friend invites, missing the fact that a large number of people playing the game will have already invited everyone they know who they think would be interested is a pretty colossal oversight, even if I can forgive them for not anticipating the backlash. Obviously I get to sit here making up rationalizations to justify my hindsight, but I do think that they were way off in their guesses both in terms of number of participants and the amount that participants could contribute on several of the goals.

  3. I contributed about 60 Quests (Reminisces) before I learned about the Gwendolyn method ;o)

    From observation, it looked like the Bonus and Supermega goals were being computed on the fly, with some minimum -- if we'd continued at the rate at which we'd achieved the Sparkly Feat's first goal, we would have hit the Supermega in a little over 30 hours. Likewise with the Deaths, the Quests, and the Incantations Feats... and note that we barely finished both of the Supermegas in time!

  4. Ooh! Interesting idea about computing on the fly! Stockpiled sparkly would explain why the supermega was so impossible. Do we have any information on how FAST the 1st bonus was reach for any of the feats?

    But I'm going to stick to my guns a bit on this-- there are a lot of people who don't like to die, even though it doesn't actually hurt a glitch. Perhaps the devs thought participation rates in that feat would be much lower than they ended up being. (And in a way, that was true-- that was, by far, the feat with the most contribution by the top 26....)


  5. [Disclaimer: These values are not *accurate* but reflect what I recall from the feat. They should be generally correct, although these values should be taken with some amount of doubt.]

    Ramble: I missed this one.
    Deaths: Minimum was achieved an hour or so into the second day.
    Call: We didn't make the minimum.
    Sparkly: We made this one in a little under two hours. Sparkly also sold for 150c in these first two hours...
    Questing: We hit the goal at around midnight PST.
    Incantations: It barely took an hour for us to reach the minimum goal.


  6. Wish I knew how to put a table in to blogger.

    Arranging the quests by the amount of time it took to get the minimum goal, with the multipliers from min to bonus, from bonus to super-mega, and from min to super-mega:

    Questing & Contesting - 6 hours - 1.9 - 1.7 - 3.4
    Deaths & Reminisces - 5 hours - 2.5 - 2.4 - 6.0
    Sparkla's - 1.75 hours - 9.0 - 3.0 - 27
    Incantations of absurdidity - 1.2 hours - 5.0 - 6.7 - 33.3
    Rambles - unknown - 9.9 - 3.8 - 37.7

    Rambles certainly hit the minimum quickly, since it was open for some time on a previous day (when I logged in for the 1st time 12 hours into the feat, I had already contributed QQs). Also, it would have had a much faster rate until they capped non-street QQs, explaining us just barely getting past bonus.

    Sparkly would also have had a super-high (comparatively) rate over the 1st 2 hours, again explaining not hitting bonus.

    Questing & Contesting might arguably be similar, as people ran through the easier quests in their logs, but it might arguably be the other way around as more and more people went to do Gwendolyn. (Or perhaps people got sick of running Gwen like I did?)

    It seems clear to me that the super-mega multiplier (super/min) is inversely related to the number of hours it took to reach the minimum goal, and that some attempt is made to scale the bonus and super-mega goals. The only inconsistency I see is that they seem to have had the 2nd of the 3 multipliers (super mega/bonus) capped up until Incantations hit, then moved to a more even distribution for Incantations.

  7. There's no clear formula there. Of course they were probably also messing with all aspects of feats after each one, so it's possible there is a simple formula that was changed or that there is a very complex formula that they didn't change.

    The idea of calculating the second two goals on the fly is very interesting and potentially quite awesome though it clearly has a problem when stockpiles are involved. It also has a big problem with different times of day. These feats all started at 21:00 EST. It is fairly easy to predict that the first couple of hours will be a very high rate, the next couple of hours there will be a steady decline and around five to six hours after the feat launches the rate will really get low. Of course if it's a Friday night then the rate will jump up again mid-morning, but if it's a weekday then the rate won't really start to hit peak again until a couple of hours before the feat closes.

    It think it might be that the bonus and super mega goals were adjusted on the fly, but by hand rather than by a formula.