Fate of Io
v5 Beta 3.1
Posted by Temporal at 2005/02/22 08:15:01 PST

Well, all I really did today was tweak the files some. The major update is that you can now rate files. Once a day, rankings will be computed for all files based on these ratings, and files (other than "recent" files) will be displayed in order of rank.

The ranking algorithm actually uses Condorcet's Method rather than a simple average of ratings. This way, it won't matter if some people are harsher judges than others. For example, even though ratings are supposed to be on a scale of 0-100, I could perfectly well rate files on a scale of 0-10 personally and it wouldn't mess up the system. The only thing that matters is which files I think are better or worse than which other files.

The other thing I added is a progress indicator for large file uploads. It's a bit of a hack, but it solves the problem of not knowing if the upload is working or not.

Please press F5 now to make sure you have the latest stylesheet and javascript.

2005/02/22 14:23:44 PST by mystik3eb [0/43]
[mystik3eb's avatar]

Just thinking: maybe we should make it so whoever submits a file can't rank it themselves? Sure if people were shrewd enough they'd go out of their way to make another account and rank from there, but for the most part, it'd be fair...what do you think?

Added at 2005/02/22 14:27:48 PST

Oh, and two thoughts that immediately came to mind after posting:

1.0 Should it be possible to see what the file's ranking is besides seeing how high it is on the chain, since it's really vague that way?

2.0 Do you get trash everytime someone edits (appends, heh heh) a post? Would you recommend further use of the Preview button, or is that not a big deal anymore? Just for clarity.

2005/02/22 14:37:30 PST by Captain Vimes [0/13]

Just so I know, do you have a new link for v5 bracket commands? I don't know all that many other than quote, italics, and bold.
And, er, if I missed it, smite me with thunderbolts.

2005/02/22 17:28:00 PST by mystik3eb [0/43]
[mystik3eb's avatar]
Added at 2005/02/22 17:29:25 PST

http://fateofio.org/news/22

Also there's the results of me testing the bracket commands in v5.0

2005/02/22 18:25:53 PST by Temporal [manager]
[Temporal's avatar]
Quote from mystik3eb:

Just thinking: maybe we should make it so whoever submits a file can't rank it themselves? Sure if people were shrewd enough they'd go out of their way to make another account and rank from there, but for the most part, it'd be fair...what do you think?

I don't think it's a problem if people can rate their own work. It's like how presidential candidates get to vote for themselves. They're citizens too and they have the right to vote.

As for being able to register a second account... Rankings will be weighted by influence. So, ratings by new accounts which have not contributed anything significant to the project will essentially be ignored.

What I really like about this system is that it means there is no advantage whatsoever to registering multiple accounts. If you split all your contributions between two accounts, you would end up with two accounts that have half the influence, and would have no more combined power than if you had done all the contributions under one account.

Quote from mystik3eb:

1.0 Should it be possible to see what the file's ranking is besides seeing how high it is on the chain, since it's really vague that way?

I was thinking just that, actually. For now, the rankings are displayed in order to assist in testing. But, in the final version, perhaps rankings should be invisible, in order to prevent people from really knowing if their file "beat" any other file, to diffuse some of the boasting.

Quote from mystik3eb:

2.0 Do you get trash everytime someone edits (appends, heh heh) a post? Would you recommend further use of the Preview button, or is that not a big deal anymore? Just for clarity.

What do you mean, "get trash"? It's up to you how to use the preview button. Since appending can't be used to correct spelling, grammar, and formatting errors (except to say "oops, I meant blah"), I'd suggest using preview to check those. But, I don't mind people repeatedly appending a post to add further thoughts.

2005/02/22 18:28:36 PST by Temporal [manager]
[Temporal's avatar]

Bug: Commenting on a non-recent file causes crazy problems with the file list. They're corrected easily by updating the ranks, but whoa. O.o

Added at 2005/02/22 18:33:47 PST

Fixed!

2005/02/23 02:47:16 PST by mystik3eb [0/43]
[mystik3eb's avatar]
Quote from Temporal:

What do you mean "get trash?"

Sorry, I meant does the system send you a log everytime someone appends? It doesn't seem like you care as much about it these days, but that's what I was wondering.

2005/02/23 04:36:52 PST by Temporal [manager]
[Temporal's avatar]

No, and it never did. I mean... with v4, I only received messages when errors occurred. With v5 I don't actually receive any sort of messages. I probably should rig something up to send me exceptions, though...

2005/02/23 12:52:11 PST by mystik3eb [0/43]
[mystik3eb's avatar]

Oh, and when I was talking about rank, I actually meant rating, sorry. If we don't want people to see what rank they are, I guess that means the same for rating, but I'm just checking. Like personally I'd like to see what overall rating my song got, for example.

2005/02/23 23:05:19 PST by Temporal [manager]
[Temporal's avatar]

Oh. Well, the problem is, there is no meaningful way to calculate an "overall rating". A simple average doesn't make sense, because everyone has a different scale. For instance, Dev and I might both think a piece of concept art is equally good, but I might rate it 90 while he rates it 75, just because he's more stingy. So what good would it do to display the average score of 83?

Thus, the ratings people assign are only usable for determining which submissions are better or worse that others.

Added at 2005/02/23 23:25:18 PST

And, again, rankings are not based on average ratings. Rankings are created by first creating a ranked list for each user (based on that user's ratings), then combining those lists using Condorcet's method.

Added at 2005/02/23 23:34:41 PST

In fact, as an example, imagine the submissions A, B, and C are rated as follows:

Dev: A 10, B 5, C (not rated)
Temporal: A (not rated), B 60, C 50

Clearly, Dev and Temporal are using wildly different rating systems. In reality, the differences would be more subtle, but there would still be differences.

Now, from the above, the average ratings would be:
A 10
B 33
C 50

Ordering by average rating, we'd have C as #1, B as #2, and A as #3.0

However, if you actually look at the ratings, you see that Dev thinks A is better than B, and Temporal thinks B is better than C. Clearly the correct order in this case, given no other information, would be A #1, B #2, and C #3.0

My algorithm will produce the correct order.

2005/02/23 23:40:22 PST by mystik3eb [0/43]
[mystik3eb's avatar]

Ok I understand. You're saying that we won't take the average, we'll take a comparison depending on what individual users have rated the submissions? That works and it makes sense.

2005/02/23 23:53:12 PST by Temporal [manager]
[Temporal's avatar]

Right. The only thing that matters is which submissions users think are superior or inferior to which other submissions, not the acutal numerical ratings they give.

2005/02/24 02:11:33 PST by Dev [manager]
[Dev's avatar]

Exactly. Except .... jeez, dude, I'm not that stingy. :P

fateofio.org © Copyright 2001-2005 Sam Pierce, Kenton Varda, and contributors
Powered by Io Community Manager, Evlan, and FreeBSD