Duel Ranking

  • 2
  • Idea
  • Updated 8 years ago
  • Implemented
I don't know how the current ranking system takes into consideration every duel, but I would like to highlight a possible issue that may arise in the long term if some of the players doesn't behave fairly.

For example, is the system able to identify players that are voting against the best images (just with the purpose to penalize competitors) in the categories they are challenging with other images?

I know it's a very delicate matter (because everyone has its own taste that cannot be judged), but I would like to discuss with you some possible rules with the goal to prevent "unfair" play.

1. Every image in a category should be matched with all the others in the same category for a predefined number of times at least (suppose at least 7 times)

2. Every duel doesn't lead to an instant Win, but only after the predefined number specified above (in the above case, after 4 victories among 7 against the same picture)

3. Each player should not be able to vote for the same image more than X times

Again... I'm sure that almost all players are playing the game correctly, but I think that some rules with the aim to "normalize" votes should be in place (maybe a sort of reliability indicator for each player to weight each vote?).
Photo of Luca Libralato

Luca Libralato

  • 43 Posts
  • 17 Reply Likes
  • undecided

Posted 9 years ago

  • 2
Photo of Justin Kifer

Justin Kifer, Official Rep

  • 949 Posts
  • 193 Reply Likes
Hey Luca,

For the very reasons you've specified above, we do not divulge specifics about how the ranking system works. This is meant to make it more difficult to game the system.

That said, I can tell you there are users that are trying to game the system. As much as we would like for everyone to play fair, vote without bias for a friend or family member's image, etc... some are just looking for the win. This is completely against our mission to help discover the best images in any given subject category and thereby the best Players in those categories.

As we are still a very young site, we are still working out the kinks and this is one area we are focusing heavily. We find the "bad apples" quickly and we're getting really efficient at it. We can correct their misuse of the ImageDuel system as well and we do.

To your specific suggestions. Unfortunately, there are not enough votes to allocate for each image to go against every other image in the same category 7 times or really any other number of times. We have designed the matching and ranking systems to be efficient and quickly weed out images that don't belong at the top of the leader board. Take a look down through the leader boards and you'll see what I'm talking about. We allocate a lot of votes to the top of the leader board so as to create definition where it counts.

We already enforce a limit on the number of times a user can vote on an image.

We are constantly pouring over the data (and there is a lot of it) to find ways to improve the logic behind matching and ranking. In fact, we're in the process of bringing a mathematician on full time!

Thanks for your thoughtful ideas!
Photo of Luca Libralato

Luca Libralato

  • 43 Posts
  • 17 Reply Likes
Maybe you better need an expert of Game Theory instead of a pure mathematician.

I would suggest to differentiate votes between at least two different user categories, something like Power and Normal Users... or you may also think to create a Jury where you add a certain number of reliable users which votes are weighted differently.

I guess this would add stability to the system.

Also another idea that you may have already considered... what about making a picture not available into photographer profiles until it reach at least 50/100 votes in contests? This could help to prevent the bias towards photographers who follow each other.
Photo of Justin Kifer

Justin Kifer, Official Rep

  • 949 Posts
  • 193 Reply Likes
Hey Luca,

You're starting to hit on stuff we're doing already and are continuing to enhance. Reliability of a user and their votes is an important factor in how we allocate votes.

Interesting idea regarding masking the player who submitted the image until after a certain duel threshold has been reached. The only problem with this is that we allow players to post their submitted images to Facebook and Twitter which means that won't entirely eliminate that issue. In addition, some images will be paused well in advance of that threshold.

Still, it's something to consider!
Photo of Marianna Armata

Marianna Armata

  • 43 Posts
  • 13 Reply Likes
I second this to the fullest extent. Although you don't show the author's name on the ImageDuel screen, you do on the Leaderboard. So - it's very easy to see who submitted what. Couldn't you just withhold all names until the voting/duel period is over? that would eliminate the (evidently rampant) favoritism.

Suspending images is really, really frustrating. I don't see why I should have to baby sit an image after just a few votes. If I submit an image it should play out the duel to the end of the week. Deserving one win or 100, regardless.
Photo of Linda Clark

Linda Clark

  • 23 Posts
  • 0 Reply Likes
Very frustrating to have an imaged paused so quickly..after just a few votes. Not saying I'm that good...but if you invite new people in the door, well then let the images play at least a min. amount.....that would appear more fair, especially with many evidently fast voting cause you've set up a system where they need points. I've boosted some images, and they go forward to do reasonable for a beginner. Fix, yes?
Photo of Luca Libralato

Luca Libralato

  • 43 Posts
  • 17 Reply Likes
I agree on the frustration of suspending images. At least an "auto renew" option should be implemented to allow the photo to be submitted again automatically for a certain number of times (3?).

I have also noticed today that one image has been matched 3 times consecutively against another photo (3 consecutive losses!) in the first 5 matches, this meant an immediate suspension.

Wouldn't be better to allow a larger minimum amount of different matching (for example 10 different matches) before being suspended the first time? It's really frustrating seeing your images being suspended because it constantly lose the duel with the same image several times just few minutes after the first submission.
Photo of Eko Suhartono

Eko Suhartono

  • 17 Posts
  • 0 Reply Likes
This topic is very interesting, and after some time I followed a similar site with pixoto, both internationally and domestically Indonesia, the issue of intent (bad?) of the contestants who use improper means to obtain the highest score.
I'm just a few days following the new pixoto and upload 3 photos. I just love to experiment (testing) something new. there is 1 of 3 photos that I uploaded was a quality entirely unfit for the contest. and in fact to date, only 290 received a perfect score, this means I am testing one of the items correctly, and my other two photos one obtains scores above 500 and only above 300. This value is appropriate for my photos.
With this initial testing I can conclude for middle-level photo quality, the system of Pixoto it can give a correct assessment. As for the 20% best, still have your system repaired. In my opinion the best for the best 20% you have to do a special scoring system, so the result is really satisfying all the contestants.
I am sure with so many photos are uploaded you have to rely on a specific software system to assess, you might not judge it manually.
Some of the obstacles that I feel in Pixoto (may be an input for future improvements) are:
1. after I participated in imageduel, within a period of 2 days, I am faced with a confusing choice for bids on imageduels, could not contested because of different categories, for example (I am less concerned about the categories) photographs of the children dueled with railroad photos fire should not be in one category, and several other imageduels sessions which I think can not be contested because of different categories.
2. a lot of photos that are not original works entered in all categories (or was acquired by the ToS?).
3. I think a image of children excluded from the category portrait & people, and became a separate category. In my opinion can not contesting between photos of the children with a photo model (adulthood), a completely different of criteria, the character and nature.
Thanks, bravo for Pixoto, go ahead and continue to improve. congratulations.
Photo of Alex Fullam

Alex Fullam

  • 11 Posts
  • 2 Reply Likes
I too am following this thread and ideas. I would like to say however that looking at the current system, it works for the most part, and I LOVE the ideas and what you are doing. Heck, the prize winnings are cool, but I am hooked without caring if I win or not. as a professional photographer, this site has allowed me to see where I can grow, and I can take even a major loss as a learning tool to make myself better, and it also gives otherwise unknown artists a chance to shine so huge applause from me! I will always judge fairly, and I think most people using Pixoto are doing the same :)

One idea, throw it out if you like, I was looking at a few photos I posted and playing with the numbers. I don't know if it is possible, but those who have had a number of badges, is there any way to have their photos ranked in the middle tier at first post? some algorithmic equation that would consider a player to have higher possibilities of a top tier photo? The logic I see is that somebody just trying to win probably won't have a stellar photo, and by placing a higher probability photo above the lower tier might cut down the false negatives. this of course may come as a double edge sword in that if your photo starts in the middle tier, you get knocked down faster.

Just thinking out loud lol. but seriously, Love the site, I am spreading the word around here and making sure people know that they need to SKIP when they see my photo, well, unless it is a clear looser / winner. and I do the same. Its great to see what people really think of my work.

Thanks again!
Alex Fullam

This conversation is no longer open for comments or replies.