How are Badges rank/order determined?

  • 3
  • Question
  • Updated 9 months ago
  • Answered
  • (Edited)
Annual Badge 2017 - Animals Cats : I received an annual badge of 10 for 2017 but according the list ( https://www.pixoto.com/players-photography/animals---cats/leading/in-year-2017 ) it should be a rating of 7. Is this a mistake?
Photo of Pieter J de Villiers

Pieter J de Villiers

  • 44 Posts
  • 4 Reply Likes

Posted 10 months ago

  • 3
Photo of SweetNana55

SweetNana55

  • 419 Posts
  • 45 Reply Likes
My badges don't line up with the list either. I'm guessing that it depends on the date the list for badges was run versus what it looks like today. I was expecting #8 in animals and was awarded #10. Likewise, I thought I'd get #10 in B&W and got #11. I was surprised to get #18 in Abstract since I am only #21 on that list.
Photo of Dave Lipchen

Dave Lipchen

  • 80 Posts
  • 21 Reply Likes
I have the same problem too.   Artistic objects missing and a tenth spot for animals instead of 12th place.
Photo of Dave Lipchen

Dave Lipchen

  • 80 Posts
  • 21 Reply Likes
Just double checking again.  My cats show 15 place when it looks like I should have 16th place.
Photo of Pieter J de Villiers

Pieter J de Villiers

  • 44 Posts
  • 4 Reply Likes
If this is not solve, I will not renew my pro-account !
Photo of Karen Mac2

Karen Mac2

  • 135 Posts
  • 33 Reply Likes
And I evidently did not get a badge for a category, even though I was listed as being #17 in it, according to the leaderboard. People who were listed after me did get a badge in the category.   It is puzzling! And a #6 place instead of a 4th place in flowers. Odd, as the listings have been the same for over a month on there.
Photo of Pieter J de Villiers

Pieter J de Villiers

  • 44 Posts
  • 4 Reply Likes
There was no changes in the placings since January 2018, Pixoto should get there house in order!
(Edited)
Photo of Kelley Hurwitz Ahr

Kelley Hurwitz Ahr

  • 137 Posts
  • 27 Reply Likes
Mine are wrong, too (I guess most/all are wrong).  
Photo of Jasenka

Jasenka, Official Rep

  • 17886 Posts
  • 1287 Reply Likes
Hi all, thank you for informing us about this. I'll forward it to be checked and let you know when I get a reply.
Photo of Scott Murphy

Scott Murphy, Official Rep

  • 583 Posts
  • 59 Reply Likes
Thank you Pieter for your inquiry. I will have an answer shortly.
Photo of Scott Murphy

Scott Murphy, Official Rep

  • 583 Posts
  • 59 Reply Likes
Badges are assigned based on total points for a category EXCLUDING points won for annual awards. Annual awards are too high of a point value and would skew the total points won in a particular category.
Photo of Mia Ikonen

Mia Ikonen

  • 295 Posts
  • 52 Reply Likes
Is this only for 2017 because the annual awards and badges were issued so late in the end of the year? How will it be with the annual awards and badges concerning images submitted in 2018?
Photo of Scott Murphy

Scott Murphy, Official Rep

  • 583 Posts
  • 59 Reply Likes
No, badges were assigned constantly according to how they have been assigned since 2011
Photo of Mia Ikonen

Mia Ikonen

  • 295 Posts
  • 52 Reply Likes
As far as I know the system changed in 2014. Previously best of the year accolades were determined by the average of user's top 10 images in a category. Since 2014 the rank has been determined as a total of the points of Daily, Weekly, Monthly and Yearly Awards. This was a welcome and fair change.
Photo of SweetNana55

SweetNana55

  • 419 Posts
  • 45 Reply Likes
Scott, I respectfully disagree with your explanation of how the annual awards have been given since 2014. I did some research regarding where people show up on the Top Contributors page and the awards they received in 2014, 2015, and 2016 versus 2017. In every instance for the three years prior to 2017, I found NO instances where a person won an award higher than where they currently show on the leaderboard.  Their placement and award matched or the award given was one less than now showing. That can happen if one of the leaders totally closed their account. Then everyone below them should move up one on the leaderboard causing the award to be one less than what is seen. Of course, the farther down you go on the leaderboard, there is a chance that a person could have moved up 2-3 slots. I checked three years of data for more than a dozen individuals in the animal and B&W category. Since I used the placement shown on the Leaderboard, either the Leaderboard does not include the annual points (which would mean that the 2017 awards should line up with it, too) or the annual points are included in the awards for 2014, 2015, and 2016.

The only time where I found people who had a higher award than what the leaderboard shows was for 2017. In Animals, Renos got the #3 award and only shows as #4 on the board. Bernard got the #8 award and only shows as #11 on the board. In the Black & Whate category, Shawn shows as #5 and won the #4 award, and S. Balaji won the #10 award and only shows as #11. Surely I would have seen the same things happening in at least one instance in 2014, 2015 and 2016 if they were calculated the same way as the 2017 awards. I know my research is limited, still I feel that it was big enough that I should have found more instances similar to 2017 in the years prior. 
Photo of Scott Murphy

Scott Murphy, Official Rep

  • 583 Posts
  • 59 Reply Likes
Hi Judy, I didn't come up with my own way of doing this. I used the exact logic that was used for 2016 and as far as I am aware of has been used for all the years prior. If you want we can look a particular category/year and I can generate the ranking based on different methodologies (excluded vs included).
Photo of Scott Murphy

Scott Murphy, Official Rep

  • 583 Posts
  • 59 Reply Likes
Also bear in mind that placement can very likely be coincidental. E.g. both strategies could yield the same result/placement.
Photo of Kelley Hurwitz Ahr

Kelley Hurwitz Ahr

  • 137 Posts
  • 27 Reply Likes
I think the point was missed - I think what Sweet Nana is saying is that her findings suggest that the way it was done in the past is NOT the way it was done for 2017.  Am I wrong?
Photo of Jasenka

Jasenka, Official Rep

  • 17886 Posts
  • 1287 Reply Likes
Kelley, I checked contributor leaderboard for things SweetNana is pointing out - I also used B&W category for 2015 ( https://www.pixoto.com/players-photog... ) - on leaderboard S. Balaji (currently 7) is positioned above Garry Chisholm (currently 8) yet Balaji got #9 accolade and Chisholm got #8 accolade for B&W category for 2015, so in previous years it is also shown as in 2017 that leaderboard is not always the same as accolades (and as Scott already explained they can be the same, but don't need to be).
You can see similar behavior in Artistic objects in 2015 - https://www.pixoto.com/players-photog...
Ian McGuirk won #18 accolade yet it is on leaderboard above Nagy who won #8 accolade and others with higher accolades them him for the same year.

I hope this helps.
Photo of Dave Lipchen

Dave Lipchen

  • 80 Posts
  • 21 Reply Likes
Hi J. why would you go back to 2015 and not what happened in 2017?

Just Curious.

Also, why wait until March to produce the badges  when all the contributions are finished for 2018 especially since the annual awards don't count for anything?
Photo of Jasenka

Jasenka, Official Rep

  • 17886 Posts
  • 1287 Reply Likes
I went back to 2015 because SweetNana pointed out that leaderboards from 2014, 2015, 2016 were exactly the same as accolades (meaning they were assigned with yearly awards). I just pointed out that they are not the same.
Regarding March to produce badges - Scott replied in this thread to you about the same question: http://community.pixoto.com/pixoto/to...
"... we have some significant changes coming out in February and want to make sure the overall system is stable before releasing awards. Also, I want to give everyone an opportunity to express their opinion on how badges are awarded prior to releasing them."
Photo of Scott Murphy

Scott Murphy, Official Rep

  • 583 Posts
  • 59 Reply Likes
Here is the ranking based on award points (excluding annual awards)
You can view a particular user by visiting httsp://www.pixoto.com/USER where USER is the number from the list below.
Photo of Scott Murphy

Scott Murphy, Official Rep

  • 583 Posts
  • 59 Reply Likes
The link you provided
https://www.pixoto.com/players-photog...
is sorted based on total points (including annual awards).

Badges are a recognition of a users contribution over time. Otherwise someone could just submit a couple images in December and get recognition for the entire year.
Photo of Dave Lipchen

Dave Lipchen

  • 80 Posts
  • 21 Reply Likes
Hey Scott, if yearly awards don't count then why bother with them.  Since your final yearly standing is based on the January - December monthly count. All points for the 2018 year should be out on the second Tuesday of the January 2019.   After that do a data dump for the badges and assign them.  Just a thought.
Photo of Scott Murphy

Scott Murphy, Official Rep

  • 583 Posts
  • 59 Reply Likes
Hi Dave, I did not come up with the badge concept so I can’t give you an explanation on why annual awards don’t count other than my interpretation after the fact.All I did was assign badges in accordance to how it has been done every year.Personally, I have no strong opinion in the matter and am willing to open it up for discussion
Photo of SweetNana55

SweetNana55

  • 419 Posts
  • 45 Reply Likes
Why assign so many points for annual awards if you must then ignore them when determining annual assignment of badges? Wouldn't it make more sense to determine points in a way that is cohesive for the whole award system?
Photo of Scott Murphy

Scott Murphy, Official Rep

  • 583 Posts
  • 59 Reply Likes
It depends how you look at it. Annual awards are redundant if they influence badges too much. I think the whole reason badges were created was to award consistent success in a particular category over the course of the year.
Photo of Scott Murphy

Scott Murphy, Official Rep

  • 583 Posts
  • 59 Reply Likes
Perhaps for clarity, we could consider re-ordering the link you provided.
Photo of Mia Ikonen

Mia Ikonen

  • 295 Posts
  • 52 Reply Likes
This reply was created from a merged topic originally titled Best of the year Photographers from now forward?.

I’m confused about the thread "How are Badges rank/order determined?" and because the topic been marked ”Answered” I will start a new one. I want to know and understand how the Best photographers of the Year from this year (2018) and forward will be determined (rank or order / badges / accolades). Since 2014 the rank has been determined, as far as I know, as a total of the points of Daily, Weekly, Monthly and Yearly Awards. According to my opinion this has been a fair way to get it done. What has been submitted and earned points from, no matter in which month although it would be December is as valid as the points earned in the beginning of the year. The rank/order, badges and the accolades should though be issued the following year preferably latest in February because otherwise the score doesn’t validly show the order correctly. This happens because the best images are dueling more or less as long as they are on the site. And there are always new images submitted that mess with the ”old rank” in the process of determining ”All time best images”. So, how will all of this be done from now forward?
Photo of Mia Ikonen

Mia Ikonen

  • 295 Posts
  • 52 Reply Likes
This reply was created from a merged topic originally titled Pixoto's ranking system 2018 and after that.

For some reason Jasenka has merged two topics regarding Pixoto's ranking system which one of was mine and I don't approve of it. I want clear answers about the ranking system regarding 2018 and after that. This Best Photographers of the Year / Badge / Accolade mess is too much to understand regarding 2017 and that's why I wanted to separate these issues. I was confused about the thread "How are Badges rank/order determined?" and also some of Scott's answers concerning the matter. Among other things he wrote "Badges are assigned based on total points for a category EXCLUDING points won for annual awards. Annual awards are too high of a point value and would skew the total points won in a particular category." What is this? Annual awards have been included in the total points for many years now. Because the topic was marked ”Answered” at this point I started a new one. I want to know and understand how the Best photographers of the Year for this year (2018) and forward will be determined (rank or order / badges / accolades). Early on in Pixoto’s history the best of the year accolades were determined by the average of user's top 10 images in a category. This was criticized and debated many times for not giving an accurate result of the whole. Members who submitted a lot of great images but didn’t make it into the top 10 felt left outside. Since 2014 the rank has been determined, as far as I know, as a total of the points of Daily, Weekly, Monthly and Yearly Awards. According to my opinion this has been a fair way to get it done. What has been submitted and earned points from, no matter in which month although it would be December is as valid as the points earned in the beginning of the year. The rank/order, badges and the accolades should though be issued the following year preferably latest in February because otherwise the score doesn’t validly show the order correctly. This happens because the best images are dueling more or less as long as they are on the site. And there are always new images submitted that change the ”old rank” in the process of determining ”All time best images”. So, how will all of this be done from now forward?
Photo of Jasenka

Jasenka, Official Rep

  • 17878 Posts
  • 1287 Reply Likes
Hi Mia, I merged the topics (as well as this one) since they have same question. Also, topic status is changed to "In progress".
We are trying to clean up the community and are merging all topics with same theme/issue/question.
Photo of Mia Ikonen

Mia Ikonen

  • 295 Posts
  • 52 Reply Likes
Ok, but hopefully people won't get even more confused than they are at this point. Because 2018 was so full of technical problems that the Yearly Awards, Badges and Accolades of 2017 were postponed to the end of this year it accumulated strange problems in the ranking as we've heard from many members now. Nobody seems to know the reason for this. Let's all hope 2018 was an extreme exception in Pixoto's history and that nothing like this will never happen again. The ranking system Pixoto has had since 2014 has otherwise worked well and been fair, so let's continue with that!
Photo of SweetNana55

SweetNana55

  • 419 Posts
  • 45 Reply Likes
Scott, I added a comment above in response to one of your statements. (Badges are assigned based on total points for a category EXCLUDING points won for annual awards. Annual awards are too high of a point value and would skew the total points won in a particular category.) I'm not sure that you saw what I posted since there is no indication that it was noted. Please take a look. I'd be happy to share more details of my research if you'd like me to do so.
Photo of Scott Murphy

Scott Murphy, Official Rep

  • 583 Posts
  • 59 Reply Likes
Did you see my comment "Here is the ranking based on award points (excluding annual awards) " ?

This was for Animals Cats
Photo of Kelley Hurwitz Ahr

Kelley Hurwitz Ahr

  • 137 Posts
  • 27 Reply Likes
I'm also interested in understanding the differences in awards this year and how they will be determined in the future
Photo of Scott Murphy

Scott Murphy, Official Rep

  • 581 Posts
  • 59 Reply Likes
I have no strong bias one way vs the other. In the end, I will leave it up to the community. If no strong argument is made for doing it one way over the other. 2018 will be done under the same ruleset of the previous years.
Photo of Mia Ikonen

Mia Ikonen

  • 295 Posts
  • 52 Reply Likes
And with previous years, do you mean since 2014?
Photo of Scott Murphy

Scott Murphy, Official Rep

  • 581 Posts
  • 59 Reply Likes
I have no set year in mind.
Photo of Mia Ikonen

Mia Ikonen

  • 295 Posts
  • 52 Reply Likes
This is what I don't understand. Is it possible to get a straight answer? Will it be done as in the PREVIOUS years (excluding for the year 2017 when everything was abnormal and the year was closed at the end of 2018)? Early on in Pixoto’s history the best of the year accolades were determined by the average of user's top 10 images in a category. This was criticized and debated many times for not giving an accurate result of the whole. Members who submitted a lot of great images but didn’t make it into the top 10 felt left outside. Since 2014 the rank has been determined, as far as I know, as a total of the points of Daily, Weekly, Monthly and Yearly Awards. According to my opinion this has been a fair way to get it done. What has been submitted and earned points from, no matter in which month although it would be late December is as valid as the points earned in the beginning of the year. The rank/order, badges and the accolades should though be issued the following year preferably latest in February because otherwise the score doesn’t validly show the order correctly. This happens because the best images are dueling more or less as long as they are on the site. And there are always new images submitted that change the ”old rank” in the process of determining ”All time best images”.
Photo of Mia Ikonen

Mia Ikonen

  • 295 Posts
  • 52 Reply Likes
Scott, please respond to our questions, thank you. I think many of us are feeling confused and a bit worried about this "radio silence". As I wrote in another thread this is the golden hour for Pixoto to show that the difficult last year is behind us and that a new, precise way of functionality will take place.
Photo of Mia Ikonen

Mia Ikonen

  • 295 Posts
  • 52 Reply Likes
Just want to point out that this subject isn't marked "In progress" in the gallery view, although the text can be seen on the first question of the thread. Is that the reason we don't get any answers? Scott, this thread contains many really important questions and we are all waiting for answers!
Photo of Mia Ikonen

Mia Ikonen

  • 295 Posts
  • 52 Reply Likes
Hi Scott! We are all still waiting... Please read thoroughly the whole thread and answer all our questions, thank you.
Photo of Scott Murphy

Scott Murphy, Official Rep

  • 583 Posts
  • 59 Reply Likes
I want everyone to understand that I did not come up with my own way of assigning the missing 2017 badges. I used the previous logic that was used for 2016 and as far as I am aware, was used for the preceding years as well.

It is my understanding that the way it works/worked is it totaled all the award points someone has earned for that particular category (excluding annual awards and awards for images that have been deleted).

If you would like to analyze how a particular category would have ranked with or without annual awards, I am more than happy to display the placement using both strategies.

If you are unhappy with how badges are/were assigned, I am more than happy to put up a new methodology for vote. In the next few months I plan on putting issues up for vote by the community.

I would say the target for 2018 badges should be March.
Photo of J-Lo

J-Lo

  • 55 Posts
  • 2 Reply Likes
I’m curious...how many places for each category are given out each year?
Photo of Jasenka

Jasenka, Official Rep

  • 17878 Posts
  • 1287 Reply Likes
Jennifer, top 20 accolades are assigned if there was at least 1000 images in category during that year.