Glitch on the ratings system

  • 3
  • Problem
  • Updated 10 months ago
  • Solved
Today an unusual number of movies had a drop in its ratings. Movies with 7.1 dropped to 7.0, movies with 7.0 dropped to 6.9 and so on. And I mean a lot of movies, which obviously points to a glitch, while there were almost no movies with crescent ratings. I know because I have a list of almost 2000 movies and I am constantly observing its ratings movements. Please, take a look and try to fix it. Thanks and sorry for my english.
Photo of Angelo Pilla

Angelo Pilla

  • 184 Posts
  • 273 Reply Likes
  • frustrated

Posted 11 months ago

  • 3
Photo of Ed Jones(XLIX)

Ed Jones(XLIX)

  • 22871 Posts
  • 27149 Reply Likes
MUCH ADO ABOUT NOTHING
Photo of Marco

Marco

  • 1628 Posts
  • 2072 Reply Likes
It's been quite a while since that film was in the Top 250: https://250.took.nl/title/tt0107616   :)

Photo of Angelo Pilla

Angelo Pilla

  • 184 Posts
  • 273 Reply Likes
Today was the worst day of all. FIVE movies from my list of classics have fallen from 7.0 to 6.9. It usually happens with five movies a YEAR, not a day. This is terribly serious. To me it is a glitch that the technical team cannot fix. Or don't care. Anyway, IMDb is going to an end. And I mean it. Soon there will be no more classics above 7.0.
The movies:
Treasure Island (1950) 6,085 votes
Spider Baby or, the Maddest Story Ever Told (1967) 5,407 votes
Harper (1966) 7,530 votes
Thoroughly Modern Millie (1967) 5,443 votes
Up in Smoke (1978) 33,804 votes

(Edited)
Photo of bderoes

bderoes, Champion

  • 2239 Posts
  • 3683 Reply Likes
Below find the 24 drops that occurred today in my 276-title sample. I've added a column to show the number of changes since ~5Aug. I found no changes on the list yesterday. Please see my prior post for links to the spreadsheet and the lists.

Please recall that this thread began as a concern about middle-rated (7.0) films, not about the top 250 films.

Let's take the 1932 film Wonder Bar (tt0026007) as an example.
It has dropped twice since 7Aug from 6.9 to 6.7, with no change in the number of votes (536), which could be from the number of lost votes matching the number of gained votes.
On 7Aug, it had a rating of 6.9, which could have meant an average as low as 6.85.
On 14Aug, it has a rating of 6.7 (it dropped 0.1 on a day in between), which can mean an average as high as 6.749.
So it dropped at least a full 0.1 points, with 536 votes. In other words, it lost at least 53.6 "ratings points" (if the average were unweighted).
Those lost "ratings points" would need to be from either multiple voters downscaling their vote, and/or some sort of IMDb intermittent update of whose votes are counted more heavily in the weighting. ...or... multiple changes in the calculation itself???
A title with this few votes seems an unlikely candidate for the number of vote changes required to drop so much in such a short period of time. And yet there are 25 titles in my sample that have also experienced a drop since ~5Aug. 94 titles total from the 276 have experienced 116 drops.
I hope IMDb will soon explain what is happening. 
Meanwhile, I'm going to export my ratings, in case hacking turns out to be the cause.


(Edited)
Photo of sunofabeach78

sunofabeach78

  • 27 Posts
  • 49 Reply Likes
"So it dropped at least a full 0.1 points, with 536 votes. In other words, it lost at least 53.6 "ratings points" (if the average were unweighted).
Those lost "ratings points" would need to be from either multiple voters downscaling their vote, and/or some sort of IMDb intermittent update of whose votes are counted more heavily in the weighting. ...or... multiple changes in the calculation itself???"

I, too, think this is what's happening: IMDB is making changes to its "regular voters" basis, by removing and adding members to it, and/or is changing the calculating formula that uses to determine the weighted average.

"I hope IMDb will soon explain what is happening."

I really hope that too, but i seriously doubt it... 

Photo of Michelle

Michelle, Official Rep

  • 13225 Posts
  • 10570 Reply Likes
Hi All -

Thank you for all your additional examples and comments.

Our voting system is constantly being improved and the formulas used to calculate the ratings and charts are tweaked regularly in order to improve our ability to detect and defeat ballot stuffing and other attempts to influence the results.

Weighted ratings are frequently recalculated, so when the formula is changed, the ratings may also change even without the addition of many votes. The changes you are seeing are normal and result from this periodic fine tuning of the weighing mechanism.
Photo of Ed Jones(XLIX)

Ed Jones(XLIX)

  • 22871 Posts
  • 27149 Reply Likes
I think I have said that already.
Angelo................The Sky is Not Falling!
They are implementing worldwide weather control.
That is bound to take a little bit of tweaking over a specific period of time.
It's not a flip a switch and it's fixed thing!
They are fooling around with the tweaks and seeing the results of those tweaks and going.........WELL THAT didn't work! Lets try this.
Give it time.
Following this during this tweaking period will just drive you nuts.
Wait for the end result when the numbers stabilize.
That may be in a day or 6 months or a year. Who knows!
Photo of Marco

Marco

  • 1628 Posts
  • 2072 Reply Likes
Following this during this tweaking period will just drive you nuts. Wait for the end result when the numbers stabilize.

I think that's the best (and only) thing to do. We'll see in a few weeks what the situation is.
Photo of Marco

Marco

  • 1628 Posts
  • 2072 Reply Likes
Following this during this tweaking period will just drive you nuts. Wait for the end result when the numbers stabilize.

I think that's the best (and only) thing to do. We'll see in a few weeks what the situation is.
Photo of sunofabeach78

sunofabeach78

  • 27 Posts
  • 49 Reply Likes
"Our voting system is constantly being improved and the formulas used to calculate the ratings and charts are tweaked regularly in order to improve our ability to detect and defeat ballot stuffing and other attempts to influence the results."

So, all these years there was BALLOT STUFFING for all these great/classic movies (Gandhi, Groundhog Day, The Maltese Falcon, Jaws, Grapes of Wrath, Les Diaboliques, The Exorcist etc), so they had to have HUGE falls and be removed from the Top 250.
And they had to be replaced by Indian movies only, because as we all know, Indian movies are the only movies in the world that ballot stuffing is not happening in IMDB.
Yeah, right...

"The changes you are seeing are normal and result from this periodic fine tuning of the weighing mechanism."

Even if this is the case and it's just a "fine tuning", then how come the vast majority of all categories of movies have both rises and falls (but mostly falls, huge in some cases) and then there is ONE category, Indian movies, that has only rises, huge in some cases?

You call this "fine tuning"?? I think it's better called "FAVOURITISM".
Photo of honolulu styles

honolulu styles

  • 66 Posts
  • 100 Reply Likes
Today.. 178 changes.
Non us or uk movies go only up.
Movies in the chart for 20 years losing 15 places in a single day..but yeah it's normal.
By the way, somebody not from India has ever know the movie "sholay"? I think you heard that name only in this list. None of the critics in the world know about that.
But hey it's better than Donnie darko and jaws and a bunch of classics for IMDB.
Photo of honolulu styles

honolulu styles

  • 66 Posts
  • 100 Reply Likes
The answer from the staff confrim what I said in the past days.

The pressing buttons to make movies goes up and down with their ratings. In the top 250 it has a major effect than with other movies. I think this maneuver is meant for the top 250 to have more Indians movies throughout the chart, and then the effect of changing algorithm change the rating of other movies as well.

I mean. Before august there were 3 movies around #80 spot, Andhandhun around #160, Rang de Basanti around #220 and then usually pop up some movies from #240 to #250.
Now i think they want a perfect distribution with 3 idiots reaching #60 andhandhun and rang de basanti going around #110 and #160, gangs of wasseypur around #200 pk aroun #210 and so on.

In this way you have Indian movies around every position. All the other ratings of all other movies are suffering these changes.
When they reach the point they want, they will stop to adjust manually the chart.

I'm an engineer, i know every kind of formula. IMDB is raping numbers in this way.
Photo of Angelo Pilla

Angelo Pilla

  • 184 Posts
  • 273 Reply Likes
And the disaster continues. Three more movies from my list of classics have dropped from 7.0 to 6.9 only today:
The Desert Fox: The Story of Rommel (1951)
The Hitch-Hiker (1953)
Testament (1983)
(there are many other drops of course, but I am only monitoring the 7.0/6.9 ones)
Photo of Ed Jones(XLIX)

Ed Jones(XLIX)

  • 22871 Posts
  • 27149 Reply Likes
No disasters
Photo of bderoes

bderoes, Champion

  • 2239 Posts
  • 3683 Reply Likes
Link to my prior data post.

Among my sample of 276 titles, today 33 dropped and 1 gained.
14 of the drops had dropped previously (highlighted in yellow).
One title, Hollywood Revue of 1929, dropped from 6.6 to 6.3; it's also had the most drops. The others were all 0.1 drops.

continued (Showing My Favorite Brunette on both pieces to confirm I didn't skip something):


Links:
Spreadsheet:
https://docs.google.com/spreadsheets/d/1y4PBDfYnk1rWIIC_aFoaC0g41Buo1XWbSTK4rPUVbh0/edit?usp=sharing
List of full sample:
https://www.imdb.com/list/ls048810523/
List of the drops:
https://www.imdb.com/list/ls048878764/
Since ~5Aug, among my 276-title sample,
110 titles have dropped, with a total of 149 drops.
7 titles had an increase in rating; 1 title increased then decreased back to the original average.

The Hitch-Hiker gained 31 votes since 7Aug.  I'm thinking film studies class.
(Edited)
Photo of Chris

Chris

  • 59 Posts
  • 130 Reply Likes
The one thing IMDB had going for it was consistency before the Amazon behemoth took it over.  Now who knows what they're going to change next.  You might be fine with that they're doing right now with the list feature, interface, the rating feature, etc.  But just wait until they change something you've always like then you'll be changing your tune.

This latest rating "tweek" puts less faith in IMDB's features not more.  People want reliability not drastic arbitrary changes just the sake of changes.  Right now I don't believe any score on IMDB reflects anything if they finally found after decades that something was wrong or should have been improved.  Maybe Shawshank isn't even a top 10 film for all we know.  They might find some new algorithm that shows that IMDB users don't even rate it a top 100 film in a few years.  Who knows at this point its anyone's guess.  I know Rotten Tomatoes has been pretty consistent and honest with their critics average ratings at least.
Photo of bderoes

bderoes, Champion

  • 2238 Posts
  • 3682 Reply Likes
As Amazon.com subsidiary (1998–present)
In 1998, Jeff Bezos, founder, owner, and CEO of Amazon.com, struck a deal with Needham and other principal shareholders to buy IMDb outright for approximately $55 million and attach it to Amazon as a subsidiary, private company.
IMDb - Wikipedia
Photo of MikeTheWhistle

MikeTheWhistle

  • 846 Posts
  • 1134 Reply Likes
I just looked at RT's top 100 movies of all time. I don't like that list either. Can I please have a #3 with fries?
Photo of Ed Jones(XLIX)

Ed Jones(XLIX)

  • 22871 Posts
  • 27149 Reply Likes
No Pickles or Onions, with Bacon please. Easy on the ice!
Photo of Jeorj Euler

Jeorj Euler

  • 7412 Posts
  • 9825 Reply Likes
They have already downgraded things that I did not like being downgraded, so I'm spent, so to speak. IMDb was rolling strong for the first eighteen years after Amazon and IMDb partnered. We could say that it is still rolling strong but yet in a completely different way from the earlier trend.
Photo of MikeTheWhistle

MikeTheWhistle

  • 846 Posts
  • 1134 Reply Likes
Look at it this way, IMDB decided that the new market offered better profit than the old market.  Dec 2018 articles indicate they very much want to expand into India. It took about 6 months to cause the impact and I think what happened is "new" "regular" raters expanded causing the existing raters weights to be lowered (explains why no vote change but lower rating).

What they did is a common occurrence for companies when they try to expand into a new market that is different than their current market. Maybe it will be good for imdb, or maybe it'll be like when Starbucks wanted to expand into Australia and it failed miserably.

There are other top 250 lists, so one just needs to find one that fits them. Perhaps imdb will do something with their top's lists to help accommodate those that have been their loyal customers.
Photo of Chris

Chris

  • 59 Posts
  • 130 Reply Likes
This reply was created from a merged topic originally titled IMDB ratings issue.

Hello, there seems to be an issue with some titles on the IMDB.  See below.  The title was a 7.1 user rating as you can see by the arithmetic mean but is now a 6.1 just a few days later.



Other titles with a strange ratings discrepancies:
https://www.imdb.com/title/tt0019788/ratings?ref_=tt_ql_op_4
https://www.imdb.com/title/tt0006333/ratings?ref_=tt_ql_op_4
https://www.imdb.com/title/tt0004181/ratings?ref_=tt_ql_op_4
https://www.imdb.com/title/tt0018618/ratings?ref_=tt_ql_op_4
https://www.imdb.com/title/tt0004707/ratings?ref_=tt_ql_op_4
https://www.imdb.com/title/tt0018440/ratings?ref_=tt_ql_op_4
https://www.imdb.com/title/tt0012675/ratings?ref_=tt_ql_op_4
https://www.imdb.com/title/tt0024067/ratings?ref_=tt_ql_op_4

And many others.  Not sure if something was accidentally deleted or added to the rating algorithm.  I believe this was a recent change/glitch.  Seems to be on a lot of silent films but other films as well.

Photo of Péter Kaszás

Péter Kaszás

  • 16 Posts
  • 38 Reply Likes
@Chris, you mean people from the US don't have access to entertainment from other countries, right? Because I guess they are not really interested in movies from other countries.

But rest assured, other countries (mainly EU) do know whether a very good movie was released in Mexico, Poland or South Korea. In the era of Internet, i wouldn't say something like people don't know about movies simply because their cinemas don't play them.
Photo of Ed Jones(XLIX)

Ed Jones(XLIX)

  • 22871 Posts
  • 27149 Reply Likes
Ever hear of a language barrier? Also there may also be no granted rights. I have tried to view certain UK titles only to be denied "ON" the internet, that streaming rights were not available in your location!
Photo of Chris

Chris

  • 59 Posts
  • 130 Reply Likes
One thing that they may also be doing is purging the really old votes with accounts that haven't been active in a really long time.  Perhaps assuming either new accounts were created or they died of old age or something along those lines.  Which is why you're seeing mostly the older films shrink in ratings over newer films.
Photo of Chris

Chris

  • 59 Posts
  • 130 Reply Likes
One thing that they may also be doing is purging the really old votes with accounts that haven't been active in a really long time.  Perhaps assuming either new accounts were created or they died of old age or something along those lines.  Which is why you're seeing mostly the older films shrink in ratings over newer films.
Photo of bderoes

bderoes, Champion

  • 2238 Posts
  • 3682 Reply Likes
Chris, 
While it's certainly possible that IMDb is purging old accounts, my 276-title sample does not support that conjecture. Only 1 title has lost votes between 7Aug and 21Aug. Lots of titles changed rating without a visible change in votes (of course, it could be that a new vote arrived to replace a purged vote). Here's a summary of changes since my last data post on 15Aug. The summary is copied from this list description: 7Aug2019 Titles that dropped since ~5Aug, which I've been updating daily.
16Aug: 3 drops
17Aug: 16 dropped; 1 rose: Gilda Live (1980)
20Aug: no changes since 17Aug
21Aug: 11 dropped; 3 rose: Night Nurse (1931), Ma and Pa Kettle at the Fair (1952), The Kettles in the Ozarks (1956)

Here's the spreadsheet so you can view the change (and lack thereof) in votes.
From the spreadsheet (currently sorted by the change in wtd avg since 7Aug), we have 100 titles down, and 8 titles up.
25 of the drops had no change in votes, as did 3 of the rises.

Feel free to make a copy of the spreadsheet to play with the data yourself:
File > Make a Copy.
Photo of Chris

Chris

  • 59 Posts
  • 130 Reply Likes
About how much longer is this fine tuning going to go on for?
Photo of bderoes

bderoes, Champion

  • 2238 Posts
  • 3682 Reply Likes
As you can see by the official replies, no estimate has been provided. Since this thread has been labeled "Solved", I doubt that we'll get another official reply. 

I'd guess this could be a perpetual tuning. If vote-stuffers catch on and change their strategies, the calculations will need to adapt.
Photo of Chris

Chris

  • 59 Posts
  • 130 Reply Likes
So I guess we'll never know what IMDB voters rate movies ever again. *shrug*
Photo of bderoes

bderoes, Champion

  • 2238 Posts
  • 3682 Reply Likes
We never did know. Ratings are always weird. Your 8 is my 5, and we (presumably) gave honest appraisals at the time. Factor in the people who always vote 10, or the vote-stuffers, and it's a very unreliable bag. If you really care about the votes on a film, the histogram is more instructive than the average, however it's weighted. No one statistic can describe a data set.

It's just like ratings on Amazon. I don't just trust a 4 average. I want to read the super-negative reviews to see if there's a legitimate common complaint. (Unfortunately I often filter out the averages lower than 4, and sometimes that's caused by silly things like people complaining the package never arrived.)
Photo of Chris

Chris

  • 59 Posts
  • 130 Reply Likes
Yeah but we had some idea. When some films drop a full point overnight then you gotta scratch your head lol
Photo of MikeTheWhistle

MikeTheWhistle

  • 846 Posts
  • 1134 Reply Likes
Is there a link on a title's page that's for the page having ratings/votes info.  The one where you add /ratings? I've looked but don't see a link so I have to type it every time which stinx.
Photo of bderoes

bderoes, Champion

  • 2238 Posts
  • 3682 Reply Likes
Click on the vote count under the average rating:


and if you're using Reference mode:

which you could have found by using your browser's Find function.
Photo of MikeTheWhistle

MikeTheWhistle

  • 846 Posts
  • 1134 Reply Likes
TY. Funny, or sad, how things right in front of you get walked into. Didn't see it and I thought I clicked on everything.
Photo of Ed Jones(XLIX)

Ed Jones(XLIX)

  • 22871 Posts
  • 27149 Reply Likes
Mike, It's even simpler than that. Below the 6.9 Rating is the # of votes (8071). It's a highlighted link that goes to that page.
(Edited)
Photo of honolulu styles

honolulu styles

  • 66 Posts
  • 100 Reply Likes
By the way guys..it's Sunday..nobody at works at IMDB so today there won't be any strange movements in the chart.
That's proving that is just a manual maneuver and a manual adjustment.

I was wondering to create our own site like ratings.com or something like that. Using a formula that avoid ballot stuffing or commercial driven manipulations. Only problem is become popular worldwide..
Photo of Angelo Pilla

Angelo Pilla

  • 184 Posts
  • 273 Reply Likes
And after 4 days or so, the tragic glitch on the ratings is back. Today the movie Nothing Sacred dropped from 7.0 to 6.9.
Please note that I am monitoring only the movies which are on my classics list and have dropped from 7.0 to 6.9.
The tragedy is way bigger than this. 
Photo of sunofabeach78

sunofabeach78

  • 27 Posts
  • 49 Reply Likes
Angelo, it's very sad for me, too, to see all these old titles (many classics included) having rating drops. But it's NOT a glitch, it's IMDB's recalculation. It's DELIBERATE action from IMDB. And i don't think they will reach the 1.0 rating, that is not going to happen. 
Photo of Jeorj Euler

Jeorj Euler

  • 7412 Posts
  • 9825 Reply Likes
In general, extrapolation does not always yield an accurate forecast. A hypothesis is not as powerful as a theory.
Photo of Ed Jones(XLIX)

Ed Jones(XLIX)

  • 22871 Posts
  • 27149 Reply Likes
bderoes, I never unfollow anything.

Photo of Péter Kaszás

Péter Kaszás

  • 16 Posts
  • 38 Reply Likes
@Ed, Oh my God, that looks something which is hard to keep up with :)
May I ask how many movies have you rated?
Photo of Ed Jones(XLIX)

Ed Jones(XLIX)

  • 22871 Posts
  • 27149 Reply Likes
Between Netflix and IMDb over the years and different accounts around 12,000. That includes TV
(Edited)