New Fake Ratings for Silent Films

  • 2
  • Question
  • Updated 2 weeks ago
  • Acknowledged
  • (Edited)
I noticed a few months back that overnight almost all silent films and many (if not most) films of the 1930s took dumps in their ratings. Many of these films dropped 2 points or more (e.g., going from a 7.5 to 5.5) overnight without a single additional rating vote. For some reason IMDb took it upon itself to manipulate the ratings of older films (not films admired by the comic-book set) to downplay their ratings popularity, especially as compared to newer films that feature super heroes and cartoon characters. IMDb's secret formula for ratings, not a simple arithmetic average, allows for any ratings number to be assigned to any film, based on the prejudices of the IMDb staff. Since your ratings are not based on what users think and vote, why even bother having ratings? Why bother showing graphs of ratings that only prove your manipulation of the final rating? In essence, your ratings system is not indicative of anything but what your staff thinks. You might as well bring in the "tomatometer."
Photo of drednm

drednm

  • 152 Posts
  • 163 Reply Likes

Posted 1 month ago

  • 2
Photo of Ed Jones(XLIX)

Ed Jones(XLIX)

  • 19993 Posts
  • 22662 Reply Likes
You are just noticing now!
IMDb did a wholesale readjustment of the weighted formula software.
Every title in the database was effected. Not just the older titles.
You can pretty much blame this on the titles from "ONE" country causing this.
It's the the second highest population in the world!

So this needs merging with all the other complaints.
Photo of drednm

drednm

  • 152 Posts
  • 163 Reply Likes
I noticed it when it happened a couple months ago. The "averages" have always been skewed oddly, but I've never seen such a large change overnight. Yes, they claim their "formula" is in response to "ballot stuffing" but their new "correction" took a major tolls on the films of the 20s and 30s because they tend to have less than 1,000 votes so the change in rating was greater. This does a huge disservice to older films by downgrading them unfairly.
Photo of Ed Jones(XLIX)

Ed Jones(XLIX)

  • 19993 Posts
  • 22662 Reply Likes
Oh, I agree....BIG TIME!
Photo of bderoes

bderoes, Champion

  • 1747 Posts
  • 2885 Reply Likes
Back around 7Aug, someone created a thread about ratings drops, so I created a list of sample titles (dated 1919 to 2017) to follow. It was quite a roller coaster, but by 21Sep I stopped posting changes to that list.
https://www.imdb.com/list/ls048810523/
I stopped because titles were wobbling (down/up/down) just by naturally being on the rounding edge.
But I kept watching privately. Even that got boring, so my last data set is from 8Dec.

Checking those 276 sample titles today, I find more changes than usual, but 13 of the 32 changes (since 21Sep) are INCREASES in the rating.

Of the 19 decreases, 10 are new compared with the 8Dec list. Many of those 10 look familiar as being among those that had been wobbling.

drednm, would you care to share your methodology for knowing that ratings dropped without new votes (and remember that people can change their votes without that creating a visible statistic - I just downgraded a 1934 film by 2 points myself this weekend), and also share which titles you observed?
Photo of drednm

drednm

  • 152 Posts
  • 163 Reply Likes
My methodology is called arithmetic. It's mathematically impossible for a film to drop even .5 in its rating overnight when it already has over 1,000 votes and hasn't upped the number of votes overnight. IMDB freely admits it plays around with the voting by excluding votes (even though the number of votes are shown). They apply whatever filters they choose to delete "ballot stuffers" so presumable they discard large numbers of high and low ratings. By doing this, the IMDB editors/staff actually become "ballot stuffers" themselves by blithely deciding which ratings to accept.Ironic, no?

It's no secret my main interest is in silent films and I used to watch the ratings closely. But a film that has an arithmetic average of 8.2 (with nearly 1,000 votes) but an overall rating of 6.4 tells you everything you need to know. They've basically discarded all the high votes and kept the low votes. Their secret methodology is trash the majority of old films. THE JOKER, surprise surprise has an arithmetic average of 8.7 and a rating of 8.6. No ballot stuffers?
Photo of bderoes

bderoes, Champion

  • 1744 Posts
  • 2871 Reply Likes
You have to collect data to perform arithmetic.
How do you know that the number of votes has not changed?
Can you please provide the URL for these titles you claim have changed so drastically?
Photo of drednm

drednm

  • 152 Posts
  • 163 Reply Likes
I don't "claim" anything. I'm stating that the ratings went down. I can't go back in time and show you that a film like WHEN KNIGHTHOOD WAS IN FLOWER had a 8.0 one day and a 6.4 the next. The film didn't get hundreds of 1 votes overnight. That would be ballot stuffing, wouldn't it. The film has an arithmetic mean average of 8.2 and a median of 9 and its rating is 6.4 with 959 votes. I defy anyone to do the math and come out with this 6.4 rating. It's rigged by IMDb Virtually ALL older films were demeaned by this new methodology overnight. A drop of that size is mathematically impossible without receiving hundreds of low votes in one day. That did not happen. And I'm not the only one who noticed this.
Photo of Ed Jones(XLIX)

Ed Jones(XLIX)

  • 19919 Posts
  • 22543 Reply Likes
What is the purpose of this diatribe?
What do you hope to accomplish?
What do you think IMDb will do?

Answer:
Nothing.....
A Change?
And finally.....Nothing.
Photo of drednm

drednm

  • 152 Posts
  • 163 Reply Likes
IMDb could restore the ratings, but I know they won't. I give IMDB a 1 on the tomatometer.
Photo of bderoes

bderoes, Champion

  • 1744 Posts
  • 2871 Reply Likes
So you weren't keeping records to substantiate WHEN the rating dropped.
You're basing your observation on memory, and/or just the logic that the weighted average is significantly lower than the arithmetic mean.
Yes, that happened in the August.
Here is the lengthy thread on GetSatisfaction with the daily details and reply by IMDb, with last reply on 23Aug: 
Glitch on the ratings system
And the histogram of the title you shared definitely fits the profile of a film impacted by the change in algorithm at the time.
https://www.imdb.com/title/tt0013750/ratings


Without your providing more data, your claim that the rating dropped overnight seems to be an assumption. I don't have any historical data on that title either. But based on my tracking the 276 sample titles, I don't believe that a recent significant adjustment has been made to the weighting algorithm. So I conclude that the change is not new, and dates back to August 2019 when several of us were observing and commenting on the changes (which were progressive) on the "Glitch" thread cited above. 

At the time I came to the conclusion that the change was appropriate, even though it negatively impacted the ratings of some of my favorites. Most films with such skewed ratings are indeed over- (or under-) rated, and IMDb is attempting to correct for that.

BTW, the Wayback Machine doesn't have a good copy of the Knighthood title page since Sep2017, and no copy of its ratings page, but back then the weighted average was 8.2 with 757 votes:
https://web.archive.org/web/2018*/https://www.imdb.com/title/tt0013750/

Photo of drednm

drednm

  • 152 Posts
  • 163 Reply Likes
Why would I have screenshots? Was I supposed to EXPECT this to happen? You screwed the ratings of most silent films with a hundred votes or more and you beefed up the comic book movies and cartoons.Yes it was likely back in August that this happened. I didn't say it was last week. By manipulating the ratings to satisfy IMDb staff and editors you became the biggest ballot stuffer of all. Weighted average my ass.

Just because I don't have photographic proof does NOT mean the ratings were not dropped. They were.

That begs the question: why bother having ratings for movies on the IMDb site when they basically reflect the taste and interests of the staff and not the users (even though they pretend they do)?

I used to follow the ratings, especially of the silent film restorations I have produced to see how they reflected the new availability via DVD and airings on TCM. Now the ratings are garbage.
Photo of bderoes

bderoes, Champion

  • 1744 Posts
  • 2871 Reply Likes
BTW, a "Champion" on GetSatisfaction is not an IMDb employee. We are just fellow users who got an extra badge for helpful replies on this forum. So the "You" in your prior message was not addressed to me. I have zero influence on such policy matters, and my statements are merely my own opinion.

I wanted to add this: Unfortunately for the Knighthood title, just over half of the 61 Top 1000 voters who rated it gave it 6 or less, with 14 1's:
https://www.imdb.com/title/tt0013750/ratings?demo=top_1000_voters

I believe their votes are part of the weighting. Note that their arithmetic mean matches their weighted average. You may want to vote for/comment on this Idea thread:
Top 1000 Voters should be expanded to Top 10%, or ...

Thanks for explaining that you are involved with restoring silent films, and thank you for those efforts! I grieve that many (most?) 40-something (and younger) people don't know who Cary Grant is, much less Marion Davies. 
Photo of drednm

drednm

  • 152 Posts
  • 163 Reply Likes
Thanks. I stand corrected on who is staff and who is a user.

Yes, the top 1,000 voter preference is another way to stuff the ballot, since they are more likely to be youngsters not interested in silent films. I think the barrage of 1 votes as shown, bears that out. I'll bet a demographic breakdown would show that.

There probably is no way to "fix" this, but the August dump is heartbreaking in that it further distances the work of many great artists by dropping their ratings to low levels.
Photo of bderoes

bderoes, Champion

  • 1744 Posts
  • 2871 Reply Likes
Photo of drednm

drednm

  • 152 Posts
  • 163 Reply Likes
Oh, and as for KNIGHTHOOD.... The precious Top 1,000 voters whose votes count more than anyone else's total 61 votes. 61 votes out of the 959 votes cast.  61 votes that helped drop the total rating from an 8.0 (if I remember right) to a 6.4 after the August purge.
(Edited)
Photo of Philip Perkins

Philip Perkins

  • 36 Posts
  • 57 Reply Likes
Here's some hard proof of how the "weighted average" means the editors can do whatever they want with ratings.Note the title of the list and the actual ratings as of today.

https://www.imdb.com/list/ls048909330/?ref_=tt_rls_1

Photo of bderoes

bderoes, Champion

  • 1734 Posts
  • 2853 Reply Likes
Indeed, this list was created Aug 3, days before the developers began tweaking the algorithm.

The good news is that "only" half the titles (14 of 27) now have ratings below the 7.5 minimum that the title of the list states was the criteria, and 1 still has 7.9, the max.

But looking at the now lowest-rated title, the algorithm might seem too heavily skewed:
https://www.imdb.com/title/tt0018413/ratings

If the arithmetic mean and the median are 7.8 and 8, why is the weighted average 4.0?

Apparently we're getting the WEIGHTED averages in the demographic breakdown, since the total is 480 votes, and the All Ages total is 480, with an "average" of 4.0, and the 45+ age group's average matches the Top 1000 exactly: 3.2. (When you look at the histogram for the 401 votes by 45+'s, their arithmetic mean is 8.0.) (Side note: we get no insight into the demographics of the Top 1000 voters; the chart immediately below remains the same when you view histograms for subgroups.)

Here's a screenshot from the Wayback Machine, June 2019 (479 votes):
https://web.archive.org/web/20190611191955/imdb.com/title/tt0018413/


The next lowest title:
https://www.imdb.com/title/tt0015312/ratings
Arithmetic mean = 7.5   Median = 8
213 IMDb users have given a weighted average vote of 5.5 / 10
Top 1000 Voters: 3.2 (19)
The Top 1000 voters don't have much credibility (to me) when over half gave rating 1, and none gave 2 or 3 but 30% (6 votes) gave 6-8 (however, I'm biased in that I can't imagine sitting through a film that deserves a 1, and I won't rate a film I can't finish):


I wish we could see the voting histograms for these Top 1000 voters. Do they vote harshly on everything? Or on all titles older than they?

The highest-rated title on the list (7.9)
https://www.imdb.com/title/tt0011117/ratings
has the approval of 4 Top 1000 voters (Arithmetic mean = 7.0   Median = 7.5).

So it's not that IMDb "editors" are doing what they want with the ratings. The algorithm is clearly tied to the Top 1000. But how much weight should 20 votes have out of 480, especially when 10 of the 28 1's come from them? Should they really be used to cut the score in half? (I'm referring to Slide, Kelly, Slide (1927), the first histogram I included in this comment.)


Photo of drednm

drednm

  • 152 Posts
  • 163 Reply Likes
The problem is that the cherished top 1,000 most likely are dominated by a small demographic group (young, male) that historically votes down old films (see the 1 votes) and for films they've never even seen. Why a vote from A is more valid than a vote from B defies logic, especially when the IMDb magic formula so obviously works against older films (as demonstrated above---thank you).

Ultimately, using such a convoluted and obviously biased ratings system raises the questions: what's the real aim here? Someone at IMDb clearly has an issue with old films having high ratings.
Photo of bderoes

bderoes, Champion

  • 1734 Posts
  • 2853 Reply Likes
No, I don't believe the IMDb algorithm is targeting low-vote-count (much less older) films. I believe their new algorithm just happens to have that side-effect.

I believe they are targeting films that have skewed histograms on the theory that any collection of opinions should have a Normal (bell-shaped) distribution. That, of course, depends on sample-size and the randomness of the sample taken. IMDb users are not a random sample. But a film with 480 votes should be able to stand statistical scrutiny.

Why they're targeting non-Normal histograms (again, my theory, I have zero inside information) is that many new films get a lot of positive and/or negative votes from biased people (apparently there are offers to get cash for votes?).

So I wonder if the Top 1000 users, whose opinions are now so valuable, have Normally-distributed histograms themselves. If they are willing to slam old films that are very difficult to find, is that their dominant vote pattern? If so, then Top 1000 should not be based on vote-count alone, but also on an analysis of their voting pattern.  

Slide, Kelly, Slide does have a skewed histogram. Perhaps a devoted William Haines fanbase gave so many 8-10 votes? But to let the algorithm cut the score in half based on 10% 1-2 ratings, and 20 Top 1000 voters with arithmetic mean 3.3, median 2, does not seem well-founded.

I know IMDb won't answer this with specifics, but I hope they'll discuss it properly behind the scenes.
Photo of drednm

drednm

  • 152 Posts
  • 163 Reply Likes
My opinion is they should have left well enough alone. But they didn't and they won't revert to the old algorithm. As I've said, all they've accomplished is creating a new bias-based rating system.