[We would like to thank Paul Jakus (@paulj) of the Dept. of Applied Economics at Utah State University for this summary of research presented at the 2024 Phish Studies Conference. -Ed.]

As many of you know, the coding/architecture of Phish.Net is currently undergoing a major overall. While the ability to rate shows has been restored, we continue to study the ratings database with the goal of improving the accuracy (and credibility) of show ratings. This is the first in a series of four blogposts about that effort.

This post will focus the raw ratings data.

All analysis is based on an anonymized database downloaded on October 26, 2023.

This date allows us to sidestep problems associated with possible ratings shenanigans in the aftermath of the NYE Gamehendge performance. Ratings for 592 dates that were soundchecks, TV appearances, side projects, false dates (on which no show was performed), and shows for which there is no surviving audiotape were deleted. The final data consist of 343,241 ratings from 16,452 users for 1,736 shows. The shows ranged from Phish’s first, December 2, 1983, through October 15, 2023.

Show ratings are tightly concentrated at high values.

A smoothed distribution for show ratings (the simple average of individual ratings) appears below. Ratings have a heavy concentration of shows packed in between 4.0 and 4.8, and a long tail of relatively fewer shows spread across the relatively low ratings (<4.0).

· The mean show rating was 3.852 (median = 3.920).

· Some 806 of the 1,736 shows (46.4%) are rated as 4.0 or higher.

· If ranked in order, the difference in rating between the tenth highest-ranked show (Fukuoka 6/14/00, 4.636) and the show ranked #106 (Alpharetta 8/3/18, 4.539) is less than 0.1 points.

· The next difference of 0.1 points takes us all the way down to show #262.

Individual Rater Data Are Also Highly Concentrated

Every time you rate a show, it generates a new entry in the dataset consisting of your user name, the show date, show rating, and time of rating (to the nearest minute). The histogram below depicts the total number of show ratings, by rating value. Fewer than 10% of all show ratings were a ‘1’ or a ‘2’, whereas over 90% were a ‘3’, ‘4’, or ‘5’. The highly-skewed distribution of individual ratings is what drives the skewed distribution of mean show ratings.

Our 16,452 raters constitute represent a fraction of all registered Phish.Net users. Among this group,

· The average number of shows rated was 21 (median = 4.)

· Roughly 80% of all ratings were provided by about 3,300 people. This means that 17% of all raters, and only about 4% of all registered Phish.Net users, provide most of the ratings.

The individual data also reveal interesting patterns; the table below reports descriptive statistics of the individual ratings, by numeric value.

· Most people have never rated a show as a ‘1’, ‘2’, or ‘3’.

· The average person has rated about one show as a ‘1’ or a ‘2’, about three shows as a ‘3’, six shows as a ‘4’, and 10 shows as a ‘5’.

· The most ‘1s’ by a single rater (Rater A) was 393, while the most ‘5s’ by a single rater (Rater B) was 947.

Before jumping all over Rater A, let's learn a bit more about their full ratings profile. It turns out that Rater A has rated 939 performances and has used all five possible ratings, including nearly 300 shows rated as a ‘4’ or a ‘5’. On the other hand, Rater B has rated a total of 948 shows, 947 as ‘5’ and one as a ‘4’. Which of these Raters, A or B, discriminates among Phish performances?

Raters A and B are only two (extreme) examples from the 16,452 people in the database, but they help illuminate issues regarding the “bombers & fluffers” debate. The next post [link?] will dig a bit deeper into different rating behaviors, and how those behaviors may affect the accuracy of show ratings.

If you liked this blog post, one way you could "like" it is to make a donation to The Mockingbird Foundation, the sponsor of Phish.net. Support music education for children, and you just might change the world.