@mschoobs said:

I am sure I will get blasted for this

I sincerely hope that no one is blasting anybody. Everyone should rate shows any way they want to.

Better definition is required to get what you are after. What is the reason we have ratings? What makes a great vs good show? Who should get to decide that?

There is no such thing as a true show rating.

The things that make a good show versus a great show is entirely up to rater, and raters may differ on what those things are. Like you, I am not aware of any formal definition what the show rating is supposed to measure, but I believe there is an informal consensus. The average show rating represents what we, as a community composed of vastly different people, thinks of a given Phish performance.

Is there a “true show rating”? Yes, in the theoretical sense that (1) everyone who hears a show has an opinion about it, and (2) we can measure a numeric rating that represents a mix of all of our opinions. My work examines how we use those opinions (numeric ratings) to construct a summary measure that best captures what the community thinks. This, of course, moves us out of the theoretical realm and into the empirical.

Without objective guidelines for scoring you cannot have a true show rating. What we have is a popular opinion score based on subjective feelings.

Yes, the average show rating is a popular opinion score based on subjective feelings. But the key element of this research is that we can link show rating to the things we, as fans, profess to love about Phish. Do we like jamming? Do we like silky segues between songs? Do we like to hear rarely performed songs?

I believe the answer to all of these questions (and more) is, “yes.” If that’s the case, then we can connect the subjective show rating to objective (quantifiable) measures of show quality. If show ratings are correlated with quantitative measures drawn from the setlist, then we have established the empirical validity of show ratings.

My research question is, “Given all of the information we have on raters and ratings, is the simple mean rating of a show the best way to measure fans’ collective opinion?”

Trying to control opinions is making yourself the arbiter of truth. That is ok I am fine with it.

I absolutely NOT okay with controlling opinions and, in my numerous discussions with fans regarding ratings, I have yet to encounter anyone who is. Please, rate any show any way you want, according to any criteria you want. But I firmly believe that any one person's show ratings, and average show ratings, should be correlated with what happens onstage at a Phish show.