How I Rate Art on 16c

alphaking · December 26, 2020, 2:11pm

Rating art on 16c is something of a sore spot with some. I like the rating system but it needs more ratings to be useful. If only a few rate works then we’ll never get a chance to see some of the best of the best float to the surface because it will be based on the opinions of just a few.

So I thought I would take a moment to provide some transparency to how I rate art on the archive and would love to hear how others do too.

The rating system on 16c as it is right now is out of 10 stars. My system for rating would be the same if it was 3 stars, 5 stars, or 10. Ultimately it should follow a bell-curve with almost all art being a 5/10 because it’s average and that is not an insult. We aren’t all LD! or Somms but can see our progression against the masters if we are honest. Much of the art I produce is a 5 and it’s a solid benchmark to work against. If I doubt a piece I am looking at I almost always give it a 5 and then rerate later if I change my mind.

I rate logos, rip, and ansi art and tend to stay away from ascii as I think it’s important to understand the medium before you critique it and I still lack a full understanding of ascii. Anything else such as jpgs or “hi Rez”, legos, or cross-stitching doesn’t belong on a text mode archive and I ignore them or rate them a 1. This is an admittedly spiteful preference and I tend to reserve 1s for anything released after the year 2000 and prior to that I ignore it. If I wanted to see poorly airbrushed 200x200 gifs of skulls I’d go to Deviantart.

I take four main pieces of criteria into account when I view ansi art. Composition, shading, lighting, and ansi aesthetic. It’s worth noting the “ansi aesthetic” is something of a nebulous concept but one I find most artists in the scene understand. It’s the difference between block art and pixel art and very deserving of its own post.

There is a fifth dimension but that is harder to articulate in that I do consider if an artist brings something totally new to the scene and I will take that into special consideration.

One note, I do not take the publish time of the piece into account. I used to do this and have been trying to rerate those pieces. It’s not that I don’t respect what people did in 1994 for ansi. However, the only way to have a fair and honest rating system is to make sure you rate people against the same criteria and making an exception for work done in 1994 means everything gets skewed.

My rating for art is as follows:

1 Star: The artist clearly does not understand the medium and whatever it is I am seeing is almost completely incomprehensible. Not much gets a 1 star but if it does it means I wish it could be deleted from the archive for the betterment of all ansi.

2 Star: The artist is learning ansi or doesn’t yet understand the ansi aesthetic. No half-blocks are used to smooth lines, shading is over-used, and the intention of the artist is barely understood.

3 Star: The artist has shown they understand the ansi aesthetic but struggle to execute on any of the four criteria. Proportions are totally off, lighting and shading are incorrectly applied, composition is not existent. A font is only barely legible and not artistically-so.

4 Star: The artist understands the ansi aesthetic but is still struggling to fully execute on their intention. This where we see a lot of artists over-use shade blocks, utilize composition that doesn’t fit the canvas at all, or fall into the floating-head* category. A font is well drawn and shaded but lacks cohesion across letters or the artist ran out of canvas.

5 Star: A truly average piece. Most art should get a 5. The artist understands the ansi aesthetic and has communicated clearly their intention in what they drew. I would expect mastery of half-block/side-block usage, decent shading even if lighting is off, only some over-use of shade blocks, and composition is forgiven if the piece simply “terminates with a logo*” or just stops without fully being realized.

I think the scene is too forgiving for art that the artist can’t finish so it terminates with a logo. That is just sloppy composition and will never get beyond a 6.

6 Star: The artist is clearly a practiced artist and is producing art that is intentional. There are many pieces that are made by artists that I do not like their style but I respect an ansi artist who understands the ansi aesthetic, shading, composition, and yet chooses to do something unique and different. In general pieces that get a 6 are simply better than most but lack in more than one of the main dimensions of criteria of composition, shading, lighting, aesthetic. The piece could be excellent but unfinished or rushed.

7 Star: Anything a 7 and above I consider produced by a real practitioner of ansi art and we simply just have to start nit-picking. Typically this is where we see composition play a larger part in criteria. The artist typically has a solid understanding shading, lighting, and the ansi aesthetic and one of them might be off while others off only slightly.

8 Star: Anything 8 and above I consider produced by an ansi master. All four criteria are well established and only one of them is off and typically only to a small degree.

9 Star: The piece is truly great. All four pillars of criteria are firmly executed. Only the smallest details are off. The only thing that separates it from a 10 is “Is this piece one of the best ansi of all time or simply just a fantastic piece?”

10 Star: Flawless. Not a single block could be made better. One of the best pieces of ansi of all time.

burps · December 27, 2020, 11:16am

You’ve got quite the methodology there but interesting to have some insight indeed. My own approach is somewhat similar. And quite likely everyone else’s too as most people with a substancial amount of ratings done do indeed have a bell curve mostly peaking at 5 or 6.

One thing is different however, I do take the year into account. A 9 in 1994 isn’t the same as a 9 in 2020. It was still top notch work back then and should be rated as such (as I see it). Who knows what will be produced in 10 years from now, the 9’s of today are still supposed to be 9’s. But anyway, that kind of small difference evens out in the averaging, as long as enough ratings are done.

Currently we could do with more ratings indeed, the data it generates is very useful but I guess that the reasons for rating to exist might be something for another topic But with regards to the how’s, currently only averages of 6 and higher are being used for automation. Casting lower rates is not very useful currently, but it would contribute to an overall good average per pack or per group for example.

hayny · December 29, 2020, 9:06am

Thanks for the insight! I always refrained from rating things because I didn’t know how other people voted, but now, it’ll be easier to contribute to the useful data.
(I admit I voted too high on one of my pieces once because I wanted to try out the feature without affecting other people’s ratings)

alphaking · December 30, 2020, 5:20pm

I think it’s fair to say if you are newer to ansi (less than 5 years experience) and you want to rate yourself then give yourself a 5, it’s likely you are average at best and then move from there. I continue to rerate my own work as I move along (usually I rate it lower as I go) with the hopes what I produce today is higher than what I did previously.

meatlotion · December 31, 2020, 10:49am

This is a great post and very glad to have read it. Being a new “artist” myself, I don’t have the experience or skill to rate art on a technical basis, when I rate art, I rate it purely on how it makes me feel.

This probably in hindsight has skewed ratings a little so I will try and refrain from rating like this in the future.

I usually rate art 5-6 as average though, I don’t think I’ve ever rated my own stuff, but I don’t want to say I haven’t as I don’t remember or know how to find out what I have rated.

If I were to collectively rate all my stuff as a whole, it’d probably not get higher than a 2 or 3 based on Ak’s rating system. This might actually be the kick I need to push myself to get better and draw more.

Thanks for explaining how you rate stuff, it’s really insightful.

CatX · January 5, 2021, 2:18pm

In some ways I feel like the current rating system is a bit too limiting… As we all know there are several different art mediums on this site (ANSI, ASCII, hirez, RIP, teletext…), and what determines whether an artpiece is good or not can often depend on very different things. Furthermore, whether it’s a logo, an object, or something more abstract can also merit different criteria when evalutating its quality.

I like how AK put it;

I think it would be interesting to expand the rating system to include individual ratings for various features. The ones AK mentioned (composition, shading, lighting, and ansi aesthetic) would be a great place to start. For example whilst a piece might have great shading and lighting, it might be lacking in the ansi aesthethic which would bring it down.

I imagine it looking something like this:

Composition:
  [*****-----] 5
Shading:
  [******----] 6
Lighting:
  [********--] 8
ANSI Aesthethic:
  [**********] 10
Resulting score (calculated automatically):
  (*******---) 7

These features would need to be different depending on the medium, for example ANSI aesthethic doesn’t apply on a hires image, and lighting might not apply to pure ASCII since it’s monochromatic.

Having a system like this would both make it easier to rate art, as well as being a source of feedback to us artists. In it’s current state, the rating system doesn’t really tell the us what could use improvement and what our strengths are, it only tells us how popular (or not) each piece is.
This would also open up the possibility of displaying average results from all pieces on an artists page, what they’re best at, their improvement over time, and so on.

Additionally, by taking the results from these category/tag ratings we could calculate a resulting score. This resulting score would be the same as the current rating system, making the two interoperable and easier to migrate between since the old data can simply be re-used.

CatX · January 5, 2021, 2:24pm

Please note that this is just a suggestion, or an idea rather, but I think it’s an interesting one to explore.

alphaking · January 5, 2021, 3:40pm

So I really really dig this idea. I can see a point of view that making ratings more complicated would deter people from voting but I think that is solvable. You could provide users a way to rate “in general” which if you rate a 5 on a piece then the different discrete scores just get the same score across the board. You rate a 5, you get a 5 in all categories. Then allow users to also rate in a more detailed way if they choose which then gets a new calculated score as you put here. I would really like better feedback on my art and if I got a 4 that doesn’t say much other than they didn’t like it much. But if I got an 8 in aesthetic and 2 on shading then I’d know where I need to improve.

I would reduce things down to three categories if we went this route to keep it streamlined. Composition, Aesthetic (specifically to the medium, doesn’t need to be ansi as ascii or petscii have their own aesthethic), & Originality (which I think really matters for some of the pieces I rate low when I should rate higher for their interestingly new way to present).

alphaking · January 5, 2021, 3:42pm

Also worth posting there are formal methods for art critique. The most common is Barrett’s Principles of Interpretation.

Barrett’s Principles of Interpretation

Artworks have “aboutness” and demand interpretation.
Interpretations are persuasive arguments.
Some interpretations are better than others.
Good interpretations of art tell more about the artwork than they tell about the critic.
Feelings are guides to interpretations.
There can be different, competing, and contradictory interpretations of the same artwork.
Interpretations are often based on a worldview.
Interpretations are not so much absolutely right, but more or less reasonable, convincing, enlightening, and informative.
Interpretations can be judged by coherence, correspondence, and inclusiveness.
An artwork is not necessarily about what the artist wanted it to be about.
A critic ought not to be the spokesperson for the artist.
Interpretations ought to present the work in its best rather than its weakest light.
The objects of interpretation are artworks, not artists.
All art is in part about the world in which it emerged.
All art is in part about other art.
No single interpretation is exhaustive of the meaning of an artwork.
The meanings of an artwork may be different from its significance to the viewer. Interpretation is ultimately a communal endeavor, and the community is ultimately self- corrective.
Good interpretations invite us to see for ourselves and to continue on our own.

Barrett, Terry. (1994) Criticizing Art: Understanding the Contemporary. Mountain View, California: Mayfield Publishing Company.

warpus · January 7, 2021, 5:59pm

I seem to rate using an algorithm that puts an emphasis on factors like “How much effort was put into this?”, “Does this look good?”, “Is this interesting?”, “Was the initial concept worth pursuing?”, etc.

It’s not at all a formal system, but the way I see it… if I came up with a formal system, it would be an attempt to encapsulate everything that’s ever been done in the medium. 1. I don’t have the time or patience for such a project 2. It doesn’t seem like it would really affect the scores I give that much 3. What about new approaches that have not been tried yet? I don’t want to revisit my framework and update it each time I see something new or something I missed.

burps · January 7, 2021, 7:23pm

The current system was supposed to be quite simple. Much like IMDB’s scoring system. You can have movies and TV-show in all sorts of variations, but still it’s easy to score on a scale from 1 to 10 whether you like it or not.

And quite frankly, I wonder who would go through the effort of rating different aspects within a single piece, some aspects might indeed not apply to all items. Lighting might be a thing in a picture but for a font or an Amiga ASCII it might not be the case at all.
Nevertheless I see how you’d like more feedback on a certain piece, maybe there are other ways to get this, the votes are viewable in any case and you could also create a topic here and ask people for feedback. You might get lengthier and more in depth responses to that as opposed to a few scores. (who dares first)

If there was an army of raters out there, it would make more sense to me to have more in depth voting, but sadly we don’t. In practice there’s only a handful. But what they are doing is very useful as what it’s currently there is very useful. As a result it’ll pick decent opengraph images for artist pages for example. Or it feeds the featured content on the main page by selecting ‘above average scored items’. It helps surfacing interesting things that otherwise would go unnoticed, as very few people really dive deep into the archive.