Firstly, AA = Anti Aliasing. Looking up aliasing and anti-aliasing on Wikipedia brings up a fairly confusing explanation, so instead of trying to get overly technical, I'll explain for you what my current understanding of AA is. Please note this is not "THE" official and only way of doing it or explaining it, and that I'm still learning about these things just like everyone else. This article is aimed at helping everyone access the conceptual idea of what AA is all about, and how to start using it themselves.
Basically, I would describe AA as;
"The extra "in between" colour pixels placed between steps in pixel lines/edges that create the illusion of a smooth curve (even though pixels are square)."
What makes this a lacking description is that it's very hard to explain what I mean by the "steps" in a pixel line or edge, and this is crucial to understanding where to add these extra pixels and where not too. Before I get into that, there is something more important to explain.
Automatic Anti-Aliasing is created by a program (such as a Photoshop's brush tool) and Manual Anti-Aliasing is created by hand (using the pencil tool). When you are creating strict Pixel Art, it is essential that you are not using any tool that creates Automatic Anti-Aliasing, otherwise, it's no longer Pixel Art, but rather a mix of Pixel Art and general Digital Art. This is because the computer will generate the "in between colour" pixels for you, thus why a brush tool's line is always so smooth compared to a pencil tool's line. When the computer guesses the AA pixels for you, it adds a LOT of extra colours and extra pixels, far more than I ever would doing it manually. This is one of the reasons that seasoned Pixel Artists can pick out a work that has automatic AA in it, because it's colour count will be huge.
"But we're here to learn how to use AA! I thought it was a good thing!" Well yes, but we must do manual AA, in order for it to continue to be classified as 100% Pixel Art (and to keep that colour count down).
So now we know that AA is to smooth edges, and that we need to use the pencil tool. Next we need to consider what is meant by "in-between colour". This is another part of my explanation that isn't a perfect and professional explanation, but rather an attempt to give people a rough idea of how to get started with the technique.
The easiest way to explain is to use an example of a black curved line on a white background. If we consider one row of 4 to be one "step" and the next row up from that to be the next "step" of 3, then we can begin to see where we start adding AA pixels. Each time a step goes up or down, we add the in between colour (in this case grey) to the inner most part of each step.
You can also choose to take this further and figure out a secondary "in between" colour for a very smooth effect, but this is usually not necessary and is a matter of trial and error per situation. What you MUST always remember, no matter how much you AA, you should always leave at least 1px worth of space, so that the AA doesn't line up with the next step.
What becomes difficult though, is when the steps go 1px, 1px, 1px etc. If you add AA to these areas, you'll just end up with a thicker line, rather than a smoother one. Therefore I personally make it a rule to apply AA pixels only when a step goes from 2px to 1px or larger.
Picking an in-between colour gets much harder once you start talking about an orange area next to a green one, or a dark one next a light one, but as you can see it can make all the difference in high contrast edges.
Watch closely to see the lines with and without AA.
For beginners, I would suggest a kind of "cheaters" method, so you can start to get an idea for how the colours combine (although it's not to say you can't get away with using this to pick the AA colour, so long as you stick with that colour).
Start out with the two colours you need to find the in between colour of. Take one and apply it on top of the other using a layer (or similar depending on how your program works). Then drop that layer to 50% opacity to see how those colours combine. This may give you a murky colour compared to a nicely organised, hue shifted colour ramp, but for a beginner, it is certainly a very good colour to start with, and can help you get your head around how certain colours combine.
Be careful to pick the new colour and add it to your palette or another part of the work, because I find it important to delete that semi-transparent layer to ensure that it doesn't accidentally create a non Pixel effect later on by accident. This also helps to stop temptation to sample lots of extra colours.
Better methods for in-between colours include using the RGB and other complex colour options within the PS colour picker (and similar for other programs), and to use the colours from other areas of the existing palette (as you become more advanced with colour use, you can start using entirely different hues to create the AA for a section you might have thought had to be entirely green for example).
Example of a hue shifted colour ramp, and where you might use each colour when pixelling.
Sometimes, AA might not actually add to your picture, and so I personally add it on a different layer, so I can zoom back out and check to see if looks right by turning the layer visibility on and off a few times at a full view. If you're using Photoshop layers, you can turn them on and off by clicking on the little eye button.
I hope this provides people with an idea of what is meant by anti-aliasing, and how to go about starting to use it in their own Pixel Art works. If you find that this explanation is too basic for where you're up to, I would recommend searching Pixel Joint.com for their tutorials and explanations, or ask some of the Pixel Artist around dA about how they do it. I find that the best learning occurs when we try out lots of different ways of approaching the same thing, and there's certainly a lot that can be added to this explanation here.
