Non core members can do this!
Here's a brief tutorial:
Click on an icon
In the corner there is a box that says "Folder options"
Click on "icon"
Click "Change icon".
You have an option to change the icon/upload a PNG file for it. Usually, 100 x 100 or 50 x 100 works best. Sometimes you can use 100 x 200. Just depends on what you want.
Once you've changed it, click "save".
Usually Photoshop works best, but any other art program can too.
If you want FULL on customization (html backgrounds too), that's a core-only option, and I don't quite know how to do that
But custom folders are available to anyone.
(Saw a person asking about this on dA, I was sure there were others who were curious about the same thing. Please let me know what you think below, or if you have any questions!)
Here is the help page on dA for this as well, if you want more details:
(edited for numbers because dyslexia XD)