Windows shows the default folder icon (like the first one pictured right) for all your folders. Exceptions include special folders like your desktop, downloads folder, etc. But you can customize the folder's icon any time you want, using either an icon built into Windows, or an icon of your choice (which needs to be in the *.ico image format). In Details or List view, Windows shows folder icons. In larger views, a folder's "preview picture" can be shown: also customizable!
To change a folder's icon, right-click on it and choose "Properties". When the dialog opens, select the "Customize" tab and click on the "Change Icon" button at the bottom. That will open a new dialog, listing all the system icons contained in the "shell32.dll" library (see step 3 screenshot). Scroll horizontally to see more of them. If you like any of those, double-click on it and click OK.
Sometimes, the icons shown don't come from that library, and you'll only see a few, limited choices. In that case, erase the text inside "Look for icons in this file", and hit Enter inside that text field to load the default icons. To use your own icon, click on the Browse button and choose your icon file. Note that icon files can contain multiple file sizes of the same icon. In the case of Windows folders, the appropriate size will be used based on the current view.
I picked the website icon for the in 5 steps logo: it looks awesome in Details view (where 16 by 16 pixels icons are used), but it doesn't look so hot in Large Icon view - that's because I didn't create multiple sizes of the logo in my icon file. Here's an awesome tip: instead of an icon file for your folder, pick any program currently installed on your PC (where are your programs stored?)
Professionally developed applications include multiple versions of their icon, to look good in different folder views (program shortcut automatically use the program's icon, though that's customizable). Notice how my folder icon now looks great in multiple sizes: the program I picked includes various dimensions of that image, all built into the same icon resource. In some cases, you'll find that executables (*.exe files) or resource files like *.dll include MANY icons!