In Windows, the "file extension" determines which app opens a particular file. Behind-the-scenes, file names are followed by a period and some letters (like ".txt" for text files) and even numbers (like ".mp3" for MP3 files). Downside? Accidentally changing extension when renaming a file makes it un-open-able ("Windows can't open this file" error), until you restore the correct one. For this reason, recent versions of Windows automatically hide extensions.
To show file extensions, click on the start button, type "extension" in the search field, and click on the "Show or hide file extensions" link under Control Panel. Near the bottom of the box (without scrolling), uncheck the "Hide extensions for known file types" checkbox (visible in the next screenshot).
Notice that Windows uses the word "known": that's because even when file extensions are hidden, you might see them at the end of a file name if it is an "unregistered file type": a valid extension is considered unregistered if it isn't common, and if you didn't install a program that handles that file type (installed apps can -optionally- register themselves as a file handler). Click OK.
From this point on, you'll see a dot and an extension next to each file name on your desktop, in open dialogs, in Windows Explorer, etc. Just be careful when renaming a file (tip: hit the F2 key when the file is selected in a folder!) Make sure that you only type over the selected text (file name), and not to the right.
If you make that mistake, Windows will warn you by showing the "If you change a file name extension, the file might become unusable. Are you sure you want to change it?" Hit the escape key (Esc) to cancel and give it another try!