For a long time, Apple bundled the functionality to force quit an unresponsive app in Mac OS X. If the app is frozen and won't exit with the close button, or shows the colorful spinning wheel of death cursor for too long, you can "kill that process", to use Windows terminology. I'll also show you the Mac equivalent of the Windows task manager, where you can view background processes.
Click on the Apple menu (top left corner), and choose "Force Quit". If you hold down the Shift key on your keyboard, you will see "Force Quit [app name]", where Mac OS X shows the name of the app currently in the foreground.
If you Shift-click to Force-Quit, the foreground app will directly exit. If you click Force Quit without Shift key, the popup shown on the screenshot will open.
Tip: you can hit Command+Option+Esc to launch Force Quit Applications!
Now select the app you want to close, and click on the Force Quit button.
To view more than the apps currently running, open a new Finder window and select Applications on the left (or hit the corresponding Command+Shift+A keyboard shortcut!) Go to the Utilities folder, and double-click on the "Activity Monitor" app, what you might know as the "Task Manager" on Windows. The View menu lets you see your (user name's) processes, all processes, or an option in between. To kill a process, select it and go to View > Quit Process!