By default, Photoshop shows black as foreground color, and white as background color. Unless you say otherwise, new PSD documents' background color is white. To create a transparent document, click on the "Background Contents" dropdown in the File > New dialog, and select "Transparent".
To change your swatch colors back to their default, hit D as keyboard shortcut on Windows or Mac OS X, and in any version of Photoshop for as long as I can remember! Or you can click the swatch icon, as shown in the screenshot.
To invert the two current colors (foreground color becomes background color, and vice versa), hit the X keyboard shortcut, or click on the swap colors icon - as I did in the screenshot. Note: the hotkey appears at the end of the tooltip.
To pick a different color, click on one of the two color squares to launch the color picker. Drag the sliders, switch color models, pick a color value. If you like that color, you can save it for later: click on the "Add to Swatches" button. Click OK. To show the color palette, which includes your custom color swatches, go to Window menu > Color. To adjust your color, just start over!
Here's a tip when you fill an entire canvas with color, or fill a selection with a color. You can hit Ctrl+Backspace -Windows- or Cmd+Delete (Mac) to fill the selection, or entire layer, with the background color (which would be blue, based on the screenshot). To fill the layer or selection with the foreground color, hit Alt+Backspace instead (Option+Delete on Mac). Or you can memorize one of these hotkeys, and just hit X to invert the colors when needed :)