In Chapter 2 (p. 24), the book says "You can't rename or delete a catalog from within Elements." There is actually a way from within Elements to do these things. Just choose File > Catalog > Open to show the standard Windows file dialog. Then right-click on a catalog you want delete or rename, and choose Delete or Rename from the context menu.
Thanks to Colin Walls for this tip