First, make sure you're running the latest version of Audacity—1.3.12 (beta). The beta is a lot nicer than the "stable" 1.2, but the beta is still stable (in fact, more so!).
You have selected your audio first, right? There are two ways you could do this. Either select want you want to remove and press Delete, or select what you want to keep and press Cmd-T (Trim).