It does have a decent archaeological and ethnographical museum with Byzantine, Hittite, Roman and Ottoman artifacts and a couple of interesting mosques that are worth a visit if you are here for the day.
It is, perhaps, less organized and a little bit more laid back than Ürgüp or Avanos and it's probably livelier than either in the season. If you're looking for somewhere to hang out for a while, meet people and maybe blend in a little then Göreme is probably the place for you.
This is the one place that everybody who comes through Cappadocia goes. It's a nicely packaged instant version of what the whole area has to offer and it's a good place to start.
The main street has numerous shops and workshops selling plain and decorated pots and plates and you can watch the potters at work using kick wheels, the design of which has remained unchanged for generations. Many of the workshops will encourage you to have a go yourself. It's harder than it looks.