I got a go pro (3+silver) late last year, and bought a bunch of accessories. The ones I'd say are the most useful are the chest mount, the lightweight frame (don't need to use the waterproof case all the time), extra batteries and plug in charger and some kind of short pole (I am going to buy this one:
http://www.heyisiton.com/product/the-bobber-floating-hand-grip-for-gopro-hero-cameras-3/). I have a round lens cap too, but I don't remember how I got that. I don't think it comes alone, but as part of some set. It's nice to protect the lens when just using the frame rather than the waterproof case. I got the handle bar and roll cage mounts for mountain biking. The handlebar mount is smaller diameters, so if you want to mount on other parts of a bike, then the roll cage one is needed.
I asked for a bunch of these at x-mas (and purchased through amazon), and one on my list was the head band. I got one of the cheaper options for this, and it works fine. But the go-pro branded head band also has a quick clip mount that is only sold with the headband. If I had known that, I would have gotten that gopro headband instead of a knockoff to get the clip (the clip would work with backpack straps I think... that would have been nice).
If you're going to use it in water, then get the floaty back door.
I built a spring clamp mount using a cheap spring clamp from a hardware store and the tripod mount. instructions on a lot of DIY mounts are all over youtube (things like using collapsible mop handles which are way cheaper than the branded poles).
The biggest accessory on my wish list is a quadcopter like the DJI Phantom or Blade 350qx.
