I've never heard the Concorde, but the B-52s make enough of a racket for me. It especially gets noisey when they're heading out for a mission. My entire house will shake and they just seem to keep coming. Try waking up to that at 2:00 am. You start having nightmares of planes accidentally dropping bombs in your backyard.
I'm lucky though, that I was born after the sonic booms. I remember going to my grandmother's house as a kid and all of her pottery and glassware was glued down to the shelves and much of it was cracked from the sonic booms. I can't even fathom that noise.