Welcome to the RTA Forum!

I don't have a lot of info on this area, but I suspect the reason you aren't finding any homes on the atlantic side is because Chincoteague Island is blocked from the Atlantic by Assateague Island, and nearly all of Assateague is a National Wildlife Refuge where development is prohibited.

I'd expect the weather to be cool, and you won't want to spend any time in the water, but I'd suspect on a typical winter day you wouldn't have any problems walking the beach, properly bundled up. I'd say your bigger concerns would be running into a storm on the days you are there, which could make it very unpleasant. Also, keep in mind, I suspect the vast majority of businesses that cater to tourists are going to be closed during your trip.

Must sees are a tough thing - as each person has a completely different idea about what that means - but if you want to do something a little different, instead of heading straight back to DC, you could look at heading down across the Chesapeake Bay Bridge/Tunnel and check out some of the interesting things around Virginia Beach/Norfolk or Williamsburg on your way back.