Welcome aboard the RoadTrip America Forums!
There are plenty of places to visit the entire length of both shores of Chesapeake Bay, although most of it is in Maryland. But let's start with Virginia and work north along the western shore. Just north of Norfolk is the Historic Triangle which includes Jamestown, Williamsburg, and Yorktown. Next up, hugging the coast at the mouth of the Potomac is Smith Point where you can grab a ferry over to either Tangier Island (where natives speak in a unique dialect) or Smith Island in the Bay. To continue on up into Maryland, you'd have to go west a bit to cross the Potomac on US-301. If you do so, there are a couple of places that are worth a visit. St. Marys City was the original capital of Maryland, but the real jewel of the western shore is Annapolis, the current capital. There you'll find a great city to walk through with interesting architecture and the U.S. the Naval Academy. You can also cross over to the Eastern Shore (Note the difference: Eastern Shore is capitalized, western shore is not.) of the Bay here using the Chesapeake Bay Bridge. (Another difference to note: the bridge at Annapolis across the middle of the Bay is the Chesapeake Bay Bridge, the bridge at Norfolk across the mouth of the Bay is the Chesapeake Bay Bridge-Tunnel.)
In any event working north from the southern end of the eastern side of the Bay, you'd have a number of small towns in the Virginia portion of the Delmarva Peninsula, with the first significant-sized town being the crabbing/oystering/fishing village of Chrisfield in Maryland. You can get to Tangier and Smith Islands by ferry from there as well. Working northward, I'd suggest that you swing to the east a bit and visit three special islands on the Atlantic side of the peninsula - Chincoteague (home to wild ponies), Wallops (a NASA launch facility), and Assateague (a bit of un-developed beach, but crowded on weekends.) Swinging back to the Bay side, there's St. Michaels, home to the Chesapeake Bay Maritime Museum, and then you're up to the Chesapeake Bay Bridge again. A bit farther north is Chestertown, another great walking town especially if the schooner Sultana is in port.
AZBuck