Peninsula State Park. You can take the car or the bike to ride throughout. There's a beach on Nicolet Bay along with camping, a camp store, they also have rentals for kayaks, small sailboats, etc. On the Lake Michigan side there's the Cana Island lighthouse. A great place for family pictures but it's out of the way and you'll need directions. Ask one of the locals in Baily's Harbor.
There's a Mariner's museum in Gill's Rock, that's on the way if you're going out to Washington Island. The ferry docks in Gill's Rock. There's Goats on the Roof of Al Johnson's restaurant in Sister Bay. Lots of shops for the ladies, antique stores, and places to eat & drink.
We've been going there for years, there's lots to see and do if you know where to look.
Gary