We have a place over on Spider Lake. I know where your grandparents place used to be just before you go into the river from Rest lake. I have been coming up to Manitowish since I was 16, 46 years ago. Beautiful area as you noted, has just about everything including some of the highest property values in the Northwoods. We were fortunate and bought our place in 2014 when the market had hit bottom. The area is very active between Memorial Day and Labor Day, then everyone leaves until next season. it's kind of refreshing when everyone leaves and the quiet returns but you would have to live up like you do to understand that, it's not for everyone.
Its nice to have a place where there is always snow to be able to enjoy winter by getting outside. We will permanently retire here in 4 years and currently share time between MW and Oconomowoc, WI.
St. Germain area is very nice as well. I have been to more places up here (Northern WI and the UP) by snowmobile that it is interesting to visit them in the summer by car and see them in full summer mode when we are not up on the MW chain.
Enjoy the melt and and the summer months, time seems to fly these days and it will be November 2019 before we know it.