I think that what we currently call retreats seem diverse enough that the events might deserve entirely different names. Organisers could put more care into choosing a term, on a case-by-case basis, that accurately sets expectations. Some retreats I've heard of do appear to entail spending a few days in the countryside forming better bonds within a group. In that case, calling it a "retreat" would be reasonable.

I agree that we should call them something other than retreats, especially because of points 1) and points 3) - lots of people in the EA community basically don't understand that a full day of "retreat" causes the same level of exhaustion as a full day of EAG.
Ideas: