I don't take a lot of quick trips. I take a lot of long trips, as in 6 to 12 hours, and I rarely get out of the boat..That would be a lot of time if you aren't comfortable. I'm thinking about a new boat myself, and it will have a seat that sits up higher. Not only for comfort, but also to stay drier. My current boat is a SOT, but the seat doesn't raise up. Swamp-ass on a ten hour float really sucks, especially at the end of a cool fall trip.
If you plan on a lot of short spur-of-the-moment trips after work, a light sit-in might be ideal, but that boat might not be suitable for long trips or camping. I guess the bottom line is: there are no perfect options.