Two ways to solve this, either A, use the light wallet server on a node that you control and just give it your view key, or B, use Monero more often, LOL.
Or use a background sync wallet like Cake or Anonero
But yea the sync times are annoying otherwise
I had issues with the anonero dev tram telling me they could not fix a bug i was seeing so i no longer personally use them. As for Cake bg sync i have heard several reports of the wallet being corrupted and having to resync the entire chain. Until that is fixed i would not use it.