There is definitely a happy middle ground. Feedback too soon can be jarring, but waiting till the end of a form can be very frustrating, especially if there are many fields and you have to reparse to find the error. I think you need both to have a good experience.
I just think that, if you're aiming for the highest UX-bar, you're going to need client-side validation at some point. For any mildly complex form, server-side validation will usually be far past the sweet-spot you're referring to.
Now, as others have mentioned, it might be questionable whether every web-app and website should be targeting the highest UX-bar.
