you wouldn't test current Gmail users against a new interface
Why not? If you're trying to assess the impact of a new interface, surely the fact that it's going to confuse existing users is important?
You'd get a group who'd never used either one
So if this group likes the new one better, that outweighs any amount of confusion caused with existing users? That's basically what you're saying here.
If you're trying to determine which interface is "better", you'd want to rule out familiarity with one over the other. You could test familiar users as well, which might let you know things like how much hand holding will be needed to transition to a new interface, but it certainly wouldn't be useful in telling you which one was easier to use. The familiar one will almost universally be easier to use.
On your second point: I'm not saying the control group picking the new interface "outweighs any amount of confusion caused with existing users", but it certainly outweighs some.
If you're trying to determine which interface is "better", you'd want to rule out familiarity with one over the other.
You're assuming that "better" is some objective quality that is independent of the history of your product and who uses it. It's not. Part of what makes a UI "good" is that it's similar to other UIs that users already know, so it takes them less time to learn it. By this criterion, the UI you have right now is "better" than any other one, because your existing users already know it. Any metric of "better" that doesn't take that into account is, IMO, wrong.
it certainly outweighs some.
How much? Your prescription gives no way of judging that that I can see, since you explicitly ruled out considering input from existing users.
Why not? If you're trying to assess the impact of a new interface, surely the fact that it's going to confuse existing users is important?
You'd get a group who'd never used either one
So if this group likes the new one better, that outweighs any amount of confusion caused with existing users? That's basically what you're saying here.