If I have 10 points in 2d space, I can always hit them all exactly so long as I have a polynomial of degree 10 or higher. This is overfitting, which is indistinguishable from memorization.
Over-fitting is a known problem and deliberately avoided. This does not prevent the curve from intersecting an actual training set.
I mean technically, if there are many copies of the same data set pair, THEN you can call it over-fitting. But removing that does not fully remove the problem. The curve will still intersect datapoints EVEN without over fitting.
