The point was that in a small town the banker knows who you are and can evaluate your risk just by being acquainted with you. If he trusts you, he might just be your friend, if he doesn't then he is probably not your banker.
(Yes the word "friend" is a bit hyperbolic in this case)