I'm Dutch and have spent several years as a scout myself, and know several others who have done the same. I've never seen anyone getting looked down upon for that. Perhaps you'll be made fun of if the scouts in your country include merit badges and you keep showing them to everyone, but in that case you kind of deserve it...