You are right about cultivated fruits, but the distance in sugar content between them and candy remains very high.

Most cultivated fruits have a sugar content of around 10%, while only a few have a sugar content slightly above 15%, e.g. grapes and (raw) figs.

Only the dried fruits have a sugar content similar to chocolate and candy, i.e. starting from around 60% to much more.

So dried fruits should be avoided, but fresh or defrosted fruits up to a few hundred grams per day should be OK (when no other sweets are eaten).