Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

(author)

Noise certainly affects the compression rate. But you are not concerned with the absolute compression rate, you are only concerned with the relative rate achieved by two theories A and B. Both theories will be negatively impacted to the same degree by the noise, so the comparison still works to select which theory is better.



…to the extent that the difference between theories dominates over the possible noise variability.


You can easily quantify the variance and do standard model-comparison/hypothesis-testing if you want statistical-significance levels. For many datasets these days, this is hardly even a consideration: even a 1% compression improvement is clear.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: