Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

genuine question. if i have a tightly-defined unit test and Claude writes a blob of code that passes it, does it matter what's in the blob?
 help



It matters for at least a few reasons:

- Depending on the nature of your application, it may be very important to be able to audit the business logic and intended behavior. For compliance reasons, for operational reasons, for moral/ethical reasons -- you very well might want to affirm what the code is actually trying to do.

- A coding agent may get very creative in order to write code that passes a tightly-defined unit test. It may come up with approaches that technically pass, but work against the overall intention of the app in the first place. This becomes an arms race rather than a productive collaboration, where the agent's increasing creativity has to be matched by a sprawling test suite.

- Eventually, inevitably, business requirements will change, and the blob will need to evolve. It will be much easier for an agent or a human alike to understand how to safely make the change, if the existing implementation is transparent and understandable.


Two possibilities:

1. Your unit tests are exacting enough to fully specify the unit. In that case, congratulations, your unit tests are the code. They're also probably much more awkward to write, maintain, etc. Also, the compilation step to go from the unit tests to the actual code is now orders of magnitude more expensive, requires a SaaS to even work, etc.

2. Your unit tests are not that exacting and still leave ambiguity, edge cases, etc. In that case it very much matters what's in the blob of code, because while it could be a correct implementation of what you wanted, it could also be something else entirely that just happens to be correct for the part you did specify.


If the test passes, you review the blob, and QA tests it. I dont see why its any different to you having copied code from StackOverflow.

It does not matter for one instance. But it does matter if you plan to make a living off it.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: