Actually, that's EXACTLY the case I had in mind. Obviously I couldn't put a 50 line function with weird data structures in my blog post, or it would be unreadable. But the point is that comments for that function are bad! They'll rot if you ever change the function or the assumptions at all. Instead, you should break the 50-line function up into smaller functions, and add assertions and test cases for all of those "magic-lik assumptions." Then get in the habit of reading tests first. Now your code-as-comments will never rot.
No offence, but this is incredibly naïve, and it's kind of obviously an opinion that simply stems from working on things that just aren't that conceptually complicated. Some things are actually conceptually complicated, and the why is not at all obvious from the what, no matter how finely you slice the what. Code, inherently, describes what. A sparse set of higher level proper sentence comments describing the purpose of the overall goal for the next part is practically required to make code like this actually comprehensible. For some things, something approaching Literate Programming is best, which is nearly the polar opposite of "comments are bad".
Implementing an advanced data structure is a good example of this. Things that took theoreticians some time to discover, and write/publish in a paper, are not things a random programmer is just going to inherently know from a completely uncommented implementation.
Extrapolating "all comments are bad" from a few examples of pointless comments on mind-numbingly simple and obvious code (which are indeed bad) is silly.
OK. But the algorithm is over a 1000 lines split up into perhaps 50 20-line functions, and it's mysterious how the whole works by looking at the parts.
I have one particular piece of code, that I wrote, in mind. It was to implement anonymous methods in the Delphi compiler. Given an AST for a method from the parser, it had to rewrite the tree to turn captured local variable references into field accesses on a heap-allocated stack frame, turn anonymous methods into methods on this heap-allocated stack frame, and all while minimizing the number of passes over the tree. And this has to work recursively, which adds a surprising number of wrinkles about the ordering in which you can do things.
So for example any given pass may be building up accounting data needed by later passes. This is somewhat mysterious, because it seems like busywork; and why is it doing this here? Why not fold one pass into another? Why not move work between passes? Well, the subtle ordering problems are not clear at all when you're looking at things at the function level. You can only understand the pieces when you already understand the whole.
And this is why there is a very long comment block describing how this stuff works. Because figuring it out by reading the code is too expensive.
(That ~1000 lines of code took perhaps 4 months to write; having to integrate with rest of a large complex codebase adds countless more wrinkles.)
I've worked on code like that, and i'm sorry but i would rather read a 50 (or even couple 100 if appropriate) line function implementing specific functionality that is well written and documented with comments then have that critical information split up into a multiple functions and test cases spread out over multiple files. Sometimes functionality belongs together and it would be far more difficult to get that all that disjointed information into coherent mental model.
Not to mention how test cases and function names would just not cut it for explaining things like why the current implantation was chosen over more obvious, simpler ones, why the magic like assumptions are necessary and documenting the bugs/quirks in the underlying library or framework that are not obvious. Things like why call object.SetValue() instead of property object.value = x? Or Free memory from this call here, but not here, because the framework takes care of it.
I prefer commented code, even if it is over commented, because at the very least it is a way for the last programmer to explain why she implemented things the way she did.
But the point is that comments for that function are bad! They'll rot if you ever change the function or the assumptions at all
Do you leave unused functions in your code when you refactor? No. So refactor your comments, or delete them if they are no longer applicable, when you refactor your code.
I have been saved many times by comments in my own code, because you know I work on a lot of stuff, sometimes coming back to it after years and I hope I have developed as a coder and so my mindset is now different and a lot of the time I do wonder WTF was I doing here!
Hey Timothy, thanks for the response. You are correct. I'll add a warning to the library and work on a solution. Unfortunately, I think it will need to involve ctypes.
Have you tried patching the now() method onto datetime.datetime instead of replacing the whole class? If that doesn't work, you could replace datetime first:
import freezegun; freezegun.monkey_patch()
from datetime import datetime
After that, freeze_time would just set a flag on your datetime class.
Correct. I've gone with his latter solution for now and added a warning about import order. I'm not very happy with this solution through and will be spending some time with ctypes in the next few days to come up with something better.
Ahh, that's clever. Overall I just don't want to tell people what variable names to use, even if they are obscure. At the very least the end developer can use the variable names for everything else other than NaN.
By far the easiest way to get similar behavior is to run your Django app via Gunicorn (proxied from Nginx). Gunicorn supports hot code reloading via SIGHUP, and it does so by forking and gracefully killing old processes.
If your requirements don't match gunicorn (not django, not python, etc) then you can use https://github.com/TimothyFitz/zdd a project I wrote to automate rewriting nginx config files to deal with changing proxied portfiles. To integrate any existing server, all you have to do is make it bind to port 0 (let the OS choose a port) and the write a foo.port file that contains the port number (like a pidfile). That's it.
I mentioned this in a thread the other day; I've yet to see a good use case for hot code reloading. Can you really not drain requests to that host via HAProxy (or similar), and then actually restart the service? The nice thing about that approach is that your choice of service runtime doesn't matter.
Your method will stall responses for server shutdown + server startup time, which for Ruby/Python apps is usually measured in tens of seconds, and for other web servers can be much worse. Hot code reloading lets you avoid any downtime at all, and with it usually being built into the framework/language specific server you get the functionality "for free".
Zdd (the project I linked to) is all about spawning a new process in parallel. All the advantages of your approach (switch to an entirely different language? Who cares) but without the stalls.
Zdd also lets you keep the old process alive through the duration of the deploy (and after), and with a little work could let you switch back in the event of a bad deploy without having to start the old version up again.
The method he's describing requires no downtime or stalled requests. Connections are drained from some pool in a load balance set if servers. The services is restarted with the new code. The hosts are given traffic again once the are initialized and healthy.
The advantages to this method include being completely platform agnostic, as well as giving you a window to verify successful update without worrying about production traffic.
Thanks for the clarification. The downsides to that approach are that you need multiple machines, and the duration of your deploys is much longer. Not to mention, you'd have to script a deploy process across multiple machines (which is not easy, in the way that "SIGHUP Gunicorn" is easy).
Personally I've found the "put new instances into a load balancer" method to make more sense for system changes (packages, kernels, OS versions) where deploying the change is inherently slow or expensive, but the method doesn't make sense for code deploys where deploy time is important.
One really impressive thing to me about 4chan's infrastructure: All traffic is served via 4 machines. Two webservers, a database and an admin/cron box. 22M uniques on 4 machines is insane. Hopefully Chris will write more about his setup some day. (Note: these boxes were directly serving all traffic up until ~6 months ago when Cloudflare was introduced)
Canvas (USV Funded) is looking for engineers #3 and #4 to join a small close team building the rich-media community platform of the future.
The job title says "Software Engineer" but really we're looking for "Software Entrepreneur" or a "Startup Engineer". Yes, your day job will be writing code. But that's the only similarity to a big company software job.
You'll be challenged to take big ideas and turn them into concrete testable hypotheses. Shipping a great feature is important, but positively changing user behavior is the ultimate success criteria. Built-to-spec takes a backseat to moves-the-metrics.
Canvas (USV Funded) is looking for engineers #3 and #4 to join a small close team building the rich-media community platform of the future.
The job title says "Software Engineer" but really we're looking for "Software Entrepreneur" or a "Startup Engineer".
Yes, your day job will be writing code. But that's the only similarity to a big company software job.
You'll be challenged to take big ideas and turn them into concrete testable hypotheses. Shipping a great feature is important, but positively changing user behavior is the ultimate success criteria. Built-to-spec takes a backseat to moves-the-metrics.
Canvas (USV Funded) is looking for engineers #3 and #4 to join a small close team building the rich-media community platform of the future.
The job title says "Software Engineer" but really we're looking for "Software Entrepreneur" or a "Startup Engineer".
Yes, your day job will be writing code. But that's the only similarity to a big company software job.
You'll be challenged to take big ideas and turn them into concrete testable hypotheses. Shipping a great feature is important, but positively changing user behavior is the ultimate success criteria. Built-to-spec takes a backseat to moves-the-metrics.
Canvas (USV Funded) is looking for engineers #3 and #4 to join a small close team building the rich-media community platform of the future.
The job title says "Software Engineer" but really we're looking for "Software Entrepreneur" or a "Startup Engineer". Yes, your day job will be writing code. But that's the only similarity to a big company software job.
You'll be challenged to take big ideas and turn them into concrete testable hypotheses. Shipping a great feature is important, but positively changing user behavior is the ultimate success criteria. Built-to-spec takes a backseat to moves-the-metrics.
