Recommendation is to validate "best practice" claims. Doesn't matter what hosting solution used. Measure, measure, measure to make sure not only are you getting said claim but also that the end result meets your expectations. An example is in the past I had 7 "instances" (as shykes points out make sure they are hosted on separate nodes!) 4 of which where load balances Python web app. One of the nodes was overloaded so 1 out of 4 requests was very slow (5-10x). This was a big ajax app so initial page load would hang on the request(s) to that one instance. My point was since I had measured I could see that the node was the problem and now that I am on dedicated EC2 each node is consistant. Good luck.
That's good advice. As the saying goes: "trust, but verify".
Regarding your performance issue - most platforms (including dotCloud) enforce ram and cpu separation between nodes, but are vulnerable to IO contention at some level. This is also true for EC2 if you use EBS: your standalone instances will almost certainly, at some point, suffer from degraded and inconsistent performance because another instance is competing for IOPS [1].
You can avoid this with the new "provisioned IOPS" volumes [2], or by skipping EBS altogether for stateless components.
There are 3 reasons I have gone running and screaming from and RDBMS.
1. Software gets large / complex to get meaning full work done. I am all about data consistency but at some point it is time to break things up into services and not have a single database.
2. If the software is popular enough everyone is running to use NoSQL (cache is NoSQL).
3. Clearly it is not a good storage solution either because for example in an address book nested list greatly simplifies everything. (right tool for the job)
I spent many years hammering away with RDBMS and by and large it was great until it wasn't. I try to look at data storage more holistically now based on best guess of the problem. I have tried to convert an application from Postgresql to MongoDB and it failed but that wasn't MongoDB's falt it was because I didn't change the data model to fit a document storage system. I have also tried to use PostgreSQL for a realtime reporting system and failed horrifically and that was not Postgesql fault it was mine. Amazing what happens when you stop pushing a chain and start pull it!
>1. Software gets large / complex to get meaning full work done. I am all about data consistency but at some point it is time to break things up into services and not have a single database.
This is true. Managing complexity is always an important task. I am not sure that NoSQL solves this however. Also the best way to break things up is to loosely couple things, and this requires to some extent that you have ACID compliance. A good RDBMS, like PostgreSQL or Oracle, will provide tools for managing that loose coupling.
>"2. If the software is popular enough everyone is running to use NoSQL (cache is NoSQL)."
Like proverbial lemmings over a cliff....
>"3. Clearly it is not a good storage solution either because for example in an address book nested list greatly simplifies everything. (right tool for the job)"
Funny, I thought nesting was what WITH RECURSIVE was for....
I am not saying there aren't use cases for MongoDB or reasons to switch some applications. For example I can think of a few really cool apps, like maybe a network back-plane for a huge LDAP directory. Also content management might be a good fit. But despite your years of experience, it doesn't sound like you have really looked at how to solve these with good RDBMS's.....
I wrote the article because I write a lot of web applications which a vast majority of the request are JSON. I am about to do a lot of post analysis on JSON sent from the web browser so it is valuable for me to know how fast a language (library) JSON parser is. For highly concurrent webserver it is good to know how fast and efficient the JSON parser is since it will be doing a lot of concurrent parsing of JSON.
When you're working with streaming JSON like this (or in a web app), it might be easier to use a decoder/encoder. See http://play.golang.org/p/TLNORK2WK9 for an example.
Also fwiw, it has never been my experience that json decoding/encoding become the bottleneck in a web app. I/O (to a database, or the filesystem) is by far the largest bottleneck in any app I've profiled. A good benchmark for a web app language is hard to write, because it tends to depend on (unreliable) IO-bound systems.
You could use one of the C json implementations such as json-c. However as has been pointed out you will still be just comparing libraries rather than the languages themselves.
It certainly could be more optimized.
For example, all numeric literals are converted to strings before being parsed. That's a mostly unnecessary copy/allocation per number.
Similarly, it doesn't make use of the fact that a []T can only legally contain Ts. It's all very clean and generic, which is great, because when working on the compiler they can work on making good code faster without being mislead by code that tries to work around lack of optimizations.
As an experiment, I replaced the []int in the Go code in the link with an IntList that implements json.Unmarshaler, and does a single-pass integer list parsing there (without overflow checking, but eh). With that quick 40 line change, the Go code was faster than the CPython, at least on my machine.
That and similar type-specific handling could be built into the json package fairly easily, and would make those cases quite a bit faster. Then again, I bet the Go team would have a better and more general idea.
I spun the JSON decoding off into a few goroutines and managed to get it down to ~1second (but it was out of order, so didn't quite solve the same problem - though this is fixable). This is was ~10 lines of Go (and only using a single core).
This was my bad I have updated the post. I should have been marsheling to a more comparable datatype (map[string]interface{}{}) and now the performance is more in line with what I would expect. I need to look into the Unmarshal code to see why this is so slow.
I appreciate why you use "golang" instead of the general "Go" or current release "Go1", but the former is the name of the open source organization, and the latter is the name of the language.
It looks more like it was lottery than anything else. If you clicked on it at the right time and got lucky in a 30 minute window. I have been to a number of them since I went to the first one (part of the android challenge before there was an android phone) I have really enjoy the conference every time I have gone. This year I mainly wanted to get some time in with the Go group and reconnecting with friends I have made over the years. Look like time to start looking for another conference with lots of web startup. Anything going on in the UK?
I suppose we all knew this day would come from Yahoo. The start of where lawyers take over the company and product make a steady march to complete crap. Pretty common pattern for companies that can no longer compete (SCO anyone). It is sad Yahoo still have a great brand and could reinvent itself. This entire thing is nonsense but I guess to try and squeeze some more revenue for the share holders is what they are obligated to do. I miss the old Yahoo.
SEEKING WORK - Remote, US, EU: Python, Golang, Flask, Mongodb, PostgreSQL, Android, jQuery, Google Web Toolkit, AppEngine (python, go)
Love building software! Specifically love working in exciting environments. Would love to work on some interesting projects my email is in the profile.
Why don't you just learn to program? It is not that difficult and in a startup everyone needs to code. Engineers learn business skills all the time but it is strange that the reverse is not true.
