I think you can still drive some insights. I clicked on the TPC-C results you shared and read their executive summaries.
The Oracle on SPARC cluster (at the top, 2010) performs 30.2M qualified tx/min vs the 16K tx/min in this blog post. The Oracle cluster also costs $30M, which is clearly higher than the Cockroach cluster's cost.
That said, the TPC-C benchmark is new to me. Happy to update this comment if I'm misreading the numbers.
A short note that the total cost of that SPARC cluster was $30 million. You're not misreading those numbers, but it requires a little context.
We're focusing today on our improvements over CockroachDB 1.1, using a small-ish cluster. We'll be showing some more scalability with larger clusters in the coming weeks. If you've found CockroachDB performance slow in the past, you will be pleasantly surprised with this release!
Sure thing. I was primarily answering the question above - in terms of how the numbers in the TPC-C benchmark fit in. I updated my comment to reflect the cost.
I think what's interesting with TPC-C is that you can sort the results based on performance or price/performance. On the price/performance metric, SPARC looks expensive. Dell has a $20K SQL Anywhere cluster that can do 113K tx/min.
I wonder if anyone tried to run these benchmarks on the cloud and how one would calculate total cost of ownership there now.
Yeah, but 1700 cores worth. That's still a lot of $300 boxes. Like qty 53 Sparc T3-2's for example. Which seem to be $1200 to $2k on eBay. And unsupported, end of life, etc.
I'd compare CockroachDB's number to some more recent result with a similar number of cores. (If you can find one)
My guess is that the benchmark setup would cost about 1m dollars to install (3 racks of commodity servers). The software is free. Naturally, Oracle aren't pushing this, when they charge 10s of millions for Oracle rack :)
TPC-C is much, much harder to do well on than that benchmark, even if you are using special hardware and even if you are using "tricks" like intentionally misreading the TPC-C invariants in order to avoid coordination. It's not directly comparable to what you posted; while TPC-C performs joins, reads, insertions, and updates, including ones that necessarily must cross partition boundaries some percentage of the time, and specifies invariants that must be maintained in order to qualify as a correct run, the benchmark you posted (flexAsynch) does only reads or only writes in the tested configuration--both options can be trivially executed without any coordination at all on as many nodes as you feel like. As such, it's more of a stress test of the overhead of the implementation than it is an indication of the actual performance you will get out of a normal workload on such a cluster.
And while I'd like to say that MySQL Cluster is nonetheless exhibiting very impressive performance, I can't really say that; they are using expensive networking and hardware that most people don't have available, microoptimizing both the client and server sides, and using a low-level API specifically designed for doing well in these sorts of benchmarks, but they still lag far behind the per-core performance of state of the art key / value stores in similar circumstances. For example, MICA can do 80 million key/value requests per second of a comparable size to the ones they listed on a single commodity server, with only regular NiCs and 10GiB ethernet (and in fact can saturate the Ethernet link). Granted, MySQL Cluster is a full-fledged database and MICA just does key/value, but I can pretty much guarantee you that on real requests MySQL Cluster's performance collapses, and in multi-master mode it's known to be inconsistent anyway.
If you really need hundreds of millions of k/v requests per second, you'll pay a lot less buying three servers and hiring all the researchers who wrote the MICA paper to manage your key / value store than you will buying MySQL Cluster :P Or, if you want a real database, you can play the same game with the many, many database systems that do much better than Oracle's cluster on TPC-C; the same person who wrote the MICA paper released one last year about Cicada, which can handle over 2 million serializable TPC-C transactions per second on a single 28-core server. Or you can try Calvin, which can do around 500,000 georeplicated TPC-C transactions per second on a commodity EC2 100-node cluster (2.5 compute units each back in 2014) and can operate on top of any existing key-value store. The database world has advanced a lot in the past ten years, and people who really need performance have no shortfall of options that aren't Oracle.
Great, thanks for the in-depth reply. Just saw it now. I agree that TPC-C is much harder than flexasynch, which is just a KV benchmark.
Here's a different benchmark. NDB got around 8m ops/sec on 8 datanodes when performing 1.2m HDFS ops/sec in https://www.usenix.org/conference/fast17/technical-sessions/... .
That is not a KV workload. It is, in fact, a very mixed workload with every transaction being a cross-partition transaction, lots of batched PK ops, lots of partition-pruned index scans. Transactions containing lots of ops. Full-table scans. Index scans. And get this, it's open-source. Research has, of course, advanced since NDB, but nothing open-source still comes close to it for some (sometimes complex) OLTP workloads - anything subscriber oriented. Not TPC-C workloads, they suck on NDB.
I'd never heard of MICA, will read up on it. Calvin, though, is extremely limited. They tried to build CalvinFS on top of Calvin, but it didn't work. Subtree ops didn't scale (at all), as the DB wasn't flexible enough. Features and maturity sometimes matter, and NDB has both.
Looking at the TPC-C page all the benchmarks seem quite old and only reflect commercial databases. Do you have any recent TPC-C benchmarks for OLTP databases such as Postgres, MySQL, and Cassandra so I can compare with CockroachDB?
Open source DBMS normally provide numbers using some open
source variant of TPC-C. One such implementation is DBT2.
TPC-C according to the standard is extremely expensive to
setup and is only interesting for databases with massive
disks. However most open source DBMS runs DBT2 with no wait
time, this means that also an in-memory DBMS can report
numbers on a TPC-C-like benchmark (e.g. DBT2).
MySQL Cluster (NDB) did such benchmarks a few years ago
where we ran 2.2M transactions per minute using DBT2.
This meant executing millions of SQL queries per second
towards the cluster and most of the CPU load is in the
set of MySQL servers.
Currently I am using the load phase of this benchmark to
test loading massive amounts of data into the new 7.6
version of MySQL Cluster using machines with 1 TB of memory.
On the Data Migration Tool (DMS), I really wish they would fix important issues related to Postgres. Right now, if you have a JSONB column in your database, DMS truncates the data.
(Semi-related) I'm an L-1A spouse. It took my work application 10 months to get processed, when the USCIS' own regulation stated that the it should have been closed in 90 days.
You call them, ask them, plead with them. Nothing. "Your application is in the system and is getting processed."
Nine months into it, it turned out that they lost my digital picture taken at a USCIS facility. How in this day and age one loses digital pictures is beyond me.
The only thing I could find was potentially a 7-0 match for Turkey vs. Korea in the 1954 World Cup, which was played in Switzerland? Hard to say, since the data seems to come from:
The only World Cup qualifier that I could find (going back to 1990) are the play-off games between Turkey and Switzerland in '06. Switzerland wins the first game 2-0, and Turkey wins the second one 4-2.
Nothing as dramatic as a 7-0 margin as suggested by the article's data though.
If you don't have access to Internet, unfortunately, it does.
I called my mom who lives in Eskisehir Saturday morning TST, and asked her about the protests. Her reaction was, "What protests?" If you get your news from TV, and if none of them broadcast the protests, then how would you know?
Sadly, when there is internet connection, the problem doesn't simply go away. In Russia, internet media are heavily censored too - so internet user has to make effort to obtain the information not biased in favour of Putin's government: there are only several "unbiased" regular online media(not counting social media like big social networks like facebook or twitter). It is much simpler to just turn on TV especially if you don't care much about politics.
The described reaction is extremely familiar to me. There were major protests in Russia during winter 2011/2012. The first protest event happened the next day after federal parliament elections(quite important event). So, after another 2 days I call my mother(she lives about 3000 km away from Moscow) and ask her if she heared about protests, and get the same reaction "What protests?".
Five days after the initial event, there was another: ~50 000 - 100 000 people gathered in the center of Moscow. Did the federal governmental media mention it? This time did, but very briefly, and understating the number of participants by order of magnitude.
If some guy is bad it doesn't automatically makes Putin good. If democracy in other countries is not a real democracy, it doesn't make Putin a democratic president instead of bloody dictator.
His actual quote about news spreading through social media is near the middle of the article.
"There is now a menace which is called Twitter," Erdogan said. "The best examples of lies can be found there. To me, social media is the worst menace to society."
In most of the world, revolutions come bottom up. In Turkey, they came top down. It was the military that toppled the "sultan" and established the parliamentary system / Turkish Republic. And later it was a former general who set up the multi-party system (and didn't get elected).
I know it sounds off, but what the Founding Fathers did for the US, the generals did for Turkey. Not a very healthy one I agree, but at least they established whatever form of democracy Turkey has today.
> It was the military that toppled the "sultan" and established the parliamentary system / Turkish Republic
The Young Turk Revolution, while supported by one army corp didn't topple the Sultan. The Sultan (Mehmed VI) was deposed by the Grand National Assembly of the Turkish Republic, a political body.
Turkey was ruled by one party until 1946 like the ones in Soviet Union. After 4-5 years of the multi party election of the first prime minister that "honest" and "non-corrupt government" hung the prime minister after the coup. The last soft coup in 1998 costed Turkey by one estimation $300Bn. Wanna talk corruption, talk the military and all the state investments...
The Oracle on SPARC cluster (at the top, 2010) performs 30.2M qualified tx/min vs the 16K tx/min in this blog post. The Oracle cluster also costs $30M, which is clearly higher than the Cockroach cluster's cost.
That said, the TPC-C benchmark is new to me. Happy to update this comment if I'm misreading the numbers.
(Edited to incorporate the reply below.)