Cloudflare not allowing benchmarks in their TOS is very sketchy, that puts them in the same tier as Oracle.
Cloudflare have pulled enough shady stuff now that they've fallen out of my favor. Their generous free product bought them a lot of community goodwill but their real face has been showing the past few years.
> Cloudflare not allowing benchmarks in their TOS is very sketchy, that puts them in the same tier as Oracle.
There's a non-sketchy way for a company to do that kind of thing which avoids putting them in Oracle territory. Simply require that anyone publishing benchmarks (1) publish complete configuration information sufficient to allow the company and other interested third parties to reproduce the benchmarks, and (2) allows benchmarking of their products that they are benchmarking against yours under similar terms.
It is nicer of course to not have any benchmarking restrictions, but if you do that you are putting yourself at a disadvantage to companies like Oracle. They can benchmark against you put you can't do the same against them.
Once one important player in a market goes down the Oracle route others tend to follow. But if they aren't asses they will follow with the kind of restriction with reciprocity I outlined above instead of an Oracle-like restriction.
I like your ideas, but they seem difficult to enforce. It assumes good faith on all sides. One of the biggest complaints about AI/ML research results: It is frequently hard/impossible to replicate the results.
One idea: The edge competitors can create a public (SourceHut?) project that runs various daily tests against themselves. This would similar to JSON library benchmarks. [1] Then allow each competitors to continuously tweak there settings to accomplish the task in the shortest amount of time.
Also: It would be nice to see a cost analysis. For years, IBM's DB2 was insanely fast if you could afford to pay outrageous hardware, software license, and consulting costs. I'm not in the edge business, but I guess there are some operators where you can just pay a lot more and get better performance -- if you really need it.
I would give them a bit more wiggle room before giving up. We’re currently living under a monopoly of AWS (GCP is the most serious “competitor”, still far far behind in market share) and I would love to see more competition in this space.
That said: this is definitely not the kind of behavior I expect from them and I’m hoping there’s a better reason than them not willing to submit their products to an impartial evaluation.
AWS is a monopoly if you're living in a HN bubble. The vast majority of "hosting" (internet or otherwise) doesn't use AWS. Large telco's (Telefonica, AT&T, China Unicom, ...) have numerous data centers. Conglomerates (Tata, ...) have numerous data centers. You have wholesalers (Equinix, Digital Realty, ...). You have dedicated providers (OVH, Hetzner, PhoenixNap, ...). VPS' (DO, Linode, Vultr,...) and shared hosting (GoDaddy, ...)
Then you have a lot of private data centers, e.g. I worked at a major bank that had its own datacenters (multiple around the world). Admittedly, these are super small compared to an AWS DC, but there's literally thousands of these around the world (I know for banking and government, but I assume other industries do this too).
I’m not forgetting Azure, it’s a deliberate omission because it is not a Tier 1 cloud product. It might be better than OCI or IBM Cloud but it simply does not have the maturity of either AWS or GCP.
Cloudflare have pulled enough shady stuff now that they've fallen out of my favor. Their generous free product bought them a lot of community goodwill but their real face has been showing the past few years.