Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

As a user of an unsigned Firefox fork, Turnstile has ruined a moderate portion of the Internet for me. The way Cloudflare doesn’t think twice about eroding user freedoms, for the sake of a gate that can be trivially bypassed with solvarr or similar, is deeply disturbing. They are no longer a force for good on the web.
 help



As bad as cloudflare is there is a reason people use it.

If you try and run a site that has content that LLMs want or expensive calls that require a lot of compute and can exhaust resources if they are over used the attack is relentless. It can be a full time job trying to stop people who are dedicated to scrapping the shit out of your site.

Even CF doesnt even really stop it any more. The agent run browsers seem to bypass it with relative ease.


Granted, but there are open source alternatives that don’t have the same obsession with meaningless digital signatures. Turnstile is just a terrible product.

Vast majority of websites today can and should be static, which makes even the aggressive llm scrapping non-issue.

One of the things that a lot of LLM scrapers are fetching are git repositories. They could just use git clone to fetch everything at once. But instead, they fetch them commit by commit. That's about as static as you can get, and it is absolutely NOT a non-issue.

No... Basically all git servers have to generate the file contents, diffs etc. on-demand because they don't store static pages for every single possible combination of view parameters. Git repositories also typically don't store full copies of all versions of a file that have ever existed either; they're incremental. You could pre-render everything statically, but that could take up gigabytes or more for any repo of non-trivial size.

> Git repositories also typically don't store full copies of all versions of a file that have ever existed either; they're incremental

This is wrong. Git does store full copies.


git stores files as objects, which are stored as full copies, unless those objects are stored in packfiles and are deltified, in which case they're stored as deltas. https://codewords.recurse.com/issues/three/unpacking-git-pac...

... which, in the context that is being discussed, is unusual.

Thank you for the insights.

that's a pretty niche issue, but fairly easy to solve.

Prebuild statically the most common commits (last XX) and heavily rate limit deeper ones


1. that doesn't appear to match the fetching patterns of the scrapers at all

2. 1M independent IPs hitting random commits from across a 25 year history is not, in fact, "easy to solve". It is addressable, but not easy ...

3. why should I have to do anything at all to deal with these scrapers? why is the onus not on them to do the right thing?


I see people saying that a lot, but I use Zen which is a fork of Firefox and I don't think I've ever had an issue with Turnstile, at least not noticeably more than I had on mobile Chrome.

Zen has been signed for close to a year.

Isn't it the opposite? They allow you to still use it when it would almost certainly be better for cloudflare and the website behind then to just block you.

How does Cloudflare know you are using the fork? Can you not just set the user agent to match firefox's (or even chrome's for that matter)

Quite likely fingerprinting detection, which is remaining firmly enabled.

How does that work technically? Presumably a fork of firefox is almost indistinguishable from firefox from Cloudflare's perspective?



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: