Is there a particular piece? I'll comment on what I think are the key points from his article:

1) Wasted energy.

2) It interferes with legitimate human visitors in certain situations. Simple example would be wanting to download a bash script via curl/wget from a repo that's using Anubis.

3A) It doesn't strictly meet the requirement of a CAPTCHA (which should be something a human can do easily, but a computer cannot) and the theoretical solution to blocking bots is a CAPTCHA.

and very related

3B) It is actually not that computationally intensive and there's no reason a bot couldn't do it.

Maybe there were more, but those are my main takeaways from the article and they're all legit. The design of Anubis is in many respects awful. It burns energy, breaks (some) functionality for legitimate users, unnecessarily challenges everyone, and probably the worst of it, it is trivial for the implementer of a crawling system to defeat.

I'll cover wasted energy quickly -- I suspect Anubis wastes less electricity than the site would waste servicing bot requests, granted this is site specific as it depends on the resources required to service a request and the rate of bot requests vs legitimate user requests. Still it's a legitimate criticism.

So why does it work and why am I a fan? It works simply because crawlers haven't implemented support to break it. It would be quite easy to do so. I'm actually shocked that Anubis isn't completely ineffective already. I actually was holding out bothering testing it out because I had assumed that it would be adopted rather quickly by sites and given the simplicity in which it can be defeated, that it would be defeated and therefore useless.

I'm quite surprised for a few reasons that it hasn't been rendered ineffective, but perhaps the crawler operators have decided that it doesn't make economic sense. I mean if you're losing say 0.01% (I have no idea) of web content, does that matter for your LLMs? Probably if it was concentrated in niche topic domains where a large amount of that niche content was inaccessible, then they would care, but I suspect that's not the case. Anyway while defeating Anubis is trivial, it's not without a (small) cost and even if it is small, it simply might not be worth it.

I think there may also be a legal element. At a certain point, I don't see how these crawlers aren't in violation of various laws related to computer access. What i mean is, these crawlers are in fact accessing computer systems without authorization. Granted, you can take the point of view that the act of connecting a computer to the internet is implying consent, that's not the way the laws are, at least in the countries I'm familiar with. Things like robots.txt can sort of be used to inform what is/isn't allowed to be accessed, but it's a separate request and mostly used to help with search engine indexing, not all sites use it, etc. Something like Anubis is very clear and in your face, and I think it would be difficult to claim that a crawler operator specifically bypassed Anubis in a way that was not also unauthorized access.

I've dealt with crawlers as part of devops tasks for years and years ago it was almost trivial to block bots with a few heuristics that would need to be updated from time to time or temporarily added. This has become quite difficult and not really practical for people running small sites and probably even for a lot of open source projects that are short on people. Cloudflare is great, but I assure you, it doesn't stop everything. Even in commercial environments years ago we used Cloudflare enterprise and it absolutely blocked some, but we'd get tons of bot traffic that wasn't being blocked by Cloudflare. So what do you do if you run a non-profit, FOSS project, or some personal niche site that doesn't have the money or volunteer time to deal with bots as they come up and those bots are using legitimate user-agents coming from thousands of random IPs (including residential! -- it used to be you could block some data center ASNs in a particular country until it stopped).

I guess the summary is, bot blocking could be done substantially better than what Anubis does and with less down side for legitimate users, but it works (for now), so maybe we should only concern ourselves with the user hostile aspect of it at this time -- preventing legitimate users from doing legitimate things. With existing tools, I don't know how else someone running a small site can deal with this easily, cheaply, without introducing things like account sign ups, and without violating people's privacy. I have some ideas related to this that could offer some big improvements, but I have a lot of other projects I'm bouncing between.

It's super weird for sure. I'm not sure how the bots have managed to use so much more bandwidth with only 30k more hits than regular traffic, I guess they probably don't rely on any caching and fetch each page from scratch?

Still going through my stats, but it doesn't look like I've gotten much traffic via any API endpoint (running WordPress). I had a few wallpapers available for download and it looks like for whatever reason the bots have latched onto those.

Negative. Our solution is completely home grown. All artisinal-like, from scratch. I can't imagine I reveal anything anyone would care about much except product specs, and our inventory and pricing really doesn't change very frequently.

Even so, you think someone bothering to run a botnet to hound our site would distribute page loads across all of our products, right? Not just one. It's nonsensical.

This dance to get access is just a minor annoyance for me, but I question how it proves I’m not a bot. These steps can be trivially and cheaply automated.

I don't think the author understands the point of Anubis. The point isn't to block bots completely from your site, bots can still get in. The point is to put up a problem at the door to the site. This problem, as the author states, is relatively trivial for the average device to solve, it's meant to be solved by a phone or any consumer device.

The actual protection mechanism is scale, the scale of this solving solution is costly. Bot farms aren't one single host or machine, they're thousands, tens of thousands of VMs running in clusters constantly trying to scrape sites. So to them, a calculating something that trivial is simple once, very very costly at scale. Say calculating the hash once takes about 5 seconds. Easy for a phone. Let's say that's 1000 scrapes of your site, that's now 5000 seconds to scrape, roughly an hour and a half. Now we're talking about real dollars and cents lost. Scraping does have a cost, and having worked at a company that does professionally scrape content they know this. Most companies will back off after trying to load a page that takes too long, or is too intensive - and that is why we see the dropoff in bot attacks. It's that it's not worth it for them to scrape the site anymore.

So for Anubis they're "judging your value" by saying "Are you willing to put your money where your mouth is to access this site?" For consumer it's a fraction of a fraction of a penny in electricity spent for that one page load, barely noticeable. For large bot farms it's real dollars wasted on my little lemmy instance/blog, and thankfully they've stopped caring.

The author demonstrated that the challenge can be solved in 17ms however, and that is only necessary once every 7 days per site. They need less than a second of compute time, per site, to be able to send unlimited requests 365 days a year.

The deterrent might work temporarily until the challenge pattern is recognised, but there's no actual protection here, just obscurity. The downside is real however for the user on an old phone that must wait 30 seconds, or like the blogger, a user of a text browser not running JavaScript. The very need to support an old phone is what defeats this approach based on compute power, as it's always a trivial amount for the data center.