Indeed. Shutoff Garbage collection completely and it can work. (And make sure your Java code creates no garbage - which is a new type of programming in and of itself)
I don't think their comment is intended to be negative really -- looks more like appreciative of the option, while cognizant of the fact that using it introduces a new challenge.
> This is because Zing uses a unique collector called C4 (Continuously Concurrent Compacting Collector) that allows pauseless garbage collection regardless of the Java heap size.
This is incorrect. Firstly, C4 triggers two types of safepoints: thread-local, and jvm-wide. The latter can easily go into the region of ~200 micros for heaps of ~32GB even when running on fast, overclocked servers.
Secondly, the design of C4 incurs performance penalty for accessing objects due to memory barriers. This impacts median latency noticeably.
You might not believe me, but ask Gil and he'll openly admit it. This article was written by someone who:
1) doesn't know how C4 works
2) doesn't analyze relevant metrics from their JVMs
To be fair, Mailinator https://www.mailinator.com has been around since it's inception in 2003. It offers other services now, but the "disposable email" part is still there as it was from the start.
Similarly spamgourmet has been around since 2001 and other than some brief outages (around the time the founder was sadly struggling with a terminal illness and figuring out succession plans) it's doing fine.
Loom sets out to give you a sane programming paradigm similar to what threads do (i.e. as opposed to programming asynchronous I/O in Java with some type of callback) without the overhead of Operating System threads.
That's a very cool and a noble pursuit. But the title of this article might as well have been "5M persistent connections with Linux" because that's where the magic 5M connections happen.
I could also attempt 5M connections at the Java level using Netty and asynchronous IO - no threads or Loom. Again, it'd take more Linux configuration than anything else. If that configuration did happen though now you can also do it in C# async/await, javascript, I'm sure Erlang and anything else that does Asynchronous I/O whether it's masked by something like Loom/Async/Await or not.
It is true that the experiment exercises the OS, but that's only part of the point. The other part is that it uses a simple, blocking, thread-per-request model with Java 1.0 networking APIs. So this is "achieving 5M persistent connections with (essentially) 26-year-old code that's fully debuggable and observable by the platform." This stresses both the OS and the Java runtime.
So while you could achieve 5M in other ways, those ways would not only be more complex, but also not really observable/debuggable by Java platform tools.
Writing the sort of applications that I get involved with, it's frequently the case whilst it's true that 1 OS thread/java thread was a theoretical scalability limitation - in practice we were never likely to hit it (and there was always the 'get a bigger computer').
But: the complexity mavens inside our company and projects we rely upon get bitten by an obsessive need to chase 'scalability' /at all costs/. Which is fine, but the downside to that is the negative consequences of coloured functions comes into play. We end up suffering having to deal with vert.x or kotlin or whatever flavour-of-the-month solution is that is /inherently/ harder to reason about than a linear piece of code. If you're in a c# project, the you get a library that's async, and boom, game over.
If loom gets even within performance shouting distance of those other models, it's ought to kill (for all but the edgiest of edge-cases) reactive programming in the java space dead. You might be able to make a case - obviously depending on your use cases which are not mine - that extracting, say, 50% more scalability is worth the downsides. If that number is, say, 5%, then for the vast majority of projects the answer is going to be 'no'.
I say 'ought to', as I fear the adage that "developers love complexity the way moths love flames - and often with the same results". I see both engineers and projects (Hibernate and keycloak, IIRC) have a great deal of themselves invested in their Rx position, and I already sense that they're not going to give it up without a fight.
So: the headline number is less important than "for virtually everyone you will no longer have to trade simplicity for scalability". I can't wait!
1. Demanding scalability for inappropriate projects and at any cost is something I've seen too, and on investigation it was usually related to former battle scars. A software system that stops scaling at the wrong time can be horrific for the business. Some of them never recover, the canonical example being MySpace, but I've heard of other examples that were less public. In finance entire multi-year IT projects by huge teams have failed and had to be scrapped because they didn't scale to even current business needs, let alone future needs. Emergency projects to make something "scale" because new customers have been on-boarded, or business requirements changed, are the sort of thing nobody wants to get caught up in. Over time these people graduate into senior management where they become architects who react to those bad experiences by insisting on making scalability a checkbox to tick.
Of course there's also trying to make easy projects more challenging, resume-driven development etc too. It's not just that. But that's one way it can happen.
2. Rx type models aren't just about the cost of threads. An abstraction over a stream of events is useful in many contexts, for example, single-threaded GUIs.
I think my point is more that you end up having to pay the costs (of Rx-style APIs) whether you need the scalability or not, because the libraries end up going down that route. This has sometimes felt that I'm being forced to do work in order to satisfy the fringe needs of some other project!
And sure, if you are living in a single-threaded environment, your choices are somewhat limited. I, personally, dislike front-end programming for exactly that reason - things like RxJS feel hideously overcomplicated to me. My guess is that most, though not all, will much prefer the loom-style threading over async/await given free choice.
One additional - as noted, it's been 26 years since Java's founding. Project Loom has been around since at least 2018 and still has no release date. It'll be cool for Java projects whenever it comes out, but I just...have a hard time caring right now. I can't use it for old codebases currently, and new codebases I'm not using one request per Java thread anyway (tbh - when it's my choice I'm not choosing the JVM at all). The space has moved, and continues to move. In no way to say the JVM shouldn't be adopting the good ideas that come along the way, that is one of the benefits of being as conservative and glacial in adoption as it is, but I just...don't get excited about them, or find myself in any position in relation to the JVM (Java specifically, but the fundamentals affect other languages) other than "ugh, this again".
JEP 425 has been proposed to target JDK 19, out September 20. It will first be a "Preview" feature, which means supported but subject to change, and if all goes well would normally be out of Preview two releases, i.e. one year, after that.
> I'm not using one request per Java thread anyway
You don't have to, but not that only the thread-per-request model offers you world-class observability/debuggability.
> other than "ugh, this again".
Ok, although in 2022, the Java platform is still among the most technologically advanced, state-of-the art, software plarform out there. It stands shoulder to shoulder with clang and V8 on compilation, and beats everything else on GC and low-overhead observability (yes, even eBPF).
The point I was making is that Loom isn't released, stable, production ready, supported, etc, and there's no still no date when it's supposed to be, so what you can do with Loom in no way affects what I can do with a production codebase, either new or legacy. I'm not sure how you missed that from my post.
I'm not defending reactive programming on the JVM. I'm also not defending threads as units of concurrency. I'm saying I can get the benefits of Project Loom -right now-, in production ready languages/libraries, outside of the JVM, and I can't reasonably pick Project Loom if I want something stable and supported by its creators.
> and there's no still no date when it's supposed to be
September 20 (in Preview)
> I'm saying I can get the benefits of Project Loom -right now-, in production ready languages/libraries, outside of the JVM
Only sort-of. The only languages offering something similar in terms of programming model are Erlang (/Elixir) and Go — both inspired virtual threads. But Erlang doesn't offer similar performance, and Go doesn't offer similar observbility. Neither offers the same popularity.
I'm not saying there aren't tradeoffs, just that if I need the benefits of virtual threads...I have other options. I'm all for this landing on the JVM, mainly so that non-Java languages there can take advantage of it rather than the hoops they currently have to jump through to offer a saner concurrency model, but that until it does...don't care. And last I saw this feature is proposed to land in preview in JDK19; not that it would, and...it's still preview. Meaning the soonest we can expect to see this safely available to production code is next year (preview in Java is a bit weird, admittedly. "This is not experimental but we can change any part of it or remove it for future versions depending how things go" was basically my take on it when I looked in the past).
Meanwhile, as you say, Erlang/Elixir gives me this model with 35+ years of history behind it (and no libraries/frameworks in use trying to provide me a leaky abstraction of something 'better'), better observability than the JVM, a safer memory model for concurrent code, a better model for reliability, with the main issue being the CPU hit (less of a concern for IO bound workloads, which is where this kind of concurrency is generally impactful anyway). Go has reduced observability than Java, sure, but a number of other tradeoffs I personally prefer (not least of all because in most of the Java shops I was in, I was the one most familiar with profiling and debugging Java. The tools are there, the experience amongst the average Java developer isn't), and will also be releasing twice between now and next year.
Again, I'm not saying virtual threads from Loom aren't cool (in fact, I said they were; the technical achievement of making it a drop in replacement is itself incredible), or that it wouldn't be useful when it releases for those choosing Java, stuck with Java due to legacy reasons, or using a JVM language that is now able to migrate to take advantage of this to remove some of the impedance mismatch between their concurrency model(s) and Java's threading and the resulting caveats. Just that I don't care until it does (because I've been hearing about it for the past 4 years), it still doesn't put it on par with the models other languages have adopted (memory model matters to me quite a bit since I tend to care about correct behavior under load more than raw performance numbers; that said, of course, nothing is preventing people from adopting safer practices there...just like nothing has been in years previous. They just...haven't), nor do I care about the claims people make about it displacing X, Y, or Z. It probably will for new code! Whenever it gets fully supported in production. But there's still all that legacy code written over the past two decades using libraries and frameworks built to work around Java's initial 1:1 threading model, and which simply due to calling conventions and architecture (i.e., reactive and etc) would have to be rewritten, which probably won't happen due to the reality of production projects, even if there were clear gains in doing so (which as the great-grandparent mentions, is not nearly so clearcut).
Erlang is very cool, and Go has certainly achieved a notable measure of popularity — both served as an inspiration here — but a super-popular language like Java plays in a different world and a completely different scale than languages with a smaller reach. Virtual threads will bring the benefits of lightweight user-mode threads to an audience that is many times that of Erlang and Go combined.
As to legacy code, Java programs have been using the thread-per-request model for over 25 years (there's been a lot of talk of reactive, but actual adoption is relatively low), and Java's threads were designed to be abstracted from day one (in fact, early versions of Java implemented them in user mode). So the right fit has been there all along. Migrating applications to use virtual threads requires relatively few changes because of those reasons, and because we designed them with easy adoption in mind. This particular experiment is about simple, "legacy" Java 1.0 code enjoying terrific scalability.
BTW, Java's observability has come a long way in recent years (largely thanks to JFR — Java Flight Recorder), and even Erlang's is no match for it, although Java still lags behind Erlang's hot-swapping capabilities.
[1]: BTW, I always find talk about the "average Java programmer" a bit out of touch. The top 1% of Java programmers, the experts, outnumber all Rust (or Haskell, or Erlang) programmers several times over, and there are many more reliable Java programs than reliable Erlang programs. The average Java (or Python, or JavaScript, the two other dominant languages these days) programmer, is just the average programmer, period.
Threads (whether lightweight or heavyweight) can’t fully replace reactive/proactive/async programming even ignoring performance and scalability. Sometimes network code simply needs to wait for more than one event as a matter of functionality. For example, a program might need to handle the availability of outgoing buffer space and also handle the availability of incoming data. And it might also need to handle completion of a database query or incoming data on a separate connection. Sure, using extra threads might do it, but it’s awkward.
Let me preface by saying I am a Johnny-come-lately loom fanboy. Amazing work and huge impact.
Re structured concurrency: I wonder if there’s any way to combine with generic exceptions such that we can not force a wrapping exception class. So maybe have an executor class that’s generic on the thrown exception type, and then have the join or get apis explicitly throw that type?
This thought process is inspired by the goto-considered-harmful trail of logic: I think it would get us even closer to concurrency encapsulated in function blocks.
Convenient polymorphism over exceptions is something I would very much like to see in Java, but it's a separate topic. Given that structured concurrency is normally used with things that can fail, and whose failures must be handled, I hope (and think) you'll find that the use of checked exceptions is not onerous at all. If we're mistaken, we can consider solutions during the incubation period.
Totally agree that we need explicit handling of the concurrency-specific exceptions like interruptedexception. It’s just that concurrent apis by their nature take callable/runnable apis which lose any formality over exceptions thrown by client code, and thus someone up the stack is always forced to write a catch( Throwable ) block. So the concurrency leaks up the stack, and forces unsafe default clauses.
You’re clearly correct that the topic is separate, but it has great impact on the leakiness of these apis.
I think we're in agreement. Ignoring under the hood - Loom's programming paradigm (from the viewpoint of control flow) is the Threading programming paradigm. (Virtual)Thread-per-connection programming is easier and far more intuitive than asynchronous (i.e. callback-esque) programming.
I still attest though - The 5M connections in this example is still a red herring.
Can we get to 6M? Can we get to 10M? Is that a question for Loom or Java's asynchronous IO system? No - it's a question for the operating system.
Loom and Java NIO can handle probably a billion connections as programmed. Java Threads cannot - although that too is a broken statement. "Linux Threads cannot" is the real statement. You can't have that many for resource reasons. Java Threads are just a thin abstraction on top of that.
Linux out of the box can't do 5M connections (last I checked). It takes Linux tuning artistry to get it there.
Don't get me wrong - I think Loom is cool. It's attempted to do the same thing as Async/Await tried - just better. But it is most definitely not the only way to achieve 5MM connections with Java or anything else. Possibly however, it's the most friendly and intuitive way to do it.
*We typically vilify Java Threads for the Ram they consume. Something like 1M per thread or something (tunable). Loom must still use "some" ram per connection although surely far far less (and of course Linux must use some amount of kernel ram per connection too).
Saying "Linux cannot handle 5M connections with one thread per connection" isn't a reasonable statement because no operating system can do that, they can't even get close. The resource usage of a kernel thread is defined by pretty fundamental limits in operating system architecture, namely, that the kernel doesn't know anything about the software using the thread. Any general purpose kernel will be unable to provision userspace with that many threads without consuming infeasible quantities of RAM.
The reason JVM virtual threads can do this is because the JVM has deep control and understanding of the stack and the heap (it compiled all the code). The reason Loom scalability gets worse if you call into native code is that then you're back to not controlling the stack.
Getting to 10M is therefore very much a question for the JVM as well as the operating system. It'll be heavily affected by GC performance with huge heaps, which luckily modern G1 excels at, it'll be affected by the performance of the JVM's userspace schedulers (ForkJoinPool etc), it'll be affected by the JVM's internal book-keeping logic and many other things. It stresses every level of the stack.
> But it is most definitely not the only way to achieve 5MM connections with Java or anything else. Possibly however, it's the most friendly and intuitive way to do it.
It is the only way to achieve that many connections with Java in a way that's debuggable and observable by the platform and its tools, regardless of its intuitiveness or friendliness to human programmers. It's important to understand that this is an objective technical difference, and one of the cornerstones of the project. Computations that are composed in the asynchronous style are invisible to the runtime. Your server could be overloaded with I/O, and yet your profile will show idle thread pools.
Virtual threads don't just allow you to write something you could do anyway in some other way. They actually do work that has simply been impossible so far at that scale: they allow the runtime and its tools to understand how your program is composed and observe it at runtime in a meaningful and helpful way.
One of the main reasons so many companies turn to Java for their most important server-side applications is that it offers unmatched observability into what the program is doing (at least among other languages/platforms with similar performance). But that ability was missing for high-scale concurrency. Virtual threads add it to the platform.
As the GP said, what's cool about this is how simple the code is. You might be able to achieve 5M connections in Java using an event loop based solution (eg Netty), but if the connection handlers need to do any async work, then they also need to be written using an event loop, which is not how most people write Java. Simply put, 5M connections was not possible using Java in the way most people write Java.
Exactly, even if you make something novel and win the indie lottery, it's going to be ripped off immediately. Apple and Google are really failing at maintaining any level of quality in their app stores.
It’s pretty much all their fault. The same way the internet is overflowing with ad-riddled blogspam due to Google, the appstores are filled with scammy/exploitative clones because that’s what makes the most money when there’s no competitive forces to drive quality control of any kind.
For example, if the Epic game store became known as a source for good mobile games, customers will stop browsing the App/Play store for games and will instead go to the curated and high quality alternative, because why the hell wouldn’t they.
If Apple and Google are forced to stop suppressing competition on iOS/Android, it may lead to an entirely new mobile gaming industry that’s actually somewhat respectable.
> Exactly, even if you make something novel and win the indie lottery, it's going to be ripped off immediately.
Yep, see any of the 200+ Wordle clones and derivatives out there, only about two months after it went viral.
One of my video games has had at least five clones made that I'm aware of (probably several more than that), and it didn't have anywhere near the same success that a game like Wordle did.
If you make a game which can be cloned quickly, you've made something extremely simple. Don't do that.
Anyway, I would certainly not choose to develop mobile games, seems like a bad idea for many reasons. But look at PC games: you can make something as absolutely weird (and arguably 'unmarketable') as Disco Elysium, which needed no "lottery", it's simply a masterpiece.
I agree, when it comes to larger indie games. But there are still simple games that are worth playing, and certainly don't deserve to have their revenue stolen by content farms. The game referenced in the video above is an example of this. And people forget that 2048 was heavily "inspired" by Threes.
It's a separate discussion, but I believe there's still a "lottery" aspect to indie games. There's a lots of survivorship bias as play there. For every Hollow Knight or Disco Elysium, how many other passion projects run out of funding, or get released and go unnoticed?
There's a huge infrastructure for game discovery - Dedicated subreddits with millions of subscribers, curators, massive infrastructure from companies like Valve (Steam) and Google to help discoverability, forums and discussion groups and Discords and Kickstarter...
People are hunting endlessly for good games. But cannot find them.
Even me! I'm a knowledgeable nerd with time, but I can't find anything worth playing. I know many people in the same situation.
There aren't really unknown gems. Great games don't go unnoticed. Many indie devs just like to think they do because they don't want to accept that their game just isn't very good.
Hollow Knight and Disco Elysium are great examples. These could never have gone unnoticed. These are professional efforts from teams of dedicated, very skilled developers, who did a lot of non-obvious things the right way. Nothing replaces a game like Disco Elysium especially. It was always going to get noticed. (I don't know HK as well but from what I've heard it is also exceptional at what it does.)
I'd be interested if you could name any unnoticed game that approaches either of these two in being compelling to play.
The biggest thing they're bringing to the table is the (supposed and impending) funding. The idea itself is nearly worthless until executed.
Good Tech Founders ARE the hardest founding team members to find.
If you accept notably less, you're always a lesser member of the team. The best founding team keeps equity and salary equal. That's the least chance for friction and resentment.
> perhaps because mailinator stopped offering that service
That is untrue. Mailinator definitely still supports pointing any domain to it's MX records and will allow all incoming email (modulo DoS protection, abuse, etc). Such email will arrive in the respective Mailinator inbox (i.e. bob@yourdomain.com goes to the "bob" inbox)
