Serverless first is a mistake. It should be "serverless where it's distinct advantages are actually useful" which is way less common than people think.
Also generally the conception that building things with containers + k8s actually takes more time than serverless is a horseshit take IMO.
It doesn't and if it's taking you longer you are probably doing something wrong.
All the points noted in the slides etc just come from poor engineering leadership. i.e decisions/choices, etc. Constaints do speed up development but they can be enforced by leadership, not by putting yourself in an endless pit of technical debt.
The only world where I can make a good case for serverless for small teams is if you have said poor leadership and serverless helps you escape their poor decision making.
Otherwise assuming you have the mandate necessary to get stuff built the right way just stick with the Boring Tech stack of JVM + PG + k8s. It solves all problems regardless of size, there are minimal decisions to be made (really just Spring or Quarkus atm) and you won't spend time dicking around with half-baked serverless integration crap. Instead you can just use battle tested high performance libraries for literally everything.
Don't get sucked into this serverless nonsense. It's always been bad and it probably always will be. When it's out of fashion Boring Tech will still be here.
Really strong language without any backing arguments whatsoever. You can’t just pick some random tech and decide this is boring tech so it’s good and so the other is bad and has always been. This type of comment is so incredibly meaningless and infuriating.
Please explain why you are saying what you’re saying so at least we can learn from it.
I do think specifying JVM/PG/k8s in that comment was a mistake. Boring tech will be different for different people and companies. Boring tech doesn't just mean it's time-tested and battle-hardened, it means you have deep knowledge across the company because you've used it for so long. I'm in a Python shop, so Python is my boring default tech stack choice. If I introduced C++ into our tech stack, it would be not-boring tech for us, despite C++ not being a cutting edge language.
k8s is funny to call "boring tech" because it's notoriously complex. But if your company has set up dozens of k8s clusters and has a bunch of k8s experts on staff, it can be considered "boring tech." It's all relative.
I think the big thing with "boring" vs "exciting" tech is people pick exciting tech for reasons other than it being a good technical fit (lets board the hype train!!!). Bad results then happen.
Boring tech is usually more stable and more mapped out. Everyone knows where its good and where its not. Other people have already had whatever problems you have.
I'll always bet on the boring tech over the hyped exciting tech.
Those last 2 really don't matter though as neither problem is that difficult if you know what you are doing. They both would have been a huge selling point in a world where k8s doesn't exist. Alas it does and thus mostly moot.
The first however is a big deal as managing very bursty loads on k8s can actually be a bit of a challenge. This is doubly true when you are using some sort of queue and don't have good queue scaling metrics for whatever reason. Serverless lets you cop out of managing that by using infinite concurrency.
Ok so now cons of serverless:
Turns all your code into spaghetti, this is probably the biggest one.
Cold boot performance is highly varible and generally worsens with added function complexity but this depends on tech choice.
Infinite concurrency is a bit of a curse. Want to use that MySQL DB that contains all your business critical data? Better use some MySQL Serverless proxy nonsense because unbounded connections isn't fun.
Lock-in is awful, migration story is awful.
Amount of Infrastructure as Code overhead, ala Terraform, CloudFormation, whatever else floats your boat to support serverless is insane. The sheer number of objects that need to defined, kept in sync etc is just bonkers. Not to mention if you have multiple services setup like this it's all boilerplate you either need to abstract or carry around.
Hides serious programming deficiencies like memory leaks because context is thrown away after servicing request.
Unpredictable runtime makes observability more complicated, probably forcing you into poor decisions as a result, i.e prom push gateway.
Runtime/execution time limits mean people trying to do all-serverless end up doing horrible things to accomplish long-running tasks.
All of the above leads to increasing pressure to make poor architectural decisions to work around limitations of the runtime.
I could probably go on but that is a reasonable set for you.
EDIT: Actually I forgot the worst con. The cost. The $/CPU/RAM cost of Serverless is beyond bonkers. When I was pricing up Cloud Functions vs Spot instances on GKE I think I came to the conclusion Cloud Functions are about 300x as expensive. If your code can run on serverless it can probably run on Spot instances which you can just add to your GKE deployment using another node pool and only schedule things that can handle nodes going away within 60s there and instantly reap 300x cost savings. This is because Serverless is sold at a massive premium to CPU list prices and Spot is sold at like ~30-40% of list prices depending on machine type (for those curious when I last checked T2D was the best for price/performance for highly single thread performance dependent workloads, ala Node.js)
> Turns all your code into spagetti, this is probably the biggest one.
Overall agree with you, but I don't think spaghetti is the right term here - even if the typo is corrected :)
Spaghetti code is more of a term for monolithic mess where you have a hard time following all the code logic (cyclomatic complexity) and dependencies.
Understanding & tracking dependencies is actually easier with serveless, it's pretty clear what dependency that one tiny micro service has.
But where it becomes worse is regarding the code logic mess, you've now turned it in a distributed system problem where, in addition to having a even hard time with cyclomatic complexity, you also have to think about transient failures/retries at each service boundaries.
That and the other problems you describe (performance hit etc)
> Spaghetti code is more of a term for monolithic mess where you have a hard time following all the code logic (cyclomatic complexity) and dependencies.
> Spaghetti code is more of a term for monolithic mess where you have a hard time following all the code logic (cyclomatic complexity) and dependencies.
The only thing worse than a monolithic mess, is the same thing but distributed
> Infinite concurrency is a bit of a curse. Want to use that MySQL DB that contains all your business critical data? Better use some MySQL Serverless proxy nonsense because unbounded connections isn't fun.
Or just set the Lambda concurrency to a fixed value proportional to your MySQL DB's capacity. It's an easy parameter.
> Unpredictable runtime makes observability more complicated, probably forcing you into poor decisions as a result, i.e prom push gateway.
This is just nonsense. What is "unpredictable" about the runtime? What is difficult about enabling X-Ray?
> Runtime/execution time limits mean people trying to do all-serverless end up doing horrible things to accomplish long-running tasks.
"All-serverless" is just as stupid an idea as "no-serverless."
Aside from lock-in, your "cons" are just the cons of anyone trying to build large-scale systems without knowing what they are doing. There is absolutely no way anyone who read the documentation would run into your cons of serverless.
> Infinite concurrency is a bit of a curse. Want to use that MySQL DB that contains all your business critical data? Better use some MySQL Serverless proxy nonsense because unbounded connections isn't fun.
> Turns all your code into spaghetti, this is probably the biggest one.
This is a nonsensical claim. Nothing about serverless requires 'spaghetti' code. Vice versa, nothing aboud non-serverless deployments suddenly ensures well written code.
> Cold boot performance is highly varible and generally worsens with added function complexity but this depends on tech choice.
For many use-cases this isn't a problem. If it is a problem, then sure, don't use serverless.
> Infinite concurrency is a bit of a curse. Want to use that MySQL DB that contains all your business critical data? Better use some MySQL Serverless proxy nonsense because unbounded connections isn't fun.
Sure, but:
A) You get a lot of scalability before that becomes an issue.
B) Setting up a proxy isn't exactly hard. (I get one for free since we use Hasura).
> Lock-in is awful, migration story is awful.
This is nonsense.
Our serverless code runs in docker containers, running perfectly normal ASP.net code. This code could be deployed to a VM, appengine, or k8s in less than a day if needed.
> Amount of Infrastructure as Code overhead, ala Terraform, CloudFormation, whatever else floats your boat to support serverless is insane. The sheer number of objects that need to defined, kept in sync etc is just bonkers. Not to mention if you have multiple services setup like this it's all boilerplate you either need to abstract or carry around.
One of the advantages of our use of serverless is that we don't yet need any of that complexity.
> Hides serious programming deficiencies like memory leaks because context is thrown away after servicing request.
What you descried is literally not a serious deficiencies. If you code leaks memory, but never runs long enough for that to have meaningful impact, it's not a problem.
> Unpredictable runtime makes observability more complicated, probably forcing you into poor decisions as a result, i.e prom push gateway.
The serverless solutions we use have observability built in for free... and datadog automatically can pull that in if you wnat a starting point for more.
> Runtime/execution time limits mean people trying to do all-serverless end up doing horrible things to accomplish long-running tasks.
If you have something that must be long running, serveless isn't a great pick for that one thing (at least, witht he solutions I'm aware of).
That said, there are some elegant ways to handle that using serverless (e.g. by breaking it into smaller tasks with shorter durations)... and going non-serverless isn't a silver bullet, I've seen terrible architectures for long-running code in all kinds of deployments.
> All of the above leads to increasing pressure to make poor architectural decisions to work around limitations of the runtime.
Completely ignoring that serverless can encourage better architectural decisions.
> EDIT: Actually I forgot the worst con. The cost. The $/CPU/RAM cost of Serverless is beyond bonkers. When I was pricing up Cloud Functions vs Spot instances on GKE I think I came to the conclusion Cloud Functions are about 300x as expensive.
Curious.
Let's have a 1s task on Cloud Run with 1 CPU and 1GB of memory:
CPU: $0.00002400/s
Memory: $0.00000250/s
Request: $0.40 / million
Total = $0.0000269 for 1 request lasting 1s.
Current spot pricing for C3 instances is:
$0.003086 / vCPU hour
$0.000413 / GB hour
Or for 1s that would be:
0.00000097/s
Which is 27x cheaper - so you're already off by an order of a magnitude.
It gets worse - that's 1GB of RAM, however the VM needs RAM for it's own OS etc, so maybe you need to provision 2GB of memory, a bit more CPU etc.
The minimum billing time for compute instances according to docs is 60s... so 60x cost if you only need to do this hourly or less.
You're entirely responsible for all time spent, including startup, shut down, idle etc. This just adds more and more cost.
The expected availability model is completely different - serverless should be available in milliseconds, seconds at the worst. Spot instances have no such expectations.
It's a fundamentally different model - a spot VM can't loadbalance, serve code or anything without devops. So now we're spending likely $XXX/hour on devops to try to save fractions of pennies an hour.
When this is all said and done, I don't think you really understand the cost or use-cases of serverless well - and dramatically underestimate the costs of the solutions you present.
As someone who went with Lambda for my current project, I don't agree with this take at all. In fact, it's as boring of an implementation as you can imagine - a single "fat function" developed in .NET serving the entire web application. Can be easily transplanted onto any other hosting paradigm, but Lambda is incredibly cheap (free at my current usage) and incredibly scalable should I hit some inflection point. Learning K8s would have been quite an overkill.
We are thinking the exact same thing (w/ Az Functions). Keeping it simple as possible means you can forklift the whole thing in one afternoon to a different provider. It also means you could potentially even go back to an on-prem stack or whatever.
We are looking at taking our .NET6 monolith and "inverting" it so the entry point is our logical HTTP resources instead of the various DI monstrosities that emerge from main.
The reasoning for us is that we don't want ownership over all of the horrible things we could get wrong. If a <TLS1.2 connection some how gets established and something naughty happens because we forgot to update some startup.cs boilerplate, it's currently our pain today. Like legally. If we are just eating pre-approved/filtered/etc HTTP requests from inside our hermetically-sealed function environment, we get to cast a lot more blame than we otherwise would be able to.
I really don't get why you'd go to serverless and then shit it up with a lot of extra complexity. The whole point in my view is to inherit the compliance packages of your hyperscalar by way of directly using their native primitives in as standard a way as possible. This means you get to 'sail' through audits and engage up-market clients that are typically out of reach for everyone else.
If you have a monolithic entrypoint, is it really serverless? You're simply using the Lambda infra as a run-on-demand server, something Heroku has been offering for 10+ years. Why would you pay extra when you can deploy to a simple VM or managed platform costing 100x less?
What you described would be more akin to using ECS Fargate to run the workload. What they described sounds like a single .NET lambda behind API Gateway, so the infrastructure of ingesting requests (and handling TLS termination, DDoS mitigation, WAF, etc.) all happens on the AWS side. All you have to do is handle the routing inside of the lambda using regular ASP.NET Core. It would also scale up infinitely and down to zero with no work on their end. This isn’t similar to running something on Heroku on an allocated server with fixed overhead at all.
Indeed it would be more like Fargate in terms of container runtime vs lambda runtime, but the line between those is pretty blurry. Fly.io for example does containers but they spawn faster than some cloud function platforms. It's not really relevant for the discussion.
> Heroku on an allocated server with fixed overhead at all
I don't think you're familiar with Heroku then? I'm not talking about dedicated or managed servers. That's exactly what their 'dynos' platform has been doing since ~2008, though less granular: https://www.heroku.com/dynos/scaling
"All you have to do is handle the routing inside of the lambda using regular ASP.NET Core" also applies to both fly.io and Heroku. That's what I'm getting at. If you don't have a collection of independent cloud functions to execute, and you are spinning up a little monolith for every request, the platform seems like overkill.
What makes it serverless is the ability to spin up instances on demand (including down to zero when not in use), while paying only when the resources are in use. Given Lambda's generous free tier, I'm literally paying nothing for it. That would not be the case for any other platform.
Because it doesn’t actually cost less? You have to have a LOT of invocations to match the cost of the entry level 6 dollar digital ocean box. And with that you have to manage that box vs serverless you don’t have to spend engineering time (which costs an order of magnitude more) on maintenance, a big deal for small shops without dedicated/capable ops teams.
I implemented self-hosted k8s in my current company and had to learn it from the ground. I'm not a professional, but I learned a thing or two.
My opinion is as follows:
Installing kubernetes with kubeadm is easy. You can do it in few hours if you're fluent with Linux.
Kubernetes storage is where it starts to be hard. We were lucky to have OpenStack hosting which allows to use network block storage (ceph) without any additional work. Using kubernetes without any storage might severely limits apps you can run there, although it's possible. Using local storage is an option but also with many "but". Rolling out ceph: I've heard it's hard.
Load balancing: again I was lucky to have OpenStack load balancer. Otherwise, I think that it's possible, but definitely not within a few hours, unless you did it before.
Kubernetes networking might be tricky. I wasn't able to implement it in a way I like. I used calico with overlay mode in the end, but I'd prefer "flat" network. Calico overlay mode was easy.
Upgrading kubernetes manually is easy. At least until it breaks, it didn't break for me yet. kubeadm just works.
Node autoscaling is hard. There're no simple solutions, at least I didn't find any. Our cluster does not autoscale. Manual scaling is not hard, though. I can implement autoscaling with loads of bash/terraform/cloudinit scripts but I'd rather not to. I think that node autoscaling is what managed kubernetes offering do well.
Now that we have kubernetes, story is not really over. We need to collect logs for nodes, for services, for pods. We need to collect metrics. We need to set up alerts. We need to have traces. Service mesh looks really nice so you can see how services interact with each other. Those things are not for given. They take lot of time and expertise. And while they're not strictly about kubernetes, usually you need them.
Agree on storage being the hard part. You either need a robust SAN to store persistent volumes or you need to store persistent volumes on the local storage of kubelets which is a major pain. Once a pod binds a local pvc to a kubelet that pod must now run on that kubelet for the rest of eternity. If you want to get rid of a kubelet with a local pvc you either need the application be durable to data loss (which many are not) or you need to implement some sort of stateful pod draining logic.
When rolling your own, I think you need to invest in a battle tested SAN or just not do stateful on k8s. For the handful of stateful services I have run on k8s it would have been better to just use a managed stateful solution.
One of my recent conclusions is that it's not so much that Kubernetes on its own is so very complicated; it's that if someone tries to glue their own container platform together from Kubernetes plus another dozen or so cloud-native projects and integrate the whole thing, it can get wickedly complex very quickly.
Right — Kubernetes was evolved in the context of being a Google Cloud service, and despite now having other deployments, still is mostly architected around the assumption that your Kubernetes cluster is surrounded by a featureful IaaS platform providing things like managed load balancers, VM auto-scaling pools, iSCSI volumes on a dedicated SAN network, non-throughput-bottlenecked object storage (and a container image repo on top of that!), a centralized logs and metrics sink, etc. A k8s cluster expects these like a computer expects a bootable storage device.
If you're using a managed Kubernetes (in GCP, or AWS, or Azure, etc) then the owner of the platform presumably has all this stuff going on somewhere (whether they expose it to you or not), and so is solving all these problems for you. But if you're not using managed Kubernetes — i.e. if you're ever typing `k3s deploy` or whatever equivalent — then you'd better realize that you're signing up to either also deploy and manage your own bare-metal OpenStack deployment for your k8s cluster to plug into; or to tediously glue together a grab-bag of services from your hosting provider + third-parties to achieve the same effects.
(And this is why Google's "Anthos" is an interesting offering to many enterprises: rather than running your own on-prem k8s cluster implying the need to own+manage on-prem supporting infra, you can instead have a GCP project that has all the managed supporting infra, and maybe some hosted k8s clusters; and then your on-prem k8s nodes can just think of themselves as part of that GCP project, relying on the same cloud infra that the hosted clusters are relying on — while still being on-prem nodes, and so still having data security guarantees, low-latency links to your other on-prem systems, etc.)
There are other container platforms like Red Hat OpenShift as well (disclaimer: I work there) either operated for you or on-prem (doesn't need to be on top of OpenStack and typically isn't these days). I'm biased but Kubernetes complexity tends to be conflated with the complexity of a DIY container platform which can be significant, especially given that Kubernetes itself has generally made the decision to keep its scope relatively narrow.
Your comment highlights that you really don't understand what "serverless" is. We run a monolith running on App Engine "NodeJS Flexible". All of our code is basically a single Node monolith, and under the covers many of GCP's newer serverless options (like Cloud Run and Cloud Functions V2) run on the same infrastructure -it's basically just containers, and they even offer versions that let you simply deploy any Docker containerized app.
But compared to the maintenance burden I've seen with other similarly sized startups that use k8s, we just spend way, way less time on infrastructure management.
On a related note, this kind of language, "The only world where I can make a good case for serverless for small teams is if you have said poor leadership and serverless helps you escape their poor decision making." which is basically "if you don't agree with me you're an idiot", says a lot more about the speaker than the speakers usually realize, and it's a huge, giant red flag.
Yeah I am familiar with those (Cloud Run, Fargate et al) and they are much less egregious than Lambda and Cloud Functions. As for if they qualify as Serverless though I don't know, it seems poorly defined.
How about I clarify that my stance mostly relates to Functions as a Service so it's clear that while I wouldn't choose Cloud Run over GKE that is hugely different to FaaS.
I think Cloud Run, Fargate and Azure Container Apps all fall more into the category of "platform-as-a-service" rather than "function-as-a-service", in the same way that EKS or GKE could be categorised as a PAAS.
EDIT: I also think they're _excellent_ to use for development. The migration from ECS + Fargate to EKS + EC2 is not a difficult migration from the application side, and can be done relatively piecemeal, in my experience.
LAMP, this is boring tech. You can switch Apache for Nginx, and MySQL for another SQL database, but that's the idea. People still develop in PHP today, and performance-wise, assuming you follow standard good development practices, and good servers, it can take a lot. If it was good enough to get Facebook started, it is most likely good enough for you, and the situation has much improved since the early days of Facebook, a lot of it thanks to Facebook itself, and also better hardware. PHP security is not as terrible as it once was too.
At work we have a couple of Spring projects, and I don't think I have hated a tech stack more than that. If you to illustrate the "action at a distance" anti-pattern, look no further. I'm sure most Spring web apps could be implement in PHP for a fraction of the size, both in terms of application code and memory. I don't do k8s, so I have no real opinion except that it looks like overkill in many cases. As for Postgres, nothing against it, it is a very good SQL database, though my opinion is that you should limit yourself to basic SQL as much as you can, so that you are not dependent on a particular database.
I don't get how a Spring app rewritten in PHP, assuming a framework is used, reduces code significantly given that PHP5+ is a dynamically-typed Java clone with the same mind-numbing boilerplate.
> All the points noted in the slides etc just come from poor engineering leadership. i.e decisions/choices, etc.
Hindsight is 20/20. It's easy to make these claims after the fact. In the moment it's far more complex and the decisions made at the time take in a lot of factors. Oversimplifying things and placing blame on a tiny thing doesn't push the discussion and learnings forward.
Lambda came out in late 2014 and Step Functions came out in late 2016. It's entirely likely that this team was an early adopter of the tech. Internally a lot of these AWS technologies have a roadmap that sound like they will solve a lot of your issues but those roadmaps get tossed fairly regularly as priorities shift at the product level and AWS-wide level.
We won't be able to get the info on what went on here (likely the team that improved things doesn't even know due to turn over). But watering it down to be "poor engineering leadership" is off the mark and way too simplistic.
I wasn't talking about the Prime Video case, I was talking about the slides in Adrians blog post which compare Serverless with traditional development.
IMO if you are facing those problems it has to be poor leadership, there is no excuse for why you shouldn't be able to make effective decisions if you know what you are doing.
I don't care for serverless, but your suggestion that the only right way to build software is by using the JVM, and the only right way to use the JVM is to use Spring or Quarkus is... puzzling to say the least.
I might as well say that you should run everything on PHP. I guess that's boring too?
You can sub JVM for something else but you should probably stick with PG and k8s.
What I am saying is the highest quality tools are still the best and will remain that way. As other things come and go they absorb the best ideas from the new stuff without losing what makes them good.
Sometimes the new stuff survives (Node/TS/Go/Rust) sometimes it doesn't (Clojure, Scala, arguably Ruby), some we are waiting to find out.
But betting on JVM and .NET has been winning strategy from a technical perspective for a long time now.
Also worth mentioning no, PHP probably doesn't work here because it's not "general purpose" enough IMO.
JVM can handle almost any workload shape, the main one it doesn't do well in is memory constrained environments (though it's possible if you really want to).
PHP, Node/TS, Python, Ruby all suffer from being single threaded so can't take advantage of properly large machines without a ton of extra complexity and you pay a high memory overhead for most of those strategies.
Go is OK, I hate writing it but it can do all the things so if that floats your boat go for it. Rust is also perfectly fine, I just find it is actually pretty slow to write because changing requirements often mean changing structure which can be harder to manage in Rust vs Java/Kotlin/C#.
Anyways, wasn't meant to be a thing about languages. Just pick a good runtime or AoT lang of choice, write everything in it, stick to conventional frameworks, choose PostgreSQL, run the lot on k8s. That is the core takeaway.
I mean, I currently work on the JVM, I just have enough experience with different languages (including Ruby, which I don't think is quite dead yet) and tech stacks to know that they all suck in their own way. The JVM, and specifically Java, and even more specifically Spring, have a lot of annoying warts, it's just that other languages have different warts. I agree that you shouldn't try to rewrite everything in an obscure, untested technology, but beyond that, there's a number of options with enough tradeoffs. But it seems we're largely in agreement there.
I don't necessarily agree with the "keep everything in one language / stack" sentiment. There's certainly companies that overdo it, having 20 different languages at the same time. But there's also valid reasons for keeping certain tech stacks separate (e.g. having to do ML / data analysis stuff in Python). In the worst case, the "use only a single language" mindset leads to disasters like GWT, which is something that my current company unfortunately still uses because some Java developers (that don't work for the company any more) thought that JavaScript was beneath them.
I also wouldn't choose PHP, for the record. I would like to trash it here, but I've never actually been forced to use it, so I can't even comment on it.
I'm not so sure about k8s. I think you can get away with much simpler deployment models, too. At some point the overhead of having to manage everything manually (or a need for zero-downtime deployments) might make the added complexity of k8s more bearable, but it's certainly added complexity and not necessary everywhere and all the time.
I 100% agree about postgres, it's so good and feature-rich that you should probably really only ever use another database if you have very specific needs (except maybe also adding redis as a cache, because redis is also very good).
I haven't yet used Quarkus, but in my opinion the solution to Spring's problems with startup times isn't to run everything in GraalVM, it's to declare dependencies statically instead of creating a mess of annotations that have to be interpreted at runtime. Kotlin IMHO provides ways of doing that rather elegantly.
I don't know about absorbing the best ideas. Java, even after the introduction of raw string literals, still can't handle a regex without escaping metacharacters. Perl had that one solved in the 90s.
> I might as well say that you should run everything on PHP. I guess that's boring too?
PHP can't run everything though. If we go by the traditional model it's per request and fails at things like websockets. It's also less performant than a JVM solution, so definitely can't serve every use case.
Serverless first is a great way to start building your business. As long as you don’t have strict latency requirements. You focus on your business, while the infrastructure requirements are handled for you.
The point of engineering is to deliver the best solution and the right cost. Cost needs to be evaluated on multiple dimensions - actual dollars, time required to build, time required to maintain.
Nothing stops you from using Postgres with Lambda. AWS has done a bunch of work to make this possible.
I would recommend, creating the correct layer of separation in your code, so that when the times comes to migrate from Lambda or other serverless functions to Fargate, cloudrun, or even containers run on VMs managed by K8S, you have to make minimal changes to get up and running.
Spring is fun until you need to do anything mildly complex and then you tear your hair out trying to find the right documentation. You have to navigate myriad of sub projects. They really really need to sort out the SEO. It’s atrocious.
The only way to learn Spring is to read the latest manuals of every subproject you think you might need, cover to cover.
Even then you might not figure out what awkward collection of beans you need to override the magic Spring does.
I found Spring to have a little bit of an understandability cliff. Coming from Guice though once I understood the container and had a feel for the general patterns it was easy.
I think if you try to use it without understanding the magic you can probably get pretty far but you will always feel lost when things don't work just like the example. Understanding how all the runtime wiring works at the DI/IoC layer generally makes it pretty easy to work out what is going on though.
Yeah I very much prefer Kotlin now. It's Boring Tech but at the same time pleasant because of it's high expressivity and tooling.
Also I find it way easier to convince people coming from a dynamic language background to try it out vs Java which they find less approachable. Which is ironic as Kotlin is by far the more complicated language vs Java.
I see the anti-K8S sentiment so much on hackernews. Sure, there’s a time and a place for K8S, but I see a lot of point-blank objection to it for often weak reasons.
Actually your comment is a great demonstration of poor engineering leadership. You've already decided on an outcome and are shifting the narrative to adhere to this outcome.
The labelling of JVM + K8 as boring, to posture why it's automatically superior to "serverless integration crap" is exactly the sort of thing poor engineering leadership does to avoid having the sort of difficult conversations necessary. There's no metrics involved, no studies, no prototypes, no financial assessments, none of that it's purely based on emotion.
You can envision the same exact thing happening (but to the preference of serverless) on the Prime Video team.
It's not emotion. I make money from porting people off of poor technical choices onto solid ones. It comes from experience working with all these various systems.
I have used (at large scale) GAE, Google Cloud Functions and Lambda, before that I also used all the previous "serverless-like" things including Beanstalk (anyone else even remember that?), Heroku and bunch of other PaaS things for various stuff.
It's all come and gone. To me serverless is just the latest in a long line of things that I get paid to replace with things that actually solve the problem and doesn't turn into a ball of spaghetti that no one is able to modify anymore.
I remember beanstalk. We used it for years at twitch. We got a shit ton of mileage out of it. It's easy to talk shit about the problems of the old tech, and beanstalk has a lot of issues. But it also let us build a ton of stuff much easier than the alternatives.
This cycle will forever be this way. You only think that the new stuff is the best if this is your first rodeo. If it's not, it's just the right or wrong tradeoff for the point in time you are at.
> I make money from porting people off of poor technical choices onto solid ones
So it was successful enough for long enough to get them to the point where someone cared about optimizing and paid you to come in, right? I don't see how you are disproving the article's argument that this is a great way to prototype ideas or get a first version of the product out there.
My opinion from doing both these ports and greenfield is this:
None of the tools that say they help you move faster matter after the first 3 months.
It's true they take less code and pieces to get started with. No k8s manifests, no Dockerfiles (usually) but if you don't hate your future self you will still use IaC to create the API Gateways/function contexts/DynamoDB/whatever, so you will still be dicking around with at least one of Terraform/Pulumi/similar.
The thing is the setup cost of a traditional lang + pg + k8s setup amortised over 3 months is essentially nothing and yet you have a setup that will last you effectively forever.
If your company wasn't going to die because of being a few days slower in that initial 3 month period you have came out ahead. You have a forever architecture and you avoided a whole bunch of costs and technical debt in the process.
Maybe the calculus is different if you don't have the same depth of infrastructure experience I do but for me it's obvious this is a faster and better way of delivering a prototype that gives it proper room to grow into production software.
The IAC piece is the same across both. Except with k8s you’re managing infrastructure across cloud, k8s, potentially frameworks like helm, etc.
And on a large team there’s very little likelihood that everyone understands the different configuration layers. There’s very little likelihood that people understand the scope that changes to configuration might touch either. But when you work with CDK, it’s one tool to understand.
Faster and better way to grow into production ignores team level problems and team level constraints. At 10, 20, 100 engineers this is fine. But when you have dozens of teams you’re almost certainly losing efficiency by being prescriptive.
I find that this type of pedantry really surrounds the k8s ecosystem. I think k8s is an enterprise scale tool that has uses in enterprise applications. But the idea of suggesting it to everyone is lunacy
I am more of a generalist than you are but I have worked with a Kubernetes setup and I've also worked with going all-in on Lambda and other AWS services managed by CDK and I don't share your intuition. Besides the initial setup, the former also has other ongoing maintenance things that are harder to deal with. It does give you more flexibility if whatever you're doing isn't a good fit for those tools, though (though obviously nothing stops you from provisioning EC2 instances or whatever through other tools).
Oddly I'm not able to reply directly to re-thc, so I hope you don't mind me commenting here that - I suspect that they're not deploying with a CDK-defined Code Pipeline (starter article[0], in-depth example[1]), which makes cross account/region deployments so ludicrously smooth and simple that it often has newbies refuse to believe that it can have worked _that_ easily.
It is true that _some_ resources don't have L2 constructs, but there's pretty damn good coverage of the most popular services - and, even when you have to drop back to L1, that's still better than hand-written CFN templates or abandoning IaC entirely!
It’s just… half baked. It feels like the CDK team constantly has to work circles around the lacking functionality in CFN. Want to upload a docker image to ECR, use this hacky lambda function. Want to create an object in an S3 bucket? Use this other hacky lambda function.
Then a lot of the resources depend on each other in funky ways, but they’re not always implicitly defined, so whether everything will be created in the right order is anyone’s guess and you often have to define the dependencies manually anyway.
Then when it fails to bring up/modify a stack you get the helpful message that resource x failed to create, but nearly never why.
Like, CDK is marginally better than plain CF, but something like Terraform or Pulumi is so much better that it’s hard to imagine anyone would want to use anything else.
Also, like someone else said, the stacks are directly bound to specific accounts/regions. When I bring domething up that is my QA environment, I’d enjoy it if I didn’t have to split every stack in twain (cloudfront certificates that can only be in us-east-1 or something).
Oh, and when you finally work through all these issues and work on your app for a few months, the dreaded message will come up… “Maximum of 500 resources in stack reached”.
> It feels like the CDK team constantly has to work circles around the lacking functionality in CFN. Want to upload a docker image to ECR, use this hacky lambda function. Want to create an object in an S3 bucket? Use this other hacky lambda function.
It feels like you have a different boundary of what constitutes "infrastructure" than the CDK team do. I'd call both of those examples content _within_ an infrastructure component. Would you expect CDK to initialize an SQS Queue with a message on it? Or a DynamoDB table with data in it? I guess you could make that argument, but to me _populating_ your infrastructure is a very different task from _making_ it - it's an application-level task, not a infrastructure-level one.
Most of the rest of your comments are pretty subjective - I can't say I agree with most of them or have experienced them, but I also don't disbelieve you that they've happened to you.
Not sure what you mean by having an issue with "stacks [being] directly bound to specific accounts/regions", though. Why is that an issue? Why would you _want_ a stack that spans multiple account/regions? I just successfully created and validated a Certificate in us-west-2, so I'm not sure what your parenthetical means.
Cross region/account usage is terrible. It's also 1/2 baked. Some things have higher level constructs but some don't. Lots of things have to be "escaped" to get it to work.
If you're just on a single region/account then yes it's better than alternatives.
I suspect that you're not deploying with a CDK-defined Code Pipeline (starter article[0], in-depth example[1]), which makes cross account/region deployments so ludicrously smooth and simple that it often has newbies refuse to believe that it can have worked _that_ easily.
It is true that _some_ resources don't have L2 constructs, but there's pretty damn good coverage of the most popular services - and, even when you have to drop back to L1, that's still better than hand-written CFN templates or abandoning IaC entirely!
"So it was successful enough for long enough " Which doesn't necessarily make it a good choice at any point. It just says it wasn't enough of a bad technical choice to make the product sink.
"It works and makes money" is at the root of so much shoddy workmanship in software I think you're making a different point than what you're trying to.
The argument of TFA is that starting with serverless and evolving away from it bit by bit as it is revealed to be necessary is a sound strategy most of the time. Whatever you may think of that argument, “I have been hired to optimize projects built using serverless and get them off of it” is not a rebuttal.
>this is a great way to prototype ideas or get a first version of the product out there
As long as it works and makes money it excuses shoddiness. Publishing a draft to fish for revenue is the reason we have so much poor software running out there. After it's in production, you will not be able to make a business case to re-make it right.
This has nothing to do with software. Same applies for things like buildings where the developers get away with so many defects even after people pay huge $$$ for it.
If you consider building your service that should take 3000 man hours to build in 300k instead, overspending by a factor if a 100, and then still succeeding, then yeah, you could consider that a success.
I use serverless successfully - handing tens of millions of annual revenue.
We do this in a handful of small services, none of which gets much traffic. The alternatives (e.g. app-engine, container in vm, or k8s) all require more infrastructure, would cost more (no scaling to zero), and are overall less suited for the kinds of problems we expect to see.
I too have a small team that built a greenfield application that generates about $100M annually for a large hotel chain. Serverless allowed us to get to market fast and everyone in the team is a feature developer — 0 infra folks. I think that’s the biggest value we have been able to derive from this compared all the other internal App teams who struggle with release cadences, experimentation and TTV.
But I fully acknowledge this is not the ideal and optimal setup and we are paying more to AWS on the OpEx. However, we have gone 3 years without any downtime at full speed, and the fact that we were able to unlock net new revenue very quickly has made all project costs and OpEx very minuscule.
But I think once the application boundaries stabilize and we are able to take a breather from full on feature development, hope to migrate some of this to simpler containerized infrastructure.
To make this transition easier, we use Serverless-framework and decouple the app component architecture from deployment architecture, so developers don’t worry whether this is getting deployed to a lambda or container.
Seconded, just got done with one of these. Literally 1200 separate lambdas, step functions, hundreds of kinesis streams, and sqs topics plus a smattering of ecs fargate tasks for when invariably something takes longer than the max lambda time. And oh god all the stupid supporting infrastructure to build layers.
There primary app was Rails, you know what replaced all of it? Running the synchronous tasks on the app servers themselves, the async ones on Sidekiq and changing the max instances on the two ASGs.
The only downside has been not being able to meme about how we're web scale anymore.
Not being able to make jokes about how fast it is to serve requests from /dev/zero must be absolutely killing your team.
However, great job. Once they get to that scale of objects it becomes really hard to clean it up - especially if most the people that wrote them have already left the company for example.
Curious if you ended up also saving money in the process?
One potential reason to advocate for serverless architecture is it is relatively stateless and event-driven and therefore modular and easy to reason about.
Can you be more specific about how serverless tends to turn into a ball of spaghetti? It is simply because the lack of state becomes more of a bug than a feature?
State can be part of it. For example it can be way simpler to use a in-memory cache than a network attached cache and by doing so you save yourself a dependency, a potential network request that can fail, etc.
Also for event processing consider the case you need to process events in order for instance. That is a ton easier with stateful consumers, batching etc which while possible in serverless is usually more difficult/off the beaten path.
However by spaghetti I'm more referring to the architecture of your application being turned into networked components, i.e a distributed system.
It's pretty much accepted fact now that distributed systems are by far the most challenging area of both computer science and software engineering.
Serverless doesn't technically -mandate- this, you could put all your code in a giant function. However it strongly incentivises you not to. a) generally it's deployment times and cold start issues get worse with function size and b) it pushes you to adopt other "building blocks" which are themselves network attached.
Essentially it leads people into a massively distributed micro-service architecture right out of the gate. For most applications this is definitely the wrong choice even if you have the folks on staff that specialise in distributed systems, usually those folk are the last to advocate for anything distributed ironically.
Getting into the weeds as for why this is hard and all the problems you can run into is probably a bit much for a single HN comment but if you really are curious I suggest reading Leslie Lamport and John Ousterhout. Both have covered the problem space in depth with foundational papers, Leslies "Time, Clocks, and the Ordering of Events in a Distributed System" in particular is foundational.
> Also for event processing consider the case you need to process events in order for instance. That is a ton easier with stateful consumers, batching etc which while possible in serverless is usually more difficult/off the beaten path
By off the beaten path you mean a few lines of yaml to configure a FIFO SQS queue and have it trigger a Lambda?
I notice the GP hasn't responded to this. If Lambdas couldn't be a target for SNS or SQS, I wouldn't bother with them either. But that has been a core architectural pattern since 2018.
(quite impressed by the sheer number of comments you have made in this thread, btw)
> For example it can be way simpler to use a in-memory cache than a network attached cache and by doing so you save yourself a dependency, a potential network request that can fail, etc.
Elsewhere you decided against using cloudflare's kv because it was a poor fit for "cache invalidation" -- "way simpler"? -- and then of course turns out you really don't know if it's a bad fit.
> by spaghetti I'm more referring to the architecture of your application being turned into networked components, i.e a distributed system.
That is a novel definition of "spagetti". It's a pejorative term in software sense. Why?
> It's pretty much accepted fact now that distributed systems are by far the most challenging area of both computer science and software engineering.
Ahem. So it caching and cache invalidation. But that didn't stop you, so exactly how is this line you seem to draw with such vocal authority regarding "bad hard", "good hard" informed technically?
> That is a novel definition of "spagetti". It's a pejorative term in software sense. Why?
I don't necessarily agree with everything OP is saying, and I certainly don't speak for them, but I completely understand what they mean by a big ball of microservice spaghetti. The spaghetti is distributed across a massive literal network of dependencies instead of being distributed off a giant graph of object dependencies, or goto statements or something. The spaghetti is real and just as messy in the microservice world as the monolith world. I should know, we inherited a massive one.
But this shouldn't be too much of a revelation since there is no silver bullet in taming software architecture. Architecting software is very subjective, and more often than not we make the wrong decisions and have to untangle the mistakes further down the road.
> Elsewhere you decided against using cloudflare's kv because it was a poor fit for "cache invalidation" -- "way simpler"? -- and then of course turns out you really don't know if it's a bad fit.
I evaluated it at the time. I don't have the context loaded to remember exactly why it didn't work for my use-case but I'm going to look again because CFs platform moves pretty fast and perhaps it could be simpler than what we are currently doing.
> That is a novel definition of "spagetti". It's a pejorative term in software sense. Why?
Elsewhere it's been coined Lambda Pinball, perhaps you find that a more pleasing description?
> Ahem. So it caching and cache invalidation. But that didn't stop you, so exactly how is this line you seem to draw with such vocal authority regarding "bad hard", "good hard" informed technically?
Yes, both of which are generally necessary in distributed systems. i.e not only is it a hard problem in and of itself it requires solving sub-problems which are also some of the hardest problems.
I think in general you’re right on the money and have been throughout your comments here. Lambda pinball is a great term as well. I agree that serverless incentivizes a “nanoservice” approach that flies in the face of patterns like classes/factories or DDD, since everything is atomic you lose the concept of a service or domain boundary and everything can get muddled as hell.
Statelessnes is really a prerequisite for proper horizontal scaling (with small exceptions), so best practices for e.g. spring boot web applications will lead to a stateless architecture as well. Same goes for event-driven -> amqp, mqtt (and jms for java) plug easily into most modern web-frameworks, you can be as event-driven as you want to (which in my experience is an overhead to plain http / rest which needs to be justified)
> Statelessnes is really a prerequisite for proper horizontal scaling
I could understand making this claim w.r.t. Idempotence. Claiming statelessnes as a requirement strikes me as non-trivial enough to require some evidence or a citation thereto.
Instead of your IDE very easily telling you on click the callers or callees of things you'd rather have events where you have no idea who might be processing or sending a particular event in your system of hundreds of microservices?
I think you're conflating problems here. IDLs allow interface definitions for interactions that cross compute and administrative boundaries, and event-driven architectures do not require compute distribution (Erlang/beam).
Different people have different strengths and anxieties. The suggestion that, with some regularity, the lack of (local) ambiguity of event message arrival may be of higher leverage relative to complexity than increasing the surface of an invokable interface does not seem beyond the pale.
Ultimately relinquishing a total temporal ordering is going to be painful regardless of whether it's local or distributed.
That's fair enough if you use something that actually does this and also keeps track of users of your event. Not saying it is unsolvable.
What I've seen though is that in many cases it is unsolved for and it's very hard to reason about "if I remove this event because it's 5 versions too old already and I don't want to keep this backwards compatibility layer, will something "at the other end of our system I never even interact with" puke?
Granted some orgs are so big that it makes sense to solve for these large landscapes of services with backwards compatibility ad absurdum in order to allow anything to get done (been at that bank, done that). But reason about the entire system it does not make easier.
At many smaller orgs these services were also not built because of organizational issues that needed to be solved but because "everyone does this now" and there's no proper registry of properly defined interfaces and trying to find out if a change you were going to make is going to take you forever because you do need to version something or if you can just change things as needed is way harder than it's supposed to be at that size org.
Fair enough that it's freeing overall if your org is so large that it is faster to always be 100% backwards compatible for aeons.
I repeat: most orgs that go all in on this are actually so small that being so compatible is more of a drag than understanding the part of the system that is attached to the interface you are trying to change ;)
Well that's what I mean about choosing the right events. If you are needing to keep multiple systems in your head all the time the boundaries are wrong. But it's true that evolving the interface of services is hard.
As a one man shop, I made a decision years ago to try Beanstalk for a particular app backend / API. It's the only "serverless" deployment I've done. It still works fine for that purpose, and it's less trouble than manually deploying to multiple servers. Forced updates are an occasional headache but at least they're all in one place. The code does a lot of serverside computation for a lot of client connections, so that part needs to scale, and since there's relatively little database stuff it just connects to a single RDS instance.
I can't think of any other code of mine for which I'd use Beanstalk or something similar now, but it doesn't hurt to evaluate each project independently and pick the right tools for the job.
Yeah Beanstalk was pretty simple and decent for it's time.
I told anyone that wasn't comfortable with Cloudformation + Packer and wanted to be on AWS to use it during that era. At that time things were pretty binary, you used PaaS like Beanstalk or Heroku or you were off in the wild west where there was no standards at all. People were doing immutable AMIs on ASGs at the high end and then everything down to pet AWS instances w/Capistrano or similar tools at the low end.
These days things are different, containers were a big deal. If you target k8s your application is pretty portable, can be run locally in something like kind to integration test the infra components, etc. You can take it to the 3 major clouds and bring most of your ingress, secrets, volume, etc configuration with you.
For me k8s has been a godsend because for once I haven't had to learn a whole new set of idioms each new place that are all equivalent but different in their own special snowflake ways.
> You can take it to the 3 major clouds and bring most of your ingress, secrets, volume, etc configuration with you.
How often have you seen someone actually do this, and was it ever actually as "easy" as they imagined? I feel like the advantages of "going with the flow" and using the stuff specific to the cloud environment you're in are substantial and I feel like the efforts to avoid vendor lock-in start to look insufficient once you try to actually execute most of the time.
I have done one massive AWS + kops > GKE migration and several smaller migrations between AWS/GCP/Azure.
My experience leads me to believe the main problems stem from adopting IaaS/PaaS specific stuff outside of k8s. If you aren't doing that they are very portable and even if you are if you are using standard stuff like managed PostgreSQL/MySQL then it's still easy.
Where that breaks down is very vendor specific stuff, ala Spanner, queues/logs like SQS/PubSub etc. If you are using those and your architecture is tied to specific semantics of them you are in for a rough time.
I personally try to avoid any and all of those. Not for vendor lock-in reasons but because I greatly dislike not being able to run everything locally in exactly the same configuration it will be deployed in.
So I run PostgreSQL in k8s using Patroni, Kafka or Pulsar for queue/log, etc and that is about it usually. I generally don't need/use other things. Maybe memcached if there is a good need for it over say Caffiene (in-memory cache).
This is awesome because it means it's super super easy to have staging/UAT environment or even preview environments on PRs and I can run integration tests against the whole stack E2E without spinning up/manipulating cloud provider resources.
One nice thing about Beanstalk was that you didn't (don't) have to deal with containers on the toolchain at all. You basically zip the directory structure you'd normally serve and it just gets rolled out to all the instances. What I figured out early (the hard way) was that it wasn't worth the time trying to screw around with leader instances that installed and ran cron jobs or anything like that. I just have an little separate ec2 instance that acts as a heartbeat for cleaning up the DB and buckets, rather than trying to get Beanstalk to do that consistently. I think it's extremely well suited for one layer, the API; and counterintuitively I find it easier not to containerize that, because it gives me more flexibility to switch DBs or storage engines without touching the API, and without having to rework a container and redeploy the API itself.
Have you tried Cloudflare Workers? I work at Cloudflare but I’m genuinely curious what your feedback is on it as it does work quite differently from Lambda/Cloudflare functions.
Yes actually, currently using in production. I actually advocated for the move to Cloudflare and we ended up adopting workers because Cloudflare unfortunately has broken stale-while-revalidate support. So I built an implementation in Workers that abuses the Cache API to store our own TTLs and also our revalidating state. It works pretty well as we don't end up with too many isolates per POP but it's not perfect.
I didn't end up using KV or other features yet, they weren't a good fit for the cache revalidation implementation.
I think at-least for now I see it differently to FaaS services like Lambda/CloudFunctions because I don't intend to use it to implement application logic, rather just augment (or in this case shore up deficiencies) in the edge.
We may consider doing more stuff at the edge later for latency advantages.
This to me changes the calculus because Lamba (well vanilla, not Lamda@Edge) and CF solve my usual problems just with higher costs.
CloudFlare workers actually let me do things I couldn't do.
So the cost is still just as high as FaaS platforms BUT importantly I actually get something for it, something I can't otherwise have and important enough I am willing to pay the costs and overheads for.
Please advocate for real state-while-revalidate support internally! It would make tons of folks super happy.
Interesting. Since I TL KV currently I’m particularly interested in the ways it’s not working for you. We’re in the process of rolling out major changes here that might help you here so hit me up on the KV Discord or my username at cloudflare.com. I actually do have a meeting with the Cache team on Monday about something related and I’ll ask about it if you could message me what specifically is broke / what the semantics you want are (I’m not intimately familiar with all the HTTP semantics).
I must admit I didn't go super deep on KV so I will come by discord at some point. I can share what we are doing and maybe it turns out KV is better suited.
> I make money from porting people off of poor technical choices
I make money by delivering features my customers care about.
We tried k8s, it was a ton of mental effort and time wasted when we could have been building more features. That particular bit of infrastructure is currently running in elastic beanstalk, where it's cheap and I don't have to think about it.
I actually think Beanstalk/Cloud Run/friends are fine. I should have specified that Functions as a Service is the main mess I think people should be avoiding.
If Beanstalk + ELB + RDS or whatever is getting you by just fine then by all means. It's super easy to port that to k8s whenever you end up needing what it does, which might be never if your money/business scales well vs infra complexity.
Has GAE come and gone? Surely it’s still there, and capable of running the JVM? K8s seems like overkill for small teams which don’t already have experience
GAE is actually still pretty good tbh. However it has it's limitations and even though there are things like Cloud Run et al I still find GKE is the best option.
> Actually your comment is a great demonstration of poor engineering leadership. You've already decided on an outcome and are shifting the narrative to adhere to this outcome.
For me it's actually the decision of a CTO of one of top 10 companies in the world in terms of software size/value/etc. to push it down to very smart people - that's bad leadership.
Why? Because I guess this thing didn't go as planned - a self fulfilling prophecy that shows how awesome serverless is. It backfired in a period where people are questioning whether all these megahypes are not actually just some marketing. Yes, he is responsible for that, for pushing it down to his people that "everything new must be started with serverless" first. When actually in some situations, as the article showed, a monolith or a (big) microservice would do the job better.
Why the hell hire super smart people when you have to tell them how to do things?
Those smart engineers should be encouraged to devote their brainpower to other aspects of the business and technology stack.
Far too often teams use an opportunity to build something new as a science project to use all the latest wiz bang cloud features when the boring stack would work just fine.
For those that truly want to innovate on that aspect of the stack, they should be part of a central infrastructure team, whose job it is to build scalable solutions for the group.
Replying to your deleted comment on another thread:
IMHO if possible, grow your business without investors and retain control. Jason Cohen has an amazing talk on bootstrapping, with some very solid advice from his own experiences: https://m.youtube.com/watch?v=otbnC2zE2rw
Look at co-working spaces near you? Our business worked in an incubator for a bit, and the problems we saw with investors in other startups were eye-watering. We retained 100% ownership, and that was the right decision for us. Learn from the mistakes of others at different stages just by being around them.
I was a founder, and I don’t know shit about VC except through reading, so be appropriately skeptical about my thoughts and only take what is of value for your circumstances.
Beware: the VC model and the whole industry can at times be parasitic, self-serving, and definitely has vibes of Ponzi or pyramid schemes, and the worst excesses are hidden from view if you are not in the industry.
> Why the hell hire super smart people when you have to tell them how to do things?
A very hard question. One side is that every smart engineer will do what's best for their use case.
The other is now you might have 100 smart engineers with 100 smart different solutions which can cause repeated work, integration issues, missed common solutions, etc.
Both approaches have lots of pros cons and tradeoffs
Generally the best approach there is to elevate a bunch of your smartest engineers into an architecture team that a) set good standards but also b) appreciates the nuance of problems sufficiently to know when to deviate.
A small team can probably only handle one tech stack, an org with 10 teams can probably handle 10 stacks if they needed to but can get efficiency by standardising on some number less than 10.
However making those calls shouldn't be coming down from the CTO, it should be made among the architects that actually design and build the systems.
Where I'd consider a (good) CTO should be somehow responsible for the high-level architectural decisions. Accumulate enough expertise and know-how (via "architects" or rather very senior engineers) and then discuss the pros/cons and at the end weigh each argument and decide what may work best.
Fully agree with this - it completely ignores the context and domain for a one size fits all solution, which is precisely what is wrong with engineering today - lack of holism.
The OP's comment "Also generally the conception that building things with containers + k8s actually takes more time than serverless is a horseshit take IMO. It doesn't and if it's taking you longer you are probably doing something wrong."
I couldn't disagree with more from a hands on practical perspective. That will 100% take anyone longer to spin up for an MVP (for more cost) than spinning up a FreeBSD server and I'm unaware of anything that needs more than that for an MVP.
This isn't even mentioning the fact that every second your data sits in an AWS/CGP/Azure datacenter - you are not only telling them what kind of data you have (don't pretend like the hosts don't know and can't tell) but you're increasing your switching/egress costs
> I couldn't disagree with more from a hands on practical perspective. That will 100% take anyone longer to spin up for an MVP (for more cost) than spinning up a FreeBSD server and I'm unaware of anything that needs more than that for an MVP.
I agree with OP. If you hand me a binary and instructions on how to run it, I can have it running on either AWS or DO in a container, connected to a managed database with snapshots enabled in about 30 minutes. If the toolchain is easy to set up, I can have CI/CD with 0-downtime deployments with auto-rollback in another 30 minutes, and you won't need to touch either of those until you're hitting thousands of concurrent users for a typical web app.
> you are not only telling them what kind of data you have (don't pretend like the hosts don't know and can't tell)
this is FUD of the n'th degree - if you suspect AWS are spying on your managed RDS data, I don't know what to tell you.
> Great…So you quickly put yourself in a position where the longer you’re on the service the more expensive it will be to migrate
With 30 minutes work, I've set up a production quality solution that I won't have to touch for months/years, and has almost no vendor lock-in. Containers are portable, you can run them on AWS, DO, GCP, or install docker on a bare metal instance, it's up to you. Until you _do_ decide to migrate, it's pretty much 0 upkeep (I checked one of the work projects I stood up and we haven't touched the infrastructure side of it for 9 months).
> And yes Amazon uses AWS utilization to keep track of promising startups or places that they want signal about market changes.
That's not what you said at first, you said "you are not only telling them what kind of data you have (don't pretend like the hosts don't know and can't tell)". You're telling AWS that you're running containers and a database.
Yes, and I’m sure after you set things up with “no vendor lock in”, you came back years later when the company grew and organized a large migration where you had to deal with budgets, transferring data, regression testing, going through architecture review boards, the PMO, permissions, compliance, training, etc.
I’m also sure after many years no vendor specific corner cases snuck in that you didn’t realize were there
OP has constrained the technology choices of the org into a very capable set of primitives. The entire org can move faster by sitting atop the same rails.
K8s + tried and true tech lets the org spin up additional compute anywhere and easily migrate off a particular vendor. Lift and shift is work, but not nearly as bad when the rails are simple.
Stop with the confident silver bullets. There are effective teams using every stack out there, including serverless. The professional chooses the right tool for the job and team. This is an instance of one change between options that happened to blow in the direction you like, but there are good options at all points of the continuum.
I've used most languages in on-prem and distributed settings, from app to web app to micro services to serverless to single DB's of all kinds to distributed everything, in all clouds, and all have their place and time and team. Sometimes I'd arrive with a perfectly confident view and then learn that actually there is another truth. Sometimes a 50-page stored procedure is perfect. Microservices is awesome in the right context, monolith in the right context, just stop with the simple answers because you have a particular lens that works for you.
What a single lens does really well is that it removes options and that makes you faster at how you solve most problems. Good for you, simplicity is excellent. But don't diss professionals and their leaders who have made choices and then adjust them based on the data.
There is a big difference between learning the nuances of engineering as you go and re-adjusting your viewpoint vs bandwagoning on a new tool because it's cool even though you can't tie any of it's advantages back to your problem statement.
That isn't making a choice and adjusting based on the data, that is just shoddy engineering.
It should have been obvious to anyone at a cursory glance that not only did serverless not offer anything to this application it would be actively detrimental. Simple napkin math of data rate of a 1080p video stream should have made that immediately clear.
Just because you can use something to get a task done doesn't make it a good choice. I can still write pretty decent Perl. It gets the job done and for some protocols that haven't changed in a long time like LDAP the existing CPAN libraries are still some of the best. But that isn't the right choice is it?
I don't think it's defensible to argue that we should build thing sub-standardly just to give people more choice. That is like arguing that real engineers should have more freedom in choice in how they design bridges. No one would agree with that.
We get away with it because our bridges are much cheaper to replace and noone dies if they break or have to be rewritten because they rotted too much to run anymore.
Right now as it stands arguments for serverless aren't technical in nature. They are a best an appeal to "the traditional way is slow" or some notion of "infinite scalability", the first of which isn't true at all and the latter doesn't matter for anyone that is arguing in favor of serverless.
Also even if it was "faster" (I don't agree but lets postulate) that probably wouldn't make it a good decision either unless it was proportionally faster vs it's costs (not just dollar costs, architecture, complexity of managed objects, etc). Which I highly doubt would be the case.
As you can see from a technical POV it's just very very hard to rationalise a position for serverless > Boring Tech. For me it would take too much mental gymnastics.
I was perhaps a little strong but my points are solid.
If it was good it would have proven advantages by now. K8s adoption curve vs serverless adoption curve tells the real story of quality IMO, especially given how much bigger of a move it was to k8s from what preceeded it.
> I don't think it's defensible to argue that we should build thing sub-standardly just to give people more choice
That's like a village of straw men. Stop going around telling everyone they're fucking idiots. We'd probably agree 90% of the time, dealing down the intensity just makes for a better discussion.
They "already owned a tool for audio/video quality inspection, but we never intended nor designed it to run at high scale (our target was to monitor thousands of concurrent streams and grow that number over time)"
Good principle #1, reuse. They launched faster than starting from scratch. They changed direction and modified some components, and then published, against what marketing would probably have wanted. Real problems are when you paint yourself into a corner and can't fix it. They fixed it.
Amazon, keep telling us what lessons you've learned because it helps the whole industry.
(Edited to remove some of my own incendiary annoyance.)
> Serverless first is a mistake. It should be "serverless where it's distinct advantages are actually useful" which is way less common than people think.
Wouldn't that approach take away the remaining advantages of Serverless while still leaving you with the cost?
My understanding was that Serverless is supposed to free you from the burden of maintaining a complete software stack, including configuring the db, server, application runtime etc and just focus on the business logic - a bit like one of the old "sharehost"/LAMPP setups from a few decades ago.
If that is true, then you'd be well-advised to either go all-in and structure your entire application as lambda functions, or don't use functions at all and go with a "traditional" kubernetes deployment.
As both strategies come with considerable complexity cost, combining them would give you the cost of both. I think you'd want to avoid that if possible.
Somewhat. I think if I was to ever use serverless for something it would be in concert with a traditional setup so in a way yes, you pay 2x the deployment complexity because now you are orchestrating 2 environments.
However if you have a usecase that is absolutely a perfect fit for serverless i.e incredibly bursty, unpredicable, very short runtime. To the point it would be to the detriment of other services hosted on the same infrastructure i.e would need say an isolated cluster anyway then yeah, serverless could be a cost worth paying.
Doesn't JVM here result in some amount of "dicking around" with respect to which JVM language to use? Or do you mean that it doesn't matter, as long as it's all JVM? Or do you mean to just pick java?
One thing I've seen in my experience generally buying this "standardize on boring tech" take, is that it's really common these days for a company to be doing some amount of "serious" data science work, and most people seem to really want to do that in python, and it also seems to benefit pretty significantly from some kind of OLAP database system. So adding those things onto your "boring" stack becomes JVM+psql+k8s+python+snowflake/databricks/duckdb(?).
But I agree with your real point about serverless.
For backend, JVM implies Java. Although Kotlin is safe choice as well, it's not as popular so I wouldn't suggest to use it, unless you have good reasons to (like team of Android developers who decided to build backend).
Reasons for Kotlin is mostly nullability typing though @NotNull gets you pretty far in Java these days with the tooling configured correctly.
I like the syntax a bit more and extension functions are a nice sugar vs util classes with static functions.
The main reason however is I have a lot more luck convincing Typescript developers to try Kotlin than Java which is relevant when most of my colleagues have primarily done Typescript.
Either or though IMO, both are great choices for backend development and they interoperate really well.
Yeah, I mean if you're already a JVM shop then you could probably run Spark yourself (which can be painful) and use SparkSQL to dump to Python (maybe on databricks, though I'm not a massive fan) and do your fancy ML stuff there.
Alternatively, you could just use a read-only PG replica and Python here, which would suffice for most DS needs (unless your core product is DS related or you have huuuuuggggggggeeee data (which you probably don't)).
Yeah that is fair too. Most people have tiny data. Then there is places I have worked... where 10TB qualified as small data that you "just query with Presto" and where more exotic things were needed for actually interacting with the "big data".
+1 that k8s really isn't that much more challenging configuration syntax than most 'serverless' solutions (which is a bit different than FaaS as in the article). The difficulty with k8s is (w/ a managed solution) generally the cost of running the control plane though most IaaS will 'give' you a zonal one for free to encourage more spend because inevitably you don't just need one, but ideally one per environment per app/product maybe even per engineer if you want eng to be able to stand up their own 'full' environments of beefier apps w/ lots of components and dependencies.
Serverless first is probably not a mistake in general. But we may have a different definition? For me Serverless means no need to administer a server (in the sense of "an operating system").
So baremetal, vps, = serverful
And lambdas, app engines, firebase, ... = serverless
There is a space where you are deploying a monolith app, and letting the cloud provider do the scaling, server management, environment management. But at any time you could run that same app happily on a server.
I think starting off there is not a mistake.
I think k8s is an advanced move. You can start off with the cloud built-in container management solution, then move to k8s later if you really need it.
yeah, I think I should have said something along the lines "Functions as a Service first is a mistake" but whatever, I didn't think people would focus so much on what they consider serverless is.
IMO managed container runtimes are PaaS, are both FaaS and PaaS serverless? Who knows, we don't seem to have hard definitions but in my mind PaaS predates serverless becoming a fashionable word so I associate with FaaS and FaaS-like things only.
Safest to say the exact thing (or list of things) I reckon!
Technically "serverless" should mean "no server" which means "no process fielding network requests" i.e. "unable to accept inbound network requests" - so some kind of batch processing system, or scheduled lambdas :-)
I can't help but laugh at this when I think of how easy it is to imagine the exact same tenor in a post dismissing Kubernetes as newfangled, overcomplicated garbage.
I disagree very much. I don't think any blanket statement like that makes sense.
I recently started building a service and need to keep costs down. Serverless is perfect for it. My service has low to moderate traffic, costs pennies to run on AWS Lambda and MongoDB Atlas. If I had gone the boring route of JVM + PG + k8s, putting aside the time it would take to defamiliarize myself with anything on the JVM, the cost to run this service would have been in the hundreds of dollars a month vs the pennies. Interestingly the most expensive part of my current setup is the NAT Router on my VPC. With JVM + PG + k8s it would have been PG or K8S depending on where I ran PG.
I do agree that there is a misconception with containers taking longer than server less. I don't think either takes longer considering the current tooling available.
Seems like you got burned on Serverless at some point, I'm sorry that happened, but for many people and teams it is a productivity multiplier and can be a big cost cutting solution.
> the cost to run this service would have been in the hundreds of dollars a month vs the pennies
And maybe this again proves the initial comment's point.
> I do agree that there is a misconception with containers taking longer than server less. I don't think either takes longer considering the current tooling available.
And (the misconception) that it costs more.
> AWS Lambda and MongoDB Atlas
If you take a managed database with a free tier and compare it there are similarly managed databases (SQL) based with a free tier. Not really fair to say PG will cost more.
This also comes down to the world is not just AWS. k8s is cheap in a lot of places and definitely not hundreds a month.
There are different shades of serverless, and you don’t start at an R620 plugged into a colo rack. You grow into it OR start with a large enough problem to demand it.
You start with GitHub pages or Cloudflare pages where all you have is CSS, HTML, and JS served from the equivalent of a global CDN that you didn’t have to build for essentially free.
For a mom and pop shop, how many times are they going to have to contact you over the next 20 years for a piece of software doing its job running on GitHub pages? Practically zero, unless it’s a feature request. Can’t say the same for a Linux box plugged into the wall under your desk or an R620 at the colo downtown.
Then you graduate to sprinkling a little bit of persistent state in. When you need a little bit of persistent state for a small mom and pop, deciding to vet colo vendors and signing that shop up to be responsible for sysadmining an R620 with a Linux kernel is irresponsible. You find a serverless vendor for your DB layer (like the Firebase of yore) or a set of CRUD handlers written on infra that fully manages the language runtime, operating system, kernel, and infra for your customer (like Cloudflare workers).
How many times is the customer going to need to contact you over the next 20 years for your Cloudflare worker? Probably only a handful. Odds are, significantly less than an R620 plugged into a server rack at the colo downtown.
If you outgrow this, you probably are starting to look at VMs or colo.
Serverless is real. It’s not lambda. It’s pervasive. It’s provides substantial value in ongoing maintenance and engineering costs if you pick the right vendor. And a majority of the market demands it as a starting point.
An engineer is expensive and most customers aren’t in a position to practice the craft themselves. The best service we can provide is to minimize their dependence on our field.
> How many times is the customer going to need to contact you over the next 20 years for your Cloudflare worker? Probably only a handful. Odds are, significantly less than an R620 plugged into a server rack at the colo downtown.
This has nothing to do with "serverless". Heroku did the same thing well before serverless was available.
Keeping a simple app running on heroku over the past 15 years has required about 3 hours admin total since I set it up (moving from an old stack to a newer one).
Architecture isn’t like Lord of the Rings where one ring rules them all.
What if the dev team doesn’t know Java? Are you suggesting they be forced to learn it?
What if the company has 20 applications with complex interactions? Are you suggesting they all be baked into k8s and then have someone manage containers full-time? What if no one in the company understands containers?
I’ve been at this for 40 years and will tell you there is no common architecture environment.
There can be any mix of business logic, services, data storage, containers, events, queues, redundancy, reports, analytics, external systems, 3rd party black boxes, skill sets and levels, budget, tech stack, and leadership.
Trying to pigeonhole any architecture into that mix is the very definition of amateur.
Both are complex internally, k8s does a better job of preventing that complexity from leaking IMO.
You need to know roughly the same amount of things to use both.
On k8s to deploy something simple you will need:
1. A deployment to actually run your app.
2. A service describe how to talk to your app.
3. An ingress to route requests to the service.
On Lambda w/API Gateway:
1. Lambda Function
2. IAM Role
3. API Gateway objects (Deployment, RestApi, Stage)
More if you expand out the policies and API gateway objects.
These become -really- burdensome if you are using IaC like Pulumi or Terraform.
In practice using k8s these days is essentially managed also, ale GKE/EKS/AKS so the real differences are down to:
1. API, the FaaS etc ones are proprietary, k8s is standardized.
2. Runtime environment and execution limits, varies by FaaS, k8s has none.
3. Access to durable storage, FaaS only object/HTTP based storage, you can attach EBS-like to your k8s containers.
4. Architecture incentives. FaaS highly incentivises microservices, k8s isn't really opinionated. FaaS is more like cgi-bin on steroids, k8s is daemons on a box but with fancy management.
5. Scaling. FaaS scale "infinitely", k8s scales depending on settings of cluster autoscaler.
So generally you get as good or better from k8s, the API complexity is about the same except now it's standard no lock-in, no enforced runtime limits, you have durable containers so you can attach a debugger, etc.
Hopefully this shows a bit why I think k8s is at least as simple as serverless if not more simple.
Anyone who says that they aren’t “locked-in” hasn’t done a large migration and dealt with both the technical and organizational complexity of any large migration.
I’ve been involved with organizations that just literally wanted to do a “lift and shift” of 100s of VMs and databases. They were definitely didn’t have any “vendor lock in”. Guess how long it took? Guess how much regression testing it took and planning?
Months probably. Took the team I was on ~6mo to complete a very large AWS -> GKE migration.
If we weren't using k8s it probably would have been impossible though. We were lucky we used fairly minimal managed services, mostly RDS and a tiny bit of SQS and Kinesis which were ported to Kafka as part of the move.
When I talked to people at Amazon it sounded like there was a huge push to dogfeed Lambda. The guy was almost worshipping it in the interview and appalled that I wasn't interested in it. :)
IMO serverless is incredibly useful when latency isn't important. Backend stuff, metrics, analytics, APIs, etc should be serverless. It allows you to scale incredibly easily at basically 0 cost.
For everything else, you have to test and examine the use cases and performance requirements. In general if there's a UI that's doing work it shouldn't be serverless. Work = latency = bad.
How are you going to throw away the competitive advantage of serverless first if you’re a startup? You’re just asking to lose to whichever of your competition that goes serverless first.
However, if you’re a massive company like Amazon working on a less-than-cutting edge project, you certainly have the resources to avoid serverless up front to avoid rewrite costs down the line.
Competitive advantage comes from understanding user needs, great UX, and a smooth running sales machine. No-one cares that you built with technology X, whatever that is. There is no silver bullet.
While I agree with the conclusion, competitive advantage is by definition very relative. It might be what you say, if you're producing some sort of saas offering. But it might be something completely different as well.
What is better than relational dbs (and for what purposes)?
I've been learning k8s fundamentals recently after using borg for a long time and then awhile kind of just poking at k8s trying to use it without learning it. Honestly I think it seems like about as simple as a solution to a broad set of real common problems could plausibly be.
Relational dbs are based on relational model. It’s a giant black box of magic. And it’s magic is based on a bunch of constraints to make the relational algebra work. It’s really quite flawed for modern web apps.
It ... really isn't a black box of magic? It's a transparent box of - as your comment points out - implementing a particular mathematical model.
You're gonna need to give me some more on why it's "quite flawed for modern web apps". I may even agree with you, but it's tough to know from what you've written so far.
Referring mostly to the query planner - you have no control over it, and you have to dive deep to figure out why it's making certain plans, and how it collects statistics to make these calls. Plus the planner essentially gives up at 6+ joins.
Also generally the conception that building things with containers + k8s actually takes more time than serverless is a horseshit take IMO. It doesn't and if it's taking you longer you are probably doing something wrong.
All the points noted in the slides etc just come from poor engineering leadership. i.e decisions/choices, etc. Constaints do speed up development but they can be enforced by leadership, not by putting yourself in an endless pit of technical debt.
The only world where I can make a good case for serverless for small teams is if you have said poor leadership and serverless helps you escape their poor decision making.
Otherwise assuming you have the mandate necessary to get stuff built the right way just stick with the Boring Tech stack of JVM + PG + k8s. It solves all problems regardless of size, there are minimal decisions to be made (really just Spring or Quarkus atm) and you won't spend time dicking around with half-baked serverless integration crap. Instead you can just use battle tested high performance libraries for literally everything.
Don't get sucked into this serverless nonsense. It's always been bad and it probably always will be. When it's out of fashion Boring Tech will still be here.