We use Nomad from Hashicorp at $dayjob and I'm quite happy with it.

https://www.nomadproject.io/

https://www.hashicorp.com/resources/introduction-nomad

https://blog.cloudflare.com/how-we-use-hashicorp-nomad/

For many simple applications, an aws autoscaling group and application load balancer is as good as k8s and far simpler.

If you're running microservices (to the point where you have so many small services that having an asg per service), than the above strategy does waste resources compared to k8s I guess... But you'd need a lot of microservices to justify the significant overhead k8s has.

But now you have to manage those VM's, the OS lifecycle, different distributions, application deployment, configuration and dependencies. And we're back at containers. Or you have to go towards lambda.

For K8s you need to manage the vm's/os as well but that part is mostly trivial as it's highly focussed on immutable/disposable infrastructure.

Not OP, and don't even have an alternative, but I find it disappointing that kubernetes basically seems to be the only answer for 'running containers in production'. Docker and its tooling is pretty development focused, and Kubernetes is probably too complex for 95% of use cases, which makes me think there's gotta be something in the middle ground that reaps the benefits of containers, wraps them with good tooling, reproducibility, reliability, and observability, and avoids the complexity of k8s.

AWS Fargate has worked really well for me. And only required an afternoon to get working. I've spent weeks learning kubernetes and still don't know enough to not hit rough edges.

VMs as usual.

Have you ever worked at a company that has more than 300 VMs that run more than 10 different apps?

Kubernetes solves a specific set of problems. Just because you personally don't have these problems, doesn't mean that Kubernetes is a fad.

A small company called Nokia Networks, on a tiny product for telecommunication cluster monitoring called NetAct.

Then there was that tiny physics research center close to Geneva creating what was one of the first grid computing platforms.

But what do I know, that was almost 15 years ago.

Great. So you also believe that Kubernetes is a fad and doesn't solve any problems for anybody and we should all stick with VMs?

I know what problems it solves, what problems it creates and how very expensive it is to learn, operate, optimize and debug.

Its features do not justify its complexity and the cost of operating it.

While I agree, I'd rather deal with the k8 nonsense than maintaining infrastructure to make multiple VMs load balance, fail and restart cleanly, etc.

In almost 30 years of development, I never worked on Web scale Google style projects, and some of them did handle TB of data in a cluster.

The golden rule is not to pretend to work at a FANNG when not having the same problems.

I agree, but man so many places are going in the opposite direction… I got lucky with my current gig that I got to implement load balancing/autoscaling with Fabric, ~1000 lines of python in a daemon using boto3, nginx and disk images…

I found AWS's autoscaling offering be better than GCP's, in that you could use custom CloudWatch metrics as a basis for scaling logic rather than just the standard metrics (CPU usage, latency, etc)

GCP does offer custom metrics, but I couldn't figure out how to tie them to the LB scaling logic.