These OSS model makers need to stop benchmarking against old models. Showing how it performs against Opus 4.5, GLM-5 when we have Opus 4.6 and GLM-5.1 just tells me that it's not comparable to SOTA.
I think there's a lot of methodological expertise that goes into collecting good eval data. For example, in many cases you need human labelers with the right expertise, well designed tasks, well defined constructs, and you need to hit interrater agreement targets and troubleshoot when you don't. Good label data is a prerequisite to the stuff that can probably be automated by the AI agent (improving the system to optimize a metric measured against ground truth labels). Data scientists and research scientists are more likely to have this skillset. And it takes time to pick up and learn the nuances.
> Claude Code's 500k LOC doesn't seem out of the ordinary.
Aren't all the other products also vibe-coded? "All vibe-coded products look like this" doesn't really seem to answer the question "Why is it so damn large?"
It's a repl, that calls out to a blackbox/endpoint for data, and does basic parsing and matching of state with specific actions.
I feel the bulk of those lines should be actions that are performed. Either this is correct or this is not:
1. If the bulk of those lines implement specific and simple actions, why is it so large compared to other software that implements single actions (coreutils, etc)
2. If the actions constitute only a small part of the codebase, wtf is the rest of it doing?
>> I feel the bulk of those lines should be actions that are performed. Either this is correct or this is not:
> You're complaining about vibe coding while also complaining about how you "feel" about the code. Do you see the irony in that?
Where did I complain about how I feel about the actual code? I have feelings, negative ones, about the size of the code given the simple functionality it has, but I have no feelings on the code because I did not look at the code.
Bad by whose definition? They work really well in my experience. They aren't perfect but the amount of hand holding has gone down dramatically and you can fix any glaring problems with a code review at the end. I work on a multimillion line code base which does not use any popular frameworks and it does a great job. I may be benefiting from the fact that the codebase is open source and all models have obviously been trained on it.
Most of their issues have been solved a long time ago, with 1000x less code. It is depressing at this point. I really had no clue IT was in the shitters this much. I knew it was theatrical but I had no idea that it was by this much.
All these AI tools teams have most valid excuse "We are just a bunch of people who only know Javascript/typescript/NodeJS. Please bear with us while we resolve 10,000 open issues."
I haven't seen the scrolling glitch in months, where previously it was happening multiple times a day. Also haven't seen anyone complain about it in quite some time. Pretty sure they have resolved that.
> I think the only reason it’s seen as good anywhere is there are a lot of tasteless and talentless people who can pretend they created whatever was curled out. This goes for code as well.
This is an oversimplification.
If you have taste and talent, then the LLM output you get is going to reflect that.
So on the one hand, yes: tasteless and talentless people won't know good output from bad output. On the other hand, people with taste and talent can actually get good output.
Well you're free to disagree but my experience has been counter to your position. I write both code and research / technical documentation. The quality of what the LLM produces is limited by the quality of ideas I give it initially (mind you, this is just a starting point), and the quality of my review of its output.
This couldn't be more wrong. The simplest refutation is just to point out that there are temperature and top-k settings, which by design, generate tokens (and by extension, ideas) that are less probable given the inputs.
Vegan for 15 years. I cook 95% of my own meals, including black bean burgers, tofu, etc... Sometimes I want something that tastes like meat and I reach for a Beyond or Impossible burger. I don't need it. But I can't recreate its texture and flavor profile on my own. It's not "better" than other things I can cook. It's just different.
But literally any decent agent can recommend existing services and help you set them up. And even help you help them set the services up for you. I do this with Claude all the time.
reply