Harnesses could have solved things like the bathroom remodel, maybe, but the main point about how LLMs don't understand is the key here. You can make chatgpt better at rendering 3d scenes but you can't make it think, not really. Reasoning was only ever a feedback loop.
Anyone who has worked with LLMs has experienced all the issues he talks about here, we're either optimistic and imagine they'll be fixed, or we're pessimistic and we say they are inherent to the nature of the technology and will never be fixed
Well we've already found out because Israel broke it. With Gaza they've gotten used to very flexible ceasefires where you can still bomb the other side repeatedly and the ceasefire "holds". Iran has shown that this will not work with them, and they closed the strait again
This is a great question. You definitely aren't training this to use it, you're training it to understand how things work. It's an educational project, if you're interested in experimenting with things like distributed training techniques in JAX, or preference optimisation, this gives you a minimal and hackable library to build on.
It's also a great base for experimentation. If you have an idea for an architecture improvement you can try it for $36 on the 20 layer nanocode setting, then for another $200 see how it holds up on the "full scale" nanocode
Kaparthy's notes on improving nanochat [1] are one of my favorite blog-like things to read. Really neat to see which features have how much influence, and how the scaling laws evolve as you improve the architecture
There's also modded-nanogpt which turns the same kind of experimentation into a training speedrun (and maybe loses some rigor on the way) [2]
I was also very confused, but after some reading I figured it out.
> In an interview with NBC News from space, NASA astronaut Christina Koch described seeing the moon out the window of the Orion capsule and realizing that it looked different from what she was accustomed to on Earth.
> “The darker parts just aren’t quite in the right place,” she said. “And something about you senses that is not the moon that I’m used to seeing.”
They are not on the other side of the moon seeing the full dark side, but from their position they're seeing the moon at a slight angle, meaning that SOME of what they now see is "the dark side", or the part we can never see from earth since the same side always faces us
This. We all thought Trump was a crazy accident but the fact that he almost beat Biden, and then did beat Harris, means we just can't trust Americans to put sensible people in charge. Assuming a democrat takes the office next, they will inherit an economy in tatters, a failing infrastructure and a broken strategic alliance. They'll have four years to try to fix all of that while the republicans blame them for everything they've inherited, and four years from that the American people will have largely forgotten how Trump and his minions trailed dog shit all through the house and they'll vote for the next right wing dick that's been groomed for the job - probably Pete Hegseth, or Don Jr, or Mark Wayne Mullin
Neither Biden nor Harris were sensible candidates. Democrats could have easily beat Trump by running a more appealing/less polarizing candidate. Didn't even have to be both. Obama was polarizing but he was appealing and he won comfortably.
As a non-American I have always wondered about the criteria used by Americans to vote for their presidents.
Clinton and Obama had various defects, but at least both of them looked like presidents and talked like presidents.
On the other hand, both George Bush Junior and Trump (of course especially the latter), looked like clowns and talked like clowns.
I have never understood their appeal to the masses. I understand the discontent of those who have voted against the Democrat "elites", but the fact that anyone could look at Trump and believe that he is the right man for the job seems unbelievable, regardless of how inept were his opponents.
Your reference to Democrat "elites" shows you have a hint of it... in this country that term never applies to a Republican -- even if they were born rich, went to Ivy League schools, and were handed a career and a professional network on a platter.
It is almost _exclusively_ used to denigrate women, minorities, or men who support progressive causes.
We always talk about what these powerful people "have done", as if it's all over. Surely Epstein's death did not bring about the end of billionaire sex trafficking? Someone stepped in. These guys are still raping people on private planes and private islands
But why are we focusing on the raping, and not on what the American government is doing that has no clear rational motive without “Israel has captured the government” and a very clear rational motive with “Israel has captured the government”?
If the American government continues to perform actions that are blatantly against the interests of America and Americans, the impact of that on Americans is going to be (and may be already) massively massively worse than the person to person level crimes we are focusing on.
Does it just feel so bad thinking about it that a lot of people have a hard time even going there mentally? I really don’t get it.
I can't shake the feeling that Trump's continual needling of Europe is intended the destroy NATO. And this is a desire of Putin's.
I know it's conspiratorial, and I hate that, but it's one of the only things that makes any of the actions of the US in the last year make any semblance of sense.
I don't like to think that, but it remains, for me, a valid scenario.
Anyone who has worked with LLMs has experienced all the issues he talks about here, we're either optimistic and imagine they'll be fixed, or we're pessimistic and we say they are inherent to the nature of the technology and will never be fixed
reply