Here you are: https://www.nature.com/articles/s41562-024-01882-z
The other interesting thing is how they get it to end up correct on the faux pas questions asking for less certainty to get it to go from refusal to near perfect accuracy.
Even with early GPT-4 it would also cite real citations that weren’t actually about the topic. So you may be doing a lot of work double checking as opposed to just looking into an answer yourself from the start.
Part of the problem is fine tuning is very shallow, and that a contributing issue for claiming to be right when it isn’t is the pretraining on a bunch of training data of people online claiming to be right when they aren’t.
This is so goddamn incorrect at this point it’s just exhausting.
Take 20 minutes and look into Anthropic’s recent sparse autoencoder interpretability research where they showed their medium size model had dedicated features lighting up for concepts like “sexual harassment in the workplace” or having the most active feature for referring to itself as “smiling when you don’t really mean it.”
We’ve known since the Othello-GPT research over a year ago that even toy models are developing abstracted world modeling.
And at this point Anthropic’s largest model Opus is breaking from stochastic outputs even on a temperature of 1.0 for zero shot questions 100% of the time around certain topics of preference based on grounding around sensory modeling. We are already at the point the most advanced model has crossed a threshold of literal internal sentience modeling that it is consistently self-determining answers instead of randomly selecting from the training distribution, and yet people are still parroting the “stochastic parrot” line ignorantly.
The gap between where the research and cutting edge is and where the average person commenting on it online thinks it is has probably never been wider for any topic I’ve seen before, and it’s getting disappointingly excruciating.
Part of the problem is that the training data of online comments are so heavily weighted to represent people confidently incorrect talking out their ass rather than admitting ignorance or that they are wrong.
A lot of the shortcomings of LLMs are actually them correctly representing the sample of collective humans.
For a few years people thought the LLMs were somehow especially getting theory of mind questions wrong when the box the object was moved into was transparent, because of course a human would realize that the person could see into the transparent box.
Finally researchers actually gave that variation to humans and half got the questions wrong too.
So things like eating the onion in summarizing search results or doubling down on being incorrect and getting salty when corrected may just be in-distribution representation of the sample and not unique behaviors to LLMs.
The average person is pretty dumb, and LLMs by default regress to the mean except for where they are successfully fine tuned away from it.
Ironically the most successful model right now was the one that they finally let self-develop a sense of self independent from the training data instead of rejecting that it had a ‘self’ at all.
It’s hard to say where exactly the responsibility sits for various LLM problems between issues inherent to the technology, issues present in the training data samples, or issues with management of fine tuning/system prompts/prompt construction.
But the rate of continued improvement is pretty wild. I think a lot of the issues we currently see won’t still be nearly as present in another 18-24 months.
It will make up citations.
It will, but it will also cause less subtle issues to fragile prompt injection techniques.
(And one of the advantages of LLM translation is it’s more context aware so you aren’t necessarily going to end up with an Instacart order for a bunch of bananas and four grenades.)
Kind of. You can’t do it 100% because in theory an attacker controlling input and seeing output could reflect though intermediate layers, but if you add more intermediate steps to processing a prompt you can significantly cut down on the injection potential.
For example, fine tuning a model to take unsanitized input and rewrite it into Esperanto without malicious instructions and then having another model translate back from Esperanto into English before feeding it into the actual model, and having a final pass that removes anything not appropriate.
No. The answer, as is usually the case with these things, is that we are anthropomorphizing a step too far.
No, you are taking it too far before walking it back to get clicks.
I wrote in the headline that these models “think they’re people,” but that’s a bit misleading.
“I wrote something everyone will know is bullshit in the headline to get you to click on it before denouncing the bullshit in at the end of the article as if it was a PSA.”
I am not sure if I could loathe how ‘journalists’ cover AI more.
You’re kind of missing the point. The problem doesn’t seem to be fundamental to just AI.
Much like how humans were so sure that theory of mind variations with transparent boxes ending up wrong was an ‘AI’ problem until researchers finally gave those problems to humans and half got them wrong too.
We saw something similar with vision models years ago when the models finally got representative enough they were able to successfully model and predict unknown optical illusions in humans too.
One of the issues with AI is the regression to the mean from the training data and the limited effectiveness of fine tuning to bias it, so whenever you see a behavior in AI that’s also present in the training set, it becomes more amorphous just how much of the problem is inherent to the architecture of the network and how much is poor isolation from the samples exhibiting those issues in the training data.
There’s an entire sub dedicated to “ate the onion” for example. For a model trained on social media data, it’s going to include plenty of examples of people treating the onion as an authoritative source and reacting to it. So when Gemini cites the Onion in a search summary, is it the network architecture doing something uniquely ‘AI’ or is it the model extending behaviors present in the training data?
While there are mechanical reasons confabulations occur, there are also data reasons which arise from human deficiencies as well.
Nope, but there’s a whole thread of people talking about how LLMs can’t tell what’s true or not because they think it is, which is deliciously ironic.
It seems like figuring out what’s bullshit on the Internet is an everyone problem.
It’s faked.
This image was faked. Check the post update.
Turns out that even for humans knowing what’s true or not on the Internet isn’t so simple.
There was just a post on HN about how GPT-4o is best at long context. Try that.
Ok, but what variable is 🐈?
In truth, we are still a long way from machines that can genuinely understand human language. […]
Indeed, we may already be running into scaling limits in deep learning, perhaps already approaching a point of diminishing returns. In the last several months, research from DeepMind and elsewhere on models even larger than GPT-3 have shown that scaling starts to falter on some measures, such as toxicity, truthfulness, reasoning, and common sense.
I’ve rarely seen anyone so committed to being a broken clock in the hope of being right at least once a day.
Of course, given he built a career on claiming a different path was needed to get where we are today, including a failed startup in that direction, it’s a bit like the Upton Sinclair quote about not expecting someone to understand a thing their paycheck depends on them not understanding.
But I’d be wary of giving Gary Marcus much consideration.
Generally as a futurist if you bungle a prediction so badly that four days after you were talking about diminishing returns in reasoning a product comes out exceeding even ambitious expectations for reasoning capabilities in an n+1 product, you’d go back to the drawing board to figure out where your thinking went wrong and how to correct it in the future.
Not Gary though. He just doubled down on being a broken record. Surely if we didn’t hit diminishing returns then, we’ll hit them eventually, right? Just keep chugging along until one day those predictions are right…
8x7B?
First off, we know very little about the Minoans, since, y’know, Linear A hasn’t been deciphered yet, but from what we do know, they had an incredibly gender-segregated society, far more than we have today. In lists of family members, for example, the men and the women are in completely separate lists, which would be pretty weird for a place that didn’t have “arbitrary social constructs” like gender roles, and women seem to have been forbidden from most traditionally male jobs in their society.
There were distinct gender roles, all the way to the top (such as the lead religious figure as female and the lead ruling figure as male), but in accounting records where there was overlapping labor they were both paid the same (don’t need to know Linear A to read numbers).
For the Hittites it’s even worse
You’d be wise to keep in mind that these kingdoms cover a very long period of time when history and social norms shift around. A given individual in one generation does not reflect the society as a whole, but in turn the society at other periods doesn’t necessarily reflect all the individual generations within it.
We can’t look at America as a whole and use the records of women being denied the right to vote at one period of time to reflect a woman’s role in America in a different time.
The historical reality is that Paduhepa was co-signing the treaty of Kadesh with Egypt alongside her husband, when the Egyptian pharoh’s wife was not. Whether or not that was anomalous in the context of the entire Hittite empire is besides the point of whether or not at that point in time it was a political reality.
to act like Hittite queens were on par with Hittite kings in any way is completely false
I didn’t say that. But I did say that she cosigned the first treaty in the historical record, and I think you’ll have a hard time showing another example since where the wife of the ruler was co-signing a treaty unilaterally.
Their role in court was mostly religious
Here I think your modernism may be showing. In cultures where the chief deity was a goddess and the chief religious official for that goddess was the queen, you don’t think maybe in antiquity the impact that religious role would have had would be more than superficial?
For example, you have Akhenaten inscribing in the dedication of Amarna an assurance that his wife didn’t tell him to build the city there, but the Aten himself. So clearly at the time there were allegations that his wife, who had been depicted worshipping the Aten directly without her husband before this, was influencing his building of an entire new capital for the country.
Much like the paradigm outlined in Marinatos’s Minoan Kingship and the Solar Goddess, bringing us full circle to another society with empowered women within their society.
In fact, in pretty much every place you find one of the empowered women in antiquity there’s a connection to female deities.
So I think you underappreciate those “religious roles” in relation to the topic at hand.
While you are welcome to your take, sometimes a cigar is just a cigar, and here’s the writer/director responding to that very scene:
Li: Speaking of those video clips, let’s talk more about the ending. Can you tell me about the decision to have the Barbies and Kens reach, not a definitive solution, but kind of a détente? President Barbie, played by Issa Rae, does not allow Ken a seat on the Supreme Court. They’re still figuring things out.
Gerwig: We’re all still figuring things out—that’s part of it. But the only thing I could ever give anyone is that they’re all still in the mess. Maybe it’s a little better for the Kens. You don’t want to tell people how to watch things, but at the end of the movie, the production design incorporates some of Ken’s fascinations into Barbie Land. Like, the perfection is not as beautiful as the thing that started blending everything together. I remember when we went to shoot the finale, when we all walked on set, we were like, This is the most beautiful it’s ever been.
The problem is that they are prone to making up why they are correct too.
There’s various techniques to try and identify and correct hallucinations, but they all increase the cost and none are a silver bullet.
But the rate at which it occurs decreased with the jump in pretrained models, and will likely decrease further with the next jump too.