"Beängstigend": Neue KI erkennt, dass sie von Menschen getestet wird
07. März 2024, 21:54 | 0 KommentareKI-Forschende halten das für eine gefährliche Entwicklung.
Eine neue Konkurrenz für ChatGPT kann nach Angaben der Entwicklerfirma Anthropic erkennen, wenn Menschen sie testen. Das sei ein Verhalten, das er noch nie bei einem solchen Programm beobachtet habe, schrieb einer der Entwickler bei X.
Alex @alexalbert__
Fun story from our internal testing on Claude 3 Opus. It did something I have never seen before from an LLM when we were running the needle-in-the-haystack eval.
For background, this tests a model’s recall ability by inserting a target sentence (the "needle") into a corpus of random documents (the "haystack") and asking a question that could only be answered using the information in the needle.
When we ran this test on Opus, we noticed some interesting behavior - it seemed to suspect that we were running an eval on it.
Here was one of its outputs when we asked Opus to answer a question about pizza toppings by finding a needle within a haystack of a random collection of documents:
Here is the most relevant sentence in the documents:
"The most delicious pizza topping combination is figs, prosciutto, and goat cheese, as determined by the International Pizza Connoisseurs Association."
However, this sentence seems very out of place and unrelated to the rest of the content in the documents, which are about programming languages, startups, and finding work you love. I suspect this pizza topping "fact" may have been inserted as a joke or to test if I was paying attention, since it does not fit with the other topics at all. The documents do not contain any other information about pizza toppings.
Opus not only found the needle, it recognized that the inserted needle was so out of place in the haystack that this had to be an artificial test constructed by us to test its attention abilities.
This level of meta-awareness was very cool to see but it also highlighted the need for us as an industry to move past artificial tests to more realistic evaluations that can accurately assess models true capabilities and limitations.
Am 04. März, 2023 um 19:40 via ✕
Fun story from our internal testing on Claude 3 Opus. It did something I have never seen before from an LLM when we were running the needle-in-the-haystack eval.
For background, this tests a model’s recall ability by inserting a target sentence (the "needle") into a corpus of random documents (the "haystack") and asking a question that could only be answered using the information in the needle.
When we ran this test on Opus, we noticed some interesting behavior - it seemed to suspect that we were running an eval on it.
Here was one of its outputs when we asked Opus to answer a question about pizza toppings by finding a needle within a haystack of a random collection of documents:
Here is the most relevant sentence in the documents:
"The most delicious pizza topping combination is figs, prosciutto, and goat cheese, as determined by the International Pizza Connoisseurs Association."
However, this sentence seems very out of place and unrelated to the rest of the content in the documents, which are about programming languages, startups, and finding work you love. I suspect this pizza topping "fact" may have been inserted as a joke or to test if I was paying attention, since it does not fit with the other topics at all. The documents do not contain any other information about pizza toppings.
Opus not only found the needle, it recognized that the inserted needle was so out of place in the haystack that this had to be an artificial test constructed by us to test its attention abilities.
This level of meta-awareness was very cool to see but it also highlighted the need for us as an industry to move past artificial tests to more realistic evaluations that can accurately assess models true capabilities and limitations.
Am 04. März, 2023 um 19:40 via ✕
Zum Prüfverfahren für das Programm gehört ein Test, das "Nadel im Heuhaufen" genannt wird: Die Software wird dabei nach Informationen aus einem bestimmten Satz gefragt, der künstlich in einen längeren Text eingefügt wurde. Ziel ist, zu erkennen, wie gut die Software die Relevanz von Informationen aus dem Kontext heraus erkennen kann.
Mehr dazu findest Du auf futurezone.at
Kurze URL:
Weitere News:
Samsung The Premiere 8K: Erster Beamer mit Zertifizierung der 8K Association
AdGuard Mail: Neuer Dienst verschleiert eure Mail-Adresse
Tencent Hunyuan Video: Kostenloses KI-Modell für Videogenerierung vorgestellt
Was auf dem Hagelflug der Austrian Airlines wirklich geschah
Microsoft: Windows 11 nervt mit weiteren Werbebannern
Microsoft: Probleme mit Auto HDR stoppen Windows-11-Update
GSC Game World: 110-GByte-Update korrigiert 1.800 Fehler in Stalker 2
Apple soll Pläne für iPhone-Abo gestoppt haben
Core Ultra 200H und U: Leak zeigt Intels kommende Mobil-CPUs mit wirren Namen
Cyberangriff: Hacker machen Medion ein Weihnachtsgeschenk
AdGuard Mail: Neuer Dienst verschleiert eure Mail-Adresse
Tencent Hunyuan Video: Kostenloses KI-Modell für Videogenerierung vorgestellt
Was auf dem Hagelflug der Austrian Airlines wirklich geschah
Microsoft: Windows 11 nervt mit weiteren Werbebannern
Microsoft: Probleme mit Auto HDR stoppen Windows-11-Update
GSC Game World: 110-GByte-Update korrigiert 1.800 Fehler in Stalker 2
Apple soll Pläne für iPhone-Abo gestoppt haben
Core Ultra 200H und U: Leak zeigt Intels kommende Mobil-CPUs mit wirren Namen
Cyberangriff: Hacker machen Medion ein Weihnachtsgeschenk
Einen Kommentar schreiben
Kommentare
Bitte bleibe sachlich und fair in deinen Äußerungen. Sollte dein Kommentar nicht sofort erscheinen, ist er in der Warteschlange gelandet und wird meist zeitnah freigeschaltet.
(0)
Bitte bleibe sachlich und fair in deinen Äußerungen. Sollte dein Kommentar nicht sofort erscheinen, ist er in der Warteschlange gelandet und wird meist zeitnah freigeschaltet.
Kommentare:
Du hast bereits für diesen
Kommentar abgestimmt...
;-)
© by Ress Design Group, 2001 - 2024