AI and legal experts told the FT this “memorization” ability could have serious ramifications on AI groups’ battle against dozens of copyright lawsuits around the world, as it undermines their core defense that LLMs “learn” from copyrighted works but do not store copies.
Sam Altman would like to remind you each Old Lady at a Library consume 284 cubic feet of Oxygen a day from the air.
Also, hey at least they made sure to probably destroy the physical copy they ripped into their hopelessly fragmented CorpoNapster fever dream, the law is the law.
Again, you are stumbling at a philosopical level in your argument.
Do you have any idea how an mp3 works? That kind of complexity barrier is EXISTENTIALLY necessary to compress audio into codecs like the mp3 format so it can be efficiently streamed over mobile connections and the internet. You are imagining an mp3 like a raw Wav file, and they are VERY much not the same.
…Nobody in audio engineering is stupid enough to claim an mp3 rip of a copyright Wav file counts as not a copyright infraction because it was done at an atrocious bitrate. That apparently takes the hubris of overconfident computer people to bullshit yourself into believing.
You’re missing the boat entirely. Think about how an AI model is trained: It reads a section of text (one context size at a time), converts it into tokens, then increases a floating point value a little bit or decreases it a little bit based on what it’s already associated with the previous token.
It does this trillions of times on zillions of books, articles, artificially-created training text (more and more, this), and other similar things. After all of that, you get a great big stream of floating point values you write out into a file. This file represents the a bazillion statistical probabilities, so that when you give it a stream of tokens, it can predict the next one.
That’s all it is. It’s not a database! It hasn’t memorized anything. It hasn’t encoded anything. You can’t decode it at all because it’s a one-way process.
Let me make an analogy: Let’s say you had a collection of dice. You roll them each, individually, 1 trillion times and record the results. Except you’re not just rolling them, you’re leaving them in their current state and tossing them up into a domed ceiling (like one of those dice popper things). After that’s all done you’ll find out that die #1 is slightly imbalanced and wants to land on the number two more than any other number. Except when the starting position is two, then it’s likely to roll a six.
With this amount of data, you could predict the next roll of any die based on its starting position and be right a lot of the time. Not 100% of the time. Just more often than would be possible if it was truly random.
That is how an AI model works. It’s a multi-gigabyte file (note: not terabytes or petabytes which would be necessary for it to be possible to contain a “memorized” collection of millions of books) containing loads of statistical probabilities.
To suggest its just a shitty form of encoding is to say that a record of 100 trillion random dice rolls can be used to reproduce reality.
Not it isn’t a one-way process, literally the point of this article is that you functionally can.
You can functionality copy Shakespeare with enough random words being generated. That’s the argument you’re making here.
If you prompt an LLM to finish sentences enough times (like the researchers did, referenced in the article) you can get it to output whatever TF you want.
Wait: Did you think the researchers got these results on the first try? You do realize they passed zillions of prompts into these LLMs until it matched the output they were looking for, right?
It’s not like they said, “spit out Harry Potter” and it did so. They gave the LLM partial sentences and just kept retrying until it generated the matching output. The output that didn’t match was discarded and then the final batch of matching outputs were thrown together in order to say, “aha! See? It can regurgitate text!”
Try it yourself: Take some sentences from any popular book, cut them in half, and tell Claude to finish them. You’ll be surprised. Or maybe not if you remember that RNG is at the core of all LLMs.
No it is not, that would be writing Shakespeare by combining random words, LLMs are not capable of that level of artistry, there is no random to them. All they can do is calculate the probabilities of pre-existing connections and give you the most boring, obvious one.