Do we want a specific thread for this topic
@Board and Seize ? I'm very lost, so I'll leave it you.
Haha, well, the previous exchange between
@Florida173 and myself was a bit of a sidetrack to the intended topic. I'm happy to keep AI-ish stuff going here, and also to decipher some of the jargony stuff.
Back to the top of me resurrecting this thread. You're likely to have had news of various "AI"*
text generators popping up in your feed over the past two years or so.
The first big one that hit the news was GPT-2, which when given a text prompt would generate new text to follow. This is what you can interact with at
Talk to Transformer. This uses a specialized kind of ML (Machine Learning)** data structure called a Neural Net (NN). One particular kind of NN is the
Generative Adversarial Network (GAN). GANs are found across almost the full spectrum of current and cutting edge AI, from computer-vision object-detection and -defeat (all those captchas you do are your contribution to creating/
munging the datasets used to train these networks) to twitter bots, deepfakes and more.
Though perhaps unintuitive, prediction and detection/recognition are fundamentally the same task - just with opposite valence (as an analogy, speakers can be used as microphones - in this case sound production and detection). So you take two NNs. Train them on the same dataset (you show them a bunch of examples of 'right' and 'wrong' at whatever task - say recognizing objects in photos, as with the recent captchas). Then you set them against each other. You task one to predict/produce, and the other to filter/detect. Then you recursively train them on the results of that. As a result, they get successively better. This can be imagined as any of the OpenAI Alpha___ game AIs. That's more approachable, because you just set them to play against each other and record the results, then fast forward and have them do this thousands, millions, or billions of times. The results of the 'learning' is basically an insane number of variables (in the algebraic/cs sense) and weights for each of them.
Okay, back to GPT-n. So this is a NN trained on a huge corpus of basically all reddit comments from the top many subreddits over a period of years. This is a devastating amount of data. Then the model has some large number of 'parameters'. This is basically the variables and weights - and these are pretty blackboxed in that a human can't really penetrate what they 'mean' or correlate to.
GPT-2 dropped in early 2019 with ~1.5 billion parameters. This was the big splash in the news, and they didn't actually release the full model for fear of what shenanigans would occur. They released successively 'larger' models (more params) and then we started seeing stuff like TtT and AI Dungeon.
Then in May of 2020,
GPT-3 came out. Basically a much larger, more refined version. It's up to something like 175 billion params now. Since the model is effectively just these params and their relationships (it's a database) it's way too large to run on typical commercial home equipment. Most of your ways to interact with these models (AI Dungeon and TtT again, as well as others) utilize cloud computing resources and provide you with a terminal via webpage.
Phew!
So the new thing that those OpenAI maniacs just dropped today is an evolution of these text generation/'understanding' AIs to
create new images - as in draw them. You can write a ranndom description like "sketch of a russian dancing bear with a scorpion tattoo" and it will generate an image that matches. And even from this first glimpse, it is shockingly good.
This took me entirely too long to respond, sorry.
*I get really pedantic over the term Artificial Intelligence, and hold a strict definition. This is othertimes called Strong AI or General AI or True AI. These terms are in reaction to marketing bs that calls every instance of ML AI. To be clear, there is no known extant instance of AI. It is a future possibility.
**Machine Learning is a more accurate umbrella term for all of the crap that gets called AI today. That just means any program that uses any one of a wide range of statistical analyses (typically regressions) to separate signal from noise or pull useful info out of a heap of uninteresting or confusing info.
edit: Oh yeah AI Dungeon! This uses the GPT-2 (or GPT-3 if you pay for it) model to act as a reactive text-based story-teller or game master. It gives you a prompt, and then you start writing, narrating your actions/reactions/thoughts/etc. And then AI Dungeon tells you what happens next. It is freaking awesome, but it isn't perfect, and garbage-in-garbage-out. It can get stuck in a loop, and if you nudge it towards teenage fantasy, it will run with it (all that reddit training yo!).
