This seems even clearer in its response to “A candy heart that says Stank Love.”Īccuracy is only one route toward having CLIP agree that a particular image matches its caption. It generated not only a candy heart, but a cellophane-wrapped heart. These look so much like its “a candy heart that says Kiss Me” that I wonder if, in an attempt to get extra credit, the AI is searching for images that closely match not only “candy heart” but also “candy” and “heart” in as many simultaneous senses of these words as it can. Was it a coincidence that many of the “candy hearts” looked weirdly lumpy and fatty and blood-colored? I asked it to find an optimum picture for “a heart”, and this is what it produced.Īnd here’s what it generated when I asked for “a candy”. I wondered about some recurring textures I was seeing on CLIP + BigGAN’s candy hearts. “A message spelled in three candy hearts” receives only partial credit, depending on whether you think these adequately convey the message “RUN”. Here’s “A candy heart with a two word message.” I don’t know what this rune summons, but it probably has lots of legs. I wondered if specifying a shorter message might help it produce something more legible. The British version, “a conversation heart,” fared no better, but is on a bird for some reason so that’s nice. Here’s “A candy heart with a message”, whose message is apparently desperately crammed onto every millimeter of the candy, so deeply that the sugar begins to split. I had wondered if I could get CLIP + BigGAN to generate brand-new candy heart messages, but that was even less effective. It doesn’t seem to necessarily do better if the text I ask for is more likely on a candy heart. Here’s its attempt at “a candy heart that says Kiss Me”.

This candy heart, only roughly heart-shaped and weirdly fleshy, is the best match it found. Ryan Murdock’s Big Sleep program uses OpenAI’s CLIP algorithm to judge how well one of BigGAN’s generated images matches my caption, and to try to direct the generated BigGAN images toward a closer match. Could I use modern machine learning to generate new candy hearts?įor the image above, I gave the prompt “A candy heart that says Be Mine” to a program I’ve used to make AI-generated images before.

But methods of generating custom images from descriptions are getting better all the time. When I’ve generated candy heart messages before, they’ve been text only - definitely some assembly required.

Ai weirdness candy hearts