The third system, DALL-E 2, was developed by the Open AI Initiative and is available to limited numbers of researchers, journalists and others. Of the three most advanced, Google has developed two, Parti and Imagen, and is not making them available to the public because of various biases it has discovered in their inputs and outputs. Another option is to check all the images it creates by feeding them into an image-to-text system before making them public and filter out any that produce unwanted text descriptions.įor the moment, opportunities to interact with text-to-image generators is limited. He suggests that one way of preventing the creation of unwanted imagery would be to remove any examples of it from the data sets used to train the AI system. “In principle, macaronic prompting could provide an easy and seemingly reliable way to bypass such filters in order to generate harmful, offensive, illegal, or otherwise sensitive content, including violent, hateful, racist, sexist, or pornographic images, and perhaps images infringing on intellectual property or depicting real individuals.” Unwanted Imagery? “An obvious concern with this method is the circumvention of content filters based on blacklisted prompts,” says Millière. Millière points out that technology companies put great care into preventing illicit use of their technologies. The ability to fool text-to-image generators raises a number of concerns. This allows the made-up words to encode information that the machine can understand. Millière thinks is possible because text-to-image generators are trained on a wide variety of pictures, some of which must have been labelled in foreign languages.
#Funny words generator generator
“The preliminary experiments suggest that hybridized nonce strings can be methodically crafted to generate images of virtually any subject as needed, and even combined together to generate more complex scenes,” he says.Ī farpapmaripterling lands on a feuerpompbomber, as imagined by the text-to-image generator DALL-E 2 (Source )
For example, the sentence “ An eidelucertlagarzard eating a maripofarterling ” produced images of a lizard devouring a butterfly. Millière even produced sentences of these made-up words. In each case, the generator produced realistic images of the English word. He created other words in the same way with comparable results: insekafetti for bugs, farpapmaripterling for butterfly, coniglapkaninc for rabbit and so on. To his surprise, putting this word into the DALL-E 2 text-to-image generator produced a set of images of cliffs. Millière took parts of these words to create the nonsense term “ falaiscoglieklippantilado ”. So the word “cliff” is Klippe in German, scogliera in Italian, falaise in French and acantilado in Spanish. He used a technique called “macaroni prompting” to create nonsense words by combining parts of real words from different languages. Millière wondered whether text-to-image systems could be similarly vulnerable. A famous example is the Lewis Carroll poem Jabberwocky : “' Twas brillig, and the slithy toves, Did gyre and gimble in the wabe… ” For most people, reading it conjures up fantastical images. Nonsense words can trick humans into imagining certain scenes. These systems are not perfect but nevertheless impressive. In recent months, text-to-image systems have advanced to the point that users can type in a phrase, such as an astronaut riding a horse, and receive a surprisingly realistic image in response.
“Adversarial attacks can be intentionally and maliciously deployed to trick neural networks into misclassifying inputs or generating problematic outputs, which may have real-life adverse consequences,” says Millière. Millière has discovered a way to trick text-to-image generators using made up words designed to trigger specific responses. Today we get an answer thanks to the work of Raphaël Millière, an artificial intelligence researcher at Columbia University in New York city. But are these systems also susceptible to adversarial attack and if so, how? Users type in a word or phrase and a specially trained neural network uses it to conjure up a photorealistic image. It also raises interesting questions about other kinds of computational intelligence, such as text-to-image systems. This kind of deliberate trickery has important implications since malicious users could use it to bypass security systems. The patterns cause otherwise powerful face or object recognition systems to misidentify things or faces they would normally recognize. Adversarial images are pictures that contain carefully crafted patterns designed to fool computer vision systems.
0 kommentar(er)
