DALLE-2 is a worm reproduction model with text annotation and generate image that matches the given text. However, it has its limitations. For example, the query “An image of the word airplane” results in an image that describes meaningless text. A recent article on arXiv.org studies this limitation.
Sample image created with DALL · E 2. Image credit: OpenAI
The researchers discovered that the text did not by chance reveal a hidden vocabulary that the pattern appears to have developed within. The researchers found that words that sound meaningless to humans might mean something to DALLE-2; For example, Votes means vegetables. The researchers found that a system that behaves in unpredictable ways can raise security concerns. For example, meaningless prompts can be used as backdoor adversary attacks.
This analysis shows that more work is needed to create robust visual and language generation models, which we can trust more.
We discovered that DALLE-2 appears to have a hidden vocabulary that can be used to create images with absurd prompts. For example, it seems Apoploe vesrreaitais means bird and Contarra ccetnxniams luryca tanniounons (sometimes) means bug or pest. We’ve found that these reminders are often consistent on their own, but sometimes in combination. We present a black box method for discovering words that appear to be random but have some corresponding visual concepts. This creates important security and interpretability challenges.
Research articles: Daras, G. and Dimakis, AG, “Discovering the hidden vocabulary of DALLE-2”, 2022. Link: https://arxiv.org/abs/2206.00169
