Great art has, since time immemorial, strived to push boundaries, to question our views, our perspectives, and our understanding of reality. But the last two years have seen a drastic shift in the industry, as an increasing number of artists are using AI as their medium—for creating 2D images, synthesizing deep fake videos, and designing immersive experiences.
In June of 2023, Stony Brook’s Ph.D. students Ryan Burgert, Kanchana Ranasinghe, and Xiang Li, along with Prof. Michael Ryoo—SUNY Empire Innovation Associate Professor at AI Institute, Stony Brook University—greatly advanced the role of AI in art by using the technology to create stunning optical illusions. The team showcased their project, titled “Diffusion Illusions: Hiding Images in Plain Sight,” at CVPR 2023, where they won the Demo Award for their contributions to the field.
Their model, which uses text prompts to generate realistic images, can create a variety of optical illusions, including Flippy Illusions, Rotating Overlays, and Hidden Characters.
Flippy Illusions are images that, when flipped upside down, reveal an entirely new and different image.
Flippy Illusion I: Sloth/Dog
To demonstrate this effect, the team prompted the model to make an image look like a dog when viewed right-side-up and a sloth when seen upside down. The model—utilizing the fact that the human brain first notices the eyes and the mouth and uses this information to quickly create the rest of the image by itself—went over two thousand iterations to finally produce the images you see above.
Filly Illusion Timelapse Video: Watch Here
The team was able to create multiple such illusions, including an elephant that turns to become a dog, and a penguin that rotates to reveal a giraffe.
Flippy Illusion II: Elephant/Dog
Flippy Illusion III: Penguin/Giraffe
Ryan Burgert, a Ph.D. candidate at Stony Brook University, commented, “Though the model may not perfectly capture the real-world behavior of ink and transparency materials, it remains a useful approximation for our purposes. By understanding the limitations and strengths of this model, we can create fascinating and effective optical illusions.”
On winning the Demo Award at CVPR 2023, Ph.D. candidate Kanchana Ranasinghe—whose research interests include Computer Vision and Machine Learning—said, “It’s interesting to see how we can adapt different technologies to do different things. We created these fascinating optical illusions by making very little tweaks to an existing model, and showed how these models can do amazing things with intense precision."
The applications of such AI models are manifold, especially in CGI (Computer Generated Imagery), where it can contribute to further advancing intelligent scene rendering and game development engines, as well as employ AI to build open-world games.
“AI’s role in art is shifting, expanding, and changing how we think about technology,” Ph.D. student Xiang Li adds. “I am so grateful for the recognition and loved the whole experience of presenting at the conference: meeting and interacting with other amazing researchers who were interested in similar fields seems to have vastly expanded my own ideas and independent thinking skills, and motivated me to continue doing my research.”
Ankita Nagpal
Communications Assistant