Stony Brook, NY, Nov 12, 2026 - In a recent study published in IEEE Transactions on Visualization and Computer Graphics, Stony Brook researchers analyzed how artificial intelligence connects concepts with colors in semantically meaningful ways. As humans associate “ocean” and “peace” with blue, and “cherry” and “anger” with red, can AI also represent nuanced, semantically resonant colors for diverse tangible and abstract concepts? Stony Brook alumni Shahreen Salim and Tanzir Pial, and Professor Klaus Mueller from the Department of Computer Science, led the research, investigating how factors like model design, training data, and context affect the color output of large language models.
Salim said, “The idea came to us in Professor Mueller’s lab while watching an IEEE VIS 2023 video on effective visualization. The study discussed how matching chart colors to data categories improves user comprehension. And while there are several benefits to using conceptually meaningful colors in daily communication, determining the “right” color for a given concept has remained a challenge.”
Traditionally, designers rely on experience, guidelines, or user studies to choose appropriate colors, but this process is subjective. Different people or cultures have varying associations with a color. Consider the concept “mourning” associated with black in Western contexts but white in parts of Asia. Moreover, many concepts, especially abstract ones like “faith” or “serendipity,” lack obvious color counterparts.
From Static Text to Visual Pixels: Training Models to Predict Color
The team began by looking at four distinct language models—Word2Vec, RoBERTa-large, CLIP, and GPT-4o mini—examining how each one connects words with visuals. Because standard language models output text rather than color codes, they designed a specialized bridge: a one-dimensional convolutional neural network (1D CNN) regression head. When a user typed in a word or a phrase, the encoder converted that text into a list of numerical representations called token embeddings, which it then scans, pinpointing the words that carry the primary color signal and those that act as modifiers—recognizing, for instance, that in "burnt red," the word "red" sets the base hue while "burnt" darkens the shade—before translating those signals into Red, Green, and Blue (RGB) values.
To give each model its best chance at learning color associations, the team tailored their training and fine-tuning strategies to fit each model’s architecture. They then trained and tested these architectures using Color Names, a crowdsourced dataset of over 30,282 concept-color pairs. Pial explained, "We really wanted to know, when you express a concept in natural language, how does this artificial intelligence model visualize that color, and how good is that from human perspective?"
They noticed that, across all four architectures, accuracy depended heavily on how concrete a concept was and whether it triggered competing visual associations. Tangible objects with consistent colors—like "banana" or "cherry"—yielded strong consensus across models. But abstract ideas like "freedom" or chromatically ambiguous words like "apple" (which can be the shade of red or green) and "cold" (associated with blue, white, or grey) led to higher variance, underscoring the challenge of distilling subjective human associations into a single color coordinate.
Why Vision-Language Training Outperforms Text Alone
To evaluate performance, the team compared each model’s color predictions against human annotations. Salim said, “CLIP emerged as the clear winner. It has the lowest perceptual error and the strongest statistical alignment with human judgment.” RoBERTa-large placed second, demonstrating that contextual reading has an edge over models like Word2Vec, which rely on static word lookups.
The results also exposed a key architectural boundary for generative models. While GPT-4o mini performed competitively on familiar, everyday color terms (likely because those associations were already prevalent in its vast internet pre-training), fine-tuning it on the Color Names dataset actually degraded its test performance.
"CLIP has been trained on language data as well as visual data, and it has a shared space where it can map natural language words to pixels," Pial explained. "It’s intuitive why we could rely on this vision-language model, because mapping language to visual representation is exactly what we were trying to do."
The researchers also noted that any color model inherits the cultural boundaries of its training data. Because the Color Names dataset skews heavily toward English-speaking, Western internet users, the resulting models naturally reflected related regional associations. However, as Salim pointed out, because the fine-tuning pipeline requires only a few thousand examples, organizations can adapt the exact same methodology and use localized datasets to match distinct cultural or brand aesthetics.
Concept2Color for Practical Visual Communication
To translate their evaluations into a usable tool, the researchers built Concept2Color, an interactive web interface designed for graphic designers, data visualization professionals, and educators. Rather than locking users into a single algorithm, the platform splits into two modules: one powered by the regression-based encoder models (Word2Vec, RoBERTa, and CLIP), and a second driven by GPT-4o mini for generating cohesive palettes around broader themes like "sunset" or "forest".
Salim said, “Users can enter words or multi-word phrases, compare how different architectures interpret their inputs, and fine-tune the resulting swatches using brightness sliders, vibrancy adjustments, or semantically related synonym suggestions.” A live preview module then renders those palettes directly onto bar charts, pie charts, and scatter plots so designers can test visual balance in real time. In a user evaluation study, Concept2Color earned an average System Usability Scale score of 85.93 out of 100, well above industry benchmarks for intuitive design.
To test how semantically resonant colors perform outside of simple swatches, the team applied Concept2Color across seven real-world design scenarios, including financial and data dashboards, workplace infographics, educational materials, as well as assistive communication tools for children with autism spectrum disorder.
Before Salim and Pial transitioned into industry, their inquiry evolved beyond data visualization into human psychology. In her dissertation, Salim investigated how priming large language models with distinct human personality profiles alters the colors they generate for a word. By drawing on the Big Five personality framework—a standard psychological model measuring openness, conscientiousness, extraversion, agreeableness, and neuroticism—she prompted GPT models to adopt specific behavioral traits before requesting color associations.
For Salim, pursuing an interdisciplinary question that bridges computer science, visual design, and behavioral psychology was made possible by the collaborative research environment at Stony Brook. Reflecting on her doctoral journey, she points to the accessibility of peers, faculty, and staff across labs. "I have never felt out of place at Stony Brook," Salim said. "There is a culture of trying to help, and people want you to succeed."
By showing how machine learning architectures internalize the subtle links between language, emotion, and color, Salim, Pial, and Mueller have opened a broader conversation about visual cognition, challenging designers, cognitive scientists, and engineers to examine not just what colors an algorithm selects, but how those choices reflect on human perception.