Art Appreciation in the Era of Artificial Intelligence
I admit to being obsessed with the details that surround us. For especially unique and captivating details, I go to art museums. The art museum has many rare, and often unique, collections of artworks — objects that are waiting to be explored. Artworks are like portals to another dimension.
The painting above is from the Cummer’s permanent collection. The painter, Paul Camille Guigou, lived a short life, having died when he was 37. While he was with us, he painted many pieces from his home in Provence. Lourmarin is a region in Provence. While the river is not explicitly named, it is likely the Aigue Brun which runs south until it merges with the Durance. Guigou also has another painting of this river where it is named. What is in this painting?
We can make out the impressive landscape with cliffs, people on a river bank, sheep grazing above on the hillside, two prominent waterfalls, birds, wildflowers, and a bright blue sky dotted with sparse clouds. This is something I made up on the spot. Now, here is a concise description generated by Artificial Intelligence (AI):
The image is a landscape painting of a river valley. The valley is surrounded by steep cliffs and hills. There is a waterfall in the foreground, and a river flows through the valley. There are trees and grass on the banks of the river, and a few people are walking in the distance. The sky is blue and there are a few clouds.
This was generated from Google’s AI Studio using the large context window large language model (LLM) Gemini Pro 1.5, still a new model that has not yet been publicly released. This was in response to my prompt “What are the elements of this image?” Here is another, somewhat more extensive, narrative, this time generated with OpenAI:
In this captivating landscape painting, the artist skillfully captures the serenity of a river winding through a lush valley, framed by a clear sky and ancient mountains, to evoke a sense of timeless beauty and the eternal dance between nature and humanity. The scene is a harmonious blend of the vibrant vegetation along the riverbanks and the dynamic flow of water, set against the backdrop of the sky and stoic mountains, symbolizing the balance between change and constancy. Subtly included human figures reflect our fascination with and impact on the natural landscape, emphasizing our connection to and the fragility of the earth. Through masterful use of color, light, and composition, the painting invites introspection on the splendor and delicacy of the environment, serving as a poignant reminder of the importance of cherishing and preserving the natural world around us, and urging a harmonious coexistence with nature.
Again using ChatGPT 4.0, I told the AI that it was a computer scientist and asked a similar question about the painting, and received:
Scene
│
├── Sky
│ ├── Color: Blue
│ └── Objects: Birds
│
├── Background
│ ├── Mountains
│ │ ├── Color: Pale Blue/Grey
│ │ └── Elevation: High
│ │
│ └── Cliffs
│ ├── Stratification
│ │ ├── Layer 1: Type, Thickness
│ │ ├── Layer 2: Type, Thickness
│ │ └── Layer 3: Type, Thickness
│ │
│ ├── Vegetation
│ │ ├── Trees
│ │ │ ├── Species
│ │ │ └── Height
│ │ │
│ │ └── Shrubs
│ │ └── Density
│ │
│ └── Erosion Level
│
├── Midground
│ ├── Waterfall
│ │ ├── Width
│ │ ├── Height
│ │ └── Water Flow: Speed
│ │
│ └── River
│ ├── Color: Reflective attributes
│ └── Flow: Speed, Direction
│
└── Foreground
├── Field
│ ├── Vegetation
│ │ ├── Grass
│ │ │ └── Height
│ │ │
│ │ └── Flowers
│ │ ├── Type
│ │ └── Color
│ │
│ └── Animals: Count, Species
│
└── People
├── Position: Coordinates
└── Activities: Descriptions
This is a scene graph, which is a data structure found in computer graphics. Such scene graphs can be employed to identify the formal components of any image. Since the painting has interesting natural colors, I asked the AI to do a color analysis:
- Palette Dominance: The painting has a dominance of natural, earthy tones. There’s a strong presence of greens and blues, which are indicative of the natural landscape — the lush vegetation and the flowing water.
- Warm and Cool Balance: There is a balance between warm and cool colors. The warm hues are seen in the earthy tones of the cliffs and the reddish and brownish colors in the foreground, providing a sense of warmth and sunlight. In contrast, the cool colors are evident in the blues of the sky and the water, which lend the painting a calming and serene atmosphere.
- Contrast and Saturation: There is moderate contrast between the light and dark areas, such as the shadows in the cliffs and the bright highlights in the sky and on the water’s surface. The saturation levels vary, with more vibrant colors in the foreground and more muted tones in the background, which creates depth and suggests atmospheric perspective.
To get at a more quantitative, and formal, analysis I asked the AI to plot a color histogram using CMYK. CMYK is a common color space for print media using subtractive color, rather than RGB which is additive. Note that the heavy use of key (K), which represents black on the left side of the plot. This matches our common sense interpretation of a fairly dark painting. This kind of palette is common in landscapes from the Dutch golden age in the 17th century through French impressionism in the 19th century, to provide two sample end points.
So far, this has an an interpretive exercise in AI methods used to better understand Guigou and his art. Here is a video created using Kaiber, where I fed in the painting with a prompt “Show people moving across the landscape” and got this:
I really enjoyed this, and it was time to get creative. When you look at an artwork, you have at least two broad paths to walk. The first path is as art historian and the second is as artist. The two perspectives are very different. The historian is a scholar and familiar with lexical rhetoric. Words mostly. The artist is understanding something by creating and synthesis. The artist understands and communicates through making. Here are 4 recreations on the style and theme of the painting, all done in MidJourney with a style reference option:
I was curious about the waterfalls. They drew me into the painting. So I asked OpenAI to generate an artistically inspired waterfall using the programming language Processing. Processing is a favorite language promoted and used by designers and artists:
I did not need to know how to program to generate this. I iterated a few times to get it right, and the AI did the rest by writing the code. I could have just as well focused on other parts of the painting to investigate the potential for dynamic presentation. I asked the AI to put on 5 hats and describe the painting while wearing each hat (Science, Technology, Engineering, Art, and Mathematics — STEAM). This used a fine-tuned GPT that I created for the GPT Store (OpenAI):
In the provided image, which appears to be a landscape painting, we can explore various aspects through the lens of Science, Technology, Engineering, Art, and Mathematics.
SCIENCE: This scene depicts a diverse ecosystem, showcasing biological diversity in both flora and fauna. The geology of the cliffs and the hydrodynamics of the waterfall can be studied to understand erosion and sediment transport. The water cycle is also represented here, where water flows from rivers to the sea, evaporates, and precipitates back to land.
TECHNOLOGY: The technology in creating a landscape painting could include the chemistry of pigments and the development of canvas and brushes. Historically, advancements in these technologies allowed artists to capture scenes with greater accuracy and variety in color and texture.
ENGINEERING: The engineering aspect could include the structural integrity of the cliff formations, which would be of interest to civil and geological engineers. If this were a real landscape, the design of any bridges or barriers to protect the natural environment would involve environmental engineering principles.
ART: As an art piece, this painting offers a snapshot of Romanticism, focusing on the sublime beauty of nature. The composition, use of light, and brushwork contribute to the aesthetic and emotional impact of the artwork, inviting viewers to reflect on their own experience with nature.
MATHEMATICS: Mathematical concepts are inherent in art through perspective, symmetry, and proportions. The painting can also be analyzed for patterns in the landscape, fractal patterns in the foliage, and the fluid dynamics of the water, which can be described using mathematical equations.
This analysis provides a STEAM perspective, combining an appreciation of the natural world with an understanding of the principles and technologies that help us interpret it.
Where is this leading and what am I claiming about AI, museums, and art? Art can be viewed from many diverse perspectives. Each perspective can be seen analogically by wearing hats or donning colored lenses. You see what you want to see and this seeing can be augmented through the technology of AI. Before the AI that we have come to read about in papers, we would have a hard time seeing these perspectives. They have always been there. But you might have needed to hire someone for a personal tour. Now, anyone can play in this space.
I tried many other experiments with art, but they are not included here for the sake of necessary brevity. For example:
- Create a story about the painting. What are the people saying to each other? I received some lengthy stories about people looking at the cliffs and the water. The woman was a geologist and talked of the formation of limestone cliffs, so common in this part of France.
- Create music from the painting. The painting’s elements can be tied to musical passages that relate to pastoral views, the grandeur of nature and flowing water.
- Relate the painting to computer science. Actually, this was previously explained in the examples of a scene graph, and generating code. But I also explored mindmaps, concept maps, and predicate logic.
AI can help us explore the world, the microcosm of that world in museums, in new ways. It is a good time to explore given the emerging interest in AI and its effects on interpretation and education.
Because I cannot help myself:
There are numerous opportunities for reflection. You may not like some of the interpretations in this article. You might find AI to be a little scary so you want to talk about it. Think of this article as a gateway to identifying your concerns. Get a group conversation going.
You may want to try other experiments. Since the AI output is by no means perfect, there are opportunities for critiquing the output on the road to understanding both art and AI.