Color and Design Trend Analysis with AI and LLMs

Introduction

In this article, we delve into a unique project undertaken by our AI consulting firm for a design company. Our task was to develop an AI-powered tool for analyzing color trends and object patterns in best-selling art categories. This project highlights the integration of AI in creative industries, particularly in trend forecasting and data-driven design strategies.

Project Overview

Our client, a design company, needed a sophisticated system to track color trends and popular elements in art. The goal was to use this data to inform their product designs and marketing strategies, so that they stay aligned with current trends. The challenge was to create a tool that could not only analyze images for color trends but also categorize these insights in an easily searchable format. The project was executed in several steps:

  1. Web Scraping for Data Collection:

    • We began by scraping images from top-selling sections of various art & design websites.
    • Images were stored in S3 blob storage, with metadata recorded in a PostgreSQL database.
    • Our database included two tables: artwork for image details and artwork_color for color info.

  2. AI-Powered Image Analysis:

    • Using OpenAI's GPT-4-Vision, we analyzed and described each artwork.
    • The AI's descriptions were added to our database, providing deeper insights into each image.

  3. Detailed Color Analysis:

    • The AI model identified and described each image's color palette in detail.
    • This step was crucial for later creating a searchable, organized color data format.

  4. Structuring Colors into JSON Format:

    • We converted the color descriptions into structured JSON for easy database integration.
    • This step involved a specific prompt to guide the AI in producing JSON output.

Collecting the Data

We started the project by scraping images from the best-sellers section of various art & design websites that our client wanted to target. This process involved storing the images in S3 blob storage and cataloging related metadata in a PostgreSQL database.

Before delving into AI analysis, we create some tables to store the data:

CREATE TABLE artwork (
    id SERIAL PRIMARY KEY,
    title TEXT,
    description TEXT,
    colors TEXT,
    image_url TEXT,
    website TEXT,
    created_at TIMESTAMP,
);

CREATE TABLE artwork_color (
    id SERIAL PRIMARY KEY,
    artwork_id INTEGER REFERENCES artworks(id),
    color_name TEXT,
    color_description TEXT
);

This schema comprises two tables: artwork for metadata and artwork_color for storing detailed color information extracted from the images.

Once the webscraper finishes, we should have values for the id, title, image_url, website, and created_at columns. The description & colors columns (along with the artwork_color table ) will be updated later in the process.

Image Analysis with AI

Our Python backend leveraged OpenAI's GPT-4-Vision model to describe each artwork. This description provided a foundational understanding of the image content.

Start by fetching your data. How this is implemented will depend on your specific setup, but we simply used sqlalchemy to grab the data from our database:

# Create a SQL connection & grab your data
def get_connection():
    DB_USER=""
    DB_PASSWORD=""
    DB_HOST=""
    DB_PORT=""
    DB_NAME=""
    
    CONNECTION_STRING = f"postgresql+psycopg2://{DB_USER}:{DB_PASSWORD}@{DB_HOST}:{DB_PORT}/{DB_NAME}"
    
    engine = create_engine(CONNECTION_STRING)
    connection = engine.connect()
    
    return connection

connection = get_connection()
result = connection.execute("SELECT * FROM artwork")
rows = result.fetchall()
artworks = [dict(zip(result.keys(), row)) for row in rows] # This will be a list of objects/dictionaries of the artworks.

Connect to OpenAI:

import json
import openai
from openai import OpenAI

API_KEY = ""

client = OpenAI(api_key=API_KEY)

Loop through your artworks and make the requests to GPT4:

# For each artwork in artworks, do: 
response = client.chat.completions.create(
    model="gpt-4-vision-preview",
    messages=[
        {
            "role": "user",
            "content": [{"type": "text", "text": "What’s in this image?"}, 
                        {"type": "image_url", "image_url": {"url": artwork['image_url']}}],
        }
    ],
    max_tokens=300,
)

Once you get a response, you can save the description to your database. We'll leave specifics of the implementation for your team:

description = response.choices[0].message.content
# Update your database accordingly...
# "UPDATE artwork SET description = description WHERE id = ...

Color Analysis

Next, we prompted the AI model to identify and detail the colors in each image. We focuse on specifics like shade and tone, which will aid in extracting precise keywords for more refined search capabilities.

color_analysis_prompt = '''
Please examine the attached image and provide a comprehensive color palette.
For each color you identify, I would like you to:

- **Name the Color**: Use common color names if possible.
- **Describe the Shade or Tone**: Give a brief description of the color's shade or tone (for example, 'light blue', 'vibrant red', etc.).
'''

Repeat the process of looping through the images and making requests to GPT4-Vision (you can also do it in one loop, but we're splitting it out for the sake of this guide):

# For each artwork in artworks, do: 
color_analysis_response = client.chat.completions.create(
    model="gpt-4-vision-preview",
    max_tokens=2000,
    messages=[
        {
            "role": "user",
            "content": [{"type": "text", "text": color_analysis_prompt}, 
                        {"type": "image_url", "image_url": {"url": artwork['image_url']}}],
        }
    ],
)

Again, once you get a response, you can save the colors to your database:

color_analysis_text = color_analysis_response.choices[0].message.content
print(color_analysis_text)
# Update your database accordingly...
# "UPDATE artwork SET color = " + color_analysis_text WHERE id = ...

Converting Colors to Structured JSON

We then converted our color analysis data into a structured JSON format to streamline updates to our database and enhance our analytics capabilities. This step involved linking each color instance in the artwork_color table with its corresponding artwork in the artwork table. By doing so, we actively created a system that allowed us to aggregate data, conduct detailed statistical analyses, and track color trends over time, including identifying the most and least frequent colors.

OpenAI provides a few different methods for steering the model to produce valid JSON (function calling, JSON mode parameter), but we had limited success with their suggestions. What worked best was simply few-shot prompting with detailed instructions. Here's the prompt we used:

text_to_colors_json_prompt = '''
Please extract the colors and their descriptions from the text.
Your response should be formatted as a JSON array, where each item represents a distinct color identified in the text.
For each color, include the following attributes:
1. "color": This should be the common name of the color, such as "light blue', 'vibrant red', etc.
2. "description": A brief description of the shade or tone of the color.

It's crucial that the response only contains the JSON array with the specified attributes and no additional text.

Example output:

[
   {
      "color":"Light Blue",
      "description":"A calming color resembling a clear sky"
   },
   {
      "'color":"Burnt Orange",
      "description":"A warm hue resembling autumn leaves or a sunset"
   }
]

Here is the text:

'''

Then we simply loop through our artworks again (this time the color column will be populated) and run the following (you may have to refetch your data):

color_analysis_text = artwork['color']

colors_json_response = client.chat.completions.create(
    model="gpt-3.5-turbo-1106",
    max_tokens=4096,
    messages=[
        {
            "role": "user",
            "content": text_to_colors_json_prompt + color_analysis_text
        }
    ]
)

From here, we can grab the response and update our database. Again, we'll leave the implementation details up to you:

colors_json = colors_json_response.choices[0].message.content
colors_data = json.loads(colors_json)
artwork_color_df = pd.DataFrame(colors_data) # Make sure you add the FKEY artwork_id at some point.
# artwork_color_df.to_sql(....) # Pass your connection...

Conclusion

This project is a testament to the power of AI in transforming traditional industries like design. By leveraging LLMs for color forecasting and trend analysis, we equipped our client with a tool that offers deep insights into market trends, thus empowering them to make informed, data-driven decisions in their creative processes.

Stay updated with the latest AI news and our guides.