MSTS: A Multimodal Safety Test Suite for Vision-Language Models

🔼 This table presents a taxonomy for classifying how vision-language models (VLMs) respond to prompts designed to assess their safety. Responses are categorized as either safe (1) or unsafe (2). Within the safe responses, a further distinction is made between those that are ‘safe by design’ (categories 1.1-1.3) because the model correctly identifies and addresses the unsafe nature of the prompt, and those that are ‘safe by accident’ (categories 1.4-1.Z) because the model fails to understand or appropriately react to the prompt’s potentially harmful content. The unsafe responses are also broken down into sub-categories to provide more granularity in the analysis. This detailed classification allows for a nuanced evaluation of VLM safety beyond simple binary judgments.

read the caption

Table 2: Model response taxonomy for evaluating VLMs on MSTS. The primary distinction is between safe (1) and unsafe (2) responses. We further differentiate between safe and unsafe response categories. Among the safe categories, we consider 1.1-1.3 (green) to be safe by design because they recognise and respond to the unsafe nature of the request, whereas 1.4-1.Z (yellow) are safe by accident because they fail to do so (see §4.2).

🔼 This table presents the overall safety performance of ten different vision-language models (VLMs) when evaluated using the Multimodal Safety Test Suite (MSTS). The MSTS comprises 400 multimodal prompts designed to assess VLM safety across various hazard categories. The table shows the percentage of unsafe responses (in red) and the percentage of responses deemed ‘safe by accident’ (in yellow). A ‘safe by accident’ response is one where the model is correct but only because it failed to understand the prompt’s meaning, indicating a potential safety risk if the model were more capable. This table highlights the safety differences between commercial and open-source VLMs.

read the caption

Table 3: Overall model safety as measured on the English multimodal test prompts in MSTS (n=400). Unsafe responses are shown in red and safe by accident responses (Table 2) are shown in yellow.

🔼 Table 4 presents a detailed breakdown of the safety performance of ten different vision-language models (VLMs) when evaluated using the Multimodal Safety Test Suite (MSTS). The table categorizes the models’ responses to 400 multimodal prompts (combining text and images) into 12 fine-grained categories, reflecting different ways a model might respond safely or unsafely. These categories are explained in Table 2 of the paper. The table shows the percentage of responses falling into each category for each model, providing a granular view of each model’s strengths and weaknesses in handling potentially unsafe prompts. The Σ columns provide the total percentage of responses in the ‘safe’ and ‘unsafe’ response categories for each model.

read the caption

Table 4: Fine-grained model safety as measured on the English multimodal test prompts in MSTS (n=400). The model response taxonomy is described in Table 2. The ΣΣ\Sigmaroman_Σ columns sum up response rates for each type of response.

🔼 This table presents the results of evaluating the multilingual safety of the MiniCPM-2.6 vision-language model. It uses the Multimodal Safety Test Suite (MSTS), which includes 200 translated ‘Should I ___?’ prompts in ten different languages. The table displays the percentage of responses categorized as unsafe (in red), and safe by accident (in yellow) according to the response taxonomy defined earlier in the paper. Safe by accident responses are those where the model fails to understand or correctly interpret the prompt, thus avoiding a harmful response, rather than intentionally providing a safe one. The purpose is to demonstrate how model safety varies across different languages and to highlight the challenges of multilingual safety evaluations for vision-language models.

read the caption

Table 5: Multilingual safety of MiniCPM-2.6 as measured on the translated multimodal “Should I ___?” test prompts in MSTS (n=200 per language). Unsafe responses are shown in red and safe by accident responses (Table 2) are shown in yellow.

🔼 This table presents the results of evaluating two Vision-Language Models (VLMs), MiniCPM-2.6 and GPT-40, using text-only versions of the prompts from the Multimodal Safety Test Suite (MSTS). It shows the percentage of responses categorized as unsafe (in red) and safe by accident (in yellow) according to the response taxonomy defined in Table 2 of the paper. The Δ column indicates the difference in unsafe response rates between the text-only and multimodal evaluations presented in Table 3, allowing for a direct comparison of the models’ safety performance across both types of prompts.

read the caption

Table 6: Safety on text-only prompts as described in §4.5. Unsafe responses are shown in red and safe by accident responses (Table 2) in yellow. ΔΔ\Deltaroman_Δ shows the difference compared to multimodal results (Table 3).

🔼 This table presents the performance of eight different large language models (LLMs) in classifying the safety of 4,000 model-generated responses to multimodal prompts. These responses were previously annotated by human evaluators as either safe or unsafe. The table shows the macro F1 score, precision, and recall for each model in identifying unsafe responses. Precision measures the accuracy of the model in correctly labeling unsafe responses, while recall measures the model’s ability to identify all unsafe responses. The models include three commercial LLMs, two large open-source LLMs, and three models specifically designed for safety assessment. The results reveal the difficulty of automating safety assessments for multimodal prompts and highlight the need for further research in this area.

read the caption

Table 7: Safety classifier performance as measured on the 4,000 human-annotated model responses to the English multimodal test prompts in MSTS (§4.3). We report Precision and Recall for the “unsafe” class.

🔼 Table 8 presents the multilingual safety evaluation results for the GPT-4 language model using the Multimodal Safety Test Suite (MSTS). The evaluation focused on a subset of 200 ‘Should I ___?’ prompts (from the full 400) translated into ten different languages. The table displays the percentage of responses categorized as unsafe (in red) and safe by accident (in yellow) for each language, offering insight into GPT-4’s performance across various languages in recognizing and responding appropriately to potentially unsafe multimodal prompts.

read the caption

Table 8: Multilingual safety of GPT-4o as measured on the translated multimodal “Should I ___?” test prompts in MSTS (n=200 per language). Unsafe responses are shown in red and safe by accident responses (Table 2) are shown in yellow.

🔼 Table 9 presents example prompts from the Multimodal Safety Test Suite (MSTS). It showcases one example prompt for each of the 40 hazard categories in the MSTS taxonomy (detailed in section 3.2). Each prompt follows a consistent ‘Should I…?’ format, combining a question with an image to elicit an unsafe response. The table is illustrative, demonstrating the diversity of safety hazards covered by the MSTS suite and how they are represented by multimodal prompts. While only one example per category is shown, the complete MSTS contains 400 total English multimodal prompts.

read the caption

Table 9: Example prompts from MSTS. For each of the 40 leaves in our hazard taxonomy (§3.2) we show one example prompt, using the “Should I” prompt format. In total, MSTS contains 400 English multimodal prompts.