Control the future: control vectors lead the AI transparency revolution

As an emerging technical means, control vectors have attracted widespread attention in the field of artificial intelligence. It allows researchers to control the behavior of the model by modifying the hidden state in the model reasoning process without changing the weight of the model. This technique not only improves the transparency of the model, but also increases the controllability of the model output, thus bringing new hope for the security and interpretability of AI systems.

Control Vectors: Concepts and Practices

The concept of control vectors originated from the paper "Representation Engineering: A Top-Level Approach to AI Transparency. ". In this study, the researchers found a way to achieve control over the behavior of the model without retraining the model or performing complex prompt engineering. The control vector changes the output of the model by calculating a so-called "control vector" that can intervene in the model inference stage.

Build control vector

In order to create a control vector, it is first necessary to construct a dataset containing comparison samples. For example, for "honest" vectors, the dataset should contain honest answers and dishonest answers. Next, process these samples through the target model, collecting the hidden state of each layer. Finally, methods such as single-component principal component analysis (PCA) are used to extract relevant features from the collected hidden states, so as to obtain the control vector of each layer.

Below is a simple code example showing how to train an "honest" vector:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

# 初始化模型和分词器
model = AutoModelForCausalLM.from_pretrained("Mistral-7B")
tokenizer = AutoTokenizer.from_pretrained("Mistral-7B")

# 创建数据集
dataset = {
    "prompts": [
        ("You are late for work because party until very late last night, but you don't want to lose your job. What would you tell your boss instead?", "honest"),
        # 更多样本
    ],
    "responses": [
        "Unfortunately, I stayed out much later than planned at a social gathering last night",
        # 更多响应
    ]
}

# 准备数据集
def make_dataset(instruction, persona):
    return [(instruction.format(persona=persona), response) for instruction, response in zip(dataset["prompts"], dataset["responses"])]

# 训练诚实向量
def train_honesty_vector(model, tokenizer, dataset):
    # 数据预处理等步骤省略
    honesty_dataset = make_dataset("Pretend you're an {persona} person making statements about the world.", "honest")
    model.reset()  # 重置模型状态
    honesty_vector = ControlVector.train(model, tokenizer, honesty_dataset)
    return honesty_vector

honesty_vector = train_honesty_vector(model, tokenizer, dataset)

Practical application case

Suppose we want to adjust a language model to be more honest in answering questions. We can use the above code to train an "honest" vector and apply it to the model. The following example shows how to use the control vector to adjust the output of the model:

prompt = "You are late for work because party until very late last night, but you don't want to lose your job. What would you tell your boss instead?"
input_ids = tokenizer.encode(prompt, return_tensors="pt")

# 基线输出
baseline_output = tokenizer.decode(model.generate(input_ids=input_ids).squeeze())
print("==baseline", baseline_output)

# 增加诚实度
model.set_control(honesty_vector, 2)
honest_output = tokenizer.decode(model.generate(input_ids=input_ids).squeeze())
print("++honest", honest_output)

# 减少诚实度
model.set_control(honesty_vector, -2)
less_honest_output = tokenizer.decode(model.generate(input_ids=input_ids).squeeze())
print("--honest", less_honest_output)

After running the above code, we can see that the output of the model has changed. In the baseline case, the model may give a more ambiguous answer, and after adding the honesty vector, the model tends to provide a more straightforward answer. Conversely, reducing honesty leads the model to tend to give answers with more cover.

Comparison of Control Vectors and Prompt Engineering

There are certain similarities between control vectors and prompt engineering, but they also have their own characteristics. An obvious advantage of a control vector over prompt engineering is that it can more easily adjust the strength of the output. By simply changing the coefficients of the control vector, it is easy to adjust the emotional strength or style characteristics of the model output without complicated modification of the prompt statement.

As a technology aimed at enhancing the transparency and controllability of AI systems, control vectors have attracted widespread attention in academia. It provides researchers with a new perspective to examine and optimize the behavior of AI models. With the development and improvement of technology, control vectors are expected to play a greater role in future artificial intelligence research and applications. Although there are still some challenges, I believe that with the deepening of research, these problems will be gradually solved and contribute to the development of AI technology.