AI Notes

A system prompt is a habit, not a lock

I can write at the top of the conversation that a certain file must not be repeated, and the model will often comply. Then a later message describes the same request in other words, and the line I was protecting shows up in the answer. The first instruction was read. It was not enforced by anything except the model's willingness to follow it.

When the boundary matters, I put it in the program around the model. The tool does not receive the file, the key, or the network call. There is then nothing for a clever rephrasing to retrieve. Asking the model to promise is how I remind myself of the rule. It is not how the rule is kept.

A prompt is a good place for taste: tone, length, what to skip. It is a bad place for a secret, and a bad place for a guarantee.

Back to the notes