What to send#
Send the input exactly as your system has it. A support message can go as a string; a ticket with its metadata reads better as JSON, because field names tell the model what each value is.
{
"state": "Customer: I was charged twice and nobody has replied for 3 days."
}state is optional. A question can stand alone when its instructions carry everything, for example a Truth check on a sentence you put in the instructions. Leaving state out, sending "" and sending null all mean the same: no state.
How the model sees it#
| You send | The model reads |
|---|---|
| A string | The string exactly as sent, never re-serialised, also when it holds a JSON document (compact, spaced or indented) |
| An object or an array | Compact JSON with your key order kept. Nested objects, arrays, numbers, booleans, null and empty values are all fine |
Nothing, "" or null | No state: the questions are answered from their own text |
A bare number or boolean, such as 42 or true | Nothing: the request is a 422. Wrap it, as "42" or { "count": 42 } |
Key order is kept, so the same object sent with its keys in another order is a different state to the model and can get a slightly different answer. Build states the same way every time.
Read once, shared by every question#
The model encodes the state and any images once, then answers every question in the request from that single read. Ten questions about one ticket cost the state's tokens once plus each question's own tokens, so ask everything you need in one request rather than one request per question.
Writing a good state#
- Label what matters.
{ "plan": "team" }says more than the word team floating in a paragraph. - Leave out what does not. Large blocks of unrelated text make every decision harder. Trim logs, signatures and quoted history that do not bear on the decision.
- Keep it stable. Send fields in the same order and format every time. The model is deterministic, so a stable state gives stable answers.
- Do not put instructions in the state. Put them in each question's
instructions, so the state stays pure data and one state serves many questions.
Size#
A request holds up to 64k tokens (65,536), counting the state, the images and every question. usage.input_tokens in the response tells you what a request used. See Limits.