Tool Calling in Chat Template
Hi, I have a question regarding how a tool call should be formatted by the chat template.
According to the OpenAI cookbook (https://cookbook.openai.com/articles/openai-harmony#prompt-format), I see that model should call tools in the following way:<|channel|>analysis<|message|>Need to use function get_current_weather.<|end|><|start|>assistant<|channel|>commentary to=functions.get_current_weather <|constrain|>json<|message|>{"location":"San Francisco"}<|call|>
i.e., <|channel|>commentary precedes to=functions.FUNCTION_NAME
However, looking at the chat template, it seems like the <|channel|> commentary is being added after the to=functions.FUNCTION_NAME call:
{{- "functions." + tool_call.name + "<|channel|>commentary " }}```
I see this difference when using the model-- the model produces the tool call in the format given in the cookbook (channel before to=functions.FUNCTION_NAME), but calling `apply_chat_template` on messages with a tool call will place the channel after the to=functions.FUNCTION_NAME
Is this a bug in the chat template, or am I misunderstanding something?
I agree I believe this might be an typo in the cookbook.
In the cookbook if you look at the very next example:
"<|start|>assistant<|channel|>commentary to=functions.get_current_weather <|constrain|>json<|message|>{"location"...."
You can see it has the Channel special token before the function call.
I also ran into a related issue while fine-tuning gpt-oss-20b for function calling.
In my case, the problem turned out to be the chat template encoding for tool calls.
I was able to resolve it by slightly adjusting the template so that the generated tokens align with the format shown in the cookbook.
Specifically, I changed the template from:
'''
{{- "<|start|>assistant to=" }}
{{- "functions." + tool_call.name + "<|channel|>commentary " }}
'''
to:
'''
{{- "<|start|>assistant<|channel|>commentary to=" }}
{{- "functions." + tool_call.name + " <|constrain|>" }}
'''
This modification ensures that:
-->the <|channel|> token is emitted immediately after <|start|>assistant, and
-->the function target (functions.) is placed in the expected constrained position.
After this change, the encoded output matches the cookbook examples.
I also ran into a related issue while fine-tuning gpt-oss-20b for function calling.
In my case, the problem turned out to be the chat template encoding for tool calls.
I was able to resolve it by slightly adjusting the template so that the generated tokens align with the format shown in the cookbook.Specifically, I changed the template from:
'''
{{- "<|start|>assistant to=" }}
{{- "functions." + tool_call.name + "<|channel|>commentary " }}
'''
to:
'''
{{- "<|start|>assistant<|channel|>commentary to=" }}
{{- "functions." + tool_call.name + " <|constrain|>" }}
'''This modification ensures that:
-->the <|channel|> token is emitted immediately after <|start|>assistant, and
-->the function target (functions.) is placed in the expected constrained position.After this change, the encoded output matches the cookbook examples.
Could you provide a script or notebook for fine-tuning a model with function-calling support?