Interleaved Thinking
Models like GLM 4.7 and MiniMax M2/M2.1 now recommend passing `reasoning_content` or `reasoning_details` back into the context for enhanced reasoning performance. See https://docs.z.ai/guides/capabilities/thinking-mode and https://platform.minimax.io/docs/guides/text-m2-function-call. Is this possible with NanoGPT's OpenAI-compatible API? Or does it perhaps need an Anthropic-compatible API? And if so, can I suggest some clarity in the docs on how this might work?
Hi! This is how we currently do this - any reasoning_content or reasoning_details are converted to the standard that the specific provider uses, nothing is dropped. - Milan
Wow, thank you for the reply! Glad to hear that you pass everything through. I'll give it a shot in my proxy layer.
It looks like passing reasoning_content works fine, however I'm getting 503 errors when I pass the reasoning in the reasoning_details field, specifically to MiniMax M2.1. It feels a bit like a timeout.
On GLM 4.7 Thinking using the same interleaved thinking technique with reasoning_content, everything works fine. No 503 errors.
Is it something wrong on my side, or is the reasoning_details field actually causing real issues?
Hi,
Really very late reply, but from everything we can see it should now work properly.