Prompt Caching for more models
Many providers do offer prompt caching nowadays and utilizing that yields great discounts. Unfortunately, through NanoGPT, we can't benefit from those (except with Claude, which is the most inconvenient to implement). That remains a big advantage of OpenRouter. For me personally, the most interesting providers are Moonshot AI, Google (Gemini) and DeepSeek. I am aware there are some suggestions like this already, but they can't be voted on, so this is my way of expressing demand. I'm also aware this might hurt your profits, if you only pay discounted rates while you charge base costs, but that would also mean the pricing isn't fair.
Thanks! So a few aspects to this. We actually implemented it for OpenAI yesterday, and want to add it for more models (likely Gemini next).
For Deepseek and co this is a bit more complicated. We generally do not run these models through the direct provider (Deepseek/Moonshot), mostly for privacy reasons. Some open source providers do offer prompt caching, so when those are used then we can pass it on, but this is a bit more complicated because they are not always the cheapest providers generally.
So have to figure out what we do there. Maybe people pass a "caching" header for those models/providers, or something.
We now haev this for Gemini as well - Milan