OctaFuse Gateway 2.3.0 has been released with a notable new feature: provider stickiness for route pools. In multi-provider setups, when a request fails over to an available upstream, subsequent requests may not consistently hit the same provider, leading to poor cache hit rates, broken sessions, and unstable provider-side state. The new stickiness feature addresses this by binding requests to a specific provider after the initial failover, ensuring continuity. This is particularly valuable for teams using multiple AI model providers or API vendors where session state and caching are critical for performance and cost. The update is a small but meaningful improvement for gateway operators looking to optimize multi-vendor traffic management.
New provider stickiness feature in OctaFuse Gateway improves cache hit rates and session continuity in multi-provider failover scenarios.