Topic hub pagination
Continue browsing this topic cluster with SEO-safe static pagination.
Fix LiteLLM rate-limit error categorization where router-side RPM/TPM throttling triggers (deployment-level and model-level) are falsely reported as vendor/third-party rate limits Includes evidence for LiteLLM troubleshooting demand.
litellm_rate_limit errors emitted as vendor_rate_limit — dashboards/callbacks attribute router-side throttles to upstream vendor instead of LiteLLM proxy Fix unknown provider error when routing through OpenRouter/LiteLLM proxy — provider auth chain misconfiguration blocks API calls Includes evidence for LiteLLM troubleshooting demand.
Unknown provider 'openai' Fix LiteLLM router strategy incorrectly attributing self-imposed rate limits (deployment over defined rpm/tpm limit) to upstream vendors, preventing correct dashboards/alerts for proxy-side throttling Includes evidence for LiteLLM troubleshooting demand.
RateLimitError silently emitted category='vendor_rate_limit' for router-side rpm/tpm enforcement instead of category='litellm_rate_limit' Fix LiteLLM proxy fallback not triggering for team-scoped BYOK models due to public vs internal model name mismatch in Router.async_function_with_fallbacks Includes evidence for LiteLLM troubleshooting demand.
No fallback model group found for original model_group=primary. Available Model Group Fallbacks=None Fix LiteLLM rate-limit categorization so router-side deployment throttling is properly labelled for correct dashboard attribution and alerting routing Includes evidence for LiteLLM troubleshooting demand.
Deployment over defined rpm limit / Deployment over defined tpm limit — dashboard/callbacks attribute these to upstream vendor (VENDOR_RATE_LIMIT) instead of LiteLLM router (LITELLM_RATE_LIMIT) Fix LiteLLM proxy incorrectly reporting rate limit errors as deployment unavailability, confusing debugging for teams using multi-provider routing Includes evidence for LiteLLM troubleshooting demand.
Rate limit error reported as "No deployments available for litellm proxy" Fix LiteLLM silently ignoring configured timeout values when proxying requests to AWS Bedrock and Google Vertex AI, causing unpredictable request latency Includes evidence for LiteLLM troubleshooting demand.
litellm.Timeout: Connection timed out. Timeout passed=Timeout(timeout=0.1), time taken=0.101 seconds Fix LiteLLM proxy to mask sensitive data (SHA-256 hashes, API keys) in rate limit error responses exposed to end-users Includes evidence for LiteLLM troubleshooting demand.
Rate limit error message body leaks full SHA-256 hash of API key via litellm/proxy/hooks/parallel_request_limiter_v3.py Fix LiteLLM retry mechanism to properly pause/respect configured delay after hitting rate limits, avoiding retry storms that waste API quota Includes evidence for LiteLLM troubleshooting demand.
Retry mechanism not pausing for set delay after rate limit is hit — immediate retries instead of respecting configured backoff window Fix LiteLLM stream_timeout enforcement so timeout errors trigger at correct time rather than being ignored on first chunk, preventing silent hangs in production API calls Includes evidence for LiteLLM troubleshooting demand.
stream_timeout not enforced on first chunk; timeout should raise error at N seconds but actual behavior delays error raising significantly beyond configured timeout Fix LiteLLM proxy timing out when proxying requests to local Ollama models, causing task execution failures in AI agent frameworks (CrewAI, AutoGen, LangGraph). Includes evidence for LiteLLM troubleshooting demand.
LiteLLM timeouts with Ollama models — request taking too long to process Fix LiteLLM proxy returning 504 Gateway Timeout errors when LLM model calls exceed default timeout threshold, causing users to see browser console errors Includes evidence for LiteLLM troubleshooting demand.
504 Gateway Timeout — users receive error in browser console after approximately 60 seconds when using certain LLM models Developers integrating third-party APIs via LiteLLM proxy encounter BadRequestError when Azure backend rejects stream_options.include_usage parameter. Need compatible configuration or workaround Includes evidence for LiteLLM troubleshooting demand.
litellm.BadRequestError: AzureException BadRequestError: Unknown parameter: stream_options.include_usage Configure automatic model group fallback in LiteLLM when primary provider hits rate limits; prevent complete service outage. Includes evidence for LiteLLM troubleshooting demand.
LiteLLM rate limit errors cause proxy to crash/fail instead of falling back to secondary model group Fix misclassified 500 error caused by passing string timeout instead of numeric value — confusing users about root cause Includes evidence for LiteLLM troubleshooting demand.
string-valued timeout produces misleading 'Security scan failed' 500 Fix crash in LiteLLM OpenTelemetry guardrail integration when guardrail mode is configured as a Python list instead of single value Includes evidence for LiteLLM troubleshooting demand.
OpenTelemetry integration crashes with 'unhashable type: ''list'' when guardrail mode is a list' Fix LiteLLM rate-limiting where TPM limits apply independently per pod replica, effectively multiplying the intended rate limit by number of replicas and allowing far more requests than configured Includes evidence for LiteLLM troubleshooting demand.
Deployment-level TPM enforcement is per-pod, not cross-pod: effective limit becomes tpm_limit × N_replica Fix LiteLLM proxy buffering Anthropic SSE streams instead of sending real-time deltas, when both OpenAI and Anthropic share same proxy config Includes evidence for LiteLLM troubleshooting demand.
Anthropic proxied response is buffered: delta JSON lines sent in one large batch despite stream=true configuration Fix LiteLLM streaming fallback mechanism failing to route to backup models when primary Gemini 3 Preview returns HTTP 429, leaving clients without any response during peak usage Includes evidence for LiteLLM troubleshooting demand.
Fallbacks on STREAMING do not work with Pydantic-AI when Gemini 3 Preview throws 429 rate limit error Fix LiteLLM proxy TPM rate limiting in distributed deployments where each replica independently enforces the limit, effectively multiplying the allowed throughput beyond intended cap Includes evidence for LiteLLM troubleshooting demand.
TPM enforcement allows tpm_limit × N_replica concurrent requests across replicas — single pod limit fails to apply correctly in multi-pod K8s deployments