NVIDIA NIM
NVIDIA NIM provides optimized inference endpoints for open-source models running on NVIDIA GPUs. Get production-quality performance from models like Llama, Mixtral, and Gemma without managing your own GPU infrastructure.
Prerequisites
- An NVIDIA account
- An API key from the NVIDIA build platform
Setup
- Go to build.nvidia.com
- Sign in with your NVIDIA account
- Click Get API Key and copy it
- In Sythoria, open Settings from the sidebar
- Select NVIDIA NIM from the Provider Preset dropdown — the API base URL and default model fill in automatically
- Paste your API key into the API Key field
- The connection indicator should turn green within a few seconds
Your API key is stored securely in your OS keychain. It is never sent to Sythoria's servers. See Privacy & Security.
Default configuration
| Setting | Value |
|---|---|
| API Base | https://integrate.api.nvidia.com/v1/chat/completions |
| Default Model | meta/llama-3.3-70b-instruct |
| API Format | OpenAI Chat Completions |
| Streaming | Supported |
Available models
NVIDIA NIM hosts optimized versions of popular open-source models. Model IDs use the provider/model-name format:
| Model ID | Base model | Best for |
|---|---|---|
meta/llama-3.3-70b-instruct | Llama 3.3 70B | General-purpose, strong reasoning |
mistralai/mixtral-8x22b-instruct-v0.1 | Mixtral 8x22B | Efficient mixture-of-experts, good throughput |
google/gemma-2-27b-it | Gemma 2 27B | Conversation and instruction following |
nvidia/llama-3.1-nemotron-70b-instruct | Nemotron 70B | NVIDIA-tuned Llama, high accuracy |
Browse all available models at build.nvidia.com/explore/discover.
Model ID format
NVIDIA NIM uses the provider/model-name naming scheme. When entering a model in Sythoria's Model ID field, use the exact ID from build.nvidia.com/explore/discover. Examples:
meta/llama-3.3-70b-instructmistralai/mixtral-8x22b-instruct-v0.1google/gemma-2-27b-it
Pricing
NVIDIA offers free credits for new accounts, with pay-per-token pricing afterward. Check build.nvidia.com for current rates and credit availability.
Troubleshooting
| Error | Cause | Fix |
|---|---|---|
| 401 Unauthorized | API key is invalid or expired | Generate a new key at build.nvidia.com |
| 404 Model not found | Incorrect model ID | Use the exact model ID from build.nvidia.com/explore/discover |
| 429 Rate Limit | Exceeded free tier or usage limits | Wait and retry, or add credits to your account |
| 503 Service unavailable | Model endpoint temporarily down | Wait a moment and retry — NVIDIA scales endpoints dynamically |
| Connection indicator red | Network issue or invalid configuration | Verify the API base URL is https://integrate.api.nvidia.com/v1/chat/completions and click Refresh |
