Groq / Llama 3 settings guide

Groq serves open models very fast on its own LPU hardware; use it when response speed matters (real-time apps, prototyping, streaming).

Use a large open model for harder tasks and a small one for simple, fast tasks; check the current model list and free-tier limits in the Groq console.

Set system_prompt in the API to match your use case — Groq doesn't have a GUI memory system, so always include context in system prompt.

Use Groq as a ChatGPT API replacement for many use cases — the API is compatible with OpenAI's SDK (just change base_url).

Handle 429 rate-limit errors and fall back to another provider; free-tier limits are real.

Use streaming (stream=true) for better UX — Groq's speed makes streaming text feel instant even on slow connections.

Combine with Perplexity's API for search — Groq handles reasoning, Perplexity handles web search.