Deepgram Audio Intelligence

Audio intelligence is not a separate endpoint. It is five query parameters on /v1/listen, the same endpoint that returns the transcript. You get the transcript and the analysis in one response, from one API call. This skill gets a verified request working and states the limits precisely.

Decide first

• Your input is audio (a file, a URL, a call recording) and you want analysis: stay here, use

POST https://api.deepgram.com/v1/listen.

• Your input is already text (a transcript, an email, a chat log, a support ticket): the

parameters below do not apply. Use the Read API, POST /v1/read. Open the text-intelligence skill.

• You only want the transcript: drop these parameters and open the speech-to-text skill.
• You are streaming live audio: see the matrix (detect_entities only).

Feature matrix

detect_entities is the odd one out twice over: it is the only feature that works on the live socket, and it is the only one that does not exist on the Read API. Streaming entity detection runs on Nova, Nova-2, Nova-3, and Enhanced; it is not available on Base models or on Flux STT. [2]

Verified request

curl -s -X POST 'https://api.deepgram.com/v1/listen?model=nova-3&smart_format=true&summarize=v2&sentiment=true&topics=true&intents=true&detect_entities=true' \
  -H "Authorization: Token $DEEPGRAM_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"url":"https://dpgr.am/spacewalk.wav"}'

Returns 200. The analysis is scattered across the response, not collected in one place:

Note the plural: the parameter is sentiment, the result key is sentiments. metadata gains a summary_info, sentiment_info, topics_info, and intents_info block per enabled feature, each with model_uuid, input_tokens, and output_tokens. sentiment_score runs from -1 to 1, and the break point between neutral and positive or negative is 0.333... either side of zero; confidence_score on topics and intents runs from 0 to 1. [3] Entity labels come back uppercased (NAME, ORGANIZATION, LOCATION_CITY, MONEY, DATE_INTERVAL, CARDINAL). value is the formatted text and raw_value is the text as spoken; raw_value is present when a formatting feature such as smart_format is on. [4] Deepgram documents 59 entity types, and redact accepts 56 of them: cardinal, ordinal, and percent are detected but are not valid redact values, and passing one to redact returns a 400. [6]

summarize needs more than 50 words of speech. For a shorter input, summary.short is the original input returned as-is, and no tokens in or out are billed as summarization usage, so a summary_info with zero tokens on a short clip is not a failure. [3]

On the live socket, detect_entities=true adds a top-level entities array to Results messages, next to channel and not inside channel.alternatives[0]. Read entities only from messages whose is_final is true. Interim results can carry the key too, usually [] and sometimes populated, and those values are not final. A final result with nothing detected carries "entities": []. Same field shape as above, raw_value included when formatting is on. [4]

To return complete entities, the server holds a final result until the speaker moves on to non-entity speech, 3 seconds of silence pass, or a Finalize message arrives. no_delay=true forces immediate finalization without that wait, and leaves entities missed or incomplete in many cases. Send no_delay=true only when latency matters more than entity accuracy. [4]

Narrowing topics and intents

custom_topic and custom_intent (repeatable, up to 100 of each) add your own labels; custom_topic_mode and custom_intent_mode take extended (default, your labels plus the model's) or strict (your labels only). strict returns "segments": [] whenever nothing matches your list, which looks like a broken request but is not. Start with extended. [3]

Common mistakes

1. 1Expecting these to work on streaming. summarize fails the WebSocket handshake with 400 "Summarization is not available for streaming.". topics and intents fail it with 403 {"err_code":"UNAUTHORIZED_FEATURES_REQUESTED","err_msg":"Project does not have access to the requested feature/s [\"topics\"]."}, which reads as a permissions problem but is not one: the same key's prerecorded topics requests succeed, and the docs matrix lists streaming as unsupported for both. [2] sentiment is worse still: the handshake succeeds and no sentiment is ever returned. Only detect_entities works.
2. 2Expecting a non-English request to fail loudly. It does not. With language=es on real Spanish audio, sentiment, topics, intents, and detect_entities return HTTP 200 with the transcript, the analysis keys silently absent, and the reason only in metadata.warnings: [{"parameter":"sentiment","type":"unsupported_language","message":"Sentiment is only supported for English."}], plus "Topics are only supported for English.", "Intents are only supported for English.", and "Entity detection is only supported for English.". Read metadata.warnings before you conclude the model found nothing.
3. 3Assuming summarize behaves the same way. It is the exception when language is explicit: language=es is a hard 400, {"err_code":"Bad Request","err_msg":"Summarization v2 not supported for non-English languages"}, and language=multi gets the same 400. With detect_language=true and non-English speech it behaves like the others: HTTP 200, a metadata.warnings entry with "parameter":"summarize" and "type":"unsupported_language", and results.summary present with "result":"failure" and a short string that says the feature is English only. Check summary.result before you use summary.short. [3]
4. 4language=multi as a workaround. It is not one. For sentiment, intents, and topics, multi returns the analysis when the detected speech is English and drops it with the same metadata.warnings when it is not, so the same request succeeds or silently degrades depending on what the caller said. summarize with language=multi is a 400.
5. 5summarize=v1. Returns 400 "To use the summarize feature, please use 'summarize=true' or 'summarize=v2'. The 'summarize=v1' parameter is deprecated." Use v2; true is accepted and returns the same summary.short shape.
6. 6Looking for results.summary.text. That is the Read API's shape. On /v1/listen the summary is at results.summary.short.
7. 7Putting any of these on Flux STT. /v2/listen rejects all five at the handshake with 400 {"err_code":"INVALID_QUERY_PARAMETER","err_msg":"Unknown query parameters: detect_entities"}, and the same message naming summarize, sentiment, topics, or intents. Transcribe with Flux STT, then send the transcript to /v1/read.
8. 8Reaching for these to mask PII. Detection returns entities, it does not remove them. Use redact for that, which is a speech-to-text parameter and rejects cardinal, ordinal, and percent (see the entity-type counts above). [6]

Pricing

Enabling these features changes what a request costs. Rates and the billing model change, so read <https://deepgram.com/pricing> rather than any figure quoted in a skill.

Use a different skill when

• Your input is text rather than audio, or you need the Read API's input limits: text-intelligence

skill (/v1/read).

• You want every parameter and the full response schema: api skill, references/listen.md.
• You only need transcription, diarization, redaction, or captions: speech-to-text skill.
• You want a runnable demo app: starters skill. Note there is no audio-intelligence starter; the

text-intelligence feature (13 frameworks) is the Read API app.

• You want a snippet under 50 lines: recipes skill, "Audio Intelligence v1" — summarize,

sentiment, topics, intents, entities, in Python, JavaScript, Go, .NET, Java, Rust, and the CLI. [7]

• You want language-idiomatic SDK code: install deepgram-{js,python,java,go,rust,dotnet}-audio-intelligence

from the matching SDK repository (npx skills add deepgram/deepgram-python-sdk, and so on).

• You want to find a docs page: docs skill. You want the docs in your editor: setup-mcp skill.

Sources

1. 1https://developers.deepgram.com/docs/audio-intelligence (getting started)
2. 2https://developers.deepgram.com/docs/stt-intelligence-feature-overview (the prerecorded/streaming/language matrix, and the streaming entity-detection model footnote)
3. 3https://developers.deepgram.com/docs/summarization, https://developers.deepgram.com/docs/sentiment-analysis, https://developers.deepgram.com/docs/topic-detection, https://developers.deepgram.com/docs/intent-recognition
4. 4https://developers.deepgram.com/docs/detect-entities
5. 5https://developers.deepgram.com/docs/language and https://developers.deepgram.com/docs/models-languages-overview
6. 6https://developers.deepgram.com/docs/supported-entity-types (59 entity types, 56 of them valid redact values) and https://developers.deepgram.com/docs/redaction
7. 7https://github.com/deepgram/recipes/blob/main/COVERAGE.md ("Audio Intelligence v1" section)
8. 8https://developers.deepgram.com/reference/speech-to-text/listen-pre-recorded and https://developers.deepgram.com/reference/speech-to-text/listen-streaming
9. 9https://developers.deepgram.com/docs/errors and https://deepgram.com/pricing