JsonCutDocs

Audio, voice and captions

Mix footage, music and narration, then add editable word-timed caption layers.

Video projects can mix multiple video and audio layers. Captions use ordinary text styling plus explicit word timing, so Studio can change the transcript, typography and active-word behavior after creation.

Rendered output

Street question reel

A vertical social format combining real footage, paced captions and independently timed graphic layers.

Video1080 × 192011 secondsEditable V2 source

Audio layer

<audio id="voice" data-jc-clip data-jc-kind="audio" data-jc-name="Voiceover"
  data-jc-start="0.4" data-jc-duration="8.2" data-jc-track="6"
  data-jc-media-start="0" data-jc-playback-rate="1" data-jc-volume="1"
  data-jc-fade-in="0.12" data-jc-fade-out="0.35"
  data-jc-normalize-audio="true"
  src="jsoncut-media://media_voice"></audio>

Audio supports source trim, speed, volume, mute, loop, fades, normalization, ducking and persistent volume keyframes. Multiple audible layers are mixed in the export.

Volume keyframes use the same named and cubic-Bézier curves as visual tracks. The final gain is limited to the supported 0–1 range, including overshooting curves. A range export retains the original clip-local envelope and fade positions rather than restarting them at the cut.

Normalize audio uses the layer's stored normalizationGain (up to 4×), matching the Studio control. Export does not run a separate loudness pass per layer; a loudness target applies to the whole mix only when the project master or the render request sets one (see Mixer below). Overlapping ducking windows reduce a track once, not repeatedly for each simultaneous speaker.

MCP video renders and playback-review drafts preserve project audio by default. Use muted: true only for an intentionally silent output. Motion review reports audio-stream presence and signal levels over a bounded sample, but these cannot establish audible quality or synchronization. Check the complete final video with sound when audio is part of the brief. Library spectrum effects are decorative unless explicitly driven by an audio analysis; they do not analyze a track automatically.

Drive motion from measured audio

Select a visual layer and open React to audio in Studio. Choose an uploaded audio/video layer and Analyze & bind. Then select overall energy or low/mid/high frequencies and drive scale, vertical position or opacity. The binding follows the source layer's timeline start, source trim, speed and loop. It measures the source before volume, fades and ducking; muting the mix does not remove the animation.

From MCP use jsoncut_bind_audio_motion with clip_id (the target layer), audio_clip_id (the source layer), channel (rms, low, mid or high), property (scale, y or opacity), amount (-1 to 1, or -1000 to 1000 pixels for y) and the current project version. It analyzes owned media on the server and applies the same editable binding; do not invent sample arrays. API clients can request POST /api/v2/media/{id}/audio-analysis. Sources are limited to 250 MB and 300 seconds; analyses share a bounded media queue on the server and decoding has a 60-second budget. Files are not sent through model text.

Analysis includes a content hash, 20 energy samples per second and transient candidates in beats. These are detected onsets, not a guaranteed musical beat or tempo grid. Replacing the source invalidates its binding: Studio shows a warning and you must reanalyze. Decorative spectrum Library effects remain independent.

Caption layer

<div id="captions" data-jc-clip data-jc-kind="caption" data-jc-name="Captions"
  data-jc-start="0" data-jc-duration="8" data-jc-track="8"
  data-jc-caption-display="active-word"
  data-jc-caption-words-per-page="4"
  data-jc-caption-active-color="#111111"
  data-jc-caption-active-background="#ff9418"
  data-jc-caption-inactive-color="#ffffff"
  data-jc-caption-word-animation="pop"
  data-jc-caption-words="[{&quot;id&quot;:&quot;w1&quot;,&quot;text&quot;:&quot;Build&quot;,&quot;start&quot;:0,&quot;end&quot;:0.42}]">
</div>

Display modes are static, typewriter, word-by-word and active-word. Typography, background, border, radius and shadow remain ordinary text styling and can be applied consistently to the whole caption track in Studio.

Audio property guide

AttributeDefaultPurpose
data-jc-media-start0Source in-point
data-jc-media-endSource durationSource out-point; loops repeat only the trimmed region
data-jc-playback-rate1Playback speed, 0.5 to 2
data-jc-volume1Base clip volume, 0 to 1
data-jc-fade-in, data-jc-fade-out0Edge fades in seconds
data-jc-normalize-audiofalseEnables stored normalization gain
data-jc-normalization-gain1Gain used when normalization is enabled, 0 to 4
data-jc-ducking1Volume factor (0 to 1) while another audible track is active
data-jc-rolenonevoice, music, sound-effect or original-sound; music and sound effects never make other sound dip
data-jc-muted, data-jc-loopfalsePlayback flags

A volume animation track adds time-varying automation. Keep values between zero and one.

Caption timing

Caption words contain stable IDs, text and start/end seconds. wordsPerPage is an integer from 1 through 12. Word animation is none, pop or rise.

Studio can turn a transcript into readable cues and can import timed JSON, WebVTT, SRT-like timing and ASS dialogue. After transcription, review wording and timing before treating the caption track as final.

Voiceover

Audio in the sidebar is the sound library, built like Media: Your audio, Open stock and Generate with Voice, Music and Sound. Mixing happens on the timeline (see below); captions have their own Captions panel.

Under Generate → Voice, Write a script, Write with AI and Translate speech open the voiceover dialog on its matching tab:

  • Write: type what should be said and pick a voice.
  • With AI: describe the video in a sentence; AI drafts a script in the chosen language that fits the video's length, using the text already on screen as context.
  • Translate: choose a clip with speech and a language. Its speech is transcribed and translated line by line; you can change every line before it is spoken. Each line is placed where the original was said, and the original clip can be muted, with its captions replaced.

The dialog shows the script with its settings (delivery, speed, captions and their style) and one list of 21 voices grouped by style, each with a sample. Captions are optional: the switch remembers your last choice for new voiceovers. A blank line in a script starts a new clip, so each paragraph can be spoken again on its own. Tags such as [excited] or [whispers] steer the delivery and never appear in captions. Voiceover clips on the timeline have their own colour and show what they say; the timeline + → Voiceover… and a right click on a voiceover clip lead to the same dialog.

With captions on, the caption track uses the script word for word, timed to the generated voice. Editing a voiceover replaces only the captions that belong to it.

Voices are spoken by Eleven v4 Turbo in every supported language; the accent is the voice's own. Voice cloning is not offered.

Select a voiceover clip on the timeline (voiceover lines have their own colour and show what they say) to edit it in the sidebar: change the words, voice, delivery or speed, listen to the change and choose Update voice. In the dialog, Listen plays the paragraph under the cursor; adding it afterwards reuses that recording.

Music added from the Audio panel ends with the video and fades out instead of making the video longer; drag its end on the timeline to hear more of the file.

Mix on the timeline

The Audio view button in the timeline tools gives audio and video tracks room for mixing, as in desktop editors:

  • Volume line: drag it up or down to change a clip's volume (shown in dB). Alt-click the line to add a point; drag a point to change time and level, double-click it or press Delete to remove it. Points are the clip's volume keyframes.
  • Fade handles at the top corners set the fade in and out; the curve shows on the clip.
  • Duck under voice lowers music and sound effects while someone speaks, with points you can adjust. Choose how far: Auto (what the sound check measured), Light (−6 dB), Medium (−12 dB), Strong (−18 dB) or any depth on the slider. Auto mix evens out the voices, puts music 12 dB and effects 8 dB under them and sets the master to −14 LUFS.
  • Mixer (in the audio view's bar): a volume for every track with sound, mute and solo, and the master for the whole video: output level, a loudness target (web & social −14 LUFS, podcast −16, broadcast −23) and a peak ceiling. The export and the sound check apply them; the preview in the editor plays at most full volume.
  • Track headers show each track's name and colour; their menu renames a track, picks a colour (its clips take it on), hides, mutes or solos it. Without a choice a track is named and coloured after what it holds.
  • Solo lets you hear a track alone while editing; the project and the export are not changed.
  • Waveforms are drawn at the same density on every clip, short or long; videos with sound show theirs in the audio view. Clips without sound (text, images, captions, silent footage) turn grey there, and small tags show what else shapes a clip's sound: normalised, ducked, speed, muted.
  • Snapping (the magnet in the timeline tools) lines up a clip edge you drag or trim with other clips, the playhead, the start and whole seconds; a dashed line shows where it lands.
  • While you move, trim or cut a clip, its time shows right under it. The ruler adapts to the zoom, from tenths of a second to hours, and the timeline glides along with the playhead while it plays.

Exports are mixed in stereo: a mono voice plays on both sides at the level you hear in the editor.

Check the sound

Every sound clip has a role (Audio section of the inspector): voice, music, sound effect or video sound. Music dips 12 dB under speech by default; sound effects never make other sound dip. Detach audio (right-click a video, or the inspector) moves a video's sound to its own clip; Fit to video length cuts longer music at the end with a fade or loops shorter music. Auto mix in the audio view evens voices, puts music and effects under them and sets the loudness for web and social. The mixer has a row per role (all voices or all music 3 dB quieter or louder), and the level meter (its button in the timeline toolbar) opens a card beside the timeline with the left and right level of the mix while it plays, including lanes and output above 0 dB; a red light marks peaks above -1 dB.

Check sound in the audio view listens to the saved video (the export dialog's Check the sound opens it): the loudness of every track and of the mix against the -14 LUFS web and social platforms play, peaks above -1 dBTP, music that covers speech, uneven voices and silent parts. The mix level sits next to the button; findings with a time appear as marks on the ruler. A mark opens the finding with its measured Fix (a volume or ducking value); after a fix the check runs again. Agents use the same check through the MCP tool jsoncut_inspect_audio (at most 180 seconds per call), which can also return a sound sheet image.

Captions from existing speech

In the Captions panel, Transcribe & review turns speech into an editable caption track. The source can be Speech (audible audio and video except music and sound effects), All audio, or one timeline clip; right-click an audio or video clip and choose Create captions from this clip for the last one. A chosen clip is heard as recorded, also when it is muted in the edit. Times always follow the timeline, including clips that start later, are trimmed or play faster. Import a subtitle file reads SRT, VTT, ASS and Whisper JSON. Once there is a caption track, the panel shows its position (top, middle, bottom), the styles and the detailed caption settings; Edit text opens the transcript, where a click on a word jumps to it. If speech recognition misheard a name that is written on screen, the review offers the correction (for example "Brüli → Brewly"), and Replace a word changes a word everywhere.

From MCP use jsoncut_list_voices, jsoncut_add_voiceover, jsoncut_translate_speech and jsoncut_add_captions. These actions belong to the MCP connection and are not part of the public V2 API reference. The write actions use AI credits and return the new project version, the clips and the spoken ranges, so other layers can be timed to the words.

  • jsoncut_add_voiceover: text, voice (default sarah), tone (natural, warm, excited, calm, serious or whisper), speed (0.7 to 1.4), at (start time in seconds), captions and style.
  • jsoncut_translate_speech: source (the clip with speech), language, voice, captions, mute_original (default true) and style.
  • jsoncut_add_captions: source (auto for the project's speech, or a clip id), optional language, start and end, and style.

Voiceover scripts and speech translation support English, German, French, Spanish, Italian, Portuguese, Dutch, Polish, Turkish, Swedish, Japanese, Korean, Chinese, Hindi and Arabic. Every voice speaks every one of these languages; write the text in the target language.

Generated music and sound

Voiceover, music and sound effects generated in Studio are saved as ordinary project audio media. Their generation settings are not embedded as hidden render behavior; the accepted audio file and editable layer are what the final project uses.