Multitrack Productions and Presets

An Auphonic Multitrack post production takes multiple parallel input audio tracks/files, analyzes and processes them individually as well as combined and creates the final mixdown automatically. Please see Multitrack Productions and Multitrack Post Production Algorithms for more details.
This page describes how to use the multitrack algorithms with our API.

Multitrack Productions

Multitrack Productions can be created in the same way as any other productions. The only difference is, that you have to specify multiple audio input files (multiple tracks) by using the parameter multi_input_files (or the corresponding separate request) as introduced in Adding Intro and Outro.

The following command creates a multitrack production with 2 tracks:

curl -X POST -H "Content-Type: application/json" \
    https://auphonic.com/api/productions.json \
    -H "Authorization: bearer {api_key}" \
    -d '{
            "multi_input_files": [
                {"type": "multitrack", "id": "speech track"},
                {"type": "multitrack", "id": "music track"}
            ],
            "is_multitrack": true
        }'

Parameters:

• is_multitrack: must be set to true for multitrack productions

• multi_input_files:

• type: every track of a multitrack production must set the type to multitrack (other possible types are intro or outro, see Adding Intro and Outro)

• id: a readable identifier for the current track

Uploading Audio Files

After the production is created, you can upload local audio files for each track. This uses the same request as in Start a Production and Upload a File, but the the track id is used as the key for each file:

curl -X POST https://auphonic.com/api/production/{uuid}/upload.json \
    -H "Authorization: bearer {api_key}" \
    -F "speech track=@/home/user/Desktop/file-for-track1.wav" \
    -F "music track=@/home/user/Desktop/file-for-track2.wav"

This uploads the local file file-for-track1.wav to "speech track" and the file file-for-track2.wav to "music track" of our previous request.

Note

For uploading files, it’s necessary to encode your POST request in multipart/form-data, not JSON!

You can now start your production:

curl -X POST https://auphonic.com/api/production/{uuid}/start.json -H "Authorization: bearer {api_key}"

Uploading Files from External Services

Now we add 3 more tracks to our production: one from an External Service, one from an HTTP source and also an intro (type is set to intro):

curl -X POST -H "Content-Type: application/json" \
    https://auphonic.com/api/production/{uuid}.json \
    -H "Authorization: bearer {api_key}" \
    -d '{
            "multi_input_files": [
                {
                    "service": "pmefeNCzkyT4TbRbDmoCDf",
                    "input_file": "my_dropbox_file.mp3",
                    "type": "multitrack",
                    "id": "speech track 2"
                },
                {
                    "input_file": "http://your_server.com/speech_fritz.mp3",
                    "type": "multitrack",
                    "id": "speech track 3"
                },
                {
                    "service": "pmefeNCzkyT4TbRbDmoCDf",
                    "input_file": "intro_from_soundcloud.m4a",
                    "type": "intro"
                }
            ],
            ...
            "action": "start"
        }'

The syntax is the same as in Adding Intro and Outro.

When the multitrack production does not involve a local file upload, it can be created and started in one go by adding "action": "start" as a parameter to the request. Omit the action parameter to just save the production without starting it.

Multitrack Audio Algorithm Settings

Audio algorithm settings can be specified for each track and for the master output file in the same way as it is done in the web interface - see Audio Algorithms for Individual Tracks and Audio Algorithms for Master Track.
If some parameters are not set, the default values are used. Please also find the list of all parameters below.

Here is a basic example with audio algorithm settings for individual tracks and for the master track:

curl -X POST -H "Content-Type: application/json" \
    https://auphonic.com/api/productions.json \
    -H "Authorization: bearer {api_key}" \
    -d '{
            "metadata": { "title": "Multitrack Production with Algorithms" },
            "output_files": [{"format": "mp3"}],
            "multi_input_files": [
                {
                    "type": "multitrack", "id": "speech track",
                    "algorithms": {"denoise": true}
                },
                {
                    "type": "multitrack", "id": "music track",
                    "algorithms": {"filtering": false, "backforeground": "background"}
                }
            ],
            "algorithms":
                {
                    "loudnesstarget": -23,
                    "leveler": true,
                    "gate": true,
                    "crossgate": true
                }
        }'

For details about our multitrack algorithms see Multitrack Post Production Algorithms.
See also (Single Track) Audio Algorithms API.

More Settings for Multitrack Audio Algorithms

You can set all Individual and Master Track Audio Algorithm Parameters using the API (see list of all options below), as you can see in the following advanced example:

curl -X POST -H "Content-Type: application/json" \
    https://auphonic.com/api/productions.json \
    -H "Authorization: bearer {api_key}" \
    -d '{
            "metadata": { "title": "Multitrack More Algorithm Parameters" },
            "output_files": [{"format": "mp3"}],
            "multi_input_files": [
                {
                    "type": "multitrack", "id": "speech track",
                    "algorithms": {
                        "denoise": true, "denoiseamount": 12, "dehum": 50, "dehumamount": 6,
                        "levelerstrength": 50, "compressor": hard, "pan": "L25"
                    }
                },
                {
                    "type": "multitrack", "id": "music track",
                    "algorithms": {
                        "backforeground": "ducking", "backgroundgain": -12, "ducking_fadetime": 1000,
                        "msclassifier": "music", "compressor": "off", "pan": "R25"
                    }
                }
            ],
            "algorithms":
                {
                    "leveler": true, "maxlra": 10, "maxs": 5,
                    "loudnesstarget": -24, "maxpeak": -2,
                    "dualmono": true, "loudnessmethod": "program",
                    "silence_cutter": true, "filler_cutter": true, "cough_cutter": true,
                    "music_cutter": true, "cut_mode": "apply_cuts",
                }
        }'

Audio Algorithms for Individual Tracks:

• Options for Filtering:

  • filtering: true (default), false

  • filtermethod: hipfilter (default), autoeq

• Options for Noise Reduction:

  • denoise: false (default), true

  • denoisemethod: static (default), dynamic, speech_isolation

  • denoiseamount (noise reduction amount): 0 (Auto, default), 3, 6, … (values in dB)

  • Static Denoising:

    • denoiseamount: 0 (Auto, default), -1 (disable denoise, dehum only), 3, 6, … (values in dB)

    • dehum (hum reduction base frequency): 0 (Auto, default), 50 or 60

    • dehumamount (hum reduction amount): 0 (Auto, default), -1 (disable dehum), 3, 6, … (values in dB)

• Options for Advanced Leveler:

  • levelerstrength: 100 (default), 90, 80, 70, … 0 (values in %)

  • compressor: auto (default), soft, medium, hard or off

  • msclassifier (use classifier or set whole track to speech/music): on (default), speech, music

  • pan: stereo panning, L100, L75, …, C, R25, …, R100

• Options for Fore/Background:

  • backforeground: Auto (default), foreground, background , unchanged, ducking (track will be reduced in volume if other speakers are active).

  • gain: 0 (default), -30, …, +6 (values in dB)

  • backgroundgain: -18 (default), -26, …, -3 (values in dB)

  • ducking_fadetime: 500 (default), 125, …, 2000 (values in ms)

Audio Algorithms for Master Track :

• Options for Adaptive Leveler:

  • leveler: true (default), false

  • Leveler Broadcast Mode:

    • maxlra: 0 (Auto, default), 5, 6, 7, 8, …, 20 (values in LU)

    • maxs: 0 (Auto, default), 3, 4, 5, 6, …, 12 (values in LU, relative to the target loudness)

    • maxm: 0 (Auto, default), 8, 9, 10, 11, …, 20 (values in LU, relative to the target loudness)

  • for details please see Adaptive Leveler Settings

• Options for Adaptive Noise Gate and Crossgate (Mic Bleed Removal):

  • gate: true (default), false

  • crossgate (removes mic bleed): true (default), false

• Options for Loudness Normalization:

  • loudnesstarget: -16 (default), -31, …, -13 (values in LUFS)

  • maxpeak (maximum true peak level): 0 (Auto, default), -0.5, -1, -1.5, -2, -3, -4, -5, -6 (values in dBTP)

  • dualmono (treat mono production as dual-mono / -3LU offset): false (default), true

  • loudnessmethod (anchor-based loudness normalization): program (default), dialog (measure dialog/voice loudness)

• Options for Automatic cutting:

  • silence_cutter: false (default), true

  • filler_cutter: false (default), true

  • cough_cutter: false (default), true

  • music_cutter: false (default), true

  • cut_mode (what happens to cut segments): apply_cuts (default), export_uncut_audio, set_cuts_to_silence

For details please see Audio Algorithms for Individual Tracks and Audio Algorithms for Master Track.

Getting back the Processed Tracks

By default, a multitrack production will create a final mixdown of all your input files. If you want to get back the individual, processed tracks, you have to add an additional tracks output format:

curl -X POST -H "Content-Type: application/json" \
    https://auphonic.com/api/productions.json \
    -H "Authorization: bearer {api_key}" \
    -d '{
            "metadata": { "title": "Multitrack Tracks Export" },
            "multi_input_files": [
                {
                    "type": "multitrack", "id": "speech track",
                    "algorithms": {"denoise": true, "denoiseamount": 12}
                },
                {
                    "type": "multitrack", "id": "music track",
                    "algorithms": {"filtering": false, "backforeground": "background"}
                }
            ],
            ...
            "output_files": [
                {"format": "tracks", "ending": "wav.zip"},
                {"format": "tracks", "ending": "flac.zip"}
            ]
        }'

Individual tracks can be exported as FLAC (default) or as WAV files. All tracks are zipped into one single file to download.

Adding Segments for Individual Tracks

Segments let you apply unique noise reduction settings to specific parts of your audio. This is ideal for files with mixed content, such as speech, music, or multiple speakers.

In multitrack productions, segments must be specified in the algorithms section of each individual multi_input_files entry, not in the global algorithms section.

Create a multitrack production with segments for individual tracks:

curl -X POST -H "Content-Type: application/json" \
    https://auphonic.com/api/productions.json \
    -H "Authorization: bearer {api_key}" \
    -d '{
            "metadata": { "title": "Multitrack Production with Segments" },
            "multi_input_files": [
                {
                    "type": "multitrack",
                    "id": "speech track",
                    "algorithms": {
                        "denoise": true,
                        "segments": [
                            {"start": 0, "stop": 20, "denoisemethod": "speech", "denoiseamount": 6}
                        ]
                    }
                },
                {
                    "type": "multitrack",
                    "id": "music track",
                    "algorithms": {
                        "denoise": true,
                        "segments": [
                            {"start": 0, "stop": 15, "denoisemethod": "music"},
                            {"start": 15, "stop": 20, "denoisemethod": "classic"}
                        ]
                    }
                }
            ],
            "is_multitrack": true
        }'

Segment Fields:

Each segment must contain at least the following required fields:

• start: (float) Start time in seconds (must be >= 0)

• stop: (float) End time in seconds (must be > start)

Optional fields:

• denoisemethod: (string) The noise reduction method to use for this segment. Valid values:

  • "speech" - Removes everything except speech

  • "music" - Preserves speech, music and sound effects but removes background noise

  • "classic" - Classic noise reduction

  • "off" - Disable noise reduction for this segment

  If not specified, the default noise reduction method of the API will be used ("classic").

• denoiseamount: (float) The amount of noise reduction to apply in dB. Valid values: 0 (full/auto), -1 (off), 3, 6, 9, 12, 15, 18, 24, 30, 36, 100. If not specified, the default amount will be used. Set to -1 to disable noise reduction for this segment.

  Available for: speech, music, classic

• deverbamount: (float) The amount of reverb reduction to apply in dB. Valid values: 0 (full/auto), -1 (off), 3, 6, 9, 12, 15, 18, 24, 30, 36, 100. If not specified, the default amount will be used. Set to -1 to disable reverb reduction for this segment.

  Available for: speech, music

• debreathamount: (float) The amount of breath sound reduction to apply in dB. Valid values: -1 (off), 3, 6, 9, 12, 15, 18, 24, 30, 36, 100. If not specified, the default amount (-1, off) will be used. Set to -1 to disable breath sound reduction for this segment.

  Available for: speech only

Note

To apply the denoising settings defined in segments for a track, you must set "denoise": true in that track’s algorithms section. If "denoise": false is set, no denoising will be performed, even if noise reduction settings are specified in the segments.

Important

Segments must be contiguous! This means:

• Segments cannot overlap with each other

• Segments cannot have gaps between them

• The start time of each segment must equal the stop time of the previous segment

For example, if one segment ends at 20 seconds, the next segment must start at exactly 20 seconds.

Warning

Segment times are relative to each individual track file, not to the final mixdown!

For more information about segments in singletrack productions, see Adding Segments.

Adding offsets to Tracks

Each track in a multitrack production can have an offset, which determines when the track starts: an offset of 3.5 would start the track 3.5 seconds after the first track started.

Add the parameter offset, which must be a positive number and in seconds, to your tracks in the following way:

curl -X POST -H "Content-Type: application/json" \
    https://auphonic.com/api/production/{uuid}.json \
    -H "Authorization: bearer {api_key}" \
    -d '{
            "multi_input_files": [
                {
                    "service": "pmefeNCzkyT4TbRbDmoCDf",
                    "input_file": "my_dropbox_file.mp3",
                    "type": "multitrack",
                    "id": "speech track 2"
                    "offset": 1.25
                },
                {
                    "input_file": "http://your_server.com/speech_fritz.mp3",
                    "type": "multitrack",
                    "id": "speech track 3"
                    "offset": 5.1
                }
                ,
                ...
            ]
        }'

Changing Tracks

Use the following endpoint to change settings for the track with the ID my track id:

curl -X POST -H "Content-Type: application/json" \
    https://auphonic.com/api/production/{uuid}/multi_input_files/my track id.json \
    -H "Authorization: bearer {api_key}" \
    -d '{
            "service": "pmefeNCzkyT4TbRbDmoCDf",
            "input_file": "my_dropbox_file2.mp3",
            "id": "my new track id",
            "offset": 5,
            "algorithms": {"denoise": true, "denoiseamount": 12}
        }'

It’s also possible to delete all tracks:

curl -X DELETE https://auphonic.com/api/production/{uuid}/multi_input_files.json -H "Authorization: bearer {api_key}"

Or one specific track (here with the track ID my new track id):

curl -X DELETE https://auphonic.com/api/production/{uuid}/multi_input_files/my new track id.json -H "Authorization: bearer {api_key}"

Multitrack Presets

The creation of multitrack presets is mostly equivalent to the creation of multitrack productions, with the exception that it’s not possible to add an input file for each track:

curl -X POST -H "Content-Type: application/json" \
    https://auphonic.com/api/presets.json \
    -H "Authorization: bearer {api_key}" \
    -d '{
            "preset_name": "My Multitrack Preset",
            "output_files": [{"format": "mp3"}],
            "multi_input_files": [
                {
                    "type": "multitrack", "id": "speech track",
                    "algorithms": {"denoise": true, "denoiseamount": 12}
                },
                {
                    "type": "multitrack", "id": "music track",
                    "algorithms": {"filtering": false}
                },
                {
                    "input_file": "http://your_server.com/intro.mp3",
                    "type": "intro", "id": ""
                },
            ],
            "algorithms":  {
                "loudnesstarget": -23,
                "leveler": true,
                "gate": false,
                "crossgate": true
            },
            "is_multitrack": true
        }'

See also Creation of Presets.

API Response of Multitrack Productions

curl https://auphonic.com/api/production/{uuid}.json -H "Authorization: bearer {api_key}"

returns the following response (some details are stripped):

{
    "status_code": 200,
    "form_errors": {},
    "error_code": null,
    "error_message": "",
    "data": {
        "status": 3,
        "status_string": "Done",
        "is_multitrack": true,
        "image": null,
        "creation_time": "2014-08-29T10:07:43.009Z",
        "chapters": [ ... ],
        "edit_page": "https://auphonic.com/engine/multitrack/edit/WMLwjD4sHJMJV5zEg6oaMh",
        "output_files": [ ... ],
        "uuid": "WMLwjD4sHJMJV5zEg6oaMh",
        "error_status": null,
        "thumbnail": null,
        "metadata": { ... },
        "warning_status": null,
        "webhook": null,
        "status_page": "https://auphonic.com/engine/status/WMLwjD4sHJMJV5zEg6oaMh",
        "length": 118.975986394558,
        "length_timestring": "00:01:58.975",
        "output_basename": "multitrack_test_production",
        "outgoing_services": [],
        "change_time": "2014-08-29T10:12:19.104Z",
        "error_message": "",
        "warning_message": "",
        "speech_recognition": null,
        "waveform_image": "https://auphonic.com/api/download/audio-result/WMLwjD4sHJMJV5zEg6oaMh/waveform.png",
        "multi_input_files": [
            {
                "input_channels": 1,
                "service": null,
                "input_file": "file_1.m4a",
                "input_samplerate": 48000,
                "algorithms": {
                    "filtering": true,
                    "denoise": false,
                    "denoiseamount": 0,
                    "backforeground": "auto"
                },
                "input_bitrate": 82.701,
                "input_filetype": "aac",
                "type": "multitrack",
                "id": "track 1",
                "input_length": 118.975986394558,
                "offset": 0
            },
            {
                "input_channels": 2,
                "service": null,
                "input_file": "File 2.mp3",
                "input_samplerate": 44100,
                "algorithms": {
                    "filtering": true,
                    "denoise": false,
                    "denoiseamount": 0,
                    "backforeground": "background"
                },
                "input_bitrate": 128.0,
                "input_filetype": "mp3",
                "type": "multitrack",
                "id": "track 2",
                "input_length": 91.6908616780045,
                "offset": 3.5
            }
        ],
        "statistics": {
            "tracks": [
                {
                    "music_speech": "speech",
                    "levels": {
                        "lra": [9.94, "LU"],
                        "gain_mean": [-25.23, "dB"],
                        "noise_level": [-76.85, "dB"],
                        "max_momentary": [-27.23, "LUFS"],
                        "signal_level": [-32.52, "dB"],
                        "gain_min": [-98.86, "dB"],
                        "snr": [44.33, "dB"],
                        "max_shortterm": [-31.27, "LUFS"],
                        "loudness": [-34.12, "LUFS"],
                        "gain_max": [17.01, "dB"]
                    },
                    "identifier": "track 2",
                    "activity": [[3.32, 7.73], [12.08, 20.67], [22.22, 28.87], ...]
                },
                {
                    "music_speech": "speech",
                    "levels": {
                        "lra": [3.52, "LU"],
                        "gain_mean": [-1.52, "dB"],
                        "noise_level": [-49.72, "dB"],
                        "max_momentary": [-10.58, "LUFS"],
                        "signal_level": [-16.2, "dB"],
                        "gain_min": [-61.3, "dB"],
                        "snr": [33.52, "dB"],
                        "max_shortterm": [-15.51, "LUFS"],
                        "loudness": [-17.64, "LUFS"],
                        "gain_max": [2.07, "dB"]
                    },
                    "identifier": "track 1",
                    "activity": [[0.11, 3.78], [7.93, 15.23], ...]
                }
            ],
            "master": {
                "levels": {
                    "lra": [3.04, "LU"],
                    "max_momentary": [-11.44, "LUFS"],
                    "peak": [-1.0, "dBTP"],
                    "max_shortterm": [-14.19, "LUFS"],
                    "loudness": [-16.0, "LUFS"]
                }
            }
        },
        "algorithms": {
            "leveler": true,
            "loudnesstarget": -16,
            "crossgate": true,
            "gate": true
        },
        "start_allowed": false,
        "change_allowed": true
    }
}

Multitrack specific parameters:

• is_multitrack: true for multitrack production

• length: output audio length in seconds (longest track with intro/outro) - IMPORTANT: if you have intro or outro files, they are included in this length!

• length_timestring: parameter length in HH:MM:SS.mmm

• multi_input_files: audio file details, identifier and algorithms for each track

• statistics: level, activity, music-speech, noise reduction (and more) statistics per track and for the master track of multitrack productions

For more information about all the other parameters please see Details about a Production.