Enhancing audio support in VidBlaster

[Mike]

We need to do some brainstorming about this topic. I realize there is one camp that desperately need this, and the other one is fine with how things are now (using external mixing).

Some things are clear, we need to be able to get audio from video sources and either mix that internally, or send it to a (dedicated) audio output.

What's open for discussion is how we do the mixing if we decide not to use an external mixer. We can do it much like it is now, i.e. all the audio is sent to an audio output (typically the default one which has the speakers connected), and instead of routing that to an external mixer use that mix as our audio mix. Basically we'd emulate the famous "What You Hear" input. (a)

Alternatively we can take all audio sources and mix them in VidBlaster, then optionally send them to a sound card output for monitoring. The advantage is that you do not need a (free) sound card output for mixing, the disadvantage of this will be a higher latency. (b)

Aside from your expert thoughts, I also would like to know how and when you would use internal mixing, do you see a need for a hybrid form of mixing, and how and where (before/after delay) would you monitor live sources like microphones?

[Jim Z]

Having an audio mixer module with maybe 4-6 stereo inputs would allow a more straight-forward audio config for the 'external mixer' case. Currently we use an external mixer for two mics to the AUX bus, then through RadioDelay, then back to mix with the player audio. The output goes to a modulator and also via USB to VidBlaster recorder and streamer. If there were an Audio Mixer module in VB, the external mixer would pre-amplify and mix the mics, then feed an input of the VB Mixer. The Player audio (each one or collectively) would occupy other Mixer module inputs. The Mixer output can feed the recorder, streaming, and exported audio. In our setup we might elect to keep the Player level control on the external mixer, in which case the Mixer module would be used to implement the delay, replacing the need for RadioDelay.

As far as monitoring before or after the delay, we would monitor the composite program (after the delay). Since our announcers are not listening to the program, the delay is not an issue.

For flexibility in interconnections, the Audio Mixer could feed a named bus, such as PGM AUDIO, AUX1, etc, and the output can occur through a separate module. Having the ability to configure a delay on each Mixer input would be a nice feature. Placing the mixed output to a named bus, and if needed a delayed version to another named bus will allow for flexible interconnection between the places where audio is needed while keeping the number of separate audio streams to a minimum. The current Video Output module could have an audio selection added to it, and possibly a tune delay configurable as well.

Having the ability to set a delay at the input to the Recorder, Stream or Video Out module provides added flexibility, although I think the delay balancing is mostly accomplished at the Mixer module inputs and the Mixer module bus output.

[Mike]

Thanks for your feedback Jim. So you'd prefer my second solution? My problem with that is the added latency may be a problem for those doing live shows (as you may need to delay video to get in sync with audio).

PS: Any chance of a profile picture? Personally I prefer to remember faces over names.

[zenvideo]

Here's a few opening thoughts, in no particular order, on this potentially complex issue:-

1) It's hard to discuss the design of any audio mixer, software-based or otherwise, without a block diagram to illustrate the switching, routing & mixing, and what happens where.

2) Is there a possibility to provide hosting capability for VST plug-ins? There are lots of useful plug-ins available, including free ones, which could then enhance the audio processing capabilities without further development within VidBlaster.

3) What sort of latency is expected from an internal software audio mixer? Would the ASIO driver model be used when available (within device drivers)?

4) I suspect the requirements of (a) those at the higher end (who are happy to use a lot of external audio equipment) are not that different from (b) those at the entry level with minimal sources. In the middle are (c) people who might want quite complex audio setups without resorting to (much) external equipment.

For my part, I'd find it useful to have a simple mixer with two stereo inputs The sources should be selectable, and include physical soundcard inputs as well a "what you hear" mixed output from the default Windows playback device. Of those two inputs, one should be routed through a variable delay line before being mixed with the other as the Main Output (virtual device?). There should also be a Secondary (monitor) Output with the mix being switchable pre or post delay.

That sort of setup would cover what I think of as a full pro configuration with a multi-group external mixer taking both live sound and VidBlaster Player sound (routed to additional, non-default, audio device outputs) and producing two audio group outputs:- (i) Live sound to go to the VidBlaster channel that adds a delay, and (ii) Player sound that goes to the other channel and needs no delay.

It might also suffice for the group (b) entry-level users for whom the "delay" channel would take its source from the PC mic input, from a camera output or a simple external mixer/pre-amp, and the delay channel would take a "what-you-hear" mix of the internal VB Players.

For the (c) group, I suspect you would have to the ability to add more channels to the mixer, able to take their input from individual players as well as audio input devices (including embedded audio in video streams) and have their routing switched to either the delay bus or non-delay bus. For the monitor mix, once you've got lots of channels, it becomes desirable to add things like Mute/Solo functions. In this scenario, having VST plug-in hosting would increase the functionality considerably, if every channel and bus could have the option of VST plug-ins inserted inline.

[Mike]

Thanks Martin, I'll dive into those features later. Getting back to my question and your subsequent question: latency in kernel mode is lowest (my a), then ASIO (but not widely supported so a coding nightmare) then mixing in user mode (my b). As far as I can see at this stage, there would be no reason not to choose my a other than you would need a dedicated audio output device for the final mix. Which I guess can be a problem if you have just one sound card installed and do not want to use that.

[kricher]

Mike,

For your option B, how much latency are we potentially talking about?

[Mike]

Hard to say at this point. It depends on the required buffer size to assure hickup free mixing, which depends on the PC. Typically a few seconds is normal, but it can probably be brought down to 1 s. Which can be an awful lot for live video presentations. And don't forget that any processing would be additional (to both a and b).

[Tony]

Is there a way to route some modules to output so once can input them on an external channel input? Some times one may want to lower the volume while they talk over the video and I know there is %25 %50 etc of the volume but a more smoother audio transition. What about a peak meeter in the player (does not have to be stereo) and a volume scrubber? OR am I lost here? =)

[Edwardgriswold]

I really need audio support for the BM cards. I would think the sync issue would be solved because the video and audio are in-sync as a result of the I/O card. I know the main problem here is that windows does not see the BM inputs as an active or inactive device. This would require special programing by Mike for this support. I have been all over the internet looking for a way to accomplish this. I have only found one software audio mixer program that supports BM cards and it's just adds to much $$ to my cost when my goal is to keep cost with in reason. In the past when working with the very first PC based video toaster before Newtek activated the audio on Video Toaster cards we used a Sound Blaster audio card which had a jumper to allow a sync signal to be jumped over from the VT card and it worked perfectly. believe me I have even looked for references to this audio card or one like it that has gen-lock. I'm hopful the Mike will get the BM SDK (it's free) and find a way to make this happen. If there is anything I can do please let me know.

Of course it would be great if there was a level control for each BM I/O audio input. then I could set individual levels and maybe a some point even have the levels follow live switch. But, first things first.
Thanks

[zenvideo]

Hard to say at this point. It depends on the required buffer size to assure hickup free mixing, which depends on the PC. Typically a few seconds is normal, but it can probably be brought down to 1 s. Which can be an awful lot for live video presentations. And don't forget that any processing would be additional (to both a and b).

That's a lot more than I was expecting. Last night (at home) and this morning (at the office) I've had a play with SAC (Software Audio Console) and measured the in-to-out latency when in "normal" (non-ASIO) driver mode at just over 100ms. The PC here doesn't have a suitable soundcard with ASIO driver for me to test with, but at home in ASIO mode I think the latency sounded even less than in normal mode, but unfortunately I didn't have an easy way to record the before/after feeds and measure the difference. I was certainly able to set the buffer size/number lower in ASIO mode than in normal mode.

[Mike]

I am running tests as we speak, and I can easily achieve 100 ms latency. However, we need something that works across the board, on any PC and while running a live video production program. I can assure you the occasional tick I hear in my headphones can be quite annoying. And before you ask, yes I know we can make it a setting and I promise you it'll be one that will give lots of support issues. So right now that's not a solution (nor is ASIO for the same reason). I am trying to find a solid solution.

[zenvideo]

I am running tests as we speak, and I can easily achieve 100 ms latency. However, we need something that works across the board, on any PC and while running a live video production program. I can assure you the occasional tick I hear in my headphones can be quite annoying. And before you ask, yes I know we can make it a setting and I promise you it'll be one that will give lots of support issues. So right now that's not a solution (nor is ASIO for the same reason). I am trying to find a solid solution.

Fair enough - I look forward to seeing what you come up with. :) I'm sure you'll let us know when you need more input on the matter.

[zenvideo]

Is there a way to route some modules to output so once can input them on an external channel input? Some times one may want to lower the volume while they talk over the video and I know there is %25 %50 etc of the volume but a more smoother audio transition.

Is this not what's possible now, using a channel on an external mixer? By default all Player audio (from multiple players) is mixed together and appears on the default audio device output, from where it can be taken to an external mixer to which microphones are also connected, such that the operator can dip the Player audio when someone talks over it. On a more advanced setup, the Player audio can be routed through a compressor (or "ducking") device in which the side-chain is fed from the microphone (sub-group) level, such that the Player audio level is automatically reduced when someone speaks (as per most music radio shows).

It's also possible right now, if you have multiple audio devices on your PC, to have each Player send its sound to a different physical output connection, each connected to a different input on an external mixer, which gives the ultimate flexibility in audio control.

[Jim Z]

When the audio production is largely outside of vidblaster, just having an available Delay module in VB would be nice. Even better, if the module can be set to match the in to out video processing delay (with a tune adjustment as well). Users can simplify the configuration and reduce startup steps by using VB Delay module instead of Radio Delay; VB Delay could start automatically, and load the ports and delay values from the VB profile.

I'm just throwing out the idea of the Delay module as a simple way to keep all the present advantages and remove a shortcoming. Perhaps this same thing can be accomplished using the proposed mixer feature, configuring the input and output and specifying an output delay value.

[weconverse]

Well, just to be transparent about my standpoint - I think VidBlaster should focus on the visual part. And, given the amount of issues we have with audio delay, few will appreciate a built in s/w audio mixer if delays are induced/added.

Given that, I can see that (a) would be useful.
(perhaps this topic makes my FQ for the possibility to adjust RTSP audio level (http://forum.vidblaster.com/showthread.php?t=2249)obsolete)

[Mike]

Thanks Jim & Richard. Unfortunately, and fortunately for some, more advanced audio features like delay can only be added if there is at least some basic audio handling in VidBlaster. From there it is not a big step to go a little further and add some internal mixing. There are many ways this can be implemented though, hence my inquiry. I do not mind building things in small steps, but I like to make sure my foundation is solid. E.g. the direct trigger for this request is the new recorder I am developing, it needs an audio part and how I build that depends on this thread. Adding audio to a digital recorder is remarkably complicated as you need to sync it with video, which means changing the speed of the stream, requiring a complex process of resampling. Reading about what you need, how you plan to use it and why helps me figure out how best to set up this process, and closely related to that the entire audio foundation. This may come as a surprise but I regularly appreciate decisions I made 5 (!) years back, because now they make my live easier and make things fall into place. As some of my childhood favourites used to say (http://www.youtube.com/watch?v=OqZVRoIoMfQ): I love it when a plan comes together!

[Sinc747]

I haven't weighed in on this discussion as the suggestions that have been made are what I was thinking. I am excited about these new features and will follow this thread very closely.