Discussion:
[Freeswitch-users] Playing multiple files simultaneously
Bryan Smart
2011-07-01 05:13:20 UTC
Permalink
Is it possible for Freeswitch to play more than one file to a channel at a time? What I've seen and tried from the dialplan and scripts either queues files to play, or will stop a currently playing file so that the newly requested file will play. This also seems to be the case in conferences. When I send multiple play commands to conferences, the files are queued.

As for how this might be used, think of an IVR that plays queued prompts, yet continuously plays looping music or a Shoutcast stream in the background. I also want to be able to play short cue tones that start at the same time as a prompt (don't want to pre-mix them in to a single file, though).

Is this currently possible through any means? Perhaps with the event socket?

Bryan
Jan Berger
2011-07-01 06:49:23 UTC
Permalink
If you play several files FS will queue them if it is the same stream - you
need to somehow set up a conference with multiple streams (callers).

I would check options around music on hold.

But, another way is to use FS to play several outgoing SIP streams and loop
them back into a conference.


Jan

-----Original Message-----
From: freeswitch-users-bounces at lists.freeswitch.org
[mailto:freeswitch-users-bounces at lists.freeswitch.org] On Behalf Of Bryan
Smart
Sent: 1. juli 2011 07:13
To: FreeSWITCH Users Help
Subject: [Freeswitch-users] Playing multiple files simultaneously

Is it possible for Freeswitch to play more than one file to a channel at a
time? What I've seen and tried from the dialplan and scripts either queues
files to play, or will stop a currently playing file so that the newly
requested file will play. This also seems to be the case in conferences.
When I send multiple play commands to conferences, the files are queued.

As for how this might be used, think of an IVR that plays queued prompts,
yet continuously plays looping music or a Shoutcast stream in the
background. I also want to be able to play short cue tones that start at the
same time as a prompt (don't want to pre-mix them in to a single file,
though).

Is this currently possible through any means? Perhaps with the event socket?

Bryan


_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE

FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
broken dash
2011-07-06 06:05:10 UTC
Permalink
I have been wanting these features and I found this...liquidsoap,
check out the transcoding section. I was trying to play multiple
shoutcast audio streams and hopefully mux them together nicely, etc..
haven't gotten around to setting freeswitch to utilize it but I'm sure
it wouldn't be hard.

http://savonet.sourceforge.net/doc-1.0.0-beta2/cookbook.html


Cheers,
Brian
Post by Jan Berger
If you play several files FS will queue them if it is the same stream - you
need to somehow set up a conference with multiple streams (callers).
I would check options around music on hold.
But, another way is to use FS to play several outgoing SIP streams and loop
them back into a conference.
Jan
-----Original Message-----
From: freeswitch-users-bounces at lists.freeswitch.org
[mailto:freeswitch-users-bounces at lists.freeswitch.org] On Behalf Of Bryan
Smart
Sent: 1. juli 2011 07:13
To: FreeSWITCH Users Help
Subject: [Freeswitch-users] Playing multiple files simultaneously
Is it possible for Freeswitch to play more than one file to a channel at a
time? What I've seen and tried from the dialplan and scripts either queues
files to play, or will stop a currently playing file so that the newly
requested file will play. This also seems to be the case in conferences.
When I send multiple play commands to conferences, the files are queued.
As for how this might be used, think of an IVR that plays queued prompts,
yet continuously plays looping music or a Shoutcast stream in the
background. I also want to be able to play short cue tones that start at the
same time as a prompt (don't want to pre-mix them in to a single file,
though).
Is this currently possible through any means? Perhaps with the event socket?
Bryan
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Bryan Smart
2011-07-06 17:35:09 UTC
Permalink
When I brought this up originally, I was considering possibilities for voice apps built on FS.

The long-standing user interface model for phone apps is like a voice version of a commandline. The app plays a linear stream of output, and then waits for you to respond. With rare exception, the only interface contexts that are widely used are a menu and a data entry prompt. If speech is involved, that increases the type of data that can be entered, but speech apps are usually the same: menus or data prompts.

With the capability of FS to globally bind digits and detect speech (so to know when to listen with a speech recognizer), it should be possible to create voice apps where digits and speech perform actions by raising events in the app, rather than being exclusively accepted by an input mechanism like a menu or data prompt. So far, I haven't run across voice apps that are designed using an event-driven model, at least in the way that desktop apps are designed. FS has got to be the place to prototype such apps, though. The whole system is event-based.

I know that it takes a stretch of the imagination to think of voice apps working in a new way. The phone has been stuck in a voice version of DOS for 25 or so years (since voicemail got a start as the first big voice app). Speech recognition has expanded possibilities for input, but has not changed the model of interface design.

I'm experimenting with what is possible with FS today. However, a more complex sound playback capability is needed to make a new type of audio interface possible.

An interface context, like a graphical desktop, can contain multiple objects. If a graphical object (icon) needs to notify the user of something, it generates a visual cue (flashes, changes color, adds a badge, and so on). Objects in an audio context need to notify, also, and they would do that by playing cues, speaking, etc. Those cues need to be able to partially overlap when played. New cues certainly shouldn't preempt older cues that haven't had a chance to play yet. Further, certain status cues should be heard, but must overlap currently playing audio, as it wouldn't be desirable for a status cue to interrupt the reading of an important message, for example.

So far, the only mixing environment in FS that I've been able to use is mod_conference, but that isn't ideal. To play multiple audio streams to a conference, it is necessary to connect multiple sessions to it, and treat them each like a sound source. One issue is that conference is meant to relay audio to everyone, and I can't help but think that these sound source sessions drain far too many resources. I've been setting them to deaf, so they should only relay audio in, but they might still be drawing more resources than necessary with detecting digits/speech, and other overhead that is part of every channel (I don't know the full list of what is involved). Plus, it just seems like a mistake to think of a voice app server where every incoming caller has their own conference, plus many sessions that exist to stream files to their conference. Maybe it wouldn't be as bad as it sounds.

I'm still trying to learn about FS internals, so I may have a good bit of the following wrong. Perhaps we could meet our needs by creating a module that exposed a new API command for playing sounds asynchronously on a channel. That way, the existing sound playback would continue to be lightweight. A voice app that worked through the event socket could use this module to handle playback and control of its sounds. Here are basic capabilities:

play: Should start playback of a sound, overlapping any existing sound previously requested through this command that has not finished playing. When playback starts, should return a unique ID for the sound instance (an event?). Optional playback parameters: volume/gain, start off set, loop count.

stop: Stops an existing sound instance from playing (by unique ID), or all sound instances.

change: Change sound instance parameters of an active sound instance: volume/gain, file position/time, loop count.

An event should be raised when any sound instance finishes playing. Should include info about cause: normal stop, manual stop, loop re-trigger, etc.

The module could handle the mixing, but it occurs to me that, looking ahead, it might be better to push a lot of that work out to another tech like OpenAL. After all, at some point, some clients might be connecting with 48Khz stereo audio. OpenAL supports positional audio and effects, in addition to raw mixing. OpenAL is available for several platforms, but not everywhere. Perhaps it would be more important to be widely available, at the cost of features.

Thoughts? Is a module the right way to go? Is using OpenAL a good idea, or should the module handle the mixing?

Bryan
Post by broken dash
I have been wanting these features and I found this...liquidsoap,
check out the transcoding section. I was trying to play multiple
shoutcast audio streams and hopefully mux them together nicely, etc..
haven't gotten around to setting freeswitch to utilize it but I'm sure
it wouldn't be hard.
http://savonet.sourceforge.net/doc-1.0.0-beta2/cookbook.html
Cheers,
Brian
Post by Jan Berger
If you play several files FS will queue them if it is the same stream - you
need to somehow set up a conference with multiple streams (callers).
I would check options around music on hold.
But, another way is to use FS to play several outgoing SIP streams and loop
them back into a conference.
Jan
-----Original Message-----
From: freeswitch-users-bounces at lists.freeswitch.org
[mailto:freeswitch-users-bounces at lists.freeswitch.org] On Behalf Of Bryan
Smart
Sent: 1. juli 2011 07:13
To: FreeSWITCH Users Help
Subject: [Freeswitch-users] Playing multiple files simultaneously
Is it possible for Freeswitch to play more than one file to a channel at a
time? What I've seen and tried from the dialplan and scripts either queues
files to play, or will stop a currently playing file so that the newly
requested file will play. This also seems to be the case in conferences.
When I send multiple play commands to conferences, the files are queued.
As for how this might be used, think of an IVR that plays queued prompts,
yet continuously plays looping music or a Shoutcast stream in the
background. I also want to be able to play short cue tones that start at the
same time as a prompt (don't want to pre-mix them in to a single file,
though).
Is this currently possible through any means? Perhaps with the event socket?
Bryan
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Jan Berger
2011-07-06 19:15:40 UTC
Permalink
Brian,

An asynchronous IVR is not exactly news. But, many of the IVR engines
implement blocking schemes on top of an asynchronous engine to simplify IVR
scripting. FS support both - I am not sure if it support all the required
events on speech, but that can be added if not. I need them as well, so your
not alone.

Sockets and ESL might be what you are looking for.

Jan

-----Original Message-----
From: freeswitch-users-bounces at lists.freeswitch.org
[mailto:freeswitch-users-bounces at lists.freeswitch.org] On Behalf Of Bryan
Smart
Sent: 6. juli 2011 19:35
To: FreeSWITCH Users Help
Subject: Re: [Freeswitch-users] Playing multiple files simultaneously

When I brought this up originally, I was considering possibilities for voice
apps built on FS.

The long-standing user interface model for phone apps is like a voice
version of a commandline. The app plays a linear stream of output, and then
waits for you to respond. With rare exception, the only interface contexts
that are widely used are a menu and a data entry prompt. If speech is
involved, that increases the type of data that can be entered, but speech
apps are usually the same: menus or data prompts.

With the capability of FS to globally bind digits and detect speech (so to
know when to listen with a speech recognizer), it should be possible to
create voice apps where digits and speech perform actions by raising events
in the app, rather than being exclusively accepted by an input mechanism
like a menu or data prompt. So far, I haven't run across voice apps that are
designed using an event-driven model, at least in the way that desktop apps
are designed. FS has got to be the place to prototype such apps, though. The
whole system is event-based.

I know that it takes a stretch of the imagination to think of voice apps
working in a new way. The phone has been stuck in a voice version of DOS for
25 or so years (since voicemail got a start as the first big voice app).
Speech recognition has expanded possibilities for input, but has not changed
the model of interface design.

I'm experimenting with what is possible with FS today. However, a more
complex sound playback capability is needed to make a new type of audio
interface possible.

An interface context, like a graphical desktop, can contain multiple
objects. If a graphical object (icon) needs to notify the user of something,
it generates a visual cue (flashes, changes color, adds a badge, and so on).
Objects in an audio context need to notify, also, and they would do that by
playing cues, speaking, etc. Those cues need to be able to partially overlap
when played. New cues certainly shouldn't preempt older cues that haven't
had a chance to play yet. Further, certain status cues should be heard, but
must overlap currently playing audio, as it wouldn't be desirable for a
status cue to interrupt the reading of an important message, for example.

So far, the only mixing environment in FS that I've been able to use is
mod_conference, but that isn't ideal. To play multiple audio streams to a
conference, it is necessary to connect multiple sessions to it, and treat
them each like a sound source. One issue is that conference is meant to
relay audio to everyone, and I can't help but think that these sound source
sessions drain far too many resources. I've been setting them to deaf, so
they should only relay audio in, but they might still be drawing more
resources than necessary with detecting digits/speech, and other overhead
that is part of every channel (I don't know the full list of what is
involved). Plus, it just seems like a mistake to think of a voice app server
where every incoming caller has their own conference, plus many sessions
that exist to stream files to their conference. Maybe it wouldn't be as bad
as it sounds.

I'm still trying to learn about FS internals, so I may have a good bit of
the following wrong. Perhaps we could meet our needs by creating a module
that exposed a new API command for playing sounds asynchronously on a
channel. That way, the existing sound playback would continue to be
lightweight. A voice app that worked through the event socket could use this
module to handle playback and control of its sounds. Here are basic
capabilities:

play: Should start playback of a sound, overlapping any existing sound
previously requested through this command that has not finished playing.
When playback starts, should return a unique ID for the sound instance (an
event?). Optional playback parameters: volume/gain, start off set, loop
count.

stop: Stops an existing sound instance from playing (by unique ID), or all
sound instances.

change: Change sound instance parameters of an active sound instance:
volume/gain, file position/time, loop count.

An event should be raised when any sound instance finishes playing. Should
include info about cause: normal stop, manual stop, loop re-trigger, etc.

The module could handle the mixing, but it occurs to me that, looking ahead,
it might be better to push a lot of that work out to another tech like
OpenAL. After all, at some point, some clients might be connecting with
48Khz stereo audio. OpenAL supports positional audio and effects, in
addition to raw mixing. OpenAL is available for several platforms, but not
everywhere. Perhaps it would be more important to be widely available, at
the cost of features.

Thoughts? Is a module the right way to go? Is using OpenAL a good idea, or
should the module handle the mixing?

Bryan
Post by broken dash
I have been wanting these features and I found this...liquidsoap,
check out the transcoding section. I was trying to play multiple
shoutcast audio streams and hopefully mux them together nicely, etc..
haven't gotten around to setting freeswitch to utilize it but I'm sure
it wouldn't be hard.
http://savonet.sourceforge.net/doc-1.0.0-beta2/cookbook.html
Cheers,
Brian
Post by Jan Berger
If you play several files FS will queue them if it is the same stream - you
need to somehow set up a conference with multiple streams (callers).
I would check options around music on hold.
But, another way is to use FS to play several outgoing SIP streams and loop
them back into a conference.
Jan
-----Original Message-----
From: freeswitch-users-bounces at lists.freeswitch.org
[mailto:freeswitch-users-bounces at lists.freeswitch.org] On Behalf Of Bryan
Smart
Sent: 1. juli 2011 07:13
To: FreeSWITCH Users Help
Subject: [Freeswitch-users] Playing multiple files simultaneously
Is it possible for Freeswitch to play more than one file to a channel at a
time? What I've seen and tried from the dialplan and scripts either queues
files to play, or will stop a currently playing file so that the newly
requested file will play. This also seems to be the case in conferences.
When I send multiple play commands to conferences, the files are queued.
As for how this might be used, think of an IVR that plays queued prompts,
yet continuously plays looping music or a Shoutcast stream in the
background. I also want to be able to play short cue tones that start at the
same time as a prompt (don't want to pre-mix them in to a single file,
though).
Is this currently possible through any means? Perhaps with the event socket?
Bryan
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE

FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Bryan Smart
2011-07-06 19:33:39 UTC
Permalink
Do you mean non-blocking IVR in the sense that speech/digits interrupt the prompts, or that the IVR app runs in a loop, handling other tasks and sending audio output on the channel, until the user raises an event with digits or speech?

If the second, then it seems like layered audio would be a necessary capability. You wouldn't have as much control of when information is presented, so, with only being able to play a single sound/prompt at a time, there is potential for sounds interrupting each other, or else a long queue of them accumulating.

Bryan
Post by Jan Berger
Brian,
An asynchronous IVR is not exactly news. But, many of the IVR engines
implement blocking schemes on top of an asynchronous engine to simplify IVR
scripting. FS support both - I am not sure if it support all the required
events on speech, but that can be added if not. I need them as well, so your
not alone.
Sockets and ESL might be what you are looking for.
Jan
-----Original Message-----
From: freeswitch-users-bounces at lists.freeswitch.org
[mailto:freeswitch-users-bounces at lists.freeswitch.org] On Behalf Of Bryan
Smart
Sent: 6. juli 2011 19:35
To: FreeSWITCH Users Help
Subject: Re: [Freeswitch-users] Playing multiple files simultaneously
When I brought this up originally, I was considering possibilities for voice
apps built on FS.
The long-standing user interface model for phone apps is like a voice
version of a commandline. The app plays a linear stream of output, and then
waits for you to respond. With rare exception, the only interface contexts
that are widely used are a menu and a data entry prompt. If speech is
involved, that increases the type of data that can be entered, but speech
apps are usually the same: menus or data prompts.
With the capability of FS to globally bind digits and detect speech (so to
know when to listen with a speech recognizer), it should be possible to
create voice apps where digits and speech perform actions by raising events
in the app, rather than being exclusively accepted by an input mechanism
like a menu or data prompt. So far, I haven't run across voice apps that are
designed using an event-driven model, at least in the way that desktop apps
are designed. FS has got to be the place to prototype such apps, though. The
whole system is event-based.
I know that it takes a stretch of the imagination to think of voice apps
working in a new way. The phone has been stuck in a voice version of DOS for
25 or so years (since voicemail got a start as the first big voice app).
Speech recognition has expanded possibilities for input, but has not changed
the model of interface design.
I'm experimenting with what is possible with FS today. However, a more
complex sound playback capability is needed to make a new type of audio
interface possible.
An interface context, like a graphical desktop, can contain multiple
objects. If a graphical object (icon) needs to notify the user of something,
it generates a visual cue (flashes, changes color, adds a badge, and so on).
Objects in an audio context need to notify, also, and they would do that by
playing cues, speaking, etc. Those cues need to be able to partially overlap
when played. New cues certainly shouldn't preempt older cues that haven't
had a chance to play yet. Further, certain status cues should be heard, but
must overlap currently playing audio, as it wouldn't be desirable for a
status cue to interrupt the reading of an important message, for example.
So far, the only mixing environment in FS that I've been able to use is
mod_conference, but that isn't ideal. To play multiple audio streams to a
conference, it is necessary to connect multiple sessions to it, and treat
them each like a sound source. One issue is that conference is meant to
relay audio to everyone, and I can't help but think that these sound source
sessions drain far too many resources. I've been setting them to deaf, so
they should only relay audio in, but they might still be drawing more
resources than necessary with detecting digits/speech, and other overhead
that is part of every channel (I don't know the full list of what is
involved). Plus, it just seems like a mistake to think of a voice app server
where every incoming caller has their own conference, plus many sessions
that exist to stream files to their conference. Maybe it wouldn't be as bad
as it sounds.
I'm still trying to learn about FS internals, so I may have a good bit of
the following wrong. Perhaps we could meet our needs by creating a module
that exposed a new API command for playing sounds asynchronously on a
channel. That way, the existing sound playback would continue to be
lightweight. A voice app that worked through the event socket could use this
module to handle playback and control of its sounds. Here are basic
play: Should start playback of a sound, overlapping any existing sound
previously requested through this command that has not finished playing.
When playback starts, should return a unique ID for the sound instance (an
event?). Optional playback parameters: volume/gain, start off set, loop
count.
stop: Stops an existing sound instance from playing (by unique ID), or all sound instances.
volume/gain, file position/time, loop count.
An event should be raised when any sound instance finishes playing. Should
include info about cause: normal stop, manual stop, loop re-trigger, etc.
The module could handle the mixing, but it occurs to me that, looking ahead,
it might be better to push a lot of that work out to another tech like
OpenAL. After all, at some point, some clients might be connecting with
48Khz stereo audio. OpenAL supports positional audio and effects, in
addition to raw mixing. OpenAL is available for several platforms, but not
everywhere. Perhaps it would be more important to be widely available, at
the cost of features.
Thoughts? Is a module the right way to go? Is using OpenAL a good idea, or
should the module handle the mixing?
Bryan
Post by broken dash
I have been wanting these features and I found this...liquidsoap,
check out the transcoding section. I was trying to play multiple
shoutcast audio streams and hopefully mux them together nicely, etc..
haven't gotten around to setting freeswitch to utilize it but I'm sure
it wouldn't be hard.
http://savonet.sourceforge.net/doc-1.0.0-beta2/cookbook.html
Cheers,
Brian
Post by Jan Berger
If you play several files FS will queue them if it is the same stream -
you
Post by broken dash
Post by Jan Berger
need to somehow set up a conference with multiple streams (callers).
I would check options around music on hold.
But, another way is to use FS to play several outgoing SIP streams and
loop
Post by broken dash
Post by Jan Berger
them back into a conference.
Jan
-----Original Message-----
From: freeswitch-users-bounces at lists.freeswitch.org
[mailto:freeswitch-users-bounces at lists.freeswitch.org] On Behalf Of Bryan
Smart
Sent: 1. juli 2011 07:13
To: FreeSWITCH Users Help
Subject: [Freeswitch-users] Playing multiple files simultaneously
Is it possible for Freeswitch to play more than one file to a channel at
a
Post by broken dash
Post by Jan Berger
time? What I've seen and tried from the dialplan and scripts either
queues
Post by broken dash
Post by Jan Berger
files to play, or will stop a currently playing file so that the newly
requested file will play. This also seems to be the case in conferences.
When I send multiple play commands to conferences, the files are queued.
As for how this might be used, think of an IVR that plays queued prompts,
yet continuously plays looping music or a Shoutcast stream in the
background. I also want to be able to play short cue tones that start at
the
Post by broken dash
Post by Jan Berger
same time as a prompt (don't want to pre-mix them in to a single file,
though).
Is this currently possible through any means? Perhaps with the event
socket?
Post by broken dash
Post by Jan Berger
Bryan
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Jan Berger
2011-07-06 21:06:42 UTC
Permalink
The later, the first is probably what I call "barge-in" unless I
misunderstand you? You can do that as well.

-----Original Message-----
From: freeswitch-users-bounces at lists.freeswitch.org
[mailto:freeswitch-users-bounces at lists.freeswitch.org] On Behalf Of Bryan
Smart
Sent: 6. juli 2011 21:34
To: FreeSWITCH Users Help
Subject: Re: [Freeswitch-users] Playing multiple files simultaneously

Do you mean non-blocking IVR in the sense that speech/digits interrupt the
prompts, or that the IVR app runs in a loop, handling other tasks and
sending audio output on the channel, until the user raises an event with
digits or speech?

If the second, then it seems like layered audio would be a necessary
capability. You wouldn't have as much control of when information is
presented, so, with only being able to play a single sound/prompt at a time,
there is potential for sounds interrupting each other, or else a long queue
of them accumulating.

Bryan
Post by Jan Berger
Brian,
An asynchronous IVR is not exactly news. But, many of the IVR engines
implement blocking schemes on top of an asynchronous engine to simplify IVR
scripting. FS support both - I am not sure if it support all the required
events on speech, but that can be added if not. I need them as well, so your
not alone.
Sockets and ESL might be what you are looking for.
Jan
-----Original Message-----
From: freeswitch-users-bounces at lists.freeswitch.org
[mailto:freeswitch-users-bounces at lists.freeswitch.org] On Behalf Of Bryan
Smart
Sent: 6. juli 2011 19:35
To: FreeSWITCH Users Help
Subject: Re: [Freeswitch-users] Playing multiple files simultaneously
When I brought this up originally, I was considering possibilities for voice
apps built on FS.
The long-standing user interface model for phone apps is like a voice
version of a commandline. The app plays a linear stream of output, and then
waits for you to respond. With rare exception, the only interface contexts
that are widely used are a menu and a data entry prompt. If speech is
involved, that increases the type of data that can be entered, but speech
apps are usually the same: menus or data prompts.
With the capability of FS to globally bind digits and detect speech (so to
know when to listen with a speech recognizer), it should be possible to
create voice apps where digits and speech perform actions by raising events
in the app, rather than being exclusively accepted by an input mechanism
like a menu or data prompt. So far, I haven't run across voice apps that are
designed using an event-driven model, at least in the way that desktop apps
are designed. FS has got to be the place to prototype such apps, though. The
whole system is event-based.
I know that it takes a stretch of the imagination to think of voice apps
working in a new way. The phone has been stuck in a voice version of DOS for
25 or so years (since voicemail got a start as the first big voice app).
Speech recognition has expanded possibilities for input, but has not changed
the model of interface design.
I'm experimenting with what is possible with FS today. However, a more
complex sound playback capability is needed to make a new type of audio
interface possible.
An interface context, like a graphical desktop, can contain multiple
objects. If a graphical object (icon) needs to notify the user of something,
it generates a visual cue (flashes, changes color, adds a badge, and so on).
Objects in an audio context need to notify, also, and they would do that by
playing cues, speaking, etc. Those cues need to be able to partially overlap
when played. New cues certainly shouldn't preempt older cues that haven't
had a chance to play yet. Further, certain status cues should be heard, but
must overlap currently playing audio, as it wouldn't be desirable for a
status cue to interrupt the reading of an important message, for example.
So far, the only mixing environment in FS that I've been able to use is
mod_conference, but that isn't ideal. To play multiple audio streams to a
conference, it is necessary to connect multiple sessions to it, and treat
them each like a sound source. One issue is that conference is meant to
relay audio to everyone, and I can't help but think that these sound source
sessions drain far too many resources. I've been setting them to deaf, so
they should only relay audio in, but they might still be drawing more
resources than necessary with detecting digits/speech, and other overhead
that is part of every channel (I don't know the full list of what is
involved). Plus, it just seems like a mistake to think of a voice app server
where every incoming caller has their own conference, plus many sessions
that exist to stream files to their conference. Maybe it wouldn't be as bad
as it sounds.
I'm still trying to learn about FS internals, so I may have a good bit of
the following wrong. Perhaps we could meet our needs by creating a module
that exposed a new API command for playing sounds asynchronously on a
channel. That way, the existing sound playback would continue to be
lightweight. A voice app that worked through the event socket could use this
module to handle playback and control of its sounds. Here are basic
play: Should start playback of a sound, overlapping any existing sound
previously requested through this command that has not finished playing.
When playback starts, should return a unique ID for the sound instance (an
event?). Optional playback parameters: volume/gain, start off set, loop
count.
stop: Stops an existing sound instance from playing (by unique ID), or all sound instances.
volume/gain, file position/time, loop count.
An event should be raised when any sound instance finishes playing. Should
include info about cause: normal stop, manual stop, loop re-trigger, etc.
The module could handle the mixing, but it occurs to me that, looking ahead,
it might be better to push a lot of that work out to another tech like
OpenAL. After all, at some point, some clients might be connecting with
48Khz stereo audio. OpenAL supports positional audio and effects, in
addition to raw mixing. OpenAL is available for several platforms, but not
everywhere. Perhaps it would be more important to be widely available, at
the cost of features.
Thoughts? Is a module the right way to go? Is using OpenAL a good idea, or
should the module handle the mixing?
Bryan
Post by broken dash
I have been wanting these features and I found this...liquidsoap,
check out the transcoding section. I was trying to play multiple
shoutcast audio streams and hopefully mux them together nicely, etc..
haven't gotten around to setting freeswitch to utilize it but I'm sure
it wouldn't be hard.
http://savonet.sourceforge.net/doc-1.0.0-beta2/cookbook.html
Cheers,
Brian
Post by Jan Berger
If you play several files FS will queue them if it is the same stream -
you
Post by broken dash
Post by Jan Berger
need to somehow set up a conference with multiple streams (callers).
I would check options around music on hold.
But, another way is to use FS to play several outgoing SIP streams and
loop
Post by broken dash
Post by Jan Berger
them back into a conference.
Jan
-----Original Message-----
From: freeswitch-users-bounces at lists.freeswitch.org
[mailto:freeswitch-users-bounces at lists.freeswitch.org] On Behalf Of Bryan
Smart
Sent: 1. juli 2011 07:13
To: FreeSWITCH Users Help
Subject: [Freeswitch-users] Playing multiple files simultaneously
Is it possible for Freeswitch to play more than one file to a channel at
a
Post by broken dash
Post by Jan Berger
time? What I've seen and tried from the dialplan and scripts either
queues
Post by broken dash
Post by Jan Berger
files to play, or will stop a currently playing file so that the newly
requested file will play. This also seems to be the case in conferences.
When I send multiple play commands to conferences, the files are queued.
As for how this might be used, think of an IVR that plays queued prompts,
yet continuously plays looping music or a Shoutcast stream in the
background. I also want to be able to play short cue tones that start at
the
Post by broken dash
Post by Jan Berger
same time as a prompt (don't want to pre-mix them in to a single file,
though).
Is this currently possible through any means? Perhaps with the event
socket?
Post by broken dash
Post by Jan Berger
Bryan
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE

FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Steve Underwood
2011-07-07 01:11:14 UTC
Permalink
Post by Bryan Smart
Is it possible for Freeswitch to play more than one file to a channel at a time? What I've seen and tried from the dialplan and scripts either queues files to play, or will stop a currently playing file so that the newly requested file will play. This also seems to be the case in conferences. When I send multiple play commands to conferences, the files are queued.
As for how this might be used, think of an IVR that plays queued prompts, yet continuously plays looping music or a Shoutcast stream in the background. I also want to be able to play short cue tones that start at the same time as a prompt (don't want to pre-mix them in to a single file, though).
Is this currently possible through any means? Perhaps with the event socket?
Be really careful making plans like this. Multiple sounds will mix
nicely with something like ALaw or uLaw. If you try it with a low bit
rate codec, like G.729 or a cellphone codec, the result will sound
awful. Modern telephony really is geared to one voice and one voice only
at any moment.

Steve
Bryan Smart
2011-07-07 16:14:07 UTC
Permalink
Thanks for the warning, Steve.

The cues are short, and I'm trying to keep them thin (simple waveforms based on multifrequency sine waves for tone, and short filtered white noise where clicks or percussive transients are required). I'm not an expert on low bit rate audio, but, from the bit that I know, I think that simple audio cues like this will mix better than sampled musical instruments playing riffs, real world sounds, and so on. Sounds alright on GSM, but I haven't tested with G729 yet. Do you think this will work for me?

I'm not actually planning an overlapping cacophony of sound. The cues are short, but I'd like them to be able to play in the background while text to speech or a recorded prompt is playing. I don't anticipate lots of them playing at once, but I wanted the mixing capability so that they could play under the speech, and so that, when two play close in a row, the second one wouldn't chop off the end of the first.

Do you know if it is better to try to extend existing commands, like playback, or if it would be better to have a separate module and commands for playing audio in this way? I realize that mixing would introduce at least a small overhead, even if all files and the channel were same format, and most people would want to avoid that little performance hit unless they specifically needed the mixing capability.

I feel cautious about poking my nose in to programming issues here. I've been programming to one degree or another for 25 years, and even had it as my main gig for several years, but all of my experience has either been personal projects, or closed source commercial development for an employer. I don't have my head around people and technical processes for contributing to an open source project, so even tweaking a single line in a source file seems to involve a lot of process in order to submit it. I know that the group uses git for version control, but, if I make a tweak, do I submit it to some sort of approval branch first? If their is a processes doc for the group, I'd be glad to read it.

Thanks
Bryan
Post by Steve Underwood
Post by Bryan Smart
Is it possible for Freeswitch to play more than one file to a channel at a time? What I've seen and tried from the dialplan and scripts either queues files to play, or will stop a currently playing file so that the newly requested file will play. This also seems to be the case in conferences. When I send multiple play commands to conferences, the files are queued.
As for how this might be used, think of an IVR that plays queued prompts, yet continuously plays looping music or a Shoutcast stream in the background. I also want to be able to play short cue tones that start at the same time as a prompt (don't want to pre-mix them in to a single file, though).
Is this currently possible through any means? Perhaps with the event socket?
Be really careful making plans like this. Multiple sounds will mix
nicely with something like ALaw or uLaw. If you try it with a low bit
rate codec, like G.729 or a cellphone codec, the result will sound
awful. Modern telephony really is geared to one voice and one voice only
at any moment.
Steve
_______________________________________________
Join us at ClueCon 2011, Aug 9-11, Chicago
http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Jan Berger
2011-07-07 21:31:25 UTC
Permalink
Get on to the dev list and make suggestions - maybe make shorter mails :) -
my own experience is that if I post long mails no-one answers, but if I
spilt them up in smaller I get plenty with response.

/Jan

I feel cautious about poking my nose in to programming issues here. I've
been programming to one degree or another for 25 years, and even had it as
my main gig for several years, but all of my experience has either been
personal projects, or closed source commercial development for an employer.
I don't have my head around people and technical processes for contributing
to an open source project, so even tweaking a single line in a source file
seems to involve a lot of process in order to submit it. I know that the
group uses git for version control, but, if I make a tweak, do I submit it
to some sort of approval branch first? If their is a processes doc for the
group, I'd be glad to read it.

Thanks
Bryan

Loading...