Discussion:
[Freeswitch-users] Choppy sound with PCMU
eaf
2009-12-01 21:51:42 UTC
Permalink
Hi,

I'm trying to migrate from Asterisk to FreeSWITCH (really like the way how
it can be programmed), but ran into one issue with sound quality that I just
cannot workaround by myself. I would describe the sound problem as being
"choppy". From time to time small portions of the other party's voice are
dropped, so the voice kind of stutters. This is not too bad, but is really
noticeable, happens in every call and I don't experience the same with
Asterisk running on the same box. I attached two files: freeswitch.wav and
asterisk.mp3 to illustrate my point.

Issue completely goes away, if I set inbound-proxy-media to true.

The way how I test is to connect SPA-2000 via 10mbps LAN to the box directly
exposed to internet, and then dial a toll-free via FutureNine (a SIP
provider).

The codec in use is PCMU. Can't really try PCMA or anything else with this
provider. Only PCMU. Tried to match ptime of provider (30) with ptime of the
SPA, didn't get any improvement. Tried turning off recording, no change
either.

What puzzles me is that even with greedy codec negotiations and with PCMU on
both sides of FreeSWITCH, it's still saying that TRANSCODING_NECESSARY. I'm
attaching relevant portion of freeswitch.log to illustrate.

The box isn't particularly fast: Linux (Debian 4), CPU - AMD Geode LX800
with 997 bogomips. 256MB RAM. Only one call in progress, so I hope that it's
not a performance issue.

http://old.nabble.com/file/p26594250/freeswitch.wav freeswitch.wav

http://old.nabble.com/file/p26594250/asterisk.mp3 asterisk.mp3

http://old.nabble.com/file/p26594250/freeswitch.log freeswitch.log

Tried both 1.0.4 and 1.0.5pre5. Same results.

What should I do next? Calls are consistently bad with FreeSWITCH, and
consistently show no glitches with Asterisk.
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26594250.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
eaf
2009-12-01 21:52:03 UTC
Permalink
I should also add, after browsing through some topics here, that my SIP
provider sends 172-byte RTP frames, which is in accordance with ptime:20
that it gives to FreeSWITCH.
Post by eaf
Hi,
I'm trying to migrate from Asterisk to FreeSWITCH (really like the way how
it can be programmed), but ran into one issue with sound quality that I
just cannot workaround by myself. I would describe the sound problem as
being "choppy". From time to time small portions of the other party's
voice are dropped, so the voice kind of stutters. This is not too bad, but
is really noticeable, happens in every call and I don't experience the
freeswitch.wav and asterisk.mp3 to illustrate my point.
Issue completely goes away, if I set inbound-proxy-media to true.
The way how I test is to connect SPA-2000 via 10mbps LAN to the box
directly exposed to internet, and then dial a toll-free via FutureNine (a
SIP provider).
The codec in use is PCMU. Can't really try PCMA or anything else with this
provider. Only PCMU. Tried to match ptime of provider (30) with ptime of
the SPA, didn't get any improvement. Tried turning off recording, no
change either.
What puzzles me is that even with greedy codec negotiations and with PCMU
on both sides of FreeSWITCH, it's still saying that
TRANSCODING_NECESSARY. I'm attaching relevant portion of freeswitch.log to
illustrate.
The box isn't particularly fast: Linux (Debian 4), CPU - AMD Geode LX800
with 997 bogomips. 256MB RAM. Only one call in progress, so I hope that
it's not a performance issue.
http://old.nabble.com/file/p26594250/freeswitch.wav freeswitch.wav
http://old.nabble.com/file/p26594250/asterisk.mp3 asterisk.mp3
http://old.nabble.com/file/p26594250/freeswitch.log freeswitch.log
Tried both 1.0.4 and 1.0.5pre5. Same results.
What should I do next? Calls are consistently bad with FreeSWITCH, and
consistently show no glitches with Asterisk.
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26599565.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Anthony Minessale
2009-12-01 22:29:26 UTC
Permalink
linksys has had a bug for eons that can be fixed by setting the ptime (or
rtp packet size in their terms)
in it's firmware to .20 instead of .30

Asterisk does not use async RTP like we do so it's never a problem
you can disable the timer by setting the channel var rtp_timer_name=none or
sofia param rtp-timer-name to none in the sofia profile.

You should also test this on latest SVN trunk or wait for pre8
Post by eaf
I should also add, after browsing through some topics here, that my SIP
provider sends 172-byte RTP frames, which is in accordance with ptime:20
that it gives to FreeSWITCH.
Post by eaf
Hi,
I'm trying to migrate from Asterisk to FreeSWITCH (really like the way
how
Post by eaf
it can be programmed), but ran into one issue with sound quality that I
just cannot workaround by myself. I would describe the sound problem as
being "choppy". From time to time small portions of the other party's
voice are dropped, so the voice kind of stutters. This is not too bad,
but
Post by eaf
is really noticeable, happens in every call and I don't experience the
freeswitch.wav and asterisk.mp3 to illustrate my point.
Issue completely goes away, if I set inbound-proxy-media to true.
The way how I test is to connect SPA-2000 via 10mbps LAN to the box
directly exposed to internet, and then dial a toll-free via FutureNine (a
SIP provider).
The codec in use is PCMU. Can't really try PCMA or anything else with
this
Post by eaf
provider. Only PCMU. Tried to match ptime of provider (30) with ptime of
the SPA, didn't get any improvement. Tried turning off recording, no
change either.
What puzzles me is that even with greedy codec negotiations and with PCMU
on both sides of FreeSWITCH, it's still saying that
TRANSCODING_NECESSARY. I'm attaching relevant portion of freeswitch.log
to
Post by eaf
illustrate.
The box isn't particularly fast: Linux (Debian 4), CPU - AMD Geode LX800
with 997 bogomips. 256MB RAM. Only one call in progress, so I hope that
it's not a performance issue.
http://old.nabble.com/file/p26594250/freeswitch.wav freeswitch.wav
http://old.nabble.com/file/p26594250/asterisk.mp3 asterisk.mp3
http://old.nabble.com/file/p26594250/freeswitch.log freeswitch.log
Tried both 1.0.4 and 1.0.5pre5. Same results.
What should I do next? Calls are consistently bad with FreeSWITCH, and
consistently show no glitches with Asterisk.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26599565.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091201/09f705a3/attachment.html
Anthony Minessale
2009-12-01 22:29:26 UTC
Permalink
linksys has had a bug for eons that can be fixed by setting the ptime (or
rtp packet size in their terms)
in it's firmware to .20 instead of .30

Asterisk does not use async RTP like we do so it's never a problem
you can disable the timer by setting the channel var rtp_timer_name=none or
sofia param rtp-timer-name to none in the sofia profile.

You should also test this on latest SVN trunk or wait for pre8
Post by eaf
I should also add, after browsing through some topics here, that my SIP
provider sends 172-byte RTP frames, which is in accordance with ptime:20
that it gives to FreeSWITCH.
Post by eaf
Hi,
I'm trying to migrate from Asterisk to FreeSWITCH (really like the way
how
Post by eaf
it can be programmed), but ran into one issue with sound quality that I
just cannot workaround by myself. I would describe the sound problem as
being "choppy". From time to time small portions of the other party's
voice are dropped, so the voice kind of stutters. This is not too bad,
but
Post by eaf
is really noticeable, happens in every call and I don't experience the
freeswitch.wav and asterisk.mp3 to illustrate my point.
Issue completely goes away, if I set inbound-proxy-media to true.
The way how I test is to connect SPA-2000 via 10mbps LAN to the box
directly exposed to internet, and then dial a toll-free via FutureNine (a
SIP provider).
The codec in use is PCMU. Can't really try PCMA or anything else with
this
Post by eaf
provider. Only PCMU. Tried to match ptime of provider (30) with ptime of
the SPA, didn't get any improvement. Tried turning off recording, no
change either.
What puzzles me is that even with greedy codec negotiations and with PCMU
on both sides of FreeSWITCH, it's still saying that
TRANSCODING_NECESSARY. I'm attaching relevant portion of freeswitch.log
to
Post by eaf
illustrate.
The box isn't particularly fast: Linux (Debian 4), CPU - AMD Geode LX800
with 997 bogomips. 256MB RAM. Only one call in progress, so I hope that
it's not a performance issue.
http://old.nabble.com/file/p26594250/freeswitch.wav freeswitch.wav
http://old.nabble.com/file/p26594250/asterisk.mp3 asterisk.mp3
http://old.nabble.com/file/p26594250/freeswitch.log freeswitch.log
Tried both 1.0.4 and 1.0.5pre5. Same results.
What should I do next? Calls are consistently bad with FreeSWITCH, and
consistently show no glitches with Asterisk.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26599565.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091201/09f705a3/attachment-0002.html
erandr-junk
2009-12-02 01:26:58 UTC
Permalink
Thanks. I tried that... Just forcing SPA to 20ms didn't change anything. Just
installing SVN trunk didn't fix it either, but setting that option afterwards
surely did the trick.

One thing I've noticed while staring at the console is that it *looks like*
that w/o the new setting the stuttering happens when FS either re-registers
itself with the provider or one of the SPA's port re-registers with FS.

------ Original Message ------
Received: Tue, 01 Dec 2009 05:33:26 PM EST
From: Anthony Minessale <anthony.minessale at gmail.com>
To: freeswitch-users at lists.freeswitch.org
Subject: Re: [Freeswitch-users] Choppy sound with PCMU
Post by Anthony Minessale
linksys has had a bug for eons that can be fixed by setting the ptime (or
rtp packet size in their terms)
in it's firmware to .20 instead of .30
Asterisk does not use async RTP like we do so it's never a problem
you can disable the timer by setting the channel var rtp_timer_name=none or
sofia param rtp-timer-name to none in the sofia profile.
You should also test this on latest SVN trunk or wait for pre8
Post by eaf
I should also add, after browsing through some topics here, that my SIP
provider sends 172-byte RTP frames, which is in accordance with ptime:20
that it gives to FreeSWITCH.
Post by eaf
Hi,
I'm trying to migrate from Asterisk to FreeSWITCH (really like the way
how
Post by eaf
it can be programmed), but ran into one issue with sound quality that I
just cannot workaround by myself. I would describe the sound problem as
being "choppy". From time to time small portions of the other party's
voice are dropped, so the voice kind of stutters. This is not too bad,
but
Post by eaf
is really noticeable, happens in every call and I don't experience the
freeswitch.wav and asterisk.mp3 to illustrate my point.
Issue completely goes away, if I set inbound-proxy-media to true.
The way how I test is to connect SPA-2000 via 10mbps LAN to the box
directly exposed to internet, and then dial a toll-free via FutureNine
(a
Post by Anthony Minessale
Post by eaf
Post by eaf
SIP provider).
The codec in use is PCMU. Can't really try PCMA or anything else with
this
Post by eaf
provider. Only PCMU. Tried to match ptime of provider (30) with ptime
of
Post by Anthony Minessale
Post by eaf
Post by eaf
the SPA, didn't get any improvement. Tried turning off recording, no
change either.
What puzzles me is that even with greedy codec negotiations and with
PCMU
Post by Anthony Minessale
Post by eaf
Post by eaf
on both sides of FreeSWITCH, it's still saying that
TRANSCODING_NECESSARY. I'm attaching relevant portion of freeswitch.log
to
Post by eaf
illustrate.
The box isn't particularly fast: Linux (Debian 4), CPU - AMD Geode
LX800
Post by Anthony Minessale
Post by eaf
Post by eaf
with 997 bogomips. 256MB RAM. Only one call in progress, so I hope that
it's not a performance issue.
http://old.nabble.com/file/p26594250/freeswitch.wav freeswitch.wav
http://old.nabble.com/file/p26594250/asterisk.mp3 asterisk.mp3
http://old.nabble.com/file/p26594250/freeswitch.log freeswitch.log
Tried both 1.0.4 and 1.0.5pre5. Same results.
What should I do next? Calls are consistently bad with FreeSWITCH, and
consistently show no glitches with Asterisk.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26599565.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
erandr-junk
2009-12-02 03:19:39 UTC
Permalink
Wow... Thinking about this timer setting and about how it converted
send()/recv() from non-blocking to blocking, I straced freeswitch when it was
supposed to be idle. It never pauses! It keeps going in and out of select()
every millisecond! Why??

------ Original Message ------
Received: Tue, 01 Dec 2009 08:31:46 PM EST
From: erandr-junk at usa.net
To: <freeswitch-users at lists.freeswitch.org>
Subject: Re: [Freeswitch-users] Choppy sound with PCMU
Post by erandr-junk
Thanks. I tried that... Just forcing SPA to 20ms didn't change anything.
Just
Post by erandr-junk
installing SVN trunk didn't fix it either, but setting that option
afterwards
Post by erandr-junk
surely did the trick.
One thing I've noticed while staring at the console is that it *looks like*
that w/o the new setting the stuttering happens when FS either re-registers
itself with the provider or one of the SPA's port re-registers with FS.
------ Original Message ------
Received: Tue, 01 Dec 2009 05:33:26 PM EST
From: Anthony Minessale <anthony.minessale at gmail.com>
To: freeswitch-users at lists.freeswitch.org
Subject: Re: [Freeswitch-users] Choppy sound with PCMU
Post by Anthony Minessale
linksys has had a bug for eons that can be fixed by setting the ptime (or
rtp packet size in their terms)
in it's firmware to .20 instead of .30
Asterisk does not use async RTP like we do so it's never a problem
you can disable the timer by setting the channel var rtp_timer_name=none
or
Post by erandr-junk
Post by Anthony Minessale
sofia param rtp-timer-name to none in the sofia profile.
You should also test this on latest SVN trunk or wait for pre8
Post by eaf
I should also add, after browsing through some topics here, that my SIP
provider sends 172-byte RTP frames, which is in accordance with
ptime:20
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
that it gives to FreeSWITCH.
Post by eaf
Hi,
I'm trying to migrate from Asterisk to FreeSWITCH (really like the
way
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
how
Post by eaf
it can be programmed), but ran into one issue with sound quality that
I
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
just cannot workaround by myself. I would describe the sound problem
as
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
being "choppy". From time to time small portions of the other party's
voice are dropped, so the voice kind of stutters. This is not too
bad,
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
but
Post by eaf
is really noticeable, happens in every call and I don't experience
the
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
freeswitch.wav and asterisk.mp3 to illustrate my point.
Issue completely goes away, if I set inbound-proxy-media to true.
The way how I test is to connect SPA-2000 via 10mbps LAN to the box
directly exposed to internet, and then dial a toll-free via
FutureNine
Post by erandr-junk
(a
Post by Anthony Minessale
Post by eaf
Post by eaf
SIP provider).
The codec in use is PCMU. Can't really try PCMA or anything else with
this
Post by eaf
provider. Only PCMU. Tried to match ptime of provider (30) with ptime
of
Post by Anthony Minessale
Post by eaf
Post by eaf
the SPA, didn't get any improvement. Tried turning off recording, no
change either.
What puzzles me is that even with greedy codec negotiations and with
PCMU
Post by Anthony Minessale
Post by eaf
Post by eaf
on both sides of FreeSWITCH, it's still saying that
TRANSCODING_NECESSARY. I'm attaching relevant portion of
freeswitch.log
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
to
Post by eaf
illustrate.
The box isn't particularly fast: Linux (Debian 4), CPU - AMD Geode
LX800
Post by Anthony Minessale
Post by eaf
Post by eaf
with 997 bogomips. 256MB RAM. Only one call in progress, so I hope
that
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
it's not a performance issue.
http://old.nabble.com/file/p26594250/freeswitch.wav freeswitch.wav
http://old.nabble.com/file/p26594250/asterisk.mp3 asterisk.mp3
http://old.nabble.com/file/p26594250/freeswitch.log freeswitch.log
Tried both 1.0.4 and 1.0.5pre5. Same results.
What should I do next? Calls are consistently bad with FreeSWITCH,
and
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
consistently show no glitches with Asterisk.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26599565.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
Post by erandr-junk
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
Post by erandr-junk
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Anthony Minessale
2009-12-02 18:32:21 UTC
Permalink
idle is a 4 letter word to a realtime application.

The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.

Your choppy audio is caused by linksys lying about the packet len that it's
using and we set our timer
to the wrong speed.
Post by erandr-junk
Wow... Thinking about this timer setting and about how it converted
send()/recv() from non-blocking to blocking, I straced freeswitch when it was
supposed to be idle. It never pauses! It keeps going in and out of select()
every millisecond! Why??
------ Original Message ------
Received: Tue, 01 Dec 2009 08:31:46 PM EST
From: erandr-junk at usa.net
To: <freeswitch-users at lists.freeswitch.org>
Subject: Re: [Freeswitch-users] Choppy sound with PCMU
Post by erandr-junk
Thanks. I tried that... Just forcing SPA to 20ms didn't change anything.
Just
Post by erandr-junk
installing SVN trunk didn't fix it either, but setting that option
afterwards
Post by erandr-junk
surely did the trick.
One thing I've noticed while staring at the console is that it *looks
like*
Post by erandr-junk
that w/o the new setting the stuttering happens when FS either
re-registers
Post by erandr-junk
itself with the provider or one of the SPA's port re-registers with FS.
------ Original Message ------
Received: Tue, 01 Dec 2009 05:33:26 PM EST
From: Anthony Minessale <anthony.minessale at gmail.com>
To: freeswitch-users at lists.freeswitch.org
Subject: Re: [Freeswitch-users] Choppy sound with PCMU
Post by Anthony Minessale
linksys has had a bug for eons that can be fixed by setting the ptime
(or
Post by erandr-junk
Post by Anthony Minessale
rtp packet size in their terms)
in it's firmware to .20 instead of .30
Asterisk does not use async RTP like we do so it's never a problem
you can disable the timer by setting the channel var
rtp_timer_name=none
or
Post by erandr-junk
Post by Anthony Minessale
sofia param rtp-timer-name to none in the sofia profile.
You should also test this on latest SVN trunk or wait for pre8
Post by eaf
I should also add, after browsing through some topics here, that my
SIP
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
provider sends 172-byte RTP frames, which is in accordance with
ptime:20
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
that it gives to FreeSWITCH.
Post by eaf
Hi,
I'm trying to migrate from Asterisk to FreeSWITCH (really like the
way
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
how
Post by eaf
it can be programmed), but ran into one issue with sound quality
that
I
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
just cannot workaround by myself. I would describe the sound
problem
as
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
being "choppy". From time to time small portions of the other
party's
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
voice are dropped, so the voice kind of stutters. This is not too
bad,
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
but
Post by eaf
is really noticeable, happens in every call and I don't experience
the
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
freeswitch.wav and asterisk.mp3 to illustrate my point.
Issue completely goes away, if I set inbound-proxy-media to true.
The way how I test is to connect SPA-2000 via 10mbps LAN to the box
directly exposed to internet, and then dial a toll-free via
FutureNine
Post by erandr-junk
(a
Post by Anthony Minessale
Post by eaf
Post by eaf
SIP provider).
The codec in use is PCMU. Can't really try PCMA or anything else
with
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
this
Post by eaf
provider. Only PCMU. Tried to match ptime of provider (30) with
ptime
Post by erandr-junk
of
Post by Anthony Minessale
Post by eaf
Post by eaf
the SPA, didn't get any improvement. Tried turning off recording,
no
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
change either.
What puzzles me is that even with greedy codec negotiations and
with
Post by erandr-junk
PCMU
Post by Anthony Minessale
Post by eaf
Post by eaf
on both sides of FreeSWITCH, it's still saying that
TRANSCODING_NECESSARY. I'm attaching relevant portion of
freeswitch.log
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
to
Post by eaf
illustrate.
The box isn't particularly fast: Linux (Debian 4), CPU - AMD Geode
LX800
Post by Anthony Minessale
Post by eaf
Post by eaf
with 997 bogomips. 256MB RAM. Only one call in progress, so I hope
that
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
it's not a performance issue.
http://old.nabble.com/file/p26594250/freeswitch.wavfreeswitch.wav
http://old.nabble.com/file/p26594250/asterisk.mp3 asterisk.mp3
http://old.nabble.com/file/p26594250/freeswitch.logfreeswitch.log
Tried both 1.0.4 and 1.0.5pre5. Same results.
What should I do next? Calls are consistently bad with FreeSWITCH,
and
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
consistently show no glitches with Asterisk.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26599565.html
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com<MSN%3Aanthony_minessale at hotmail.com><
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by erandr-junk
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org<sip%3A888 at conference.freeswitch.org><
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
Post by erandr-junk
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by erandr-junk
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by erandr-junk
Post by Anthony Minessale
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091202/a320b89e/attachment.html
eaf
2009-12-03 00:31:31 UTC
Permalink
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?

That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-friendly? I.e.
make it stuck in select() or recv() when there is nothing to do, call
clock_gettime() right from the thread that wants and when it wants to know
current time?

Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some other
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-between
the calls, when there is no RTP or IVR, it will all go silent? I mean,
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len that it's
using and we set our timer
to the wrong speed.
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
eaf
2009-12-03 03:35:39 UTC
Permalink
Oh, looks like the timers are also used for streaming local data in
read_stream_thread(). Due to this there is always one timer active with 20ms
interval.

But wait a sec, why is freeswitch periodically trying to stream
/opt/freeswitch/sounds/music/8000/ponce-preludio-in-e-major.wav somewhere?
Every minute or so? Did I misconfigure it?
Post by eaf
Say, what if that thread is made to suspend on a condition variable in
case if there are no timers registered in TIMER_MATRIX? Then, if some
other thread comes up and adds its timer into the matrix, it could wake up
the timer thread and enjoy accurate timing as needed, on demand? And
in-between the calls, when there is no RTP or IVR, it will all go silent?
I mean, sitting on a wait queue in the kernel is way better than go back
and forth incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len that it's
using and we set our timer
to the wrong speed.
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26620518.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
eaf
2009-12-03 03:47:42 UTC
Permalink
OK, I'm slow. It's music-on-hold, and it's playing non-stop like that timer
thread. Even when there are no calls. Why?
Post by eaf
Oh, looks like the timers are also used for streaming local data in
read_stream_thread(). Due to this there is always one timer active with
20ms interval.
But wait a sec, why is freeswitch periodically trying to stream
/opt/freeswitch/sounds/music/8000/ponce-preludio-in-e-major.wav somewhere?
Every minute or so? Did I misconfigure it?
Post by eaf
Say, what if that thread is made to suspend on a condition variable in
case if there are no timers registered in TIMER_MATRIX? Then, if some
other thread comes up and adds its timer into the matrix, it could wake
up the timer thread and enjoy accurate timing as needed, on demand? And
in-between the calls, when there is no RTP or IVR, it will all go silent?
I mean, sitting on a wait queue in the kernel is way better than go back
and forth incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len that it's
using and we set our timer
to the wrong speed.
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26620610.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Michael Jerris
2009-12-03 03:57:06 UTC
Permalink
This is keeping track of a place in the music on hold so your hold
music does not start back up at the same place every time. If you
don't want to do this it is a module that you don't need to load and
you can get your moh from any soundfile at your choice in configuration.

Mike
Post by eaf
Oh, looks like the timers are also used for streaming local data in
read_stream_thread(). Due to this there is always one timer active with 20ms
interval.
But wait a sec, why is freeswitch periodically trying to stream
/opt/freeswitch/sounds/music/8000/ponce-preludio-in-e-major.wav somewhere?
Every minute or so? Did I misconfigure it?
Post by eaf
Say, what if that thread is made to suspend on a condition variable in
case if there are no timers registered in TIMER_MATRIX? Then, if some
other thread comes up and adds its timer into the matrix, it could wake up
the timer thread and enjoy accurate timing as needed, on demand? And
in-between the calls, when there is no RTP or IVR, it will all go silent?
I mean, sitting on a wait queue in the kernel is way better than go back
and forth incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26620518.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
eaf
2009-12-03 03:47:42 UTC
Permalink
OK, I'm slow. It's music-on-hold, and it's playing non-stop like that timer
thread. Even when there are no calls. Why?
Post by eaf
Oh, looks like the timers are also used for streaming local data in
read_stream_thread(). Due to this there is always one timer active with
20ms interval.
But wait a sec, why is freeswitch periodically trying to stream
/opt/freeswitch/sounds/music/8000/ponce-preludio-in-e-major.wav somewhere?
Every minute or so? Did I misconfigure it?
Post by eaf
Say, what if that thread is made to suspend on a condition variable in
case if there are no timers registered in TIMER_MATRIX? Then, if some
other thread comes up and adds its timer into the matrix, it could wake
up the timer thread and enjoy accurate timing as needed, on demand? And
in-between the calls, when there is no RTP or IVR, it will all go silent?
I mean, sitting on a wait queue in the kernel is way better than go back
and forth incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len that it's
using and we set our timer
to the wrong speed.
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26620610.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Michael Jerris
2009-12-03 03:57:06 UTC
Permalink
This is keeping track of a place in the music on hold so your hold
music does not start back up at the same place every time. If you
don't want to do this it is a module that you don't need to load and
you can get your moh from any soundfile at your choice in configuration.

Mike
Post by eaf
Oh, looks like the timers are also used for streaming local data in
read_stream_thread(). Due to this there is always one timer active with 20ms
interval.
But wait a sec, why is freeswitch periodically trying to stream
/opt/freeswitch/sounds/music/8000/ponce-preludio-in-e-major.wav somewhere?
Every minute or so? Did I misconfigure it?
Post by eaf
Say, what if that thread is made to suspend on a condition variable in
case if there are no timers registered in TIMER_MATRIX? Then, if some
other thread comes up and adds its timer into the matrix, it could wake up
the timer thread and enjoy accurate timing as needed, on demand? And
in-between the calls, when there is no RTP or IVR, it will all go silent?
I mean, sitting on a wait queue in the kernel is way better than go back
and forth incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26620518.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
Michael Jerris
2009-12-03 04:00:00 UTC
Permalink
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You can
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.

Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do, call
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some other
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-
between
the calls, when there is no RTP or IVR, it will all go silent? I mean,
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
eaf
2009-12-03 04:58:39 UTC
Permalink
As I see it, switch_cond_next() currently is just a do_sleep(1000). Yes, it
could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND" overrides
that.

Yeah, there is a global timestamp... It's easy to workaround that for RTP
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that. If only
this was C++...

I'll play around. Never liked polling too much. Never could've guessed that
polling could be so useful for scalability ;) My naive implementation
would've pulled timestamp via system calls and would've done sleeping by
passing exact interval to select() instead of syncing with a pacing thread.
Which would be dead-quiet at idle time, but, of course, would stop scaling
at some point due to excessive number of system calls.

Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You can
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do, call
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some other
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-
between
the calls, when there is no RTP or IVR, it will all go silent? I mean,
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26621005.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Michael Jerris
2009-12-03 13:48:55 UTC
Permalink
First off, maybe this conversation is better suited to the dev list, and second off, the current setup of where we do timers, where we poll, polling frequency and architecture is the result of 4+ years of ongoing testing and optimization. We have tried all different methods throughout. Sometimes what we found to be most efficient is not what we thought at first would be, but testing showed otherwise. We have always optimized the general case as to if there are many calls, and no suggestion would be implemented that hurts this case. That being said, if you could really come up with a way for this to be more efficient in any case, without sacrificing performance int he other cases, you are able to prove this with extensive test results, and you are able to prove that it does not impact for example call quality in any of the hundreds of edge cases that have led us to the point we are now, then we may be interested in taking such a patch.

Mike
Post by eaf
As I see it, switch_cond_next() currently is just a do_sleep(1000). Yes, it
could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND" overrides
that.
Yeah, there is a global timestamp... It's easy to workaround that for RTP
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that. If only
this was C++...
I'll play around. Never liked polling too much. Never could've guessed that
polling could be so useful for scalability ;) My naive implementation
would've pulled timestamp via system calls and would've done sleeping by
passing exact interval to select() instead of syncing with a pacing thread.
Which would be dead-quiet at idle time, but, of course, would stop scaling
at some point due to excessive number of system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You can
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do, call
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some other
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-
between
the calls, when there is no RTP or IVR, it will all go silent? I mean,
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26621005.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
eaf
2009-12-03 14:17:54 UTC
Permalink
Oh, it's not just one timer thread... Why, why is sql_thread keeps on
checking for messages every millisecond? Couldn't there be some signalling
implemented that will make the thread suspend on condition variable or a
socket/pipe in between?

#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0) at
src/switch_core_sqldb.c:783

Why does this sofia_profile_worker_thread keeps on looping checking for the
queue? Have a semaphore!

#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30,
obj=0x80f2490) at sofia.c:978

Nothing's happening on the box, but there are three threads that pretend to
be actively busy with smth. Others at least sleep for hundreds of
milliseconds, not for one.

And there is even infrastructure present to do blocking pops: i.e. why
couldn't sqldb thread do queue_pop() instead of queue_trypop() intermixed
with 1ms sleeps? This looping is such a waste...
Post by eaf
As I see it, switch_cond_next() currently is just a do_sleep(1000). Yes,
it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND"
overrides that.
Yeah, there is a global timestamp... It's easy to workaround that for RTP
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that. If only
this was C++...
I'll play around. Never liked polling too much. Never could've guessed
that polling could be so useful for scalability ;) My naive implementation
would've pulled timestamp via system calls and would've done sleeping by
passing exact interval to select() instead of syncing with a pacing
thread. Which would be dead-quiet at idle time, but, of course, would stop
scaling at some point due to excessive number of system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You can
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do, call
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some other
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-
between
the calls, when there is no RTP or IVR, it will all go silent? I mean,
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26626634.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
eaf
2009-12-03 14:55:21 UTC
Permalink
Btw, I have these popping up in my logs from time to time:

2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314
(sofia/external/xxxxx at 4.68.250.148) Running State Change CS_HANGUP
2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration
Detected! Syncing Clock

In this case an incoming call rang to both FS and Asterisk, Asterisk picked
up, but the surge of activity made FS timer thread miss a beat or two.
Post by eaf
Oh, it's not just one timer thread... Why, why is sql_thread keeps on
checking for messages every millisecond? Couldn't there be some signalling
implemented that will make the thread suspend on condition variable or a
socket/pipe in between?
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0) at
src/switch_core_sqldb.c:783
Why does this sofia_profile_worker_thread keeps on looping checking for
the queue? Have a semaphore!
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30,
obj=0x80f2490) at sofia.c:978
Nothing's happening on the box, but there are three threads that pretend
to be actively busy with smth. Others at least sleep for hundreds of
milliseconds, not for one.
And there is even infrastructure present to do blocking pops: i.e. why
couldn't sqldb thread do queue_pop() instead of queue_trypop() intermixed
with 1ms sleeps? This looping is such a waste...
Post by eaf
As I see it, switch_cond_next() currently is just a do_sleep(1000). Yes,
it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND"
overrides that.
Yeah, there is a global timestamp... It's easy to workaround that for RTP
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that. If only
this was C++...
I'll play around. Never liked polling too much. Never could've guessed
that polling could be so useful for scalability ;) My naive
implementation would've pulled timestamp via system calls and would've
done sleeping by passing exact interval to select() instead of syncing
with a pacing thread. Which would be dead-quiet at idle time, but, of
course, would stop scaling at some point due to excessive number of
system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You can
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do, call
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some other
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-
between
the calls, when there is no RTP or IVR, it will all go silent? I mean,
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Anthony Minessale
2009-12-03 16:16:26 UTC
Permalink
If you see that message then your machine/os/combo is having some problems
keeping up.
It's not the timer missing anything its the monotonic clock detecting a 1
second or more differential from what its next prediction for the time
should be. The best way to trigger this would be to suspend FS with
control-z or attach to it with gdb blocking the entire process, that 1ms
thread would have to miss 1000 iterations to trigger that warning.

Btw, that error message is at line 471 not 473 so you are using modified
code.

Its possible your box has a bad monotonic timer, you can set

<param name="disable-monotonic-timing" value="true"/>

under <settings> in switch.conf.xml

We are now starting to guess you are using some small embedded type platform
perhaps?
I've run FS even on a nokia n810 and never caused that message to fire.

if 1 call can interrupt the cpu enough to cause noticeable issues you might
want to consider running the process at a
greater priority by using the -hp command line arg or at least nice it

Why don't you tell us the whole story about what OS/platform you are using
here rather that form conjectures about what is wrong with our code that
thousands of people are happy with.
Post by eaf
2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314
(sofia/external/xxxxx at 4.68.250.148) Running State Change CS_HANGUP
2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration
Detected! Syncing Clock
In this case an incoming call rang to both FS and Asterisk, Asterisk picked
up, but the surge of activity made FS timer thread miss a beat or two.
Post by eaf
Oh, it's not just one timer thread... Why, why is sql_thread keeps on
checking for messages every millisecond? Couldn't there be some
signalling
Post by eaf
implemented that will make the thread suspend on condition variable or a
socket/pipe in between?
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0) at
src/switch_core_sqldb.c:783
Why does this sofia_profile_worker_thread keeps on looping checking for
the queue? Have a semaphore!
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30,
obj=0x80f2490) at sofia.c:978
Nothing's happening on the box, but there are three threads that pretend
to be actively busy with smth. Others at least sleep for hundreds of
milliseconds, not for one.
And there is even infrastructure present to do blocking pops: i.e. why
couldn't sqldb thread do queue_pop() instead of queue_trypop() intermixed
with 1ms sleeps? This looping is such a waste...
Post by eaf
As I see it, switch_cond_next() currently is just a do_sleep(1000). Yes,
it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND"
overrides that.
Yeah, there is a global timestamp... It's easy to workaround that for
RTP
Post by eaf
Post by eaf
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that. If
only
Post by eaf
Post by eaf
this was C++...
I'll play around. Never liked polling too much. Never could've guessed
that polling could be so useful for scalability ;) My naive
implementation would've pulled timestamp via system calls and would've
done sleeping by passing exact interval to select() instead of syncing
with a pacing thread. Which would be dead-quiet at idle time, but, of
course, would stop scaling at some point due to excessive number of
system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You can
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do, call
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some other
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-
between
the calls, when there is no RTP or IVR, it will all go silent? I mean,
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by eaf
Post by eaf
Post by Michael Jerris
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091203/061f041e/attachment.html
eaf
2009-12-03 17:29:46 UTC
Permalink
I'm sorry if I sounded that way. Did mean to. :)

Yes, it's an embedded platform. It's an ALIX board with AMD Geode LX800 chip
and 256MB of RAM. http://www.pcengines.ch/alix2d3.htm

Line offset difference is due to some minor logging changes I made to see
who's allocating timers and how often. This way I found MOH streaming and
that RTP still allocates timers even when it's set to none in the profile.

I feel that this platform turned out to be underpowered for FS because it
cannot meet its scheduling expectations. I guess, some degree of kernel
tweaking or setting priorities will fix that. Meanwhile I just got rid of
the SQLDB 1ms thread via -nosql command line option, split sofia worker 1ms
thread in two (one blocked and waiting for new commands in the SQL queue,
the other one checking registrations and gateways with 1sec interval), and
don't know yet what to do about the timer thread.

Again, I apologize for stupid or accusing questions, I'm just trying to see
how FS can be made friendlier to this board. Or the board be made friendlier
to FS ;)
Post by Anthony Minessale
If you see that message then your machine/os/combo is having some problems
keeping up.
It's not the timer missing anything its the monotonic clock detecting a 1
second or more differential from what its next prediction for the time
should be. The best way to trigger this would be to suspend FS with
control-z or attach to it with gdb blocking the entire process, that 1ms
thread would have to miss 1000 iterations to trigger that warning.
Btw, that error message is at line 471 not 473 so you are using modified
code.
Its possible your box has a bad monotonic timer, you can set
under <settings> in switch.conf.xml
We are now starting to guess you are using some small embedded type platform
perhaps?
I've run FS even on a nokia n810 and never caused that message to fire.
if 1 call can interrupt the cpu enough to cause noticeable issues you might
want to consider running the process at a
greater priority by using the -hp command line arg or at least nice it
Why don't you tell us the whole story about what OS/platform you are using
here rather that form conjectures about what is wrong with our code that
thousands of people are happy with.
Post by eaf
2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314
(sofia/external/xxxxx at 4.68.250.148) Running State Change CS_HANGUP
2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration
Detected! Syncing Clock
In this case an incoming call rang to both FS and Asterisk, Asterisk picked
up, but the surge of activity made FS timer thread miss a beat or two.
Post by eaf
Oh, it's not just one timer thread... Why, why is sql_thread keeps on
checking for messages every millisecond? Couldn't there be some
signalling
Post by eaf
implemented that will make the thread suspend on condition variable or
a
Post by eaf
socket/pipe in between?
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0)
at
Post by eaf
src/switch_core_sqldb.c:783
Why does this sofia_profile_worker_thread keeps on looping checking for
the queue? Have a semaphore!
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30,
obj=0x80f2490) at sofia.c:978
Nothing's happening on the box, but there are three threads that
pretend
Post by eaf
to be actively busy with smth. Others at least sleep for hundreds of
milliseconds, not for one.
And there is even infrastructure present to do blocking pops: i.e. why
couldn't sqldb thread do queue_pop() instead of queue_trypop()
intermixed
Post by eaf
with 1ms sleeps? This looping is such a waste...
Post by eaf
As I see it, switch_cond_next() currently is just a do_sleep(1000).
Yes,
Post by eaf
Post by eaf
it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND"
overrides that.
Yeah, there is a global timestamp... It's easy to workaround that for
RTP
Post by eaf
Post by eaf
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that. If
only
Post by eaf
Post by eaf
this was C++...
I'll play around. Never liked polling too much. Never could've guessed
that polling could be so useful for scalability ;) My naive
implementation would've pulled timestamp via system calls and would've
done sleeping by passing exact interval to select() instead of syncing
with a pacing thread. Which would be dead-quiet at idle time, but, of
course, would stop scaling at some point due to excessive number of
system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You
can
Post by eaf
Post by eaf
Post by Michael Jerris
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do,
call
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some
other
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-
between
the calls, when there is no RTP or IVR, it will all go silent? I
mean,
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by eaf
Post by eaf
Post by Michael Jerris
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26629856.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Kristian Kielhofner
2009-12-03 17:44:27 UTC
Permalink
I don't think it's the board itself...

We have extensively tested FreeSwitch (no modifications) on that exact
board with AstLinux and have it running at multiple customer
locations.

No timing errors, no warnings or errors of any kind. Pretty standard
really just don't expect too much from the LX800 (transcoding,
resampling, massive numbers of calls, etc).
Post by eaf
I'm sorry if I sounded that way. Did mean to. :)
Yes, it's an embedded platform. It's an ALIX board with AMD Geode LX800 chip
and 256MB of RAM. http://www.pcengines.ch/alix2d3.htm
Line offset difference is due to some minor logging changes I made to see
who's allocating timers and how often. This way I found MOH streaming and
that RTP still allocates timers even when it's set to none in the profile.
I feel that this platform turned out to be underpowered for FS because it
cannot meet its scheduling expectations. I guess, some degree of kernel
tweaking or setting priorities will fix that. Meanwhile I just got rid of
the SQLDB 1ms thread via -nosql command line option, split sofia worker 1ms
thread in two (one blocked and waiting for new commands in the SQL queue,
the other one checking registrations and gateways with 1sec interval), and
don't know yet what to do about the timer thread.
Again, I apologize for stupid or accusing questions, I'm just trying to see
how FS can be made friendlier to this board. Or the board be made friendlier
to FS ;)
--
Kristian Kielhofner
http://www.astlinux.org
http://blog.krisk.org
http://www.star2star.com
http://www.submityoursip.com
http://www.voalte.com
Michael Jerris
2009-12-03 17:50:26 UTC
Permalink
I know people with hardware out there in production based on arm11 and those are pretty small processors, not sure how they compare to this. In regards to the DISABLE_1MS_COND, try getting rid of that, it did increase performance on the high end but may be better for you on the low end with lower compute on idle busy loops.

Mike
Post by eaf
I'm sorry if I sounded that way. Did mean to. :)
Yes, it's an embedded platform. It's an ALIX board with AMD Geode LX800 chip
and 256MB of RAM. http://www.pcengines.ch/alix2d3.htm
Line offset difference is due to some minor logging changes I made to see
who's allocating timers and how often. This way I found MOH streaming and
that RTP still allocates timers even when it's set to none in the profile.
I feel that this platform turned out to be underpowered for FS because it
cannot meet its scheduling expectations. I guess, some degree of kernel
tweaking or setting priorities will fix that. Meanwhile I just got rid of
the SQLDB 1ms thread via -nosql command line option, split sofia worker 1ms
thread in two (one blocked and waiting for new commands in the SQL queue,
the other one checking registrations and gateways with 1sec interval), and
don't know yet what to do about the timer thread.
Again, I apologize for stupid or accusing questions, I'm just trying to see
how FS can be made friendlier to this board. Or the board be made friendlier
to FS ;)
Post by Anthony Minessale
If you see that message then your machine/os/combo is having some problems
keeping up.
It's not the timer missing anything its the monotonic clock detecting a 1
second or more differential from what its next prediction for the time
should be. The best way to trigger this would be to suspend FS with
control-z or attach to it with gdb blocking the entire process, that 1ms
thread would have to miss 1000 iterations to trigger that warning.
Btw, that error message is at line 471 not 473 so you are using modified
code.
Its possible your box has a bad monotonic timer, you can set
under <settings> in switch.conf.xml
We are now starting to guess you are using some small embedded type platform
perhaps?
I've run FS even on a nokia n810 and never caused that message to fire.
if 1 call can interrupt the cpu enough to cause noticeable issues you might
want to consider running the process at a
greater priority by using the -hp command line arg or at least nice it
Why don't you tell us the whole story about what OS/platform you are using
here rather that form conjectures about what is wrong with our code that
thousands of people are happy with.
Post by eaf
2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314
(sofia/external/xxxxx at 4.68.250.148) Running State Change CS_HANGUP
2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration
Detected! Syncing Clock
In this case an incoming call rang to both FS and Asterisk, Asterisk picked
up, but the surge of activity made FS timer thread miss a beat or two.
Post by eaf
Oh, it's not just one timer thread... Why, why is sql_thread keeps on
checking for messages every millisecond? Couldn't there be some
signalling
Post by eaf
implemented that will make the thread suspend on condition variable or
a
Post by eaf
socket/pipe in between?
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0)
at
Post by eaf
src/switch_core_sqldb.c:783
Why does this sofia_profile_worker_thread keeps on looping checking for
the queue? Have a semaphore!
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30,
obj=0x80f2490) at sofia.c:978
Nothing's happening on the box, but there are three threads that
pretend
Post by eaf
to be actively busy with smth. Others at least sleep for hundreds of
milliseconds, not for one.
And there is even infrastructure present to do blocking pops: i.e. why
couldn't sqldb thread do queue_pop() instead of queue_trypop()
intermixed
Post by eaf
with 1ms sleeps? This looping is such a waste...
Post by eaf
As I see it, switch_cond_next() currently is just a do_sleep(1000).
Yes,
Post by eaf
Post by eaf
it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND"
overrides that.
Yeah, there is a global timestamp... It's easy to workaround that for
RTP
Post by eaf
Post by eaf
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that. If
only
Post by eaf
Post by eaf
this was C++...
I'll play around. Never liked polling too much. Never could've guessed
that polling could be so useful for scalability ;) My naive
implementation would've pulled timestamp via system calls and would've
done sleeping by passing exact interval to select() instead of syncing
with a pacing thread. Which would be dead-quiet at idle time, but, of
course, would stop scaling at some point due to excessive number of
system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You
can
Post by eaf
Post by eaf
Post by Michael Jerris
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do,
call
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some
other
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-
between
the calls, when there is no RTP or IVR, it will all go silent? I
mean,
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by eaf
Post by eaf
Post by Michael Jerris
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26629856.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Anthony Minessale
2009-12-03 18:10:05 UTC
Permalink
What about the things I spent time suggesting in my last email?
Did you try them because I was actually curious if they made any impact.
Post by eaf
I'm sorry if I sounded that way. Did mean to. :)
Yes, it's an embedded platform. It's an ALIX board with AMD Geode LX800 chip
and 256MB of RAM. http://www.pcengines.ch/alix2d3.htm
Line offset difference is due to some minor logging changes I made to see
who's allocating timers and how often. This way I found MOH streaming and
that RTP still allocates timers even when it's set to none in the profile.
I feel that this platform turned out to be underpowered for FS because it
cannot meet its scheduling expectations. I guess, some degree of kernel
tweaking or setting priorities will fix that. Meanwhile I just got rid of
the SQLDB 1ms thread via -nosql command line option, split sofia worker 1ms
thread in two (one blocked and waiting for new commands in the SQL queue,
the other one checking registrations and gateways with 1sec interval), and
don't know yet what to do about the timer thread.
Again, I apologize for stupid or accusing questions, I'm just trying to see
how FS can be made friendlier to this board. Or the board be made friendlier
to FS ;)
Post by Anthony Minessale
If you see that message then your machine/os/combo is having some
problems
Post by Anthony Minessale
keeping up.
It's not the timer missing anything its the monotonic clock detecting a 1
second or more differential from what its next prediction for the time
should be. The best way to trigger this would be to suspend FS with
control-z or attach to it with gdb blocking the entire process, that 1ms
thread would have to miss 1000 iterations to trigger that warning.
Btw, that error message is at line 471 not 473 so you are using modified
code.
Its possible your box has a bad monotonic timer, you can set
under <settings> in switch.conf.xml
We are now starting to guess you are using some small embedded type platform
perhaps?
I've run FS even on a nokia n810 and never caused that message to fire.
if 1 call can interrupt the cpu enough to cause noticeable issues you might
want to consider running the process at a
greater priority by using the -hp command line arg or at least nice it
Why don't you tell us the whole story about what OS/platform you are
using
Post by Anthony Minessale
here rather that form conjectures about what is wrong with our code that
thousands of people are happy with.
Post by eaf
2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314
(sofia/external/xxxxx at 4.68.250.148) Running State Change CS_HANGUP
2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration
Detected! Syncing Clock
In this case an incoming call rang to both FS and Asterisk, Asterisk picked
up, but the surge of activity made FS timer thread miss a beat or two.
Post by eaf
Oh, it's not just one timer thread... Why, why is sql_thread keeps on
checking for messages every millisecond? Couldn't there be some
signalling
Post by eaf
implemented that will make the thread suspend on condition variable or
a
Post by eaf
socket/pipe in between?
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0)
at
Post by eaf
src/switch_core_sqldb.c:783
Why does this sofia_profile_worker_thread keeps on looping checking
for
Post by Anthony Minessale
Post by eaf
Post by eaf
the queue? Have a semaphore!
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30,
obj=0x80f2490) at sofia.c:978
Nothing's happening on the box, but there are three threads that
pretend
Post by eaf
to be actively busy with smth. Others at least sleep for hundreds of
milliseconds, not for one.
And there is even infrastructure present to do blocking pops: i.e. why
couldn't sqldb thread do queue_pop() instead of queue_trypop()
intermixed
Post by eaf
with 1ms sleeps? This looping is such a waste...
Post by eaf
As I see it, switch_cond_next() currently is just a do_sleep(1000).
Yes,
Post by eaf
Post by eaf
it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND"
overrides that.
Yeah, there is a global timestamp... It's easy to workaround that for
RTP
Post by eaf
Post by eaf
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that. If
only
Post by eaf
Post by eaf
this was C++...
I'll play around. Never liked polling too much. Never could've
guessed
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
that polling could be so useful for scalability ;) My naive
implementation would've pulled timestamp via system calls and
would've
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
done sleeping by passing exact interval to select() instead of
syncing
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
with a pacing thread. Which would be dead-quiet at idle time, but, of
course, would stop scaling at some point due to excessive number of
system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You
can
Post by eaf
Post by eaf
Post by Michael Jerris
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread.
RTP timers
should be eliminated by that setting you've suggested. IVR timers
are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end
setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do,
call
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some
other
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
thread comes up and adds its timer into the matrix, it could wake
up
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
the
timer thread and enjoy accurate timing as needed, on demand? And
in-
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
between
the calls, when there is no RTP or IVR, it will all go silent? I
mean,
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
sitting on a wait queue in the kernel is way better than go back
and
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing
and
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Post by Anthony Minessale
expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by eaf
Post by eaf
Post by Michael Jerris
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by eaf
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com><
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
Post by Anthony Minessale
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org><
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26629856.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091203/d9ff3e1b/attachment-0001.html
eaf
2009-12-03 18:43:40 UTC
Permalink
You mean, upgrading to the trunk and disabling RTP timers? Yes, I did. I
thought I responded back. Perhaps it didn't make through though, as I just
emailed back to the list instead of using nabble.com...

Anyway, upgrading to the trunk didn't change much, forcing SPA to 30ms went
w/o any effect either, but disabling RTP timers did the trick. I don't have
the original "choppy sound with PCMU" problem any more, thanks a lot for the
quick turnaround on that question.

But your suggestions made me look, into logs, strace, code, etc, so now I'm
just checking on how to quiet down those busy loops a little and how to get
rid of periodic CRIT messages about Virtual Machine Migration.
Post by Anthony Minessale
What about the things I spent time suggesting in my last email?
Did you try them because I was actually curious if they made any impact.
Post by eaf
I'm sorry if I sounded that way. Did mean to. :)
Yes, it's an embedded platform. It's an ALIX board with AMD Geode LX800 chip
and 256MB of RAM. http://www.pcengines.ch/alix2d3.htm
Line offset difference is due to some minor logging changes I made to see
who's allocating timers and how often. This way I found MOH streaming and
that RTP still allocates timers even when it's set to none in the profile.
I feel that this platform turned out to be underpowered for FS because it
cannot meet its scheduling expectations. I guess, some degree of kernel
tweaking or setting priorities will fix that. Meanwhile I just got rid of
the SQLDB 1ms thread via -nosql command line option, split sofia worker 1ms
thread in two (one blocked and waiting for new commands in the SQL queue,
the other one checking registrations and gateways with 1sec interval), and
don't know yet what to do about the timer thread.
Again, I apologize for stupid or accusing questions, I'm just trying to see
how FS can be made friendlier to this board. Or the board be made friendlier
to FS ;)
Post by Anthony Minessale
If you see that message then your machine/os/combo is having some
problems
Post by Anthony Minessale
keeping up.
It's not the timer missing anything its the monotonic clock detecting a
1
Post by Anthony Minessale
second or more differential from what its next prediction for the time
should be. The best way to trigger this would be to suspend FS with
control-z or attach to it with gdb blocking the entire process, that
1ms
Post by Anthony Minessale
thread would have to miss 1000 iterations to trigger that warning.
Btw, that error message is at line 471 not 473 so you are using
modified
Post by Anthony Minessale
code.
Its possible your box has a bad monotonic timer, you can set
under <settings> in switch.conf.xml
We are now starting to guess you are using some small embedded type platform
perhaps?
I've run FS even on a nokia n810 and never caused that message to fire.
if 1 call can interrupt the cpu enough to cause noticeable issues you might
want to consider running the process at a
greater priority by using the -hp command line arg or at least nice it
Why don't you tell us the whole story about what OS/platform you are
using
Post by Anthony Minessale
here rather that form conjectures about what is wrong with our code
that
Post by Anthony Minessale
thousands of people are happy with.
Post by eaf
2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314
(sofia/external/xxxxx at 4.68.250.148) Running State Change CS_HANGUP
2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration
Detected! Syncing Clock
In this case an incoming call rang to both FS and Asterisk, Asterisk picked
up, but the surge of activity made FS timer thread miss a beat or two.
Post by eaf
Oh, it's not just one timer thread... Why, why is sql_thread keeps
on
Post by Anthony Minessale
Post by eaf
Post by eaf
checking for messages every millisecond? Couldn't there be some
signalling
Post by eaf
implemented that will make the thread suspend on condition variable
or
Post by Anthony Minessale
Post by eaf
a
Post by eaf
socket/pipe in between?
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8,
obj=0x0)
Post by Anthony Minessale
Post by eaf
at
Post by eaf
src/switch_core_sqldb.c:783
Why does this sofia_profile_worker_thread keeps on looping checking
for
Post by Anthony Minessale
Post by eaf
Post by eaf
the queue? Have a semaphore!
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30,
obj=0x80f2490) at sofia.c:978
Nothing's happening on the box, but there are three threads that
pretend
Post by eaf
to be actively busy with smth. Others at least sleep for hundreds of
milliseconds, not for one.
And there is even infrastructure present to do blocking pops: i.e.
why
Post by Anthony Minessale
Post by eaf
Post by eaf
couldn't sqldb thread do queue_pop() instead of queue_trypop()
intermixed
Post by eaf
with 1ms sleeps? This looping is such a waste...
Post by eaf
As I see it, switch_cond_next() currently is just a do_sleep(1000).
Yes,
Post by eaf
Post by eaf
it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND"
overrides that.
Yeah, there is a global timestamp... It's easy to workaround that
for
Post by Anthony Minessale
Post by eaf
RTP
Post by eaf
Post by eaf
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that.
If
Post by Anthony Minessale
Post by eaf
only
Post by eaf
Post by eaf
this was C++...
I'll play around. Never liked polling too much. Never could've
guessed
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
that polling could be so useful for scalability ;) My naive
implementation would've pulled timestamp via system calls and
would've
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
done sleeping by passing exact interval to select() instead of
syncing
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
with a pacing thread. Which would be dead-quiet at idle time, but,
of
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
course, would stop scaling at some point due to excessive number of
system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You
can
Post by eaf
Post by eaf
Post by Michael Jerris
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?)
RTP
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
and IVR
set up their timers that are subsequently managed by this thread.
RTP timers
should be eliminated by that setting you've suggested. IVR timers
are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of
the
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
reasons
why FS scales so much better than Asterisk. But for poor low-end
setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do,
call
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
clock_gettime() right from the thread that wants and when it
wants
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
to know
current time?
Say, what if that thread is made to suspend on a condition
variable
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
in case
if there are no timers registered in TIMER_MATRIX? Then, if some
other
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
thread comes up and adds its timer into the matrix, it could wake
up
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
the
timer thread and enjoy accurate timing as needed, on demand? And
in-
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
between
the calls, when there is no RTP or IVR, it will all go silent? I
mean,
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
sitting on a wait queue in the kernel is way better than go back
and
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing
and
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Post by Anthony Minessale
expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet
len
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Post by Anthony Minessale
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Sent from the Freeswitch-users mailing list archive at
Nabble.com.
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by eaf
Post by eaf
Post by Michael Jerris
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by eaf
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com
<MSN%3Aanthony_minessale at hotmail.com><
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org
<sip%3A888 at conference.freeswitch.org><
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26629856.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26630994.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Anthony Minessale
2009-12-03 20:40:59 UTC
Permalink
no,

I mean the one after that that you must have completely skipped with a
command line option to try and a param to set in the config. It somewhat
annoys me for taking the time to compose it now. I wrote all of the code
you are talking about myself and I was trying to give you some
suggestions....

Well, actually, you did answer my question about the platform so you must
have seen it.....

The loops are not the cause of that migration message, something wrong with
the hardware or the kernel is.
Another guy just told you he does not see that problem on the same exact
hardware.

Even if you have a point about the sql threads, you could make a patch to
slow them down but you cant slow down too much or you will not be able to
handle 400 cps all asking to send updates to transactions in batches of
thousands of sql stmts. Every line of that code is carefully designed so I
don't know what else to tell you but to stop being so arrogant and re-read
this thread for all the advice you have totally ignored. I started out
trying to help you but I have a lot of work to do. I thoroughly explained
it to you and you are choosing to ignore me so I guess I'm done.
You can do whatever you want with your working copy, i'll see you in 3 or 4
years when you get up to speed with the rest of us........
Post by eaf
You mean, upgrading to the trunk and disabling RTP timers? Yes, I did. I
thought I responded back. Perhaps it didn't make through though, as I just
emailed back to the list instead of using nabble.com...
Anyway, upgrading to the trunk didn't change much, forcing SPA to 30ms went
w/o any effect either, but disabling RTP timers did the trick. I don't have
the original "choppy sound with PCMU" problem any more, thanks a lot for the
quick turnaround on that question.
But your suggestions made me look, into logs, strace, code, etc, so now I'm
just checking on how to quiet down those busy loops a little and how to get
rid of periodic CRIT messages about Virtual Machine Migration.
Post by Anthony Minessale
What about the things I spent time suggesting in my last email?
Did you try them because I was actually curious if they made any impact.
Post by eaf
I'm sorry if I sounded that way. Did mean to. :)
Yes, it's an embedded platform. It's an ALIX board with AMD Geode LX800 chip
and 256MB of RAM. http://www.pcengines.ch/alix2d3.htm
Line offset difference is due to some minor logging changes I made to
see
Post by Anthony Minessale
Post by eaf
who's allocating timers and how often. This way I found MOH streaming
and
Post by Anthony Minessale
Post by eaf
that RTP still allocates timers even when it's set to none in the profile.
I feel that this platform turned out to be underpowered for FS because
it
Post by Anthony Minessale
Post by eaf
cannot meet its scheduling expectations. I guess, some degree of kernel
tweaking or setting priorities will fix that. Meanwhile I just got rid
of
Post by Anthony Minessale
Post by eaf
the SQLDB 1ms thread via -nosql command line option, split sofia worker 1ms
thread in two (one blocked and waiting for new commands in the SQL
queue,
Post by Anthony Minessale
Post by eaf
the other one checking registrations and gateways with 1sec interval), and
don't know yet what to do about the timer thread.
Again, I apologize for stupid or accusing questions, I'm just trying to see
how FS can be made friendlier to this board. Or the board be made friendlier
to FS ;)
Post by Anthony Minessale
If you see that message then your machine/os/combo is having some
problems
Post by Anthony Minessale
keeping up.
It's not the timer missing anything its the monotonic clock detecting
a
Post by Anthony Minessale
Post by eaf
1
Post by Anthony Minessale
second or more differential from what its next prediction for the time
should be. The best way to trigger this would be to suspend FS with
control-z or attach to it with gdb blocking the entire process, that
1ms
Post by Anthony Minessale
thread would have to miss 1000 iterations to trigger that warning.
Btw, that error message is at line 471 not 473 so you are using
modified
Post by Anthony Minessale
code.
Its possible your box has a bad monotonic timer, you can set
under <settings> in switch.conf.xml
We are now starting to guess you are using some small embedded type platform
perhaps?
I've run FS even on a nokia n810 and never caused that message to
fire.
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
if 1 call can interrupt the cpu enough to cause noticeable issues you might
want to consider running the process at a
greater priority by using the -hp command line arg or at least nice it
Why don't you tell us the whole story about what OS/platform you are
using
Post by Anthony Minessale
here rather that form conjectures about what is wrong with our code
that
Post by Anthony Minessale
thousands of people are happy with.
Post by eaf
2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314
(sofia/external/xxxxx at 4.68.250.148) Running State Change CS_HANGUP
2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration
Detected! Syncing Clock
In this case an incoming call rang to both FS and Asterisk, Asterisk picked
up, but the surge of activity made FS timer thread miss a beat or
two.
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Oh, it's not just one timer thread... Why, why is sql_thread keeps
on
Post by Anthony Minessale
Post by eaf
Post by eaf
checking for messages every millisecond? Couldn't there be some
signalling
Post by eaf
implemented that will make the thread suspend on condition variable
or
Post by Anthony Minessale
Post by eaf
a
Post by eaf
socket/pipe in between?
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8,
obj=0x0)
Post by Anthony Minessale
Post by eaf
at
Post by eaf
src/switch_core_sqldb.c:783
Why does this sofia_profile_worker_thread keeps on looping checking
for
Post by Anthony Minessale
Post by eaf
Post by eaf
the queue? Have a semaphore!
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb73a4701 in sofia_profile_worker_thread_run
(thread=0x80f3a30,
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
obj=0x80f2490) at sofia.c:978
Nothing's happening on the box, but there are three threads that
pretend
Post by eaf
to be actively busy with smth. Others at least sleep for hundreds
of
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
milliseconds, not for one.
And there is even infrastructure present to do blocking pops: i.e.
why
Post by Anthony Minessale
Post by eaf
Post by eaf
couldn't sqldb thread do queue_pop() instead of queue_trypop()
intermixed
Post by eaf
with 1ms sleeps? This looping is such a waste...
Post by eaf
As I see it, switch_cond_next() currently is just a
do_sleep(1000).
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Yes,
Post by eaf
Post by eaf
it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND"
overrides that.
Yeah, there is a global timestamp... It's easy to workaround that
for
Post by Anthony Minessale
Post by eaf
RTP
Post by eaf
Post by eaf
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that.
If
Post by Anthony Minessale
Post by eaf
only
Post by eaf
Post by eaf
this was C++...
I'll play around. Never liked polling too much. Never could've
guessed
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
that polling could be so useful for scalability ;) My naive
implementation would've pulled timestamp via system calls and
would've
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
done sleeping by passing exact interval to select() instead of
syncing
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
with a pacing thread. Which would be dead-quiet at idle time, but,
of
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
course, would stop scaling at some point due to excessive number
of
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic
is
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
used throughout the code even when there is not any calls up.
You
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
can
Post by eaf
Post by eaf
Post by Michael Jerris
grep for switch_cond_next if you would like to see where but it
is
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
required to keep our global timestamp and for pacing the
scheduler
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I
glanced
through the code, and see that among others (are there others?)
RTP
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
and IVR
set up their timers that are subsequently managed by this
thread.
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
RTP timers
should be eliminated by that setting you've suggested. IVR
timers
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
are set at
20ms... So, if the thread is set to wake up every 10ms instead
of
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains
accurate
timing and then broadcasts via condition variables to hundreds
of
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
other
threads events that they can register for. I'm sure it's one of
the
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
reasons
why FS scales so much better than Asterisk. But for poor low-end
setups that
sit in the closet, eat only 6W of power and hardly ever run more
than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do,
call
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
clock_gettime() right from the thread that wants and when it
wants
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
to know
current time?
Say, what if that thread is made to suspend on a condition
variable
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
in case
if there are no timers registered in TIMER_MATRIX? Then, if some
other
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
thread comes up and adds its timer into the matrix, it could
wake
Post by Anthony Minessale
Post by eaf
up
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
the
timer thread and enjoy accurate timing as needed, on demand? And
in-
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
between
the calls, when there is no RTP or IVR, it will all go silent? I
mean,
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
sitting on a wait queue in the kernel is way better than go back
and
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing
and
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Post by Anthony Minessale
expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet
len
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Post by Anthony Minessale
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Sent from the Freeswitch-users mailing list archive at
Nabble.com.
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by eaf
Post by eaf
Post by Michael Jerris
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by eaf
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com<MSN%3Aanthony_minessale at hotmail.com>
<MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
<
Post by eaf
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
<MSN%253Aanthony_minessale at hotmail.com<MSN%25253Aanthony_minessale at hotmail.com>
Post by Anthony Minessale
Post by eaf
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
Post by eaf
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
<PAYPAL%253Aanthony.minessale at gmail.com<PAYPAL%25253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org<sip%3A888 at conference.freeswitch.org>
<sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
<
Post by eaf
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
<sip%253A888 at conference.freeswitch.org<sip%25253A888 at conference.freeswitch.org>
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by Anthony Minessale
Post by eaf
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
<googletalk%253Aconf%252B888 at conference.freeswitch.org<googletalk%25253Aconf%25252B888 at conference.freeswitch.org>
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26629856.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by eaf
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com><
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
Post by Anthony Minessale
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org><
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26630994.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091203/b872350e/attachment-0001.html
eaf
2009-12-03 21:44:15 UTC
Permalink
Oh, you mean giving FS higher priority? Yeah, as a last resort I'll do that.
At the moment, I hope it won't be necessary as I can make those "hyper"
threads behave, and will see how that goes first. I see where your
implementation could be coming from. There is a queue of SQL queries in
sofia.c processed by the worker thread. There are only two pop functions
available in APR: queue_pop() and queue_trypop(), so alas no option with a
timeout here. You don't want to block the thread in pop() indefinitely
because you chose that same worker needs to do ireg and gw processing once
in a while (separated by tens or hundreds of seconds, btw). You also want to
be able to detect shutdown condition so that the worker doesn't hold up
profile thread. So you chose to poll for events every millisecond instead of
just creating an apr_thread_cond_t for resource friendly signalling.

I agree that the timer thread philosophy is great and was the right choice
for scaling, but I just don't comprehend responses to things like these
other SQL or sofia worker threads. Did somebody even remotely acknowledge
that busy loops at least in those areas that I showed may probably be a bad
idea and could've been eliminated? I've heard suggestions to bump up
priority, I've heard that the code was perfect already, that it's the result
of 4-year effort, that I am arrogant, don't listen and don't understand
squat.

I'm sorry if I gave you impression that I was looking for the bad parts in
the software. I apologized for that already. All I wanted was to have
constructive conversation, perhaps I'm not too good at it. Code is already
perfect according to you? Fine with me.
Post by Anthony Minessale
no,
I mean the one after that that you must have completely skipped with a
command line option to try and a param to set in the config. It somewhat
annoys me for taking the time to compose it now. I wrote all of the code
you are talking about myself and I was trying to give you some
suggestions....
Well, actually, you did answer my question about the platform so you must
have seen it.....
The loops are not the cause of that migration message, something wrong with
the hardware or the kernel is.
Another guy just told you he does not see that problem on the same exact
hardware.
Even if you have a point about the sql threads, you could make a patch to
slow them down but you cant slow down too much or you will not be able to
handle 400 cps all asking to send updates to transactions in batches of
thousands of sql stmts. Every line of that code is carefully designed so I
don't know what else to tell you but to stop being so arrogant and re-read
this thread for all the advice you have totally ignored. I started out
trying to help you but I have a lot of work to do. I thoroughly explained
it to you and you are choosing to ignore me so I guess I'm done.
You can do whatever you want with your working copy, i'll see you in 3 or 4
years when you get up to speed with the rest of us........
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26633739.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Anthony Minessale
2009-12-03 23:49:20 UTC
Permalink
Sigh,

You just took it up a notch in terms of disdain and sarcasm.
Why do people always only apologize sarcastically?

I asked you to try the -hp and turn off the monotonic clock just to gather
the results to help you. You completely missed it and just went on about
the threads. Please save the "ok fine the code is perfect, blah blah" if
you would have just read the email and answered the question I might have
cared more about the status of your problem.

I told you both of those threads need to be on their toes because they try
to balance between a certian number of sql stmts or 500ms whatever comes
first. When there are thousands of events per second being turned into SQL
statements which are in turn compiled into large sql transactions.

If you want to come up with a way that they can sleep longer until there is
a sign of activity and stay busy for a few seconds then slow down again,
that's probably possible but the process is already idle at 0% cpu so maybe
you can appreciate why we are not rushing to work on it. Maybe I'll give it
a go just to show you it has nothing to do with your problem.

Please don't mock our comment about several years. You have no idea how
hard this code was to develop and it's truly insulting. Its clear to see
you are locked into assuming that the busy threads that are not all that
busy because they are constantly yielding to the scheduler is breaking the
timing code. I begged you to understand me when i told you that the err is
not normal, most boxes do not see it doing nothing and there has to be a
specific problem on your box or configuration. So instead of working with
us you want to escalate to snotty comments. That's pretty normal on the
internet I guess..... If you want to have a constructive conversation about
our core, install FS on a normal box, use it for a few weeks, figure out
everything about how it works then try.... There was pure speculation and
conjecture in your original emails and I never said a word about it until
you kept pushing.

Kristian mentioned he never sees that on that same hardware did you even
consider following up on why that is?

I don't have your device, but I assume if you get it working well it will
certainly help you more than it helps me so you could at least have the
decency to believe what we are trying to tell you.
Post by eaf
Oh, you mean giving FS higher priority? Yeah, as a last resort I'll do that.
At the moment, I hope it won't be necessary as I can make those "hyper"
threads behave, and will see how that goes first. I see where your
implementation could be coming from. There is a queue of SQL queries in
sofia.c processed by the worker thread. There are only two pop functions
available in APR: queue_pop() and queue_trypop(), so alas no option with a
timeout here. You don't want to block the thread in pop() indefinitely
because you chose that same worker needs to do ireg and gw processing once
in a while (separated by tens or hundreds of seconds, btw). You also want to
be able to detect shutdown condition so that the worker doesn't hold up
profile thread. So you chose to poll for events every millisecond instead of
just creating an apr_thread_cond_t for resource friendly signalling.
I agree that the timer thread philosophy is great and was the right choice
for scaling, but I just don't comprehend responses to things like these
other SQL or sofia worker threads. Did somebody even remotely acknowledge
that busy loops at least in those areas that I showed may probably be a bad
idea and could've been eliminated? I've heard suggestions to bump up
priority, I've heard that the code was perfect already, that it's the result
of 4-year effort, that I am arrogant, don't listen and don't understand
squat.
I'm sorry if I gave you impression that I was looking for the bad parts in
the software. I apologized for that already. All I wanted was to have
constructive conversation, perhaps I'm not too good at it. Code is already
perfect according to you? Fine with me.
Post by Anthony Minessale
no,
I mean the one after that that you must have completely skipped with a
command line option to try and a param to set in the config. It somewhat
annoys me for taking the time to compose it now. I wrote all of the code
you are talking about myself and I was trying to give you some
suggestions....
Well, actually, you did answer my question about the platform so you
must
Post by Anthony Minessale
have seen it.....
The loops are not the cause of that migration message, something wrong with
the hardware or the kernel is.
Another guy just told you he does not see that problem on the same exact
hardware.
Even if you have a point about the sql threads, you could make a patch to
slow them down but you cant slow down too much or you will not be able to
handle 400 cps all asking to send updates to transactions in batches of
thousands of sql stmts. Every line of that code is carefully designed so I
don't know what else to tell you but to stop being so arrogant and
re-read
Post by Anthony Minessale
this thread for all the advice you have totally ignored. I started out
trying to help you but I have a lot of work to do. I thoroughly
explained
Post by Anthony Minessale
it to you and you are choosing to ignore me so I guess I'm done.
You can do whatever you want with your working copy, i'll see you in 3 or 4
years when you get up to speed with the rest of us........
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26633739.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091203/538b8cbc/attachment-0001.html
Yossi Neiman
2009-12-04 19:32:56 UTC
Permalink
A word to the wise to the general FreeSWITCH community: If Anthony
Minessale suggests that you try to do any number of things, it's a very
good idea to try all those ideas before continuing on. I've known him,
MikeJ, and bkw for several years, and they almost always have very good
ideas as to troubleshoot a problem in FreeSWITCH. It's extremely
frustrating to try to help people out who won't try the provided
suggestions first.

And note directly to "eaf" - bogomips is quite possibly the least
significant bit of data about a cpu that you will get out of
/proc/cpuinfo... The name itself - bogo, means bogus.
http://en.wikipedia.org/wiki/Bogomips

-Yossi
Anthony Minessale
2009-12-04 19:48:08 UTC
Permalink
There is another user here with a 300mhz box. I am willing to investigate
this improved performance for weak devices but I need to do it in a sane
cross-platform way.


On Fri, Dec 4, 2009 at 1:32 PM, Yossi Neiman <freeswitch at cartissolutions.com
Post by Yossi Neiman
A word to the wise to the general FreeSWITCH community: If Anthony
Minessale suggests that you try to do any number of things, it's a very
good idea to try all those ideas before continuing on. I've known him,
MikeJ, and bkw for several years, and they almost always have very good
ideas as to troubleshoot a problem in FreeSWITCH. It's extremely
frustrating to try to help people out who won't try the provided
suggestions first.
And note directly to "eaf" - bogomips is quite possibly the least
significant bit of data about a cpu that you will get out of
/proc/cpuinfo... The name itself - bogo, means bogus.
http://en.wikipedia.org/wiki/Bogomips
-Yossi
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091204/1b13c721/attachment.html
eaf
2009-12-07 15:28:52 UTC
Permalink
Here is what I found...

I tried high-priority scheduling as per your suggestion, reniced the program
explicitly, rewrote timer thread to sleep on cond. variable and activate
only when there are timers and only when the timer actually had to be
clicked, turned off SQL thread and removed polling from sofia profile
thread.

That pretty much eliminated all idle 1ms sleepers that were there except for
three in sofia itself (su_epoll_port). And when I was about to be happy, I
found that two outgoing calls through my VOIP providers when bridged
together showed terrible distortions. I undid all my changes, tried 1.0.4,
trunk (noticed btw that when I bridge two calls via loopback in JS in the
trunk I must keep JS running, or the calls get terminated - NOT the same as
in 1.0.4 where exitting JS left calls running), got pretty much the same sad
results. At the same time calls bridged by freeswitch between LAN and any of
the VOIP providers behaved just fine. And calls bridged by Asterisk any way
were fine too. So that pretty much looked like the end of the freeswitch
trials for me.

But then I timed your code, mine and found that all those 1ms sleeps that
your timer thread was doing (and all those pollers were doing as well) were
actually 4ms sleeps because you know what unless kernel is configured with
HZ=1000, you can't sleep for less than 4ms (HZ=250) or perhaps even 10ms
(HZ=100). Mine was 250.

This actually meant that the original timer thread was firing once, sleeping
for 4ms, firing 3 more times back-to-back, sleeping for 4ms more, firing 4
times back-to-back, etc. It was still firing 20ms timers on time, but 30ms
ones of course were not, since 30ms doesn't divide by 4 evenly. Plus whoever
relied on runtime.reference or switch_micro_time_now() were kind of screwed
because both were running jumpy. Plus whoever assumed that apr_sleep(1000)
or cond_yield() was sleeping for 1ms were also in for a surprise. It felt
satisfying to find that, however it didn't explain why the same distortions
were observed with rewritten timer thread and disabled RTP timers.

Anyway, I sighed (pretty much like you) and recompiled the kernel with
HZ=1000. Recompiling kernel on these ALIX boards is fun. If smth goes south,
you need to hook up serial console and see what the heck went wrong.

That eliminated distortions, ha! But made freeswitch more CPU hungry. Now
the remaining 1ms threads sitting in sofia epoll were really polling for
1ms, not 4, and freeswitch was consistently sitting in the first line of the
top chart showing 3% CPU utilization when idle.

Don't know whether it's because of the remaining epolls in sofia or whether
it's because there are still some threads left in freeswitch that I
neglected to change because they were sleeping with 100ms interval, so I
figured, who cares. Maybe when all things come together (sofia, 100ms*N)
freeswitch ends up spending 3% of CPU while doing pretty much nothing.

Btw, compared with Asterisk, the latter is not even visible on the first
top's screen and spends 1% CPU when bridging two G711 calls and recording
them to disk.

So, at this time I have both original Asterisk and FS setups running. One is
seemless but clumsy in configuration, the other one is neat and stylish but
too preoccupied with smth... Should I look into sofia epollers? That's kind
of deep in the code. Or should I just stick with Asterisk?
Post by Anthony Minessale
There is another user here with a 300mhz box. I am willing to investigate
this improved performance for weak devices but I need to do it in a sane
cross-platform way.
On Fri, Dec 4, 2009 at 1:32 PM, Yossi Neiman
<freeswitch at cartissolutions.com
Post by Yossi Neiman
A word to the wise to the general FreeSWITCH community: If Anthony
Minessale suggests that you try to do any number of things, it's a very
good idea to try all those ideas before continuing on. I've known him,
MikeJ, and bkw for several years, and they almost always have very good
ideas as to troubleshoot a problem in FreeSWITCH. It's extremely
frustrating to try to help people out who won't try the provided
suggestions first.
And note directly to "eaf" - bogomips is quite possibly the least
significant bit of data about a cpu that you will get out of
/proc/cpuinfo... The name itself - bogo, means bogus.
http://en.wikipedia.org/wiki/Bogomips
-Yossi
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26678873.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Anthony Minessale
2009-12-07 16:00:17 UTC
Permalink
Did you do each thing alone too to tell the difference?
-hp alone, disable monotonic alone (i did not see you mention the disable
monotonic)

as for your 4ms thing, yes we require high resolution timing, if we ask to
sleep 1000 microseconds that is what we need it to sleep for or at least as
close as possible, and the main reason that thread is never sleeping is
because you can't actually count on it to run every 1ms but you mostly can.
Hence the whole philosophy on only making 1 thread run hot all the time to
ensure that the rest don't have to repeat the same algorithm. We focus on
high end performance this was the point of your experimentation because we
will need to use a compile time defines and other logic to make it more
efficient on your platform, a platform which we are not using. I am curious
what would happen if you install Kristian's astlinux on one of your devices,
i think you should also compare the kernel versions.


What OS are you running anyway?

Here are some more things to try (running plain trunk with no mods) do these
systematically each alone and all together with/without -hp or disable
monotonic etc to see what different combos create

comment out this line (line 10)
#define DISABLE_1MS_COND

rebuild, this tells it to run a conditional at 1ms in the same timer thread
which will make all the switch_cond_next share a 1ms conditional instead of
doing microsleeps

next

some kernels/devices work better using select(0) for sleep where others work
better using usleep.
comment out line 109
apr_sleep(t);

and try
usleep(t)

also mac works better using nanosleep so you could try changing it so it
uses the code starting at 101 instead.


also your claim about JS should be investigated because I do not think it
should be the case.
but you may want to move this to a jira http://jira.freeswitch.org

As for the asterisk comparison,
not sure how to answer you, that's your decision.
Post by eaf
Here is what I found...
I tried high-priority scheduling as per your suggestion, reniced the program
explicitly, rewrote timer thread to sleep on cond. variable and activate
only when there are timers and only when the timer actually had to be
clicked, turned off SQL thread and removed polling from sofia profile
thread.
That pretty much eliminated all idle 1ms sleepers that were there except for
three in sofia itself (su_epoll_port). And when I was about to be happy, I
found that two outgoing calls through my VOIP providers when bridged
together showed terrible distortions. I undid all my changes, tried 1.0.4,
trunk (noticed btw that when I bridge two calls via loopback in JS in the
trunk I must keep JS running, or the calls get terminated - NOT the same as
in 1.0.4 where exitting JS left calls running), got pretty much the same sad
results. At the same time calls bridged by freeswitch between LAN and any of
the VOIP providers behaved just fine. And calls bridged by Asterisk any way
were fine too. So that pretty much looked like the end of the freeswitch
trials for me.
But then I timed your code, mine and found that all those 1ms sleeps that
your timer thread was doing (and all those pollers were doing as well) were
actually 4ms sleeps because you know what unless kernel is configured with
HZ=1000, you can't sleep for less than 4ms (HZ=250) or perhaps even 10ms
(HZ=100). Mine was 250.
This actually meant that the original timer thread was firing once, sleeping
for 4ms, firing 3 more times back-to-back, sleeping for 4ms more, firing 4
times back-to-back, etc. It was still firing 20ms timers on time, but 30ms
ones of course were not, since 30ms doesn't divide by 4 evenly. Plus whoever
relied on runtime.reference or switch_micro_time_now() were kind of screwed
because both were running jumpy. Plus whoever assumed that apr_sleep(1000)
or cond_yield() was sleeping for 1ms were also in for a surprise. It felt
satisfying to find that, however it didn't explain why the same distortions
were observed with rewritten timer thread and disabled RTP timers.
Anyway, I sighed (pretty much like you) and recompiled the kernel with
HZ=1000. Recompiling kernel on these ALIX boards is fun. If smth goes south,
you need to hook up serial console and see what the heck went wrong.
That eliminated distortions, ha! But made freeswitch more CPU hungry. Now
the remaining 1ms threads sitting in sofia epoll were really polling for
1ms, not 4, and freeswitch was consistently sitting in the first line of the
top chart showing 3% CPU utilization when idle.
Don't know whether it's because of the remaining epolls in sofia or whether
it's because there are still some threads left in freeswitch that I
neglected to change because they were sleeping with 100ms interval, so I
figured, who cares. Maybe when all things come together (sofia, 100ms*N)
freeswitch ends up spending 3% of CPU while doing pretty much nothing.
Btw, compared with Asterisk, the latter is not even visible on the first
top's screen and spends 1% CPU when bridging two G711 calls and recording
them to disk.
So, at this time I have both original Asterisk and FS setups running. One is
seemless but clumsy in configuration, the other one is neat and stylish but
too preoccupied with smth... Should I look into sofia epollers? That's kind
of deep in the code. Or should I just stick with Asterisk?
Post by Anthony Minessale
There is another user here with a 300mhz box. I am willing to
investigate
Post by Anthony Minessale
this improved performance for weak devices but I need to do it in a sane
cross-platform way.
On Fri, Dec 4, 2009 at 1:32 PM, Yossi Neiman
<freeswitch at cartissolutions.com
Post by Yossi Neiman
A word to the wise to the general FreeSWITCH community: If Anthony
Minessale suggests that you try to do any number of things, it's a very
good idea to try all those ideas before continuing on. I've known him,
MikeJ, and bkw for several years, and they almost always have very good
ideas as to troubleshoot a problem in FreeSWITCH. It's extremely
frustrating to try to help people out who won't try the provided
suggestions first.
And note directly to "eaf" - bogomips is quite possibly the least
significant bit of data about a cpu that you will get out of
/proc/cpuinfo... The name itself - bogo, means bogus.
http://en.wikipedia.org/wiki/Bogomips
-Yossi
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by Yossi Neiman
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com><
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
Post by Anthony Minessale
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org><
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26678873.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091207/a83d6006/attachment-0001.html
Michael Jerris
2009-12-07 16:16:30 UTC
Permalink
Also I have seen some people reporting that the new tickless timers in newer kernels work better. You may want to try those.

Mike
Post by Anthony Minessale
Did you do each thing alone too to tell the difference?
-hp alone, disable monotonic alone (i did not see you mention the disable monotonic)
as for your 4ms thing, yes we require high resolution timing, if we ask to sleep 1000 microseconds that is what we need it to sleep for or at least as close as possible, and the main reason that thread is never sleeping is because you can't actually count on it to run every 1ms but you mostly can. Hence the whole philosophy on only making 1 thread run hot all the time to ensure that the rest don't have to repeat the same algorithm. We focus on high end performance this was the point of your experimentation because we will need to use a compile time defines and other logic to make it more efficient on your platform, a platform which we are not using. I am curious what would happen if you install Kristian's astlinux on one of your devices, i think you should also compare the kernel versions.
What OS are you running anyway?
Here are some more things to try (running plain trunk with no mods) do these systematically each alone and all together with/without -hp or disable monotonic etc to see what different combos create
comment out this line (line 10)
#define DISABLE_1MS_COND
rebuild, this tells it to run a conditional at 1ms in the same timer thread which will make all the switch_cond_next share a 1ms conditional instead of doing microsleeps
next
some kernels/devices work better using select(0) for sleep where others work better using usleep.
comment out line 109
apr_sleep(t);
and try
usleep(t)
also mac works better using nanosleep so you could try changing it so it
uses the code starting at 101 instead.
also your claim about JS should be investigated because I do not think it should be the case.
but you may want to move this to a jira http://jira.freeswitch.org
As for the asterisk comparison,
not sure how to answer you, that's your decision.
Here is what I found...
I tried high-priority scheduling as per your suggestion, reniced the program
explicitly, rewrote timer thread to sleep on cond. variable and activate
only when there are timers and only when the timer actually had to be
clicked, turned off SQL thread and removed polling from sofia profile
thread.
That pretty much eliminated all idle 1ms sleepers that were there except for
three in sofia itself (su_epoll_port). And when I was about to be happy, I
found that two outgoing calls through my VOIP providers when bridged
together showed terrible distortions. I undid all my changes, tried 1.0.4,
trunk (noticed btw that when I bridge two calls via loopback in JS in the
trunk I must keep JS running, or the calls get terminated - NOT the same as
in 1.0.4 where exitting JS left calls running), got pretty much the same sad
results. At the same time calls bridged by freeswitch between LAN and any of
the VOIP providers behaved just fine. And calls bridged by Asterisk any way
were fine too. So that pretty much looked like the end of the freeswitch
trials for me.
But then I timed your code, mine and found that all those 1ms sleeps that
your timer thread was doing (and all those pollers were doing as well) were
actually 4ms sleeps because you know what unless kernel is configured with
HZ=1000, you can't sleep for less than 4ms (HZ=250) or perhaps even 10ms
(HZ=100). Mine was 250.
This actually meant that the original timer thread was firing once, sleeping
for 4ms, firing 3 more times back-to-back, sleeping for 4ms more, firing 4
times back-to-back, etc. It was still firing 20ms timers on time, but 30ms
ones of course were not, since 30ms doesn't divide by 4 evenly. Plus whoever
relied on runtime.reference or switch_micro_time_now() were kind of screwed
because both were running jumpy. Plus whoever assumed that apr_sleep(1000)
or cond_yield() was sleeping for 1ms were also in for a surprise. It felt
satisfying to find that, however it didn't explain why the same distortions
were observed with rewritten timer thread and disabled RTP timers.
Anyway, I sighed (pretty much like you) and recompiled the kernel with
HZ=1000. Recompiling kernel on these ALIX boards is fun. If smth goes south,
you need to hook up serial console and see what the heck went wrong.
That eliminated distortions, ha! But made freeswitch more CPU hungry. Now
the remaining 1ms threads sitting in sofia epoll were really polling for
1ms, not 4, and freeswitch was consistently sitting in the first line of the
top chart showing 3% CPU utilization when idle.
Don't know whether it's because of the remaining epolls in sofia or whether
it's because there are still some threads left in freeswitch that I
neglected to change because they were sleeping with 100ms interval, so I
figured, who cares. Maybe when all things come together (sofia, 100ms*N)
freeswitch ends up spending 3% of CPU while doing pretty much nothing.
Btw, compared with Asterisk, the latter is not even visible on the first
top's screen and spends 1% CPU when bridging two G711 calls and recording
them to disk.
So, at this time I have both original Asterisk and FS setups running. One is
seemless but clumsy in configuration, the other one is neat and stylish but
too preoccupied with smth... Should I look into sofia epollers? That's kind
of deep in the code. Or should I just stick with Asterisk?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091207/31a8a586/attachment.html
Anthony Minessale
2009-12-07 17:32:25 UTC
Permalink
oh and also

use top -H to see which threads are using specific CPU and try to cross
reference them by attaching with gdb and dumping all the thread bt
Post by Michael Jerris
Also I have seen some people reporting that the new tickless timers in
newer kernels work better. You may want to try those.
Mike
Did you do each thing alone too to tell the difference?
-hp alone, disable monotonic alone (i did not see you mention the disable monotonic)
as for your 4ms thing, yes we require high resolution timing, if we ask to
sleep 1000 microseconds that is what we need it to sleep for or at least as
close as possible, and the main reason that thread is never sleeping is
because you can't actually count on it to run every 1ms but you mostly can.
Hence the whole philosophy on only making 1 thread run hot all the time to
ensure that the rest don't have to repeat the same algorithm. We focus on
high end performance this was the point of your experimentation because we
will need to use a compile time defines and other logic to make it more
efficient on your platform, a platform which we are not using. I am curious
what would happen if you install Kristian's astlinux on one of your devices,
i think you should also compare the kernel versions.
What OS are you running anyway?
Here are some more things to try (running plain trunk with no mods) do
these systematically each alone and all together with/without -hp or disable
monotonic etc to see what different combos create
comment out this line (line 10)
#define DISABLE_1MS_COND
rebuild, this tells it to run a conditional at 1ms in the same timer thread
which will make all the switch_cond_next share a 1ms conditional instead of
doing microsleeps
next
some kernels/devices work better using select(0) for sleep where others
work better using usleep.
comment out line 109
apr_sleep(t);
and try
usleep(t)
also mac works better using nanosleep so you could try changing it so it
uses the code starting at 101 instead.
also your claim about JS should be investigated because I do not think it
should be the case.
but you may want to move this to a jira http://jira.freeswitch.org
As for the asterisk comparison,
not sure how to answer you, that's your decision.
Post by eaf
Here is what I found...
I tried high-priority scheduling as per your suggestion, reniced the program
explicitly, rewrote timer thread to sleep on cond. variable and activate
only when there are timers and only when the timer actually had to be
clicked, turned off SQL thread and removed polling from sofia profile
thread.
That pretty much eliminated all idle 1ms sleepers that were there except for
three in sofia itself (su_epoll_port). And when I was about to be happy, I
found that two outgoing calls through my VOIP providers when bridged
together showed terrible distortions. I undid all my changes, tried 1.0.4,
trunk (noticed btw that when I bridge two calls via loopback in JS in the
trunk I must keep JS running, or the calls get terminated - NOT the same as
in 1.0.4 where exitting JS left calls running), got pretty much the same sad
results. At the same time calls bridged by freeswitch between LAN and any of
the VOIP providers behaved just fine. And calls bridged by Asterisk any way
were fine too. So that pretty much looked like the end of the freeswitch
trials for me.
But then I timed your code, mine and found that all those 1ms sleeps that
your timer thread was doing (and all those pollers were doing as well) were
actually 4ms sleeps because you know what unless kernel is configured with
HZ=1000, you can't sleep for less than 4ms (HZ=250) or perhaps even 10ms
(HZ=100). Mine was 250.
This actually meant that the original timer thread was firing once, sleeping
for 4ms, firing 3 more times back-to-back, sleeping for 4ms more, firing 4
times back-to-back, etc. It was still firing 20ms timers on time, but 30ms
ones of course were not, since 30ms doesn't divide by 4 evenly. Plus whoever
relied on runtime.reference or switch_micro_time_now() were kind of screwed
because both were running jumpy. Plus whoever assumed that apr_sleep(1000)
or cond_yield() was sleeping for 1ms were also in for a surprise. It felt
satisfying to find that, however it didn't explain why the same distortions
were observed with rewritten timer thread and disabled RTP timers.
Anyway, I sighed (pretty much like you) and recompiled the kernel with
HZ=1000. Recompiling kernel on these ALIX boards is fun. If smth goes south,
you need to hook up serial console and see what the heck went wrong.
That eliminated distortions, ha! But made freeswitch more CPU hungry. Now
the remaining 1ms threads sitting in sofia epoll were really polling for
1ms, not 4, and freeswitch was consistently sitting in the first line of the
top chart showing 3% CPU utilization when idle.
Don't know whether it's because of the remaining epolls in sofia or whether
it's because there are still some threads left in freeswitch that I
neglected to change because they were sleeping with 100ms interval, so I
figured, who cares. Maybe when all things come together (sofia, 100ms*N)
freeswitch ends up spending 3% of CPU while doing pretty much nothing.
Btw, compared with Asterisk, the latter is not even visible on the first
top's screen and spends 1% CPU when bridging two G711 calls and recording
them to disk.
So, at this time I have both original Asterisk and FS setups running. One is
seemless but clumsy in configuration, the other one is neat and stylish but
too preoccupied with smth... Should I look into sofia epollers? That's kind
of deep in the code. Or should I just stick with Asterisk?
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091207/89804735/attachment.html
Anthony Minessale
2009-12-07 17:32:25 UTC
Permalink
oh and also

use top -H to see which threads are using specific CPU and try to cross
reference them by attaching with gdb and dumping all the thread bt
Post by Michael Jerris
Also I have seen some people reporting that the new tickless timers in
newer kernels work better. You may want to try those.
Mike
Did you do each thing alone too to tell the difference?
-hp alone, disable monotonic alone (i did not see you mention the disable monotonic)
as for your 4ms thing, yes we require high resolution timing, if we ask to
sleep 1000 microseconds that is what we need it to sleep for or at least as
close as possible, and the main reason that thread is never sleeping is
because you can't actually count on it to run every 1ms but you mostly can.
Hence the whole philosophy on only making 1 thread run hot all the time to
ensure that the rest don't have to repeat the same algorithm. We focus on
high end performance this was the point of your experimentation because we
will need to use a compile time defines and other logic to make it more
efficient on your platform, a platform which we are not using. I am curious
what would happen if you install Kristian's astlinux on one of your devices,
i think you should also compare the kernel versions.
What OS are you running anyway?
Here are some more things to try (running plain trunk with no mods) do
these systematically each alone and all together with/without -hp or disable
monotonic etc to see what different combos create
comment out this line (line 10)
#define DISABLE_1MS_COND
rebuild, this tells it to run a conditional at 1ms in the same timer thread
which will make all the switch_cond_next share a 1ms conditional instead of
doing microsleeps
next
some kernels/devices work better using select(0) for sleep where others
work better using usleep.
comment out line 109
apr_sleep(t);
and try
usleep(t)
also mac works better using nanosleep so you could try changing it so it
uses the code starting at 101 instead.
also your claim about JS should be investigated because I do not think it
should be the case.
but you may want to move this to a jira http://jira.freeswitch.org
As for the asterisk comparison,
not sure how to answer you, that's your decision.
Post by eaf
Here is what I found...
I tried high-priority scheduling as per your suggestion, reniced the program
explicitly, rewrote timer thread to sleep on cond. variable and activate
only when there are timers and only when the timer actually had to be
clicked, turned off SQL thread and removed polling from sofia profile
thread.
That pretty much eliminated all idle 1ms sleepers that were there except for
three in sofia itself (su_epoll_port). And when I was about to be happy, I
found that two outgoing calls through my VOIP providers when bridged
together showed terrible distortions. I undid all my changes, tried 1.0.4,
trunk (noticed btw that when I bridge two calls via loopback in JS in the
trunk I must keep JS running, or the calls get terminated - NOT the same as
in 1.0.4 where exitting JS left calls running), got pretty much the same sad
results. At the same time calls bridged by freeswitch between LAN and any of
the VOIP providers behaved just fine. And calls bridged by Asterisk any way
were fine too. So that pretty much looked like the end of the freeswitch
trials for me.
But then I timed your code, mine and found that all those 1ms sleeps that
your timer thread was doing (and all those pollers were doing as well) were
actually 4ms sleeps because you know what unless kernel is configured with
HZ=1000, you can't sleep for less than 4ms (HZ=250) or perhaps even 10ms
(HZ=100). Mine was 250.
This actually meant that the original timer thread was firing once, sleeping
for 4ms, firing 3 more times back-to-back, sleeping for 4ms more, firing 4
times back-to-back, etc. It was still firing 20ms timers on time, but 30ms
ones of course were not, since 30ms doesn't divide by 4 evenly. Plus whoever
relied on runtime.reference or switch_micro_time_now() were kind of screwed
because both were running jumpy. Plus whoever assumed that apr_sleep(1000)
or cond_yield() was sleeping for 1ms were also in for a surprise. It felt
satisfying to find that, however it didn't explain why the same distortions
were observed with rewritten timer thread and disabled RTP timers.
Anyway, I sighed (pretty much like you) and recompiled the kernel with
HZ=1000. Recompiling kernel on these ALIX boards is fun. If smth goes south,
you need to hook up serial console and see what the heck went wrong.
That eliminated distortions, ha! But made freeswitch more CPU hungry. Now
the remaining 1ms threads sitting in sofia epoll were really polling for
1ms, not 4, and freeswitch was consistently sitting in the first line of the
top chart showing 3% CPU utilization when idle.
Don't know whether it's because of the remaining epolls in sofia or whether
it's because there are still some threads left in freeswitch that I
neglected to change because they were sleeping with 100ms interval, so I
figured, who cares. Maybe when all things come together (sofia, 100ms*N)
freeswitch ends up spending 3% of CPU while doing pretty much nothing.
Btw, compared with Asterisk, the latter is not even visible on the first
top's screen and spends 1% CPU when bridging two G711 calls and recording
them to disk.
So, at this time I have both original Asterisk and FS setups running. One is
seemless but clumsy in configuration, the other one is neat and stylish but
too preoccupied with smth... Should I look into sofia epollers? That's kind
of deep in the code. Or should I just stick with Asterisk?
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091207/89804735/attachment-0002.html
eaf
2009-12-07 20:58:55 UTC
Permalink
What do you want me to check while running these tests? Sound quality (it's
good now even with original 1.0.4). Or CPU utilization?

It's Debian 4.
Post by Anthony Minessale
Did you do each thing alone too to tell the difference?
-hp alone, disable monotonic alone (i did not see you mention the disable
monotonic)
as for your 4ms thing, yes we require high resolution timing, if we ask to
sleep 1000 microseconds that is what we need it to sleep for or at least as
close as possible, and the main reason that thread is never sleeping is
because you can't actually count on it to run every 1ms but you mostly can.
Hence the whole philosophy on only making 1 thread run hot all the time to
ensure that the rest don't have to repeat the same algorithm. We focus on
high end performance this was the point of your experimentation because we
will need to use a compile time defines and other logic to make it more
efficient on your platform, a platform which we are not using. I am curious
what would happen if you install Kristian's astlinux on one of your devices,
i think you should also compare the kernel versions.
What OS are you running anyway?
Here are some more things to try (running plain trunk with no mods) do these
systematically each alone and all together with/without -hp or disable
monotonic etc to see what different combos create
comment out this line (line 10)
#define DISABLE_1MS_COND
rebuild, this tells it to run a conditional at 1ms in the same timer thread
which will make all the switch_cond_next share a 1ms conditional instead of
doing microsleeps
next
some kernels/devices work better using select(0) for sleep where others work
better using usleep.
comment out line 109
apr_sleep(t);
and try
usleep(t)
also mac works better using nanosleep so you could try changing it so it
uses the code starting at 101 instead.
also your claim about JS should be investigated because I do not think it
should be the case.
but you may want to move this to a jira http://jira.freeswitch.org
As for the asterisk comparison,
not sure how to answer you, that's your decision.
Post by eaf
Here is what I found...
I tried high-priority scheduling as per your suggestion, reniced the program
explicitly, rewrote timer thread to sleep on cond. variable and activate
only when there are timers and only when the timer actually had to be
clicked, turned off SQL thread and removed polling from sofia profile
thread.
That pretty much eliminated all idle 1ms sleepers that were there except for
three in sofia itself (su_epoll_port). And when I was about to be happy, I
found that two outgoing calls through my VOIP providers when bridged
together showed terrible distortions. I undid all my changes, tried 1.0.4,
trunk (noticed btw that when I bridge two calls via loopback in JS in the
trunk I must keep JS running, or the calls get terminated - NOT the same as
in 1.0.4 where exitting JS left calls running), got pretty much the same sad
results. At the same time calls bridged by freeswitch between LAN and any of
the VOIP providers behaved just fine. And calls bridged by Asterisk any way
were fine too. So that pretty much looked like the end of the freeswitch
trials for me.
But then I timed your code, mine and found that all those 1ms sleeps that
your timer thread was doing (and all those pollers were doing as well) were
actually 4ms sleeps because you know what unless kernel is configured with
HZ=1000, you can't sleep for less than 4ms (HZ=250) or perhaps even 10ms
(HZ=100). Mine was 250.
This actually meant that the original timer thread was firing once, sleeping
for 4ms, firing 3 more times back-to-back, sleeping for 4ms more, firing 4
times back-to-back, etc. It was still firing 20ms timers on time, but 30ms
ones of course were not, since 30ms doesn't divide by 4 evenly. Plus whoever
relied on runtime.reference or switch_micro_time_now() were kind of screwed
because both were running jumpy. Plus whoever assumed that
apr_sleep(1000)
or cond_yield() was sleeping for 1ms were also in for a surprise. It felt
satisfying to find that, however it didn't explain why the same distortions
were observed with rewritten timer thread and disabled RTP timers.
Anyway, I sighed (pretty much like you) and recompiled the kernel with
HZ=1000. Recompiling kernel on these ALIX boards is fun. If smth goes south,
you need to hook up serial console and see what the heck went wrong.
That eliminated distortions, ha! But made freeswitch more CPU hungry. Now
the remaining 1ms threads sitting in sofia epoll were really polling for
1ms, not 4, and freeswitch was consistently sitting in the first line of the
top chart showing 3% CPU utilization when idle.
Don't know whether it's because of the remaining epolls in sofia or whether
it's because there are still some threads left in freeswitch that I
neglected to change because they were sleeping with 100ms interval, so I
figured, who cares. Maybe when all things come together (sofia, 100ms*N)
freeswitch ends up spending 3% of CPU while doing pretty much nothing.
Btw, compared with Asterisk, the latter is not even visible on the first
top's screen and spends 1% CPU when bridging two G711 calls and recording
them to disk.
So, at this time I have both original Asterisk and FS setups running. One is
seemless but clumsy in configuration, the other one is neat and stylish but
too preoccupied with smth... Should I look into sofia epollers? That's kind
of deep in the code. Or should I just stick with Asterisk?
Post by Anthony Minessale
There is another user here with a 300mhz box. I am willing to
investigate
Post by Anthony Minessale
this improved performance for weak devices but I need to do it in a
sane
Post by Anthony Minessale
cross-platform way.
On Fri, Dec 4, 2009 at 1:32 PM, Yossi Neiman
<freeswitch at cartissolutions.com
Post by Yossi Neiman
A word to the wise to the general FreeSWITCH community: If Anthony
Minessale suggests that you try to do any number of things, it's a
very
Post by Anthony Minessale
Post by Yossi Neiman
good idea to try all those ideas before continuing on. I've known
him,
Post by Anthony Minessale
Post by Yossi Neiman
MikeJ, and bkw for several years, and they almost always have very
good
Post by Anthony Minessale
Post by Yossi Neiman
ideas as to troubleshoot a problem in FreeSWITCH. It's extremely
frustrating to try to help people out who won't try the provided
suggestions first.
And note directly to "eaf" - bogomips is quite possibly the least
significant bit of data about a cpu that you will get out of
/proc/cpuinfo... The name itself - bogo, means bogus.
http://en.wikipedia.org/wiki/Bogomips
-Yossi
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by Yossi Neiman
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com
<MSN%3Aanthony_minessale at hotmail.com><
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org
<sip%3A888 at conference.freeswitch.org><
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26678873.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26684048.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Anthony Minessale
2009-12-07 21:29:15 UTC
Permalink
Both,
if it always sounds ok then I guess CPU usage.
Post by eaf
What do you want me to check while running these tests? Sound quality (it's
good now even with original 1.0.4). Or CPU utilization?
It's Debian 4.
Post by Anthony Minessale
Did you do each thing alone too to tell the difference?
-hp alone, disable monotonic alone (i did not see you mention the disable
monotonic)
as for your 4ms thing, yes we require high resolution timing, if we ask
to
Post by Anthony Minessale
sleep 1000 microseconds that is what we need it to sleep for or at least as
close as possible, and the main reason that thread is never sleeping is
because you can't actually count on it to run every 1ms but you mostly can.
Hence the whole philosophy on only making 1 thread run hot all the time
to
Post by Anthony Minessale
ensure that the rest don't have to repeat the same algorithm. We focus
on
Post by Anthony Minessale
high end performance this was the point of your experimentation because
we
Post by Anthony Minessale
will need to use a compile time defines and other logic to make it more
efficient on your platform, a platform which we are not using. I am curious
what would happen if you install Kristian's astlinux on one of your devices,
i think you should also compare the kernel versions.
What OS are you running anyway?
Here are some more things to try (running plain trunk with no mods) do these
systematically each alone and all together with/without -hp or disable
monotonic etc to see what different combos create
comment out this line (line 10)
#define DISABLE_1MS_COND
rebuild, this tells it to run a conditional at 1ms in the same timer thread
which will make all the switch_cond_next share a 1ms conditional instead of
doing microsleeps
next
some kernels/devices work better using select(0) for sleep where others work
better using usleep.
comment out line 109
apr_sleep(t);
and try
usleep(t)
also mac works better using nanosleep so you could try changing it so it
uses the code starting at 101 instead.
also your claim about JS should be investigated because I do not think it
should be the case.
but you may want to move this to a jira http://jira.freeswitch.org
As for the asterisk comparison,
not sure how to answer you, that's your decision.
Post by eaf
Here is what I found...
I tried high-priority scheduling as per your suggestion, reniced the program
explicitly, rewrote timer thread to sleep on cond. variable and activate
only when there are timers and only when the timer actually had to be
clicked, turned off SQL thread and removed polling from sofia profile
thread.
That pretty much eliminated all idle 1ms sleepers that were there except for
three in sofia itself (su_epoll_port). And when I was about to be happy, I
found that two outgoing calls through my VOIP providers when bridged
together showed terrible distortions. I undid all my changes, tried 1.0.4,
trunk (noticed btw that when I bridge two calls via loopback in JS in
the
Post by Anthony Minessale
Post by eaf
trunk I must keep JS running, or the calls get terminated - NOT the same as
in 1.0.4 where exitting JS left calls running), got pretty much the same sad
results. At the same time calls bridged by freeswitch between LAN and
any
Post by Anthony Minessale
Post by eaf
of
the VOIP providers behaved just fine. And calls bridged by Asterisk any way
were fine too. So that pretty much looked like the end of the freeswitch
trials for me.
But then I timed your code, mine and found that all those 1ms sleeps
that
Post by Anthony Minessale
Post by eaf
your timer thread was doing (and all those pollers were doing as well) were
actually 4ms sleeps because you know what unless kernel is configured with
HZ=1000, you can't sleep for less than 4ms (HZ=250) or perhaps even 10ms
(HZ=100). Mine was 250.
This actually meant that the original timer thread was firing once, sleeping
for 4ms, firing 3 more times back-to-back, sleeping for 4ms more, firing 4
times back-to-back, etc. It was still firing 20ms timers on time, but 30ms
ones of course were not, since 30ms doesn't divide by 4 evenly. Plus whoever
relied on runtime.reference or switch_micro_time_now() were kind of screwed
because both were running jumpy. Plus whoever assumed that
apr_sleep(1000)
or cond_yield() was sleeping for 1ms were also in for a surprise. It
felt
Post by Anthony Minessale
Post by eaf
satisfying to find that, however it didn't explain why the same distortions
were observed with rewritten timer thread and disabled RTP timers.
Anyway, I sighed (pretty much like you) and recompiled the kernel with
HZ=1000. Recompiling kernel on these ALIX boards is fun. If smth goes south,
you need to hook up serial console and see what the heck went wrong.
That eliminated distortions, ha! But made freeswitch more CPU hungry.
Now
Post by Anthony Minessale
Post by eaf
the remaining 1ms threads sitting in sofia epoll were really polling for
1ms, not 4, and freeswitch was consistently sitting in the first line of the
top chart showing 3% CPU utilization when idle.
Don't know whether it's because of the remaining epolls in sofia or whether
it's because there are still some threads left in freeswitch that I
neglected to change because they were sleeping with 100ms interval, so I
figured, who cares. Maybe when all things come together (sofia, 100ms*N)
freeswitch ends up spending 3% of CPU while doing pretty much nothing.
Btw, compared with Asterisk, the latter is not even visible on the first
top's screen and spends 1% CPU when bridging two G711 calls and
recording
Post by Anthony Minessale
Post by eaf
them to disk.
So, at this time I have both original Asterisk and FS setups running.
One
Post by Anthony Minessale
Post by eaf
is
seemless but clumsy in configuration, the other one is neat and stylish but
too preoccupied with smth... Should I look into sofia epollers? That's kind
of deep in the code. Or should I just stick with Asterisk?
Post by Anthony Minessale
There is another user here with a 300mhz box. I am willing to
investigate
Post by Anthony Minessale
this improved performance for weak devices but I need to do it in a
sane
Post by Anthony Minessale
cross-platform way.
On Fri, Dec 4, 2009 at 1:32 PM, Yossi Neiman
<freeswitch at cartissolutions.com
Post by Yossi Neiman
A word to the wise to the general FreeSWITCH community: If Anthony
Minessale suggests that you try to do any number of things, it's a
very
Post by Anthony Minessale
Post by Yossi Neiman
good idea to try all those ideas before continuing on. I've known
him,
Post by Anthony Minessale
Post by Yossi Neiman
MikeJ, and bkw for several years, and they almost always have very
good
Post by Anthony Minessale
Post by Yossi Neiman
ideas as to troubleshoot a problem in FreeSWITCH. It's extremely
frustrating to try to help people out who won't try the provided
suggestions first.
And note directly to "eaf" - bogomips is quite possibly the least
significant bit of data about a cpu that you will get out of
/proc/cpuinfo... The name itself - bogo, means bogus.
http://en.wikipedia.org/wiki/Bogomips
-Yossi
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by Yossi Neiman
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com<MSN%3Aanthony_minessale at hotmail.com>
<MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
<
Post by eaf
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
<MSN%253Aanthony_minessale at hotmail.com<MSN%25253Aanthony_minessale at hotmail.com>
Post by Anthony Minessale
Post by eaf
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
Post by eaf
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
<PAYPAL%253Aanthony.minessale at gmail.com<PAYPAL%25253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org<sip%3A888 at conference.freeswitch.org>
<sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
<
Post by eaf
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
<sip%253A888 at conference.freeswitch.org<sip%25253A888 at conference.freeswitch.org>
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by Anthony Minessale
Post by eaf
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
<googletalk%253Aconf%252B888 at conference.freeswitch.org<googletalk%25253Aconf%25252B888 at conference.freeswitch.org>
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26678873.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by eaf
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com><
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
Post by Anthony Minessale
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org><
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26684048.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091207/625b8007/attachment-0001.html
Anthony Minessale
2009-12-07 21:29:15 UTC
Permalink
Both,
if it always sounds ok then I guess CPU usage.
Post by eaf
What do you want me to check while running these tests? Sound quality (it's
good now even with original 1.0.4). Or CPU utilization?
It's Debian 4.
Post by Anthony Minessale
Did you do each thing alone too to tell the difference?
-hp alone, disable monotonic alone (i did not see you mention the disable
monotonic)
as for your 4ms thing, yes we require high resolution timing, if we ask
to
Post by Anthony Minessale
sleep 1000 microseconds that is what we need it to sleep for or at least as
close as possible, and the main reason that thread is never sleeping is
because you can't actually count on it to run every 1ms but you mostly can.
Hence the whole philosophy on only making 1 thread run hot all the time
to
Post by Anthony Minessale
ensure that the rest don't have to repeat the same algorithm. We focus
on
Post by Anthony Minessale
high end performance this was the point of your experimentation because
we
Post by Anthony Minessale
will need to use a compile time defines and other logic to make it more
efficient on your platform, a platform which we are not using. I am curious
what would happen if you install Kristian's astlinux on one of your devices,
i think you should also compare the kernel versions.
What OS are you running anyway?
Here are some more things to try (running plain trunk with no mods) do these
systematically each alone and all together with/without -hp or disable
monotonic etc to see what different combos create
comment out this line (line 10)
#define DISABLE_1MS_COND
rebuild, this tells it to run a conditional at 1ms in the same timer thread
which will make all the switch_cond_next share a 1ms conditional instead of
doing microsleeps
next
some kernels/devices work better using select(0) for sleep where others work
better using usleep.
comment out line 109
apr_sleep(t);
and try
usleep(t)
also mac works better using nanosleep so you could try changing it so it
uses the code starting at 101 instead.
also your claim about JS should be investigated because I do not think it
should be the case.
but you may want to move this to a jira http://jira.freeswitch.org
As for the asterisk comparison,
not sure how to answer you, that's your decision.
Post by eaf
Here is what I found...
I tried high-priority scheduling as per your suggestion, reniced the program
explicitly, rewrote timer thread to sleep on cond. variable and activate
only when there are timers and only when the timer actually had to be
clicked, turned off SQL thread and removed polling from sofia profile
thread.
That pretty much eliminated all idle 1ms sleepers that were there except for
three in sofia itself (su_epoll_port). And when I was about to be happy, I
found that two outgoing calls through my VOIP providers when bridged
together showed terrible distortions. I undid all my changes, tried 1.0.4,
trunk (noticed btw that when I bridge two calls via loopback in JS in
the
Post by Anthony Minessale
Post by eaf
trunk I must keep JS running, or the calls get terminated - NOT the same as
in 1.0.4 where exitting JS left calls running), got pretty much the same sad
results. At the same time calls bridged by freeswitch between LAN and
any
Post by Anthony Minessale
Post by eaf
of
the VOIP providers behaved just fine. And calls bridged by Asterisk any way
were fine too. So that pretty much looked like the end of the freeswitch
trials for me.
But then I timed your code, mine and found that all those 1ms sleeps
that
Post by Anthony Minessale
Post by eaf
your timer thread was doing (and all those pollers were doing as well) were
actually 4ms sleeps because you know what unless kernel is configured with
HZ=1000, you can't sleep for less than 4ms (HZ=250) or perhaps even 10ms
(HZ=100). Mine was 250.
This actually meant that the original timer thread was firing once, sleeping
for 4ms, firing 3 more times back-to-back, sleeping for 4ms more, firing 4
times back-to-back, etc. It was still firing 20ms timers on time, but 30ms
ones of course were not, since 30ms doesn't divide by 4 evenly. Plus whoever
relied on runtime.reference or switch_micro_time_now() were kind of screwed
because both were running jumpy. Plus whoever assumed that
apr_sleep(1000)
or cond_yield() was sleeping for 1ms were also in for a surprise. It
felt
Post by Anthony Minessale
Post by eaf
satisfying to find that, however it didn't explain why the same distortions
were observed with rewritten timer thread and disabled RTP timers.
Anyway, I sighed (pretty much like you) and recompiled the kernel with
HZ=1000. Recompiling kernel on these ALIX boards is fun. If smth goes south,
you need to hook up serial console and see what the heck went wrong.
That eliminated distortions, ha! But made freeswitch more CPU hungry.
Now
Post by Anthony Minessale
Post by eaf
the remaining 1ms threads sitting in sofia epoll were really polling for
1ms, not 4, and freeswitch was consistently sitting in the first line of the
top chart showing 3% CPU utilization when idle.
Don't know whether it's because of the remaining epolls in sofia or whether
it's because there are still some threads left in freeswitch that I
neglected to change because they were sleeping with 100ms interval, so I
figured, who cares. Maybe when all things come together (sofia, 100ms*N)
freeswitch ends up spending 3% of CPU while doing pretty much nothing.
Btw, compared with Asterisk, the latter is not even visible on the first
top's screen and spends 1% CPU when bridging two G711 calls and
recording
Post by Anthony Minessale
Post by eaf
them to disk.
So, at this time I have both original Asterisk and FS setups running.
One
Post by Anthony Minessale
Post by eaf
is
seemless but clumsy in configuration, the other one is neat and stylish but
too preoccupied with smth... Should I look into sofia epollers? That's kind
of deep in the code. Or should I just stick with Asterisk?
Post by Anthony Minessale
There is another user here with a 300mhz box. I am willing to
investigate
Post by Anthony Minessale
this improved performance for weak devices but I need to do it in a
sane
Post by Anthony Minessale
cross-platform way.
On Fri, Dec 4, 2009 at 1:32 PM, Yossi Neiman
<freeswitch at cartissolutions.com
Post by Yossi Neiman
A word to the wise to the general FreeSWITCH community: If Anthony
Minessale suggests that you try to do any number of things, it's a
very
Post by Anthony Minessale
Post by Yossi Neiman
good idea to try all those ideas before continuing on. I've known
him,
Post by Anthony Minessale
Post by Yossi Neiman
MikeJ, and bkw for several years, and they almost always have very
good
Post by Anthony Minessale
Post by Yossi Neiman
ideas as to troubleshoot a problem in FreeSWITCH. It's extremely
frustrating to try to help people out who won't try the provided
suggestions first.
And note directly to "eaf" - bogomips is quite possibly the least
significant bit of data about a cpu that you will get out of
/proc/cpuinfo... The name itself - bogo, means bogus.
http://en.wikipedia.org/wiki/Bogomips
-Yossi
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by Yossi Neiman
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com<MSN%3Aanthony_minessale at hotmail.com>
<MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
<
Post by eaf
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
<MSN%253Aanthony_minessale at hotmail.com<MSN%25253Aanthony_minessale at hotmail.com>
Post by Anthony Minessale
Post by eaf
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
Post by eaf
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
<PAYPAL%253Aanthony.minessale at gmail.com<PAYPAL%25253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org<sip%3A888 at conference.freeswitch.org>
<sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
<
Post by eaf
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
<sip%253A888 at conference.freeswitch.org<sip%25253A888 at conference.freeswitch.org>
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by Anthony Minessale
Post by eaf
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
<googletalk%253Aconf%252B888 at conference.freeswitch.org<googletalk%25253Aconf%25252B888 at conference.freeswitch.org>
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26678873.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by eaf
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com><
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
Post by Anthony Minessale
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org><
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26684048.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091207/625b8007/attachment-0002.html
Kristian Kielhofner
2009-12-08 18:30:12 UTC
Permalink
For reference, here is the AstLinux kernel config for the ALIX:

http://astlinux.svn.sourceforge.net/viewvc/astlinux/trunk/target/device/alix/linux.config?view=markup

We've got what I consider to be excellent support for the ALIX - most
of the developers use them and they are very popular in the community.

On Mon, Dec 7, 2009 at 11:00 AM, Anthony Minessale
Post by Anthony Minessale
Did you do each thing alone too to tell the difference?
-hp alone, disable monotonic alone (i did not see you mention the disable
monotonic)
as for your 4ms thing, yes we require high resolution timing, if we ask to
sleep 1000 microseconds that is what we need it to sleep for or at least as
close as possible, and the main reason that thread is never sleeping is
because you can't actually count on it to run every 1ms but you mostly can.
Hence the whole philosophy on only making 1 thread run hot all the time to
ensure that the rest don't have to repeat the same algorithm.? We focus on
high end performance this was the point of your experimentation because we
will need to use a compile time defines and other logic to make it more
efficient on your platform, a platform which we are not using.? I am curious
what would happen if you install Kristian's astlinux on one of your devices,
i think you should also compare the kernel versions.
What OS are you running anyway?
Here are some more things to try (running plain trunk with no mods) do these
systematically each alone and all together with/without -hp or disable
monotonic etc to see what different combos create
comment out this line (line 10)
#define DISABLE_1MS_COND
rebuild, this tells it to run a conditional at 1ms in the same timer thread
which will make all the switch_cond_next share a 1ms conditional instead of
doing microsleeps
next
some kernels/devices work better using select(0) for sleep where others work
better using usleep.
comment out line 109
apr_sleep(t);
and try
usleep(t)
also mac works better using nanosleep so you could try changing it so it
uses the code starting at 101 instead.
also your claim about JS should be investigated because I do not think it
should be the case.
but you may want to move this to a jira http://jira.freeswitch.org
As for the asterisk comparison,
not sure how to answer you, that's your decision.
--
Kristian Kielhofner
http://www.astlinux.org
http://blog.krisk.org
http://www.star2star.com
http://www.submityoursip.com
http://www.voalte.com
Michael Jerris
2009-12-07 16:16:30 UTC
Permalink
Also I have seen some people reporting that the new tickless timers in newer kernels work better. You may want to try those.

Mike
Post by Anthony Minessale
Did you do each thing alone too to tell the difference?
-hp alone, disable monotonic alone (i did not see you mention the disable monotonic)
as for your 4ms thing, yes we require high resolution timing, if we ask to sleep 1000 microseconds that is what we need it to sleep for or at least as close as possible, and the main reason that thread is never sleeping is because you can't actually count on it to run every 1ms but you mostly can. Hence the whole philosophy on only making 1 thread run hot all the time to ensure that the rest don't have to repeat the same algorithm. We focus on high end performance this was the point of your experimentation because we will need to use a compile time defines and other logic to make it more efficient on your platform, a platform which we are not using. I am curious what would happen if you install Kristian's astlinux on one of your devices, i think you should also compare the kernel versions.
What OS are you running anyway?
Here are some more things to try (running plain trunk with no mods) do these systematically each alone and all together with/without -hp or disable monotonic etc to see what different combos create
comment out this line (line 10)
#define DISABLE_1MS_COND
rebuild, this tells it to run a conditional at 1ms in the same timer thread which will make all the switch_cond_next share a 1ms conditional instead of doing microsleeps
next
some kernels/devices work better using select(0) for sleep where others work better using usleep.
comment out line 109
apr_sleep(t);
and try
usleep(t)
also mac works better using nanosleep so you could try changing it so it
uses the code starting at 101 instead.
also your claim about JS should be investigated because I do not think it should be the case.
but you may want to move this to a jira http://jira.freeswitch.org
As for the asterisk comparison,
not sure how to answer you, that's your decision.
Here is what I found...
I tried high-priority scheduling as per your suggestion, reniced the program
explicitly, rewrote timer thread to sleep on cond. variable and activate
only when there are timers and only when the timer actually had to be
clicked, turned off SQL thread and removed polling from sofia profile
thread.
That pretty much eliminated all idle 1ms sleepers that were there except for
three in sofia itself (su_epoll_port). And when I was about to be happy, I
found that two outgoing calls through my VOIP providers when bridged
together showed terrible distortions. I undid all my changes, tried 1.0.4,
trunk (noticed btw that when I bridge two calls via loopback in JS in the
trunk I must keep JS running, or the calls get terminated - NOT the same as
in 1.0.4 where exitting JS left calls running), got pretty much the same sad
results. At the same time calls bridged by freeswitch between LAN and any of
the VOIP providers behaved just fine. And calls bridged by Asterisk any way
were fine too. So that pretty much looked like the end of the freeswitch
trials for me.
But then I timed your code, mine and found that all those 1ms sleeps that
your timer thread was doing (and all those pollers were doing as well) were
actually 4ms sleeps because you know what unless kernel is configured with
HZ=1000, you can't sleep for less than 4ms (HZ=250) or perhaps even 10ms
(HZ=100). Mine was 250.
This actually meant that the original timer thread was firing once, sleeping
for 4ms, firing 3 more times back-to-back, sleeping for 4ms more, firing 4
times back-to-back, etc. It was still firing 20ms timers on time, but 30ms
ones of course were not, since 30ms doesn't divide by 4 evenly. Plus whoever
relied on runtime.reference or switch_micro_time_now() were kind of screwed
because both were running jumpy. Plus whoever assumed that apr_sleep(1000)
or cond_yield() was sleeping for 1ms were also in for a surprise. It felt
satisfying to find that, however it didn't explain why the same distortions
were observed with rewritten timer thread and disabled RTP timers.
Anyway, I sighed (pretty much like you) and recompiled the kernel with
HZ=1000. Recompiling kernel on these ALIX boards is fun. If smth goes south,
you need to hook up serial console and see what the heck went wrong.
That eliminated distortions, ha! But made freeswitch more CPU hungry. Now
the remaining 1ms threads sitting in sofia epoll were really polling for
1ms, not 4, and freeswitch was consistently sitting in the first line of the
top chart showing 3% CPU utilization when idle.
Don't know whether it's because of the remaining epolls in sofia or whether
it's because there are still some threads left in freeswitch that I
neglected to change because they were sleeping with 100ms interval, so I
figured, who cares. Maybe when all things come together (sofia, 100ms*N)
freeswitch ends up spending 3% of CPU while doing pretty much nothing.
Btw, compared with Asterisk, the latter is not even visible on the first
top's screen and spends 1% CPU when bridging two G711 calls and recording
them to disk.
So, at this time I have both original Asterisk and FS setups running. One is
seemless but clumsy in configuration, the other one is neat and stylish but
too preoccupied with smth... Should I look into sofia epollers? That's kind
of deep in the code. Or should I just stick with Asterisk?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091207/31a8a586/attachment-0002.html
eaf
2009-12-07 20:58:55 UTC
Permalink
What do you want me to check while running these tests? Sound quality (it's
good now even with original 1.0.4). Or CPU utilization?

It's Debian 4.
Post by Anthony Minessale
Did you do each thing alone too to tell the difference?
-hp alone, disable monotonic alone (i did not see you mention the disable
monotonic)
as for your 4ms thing, yes we require high resolution timing, if we ask to
sleep 1000 microseconds that is what we need it to sleep for or at least as
close as possible, and the main reason that thread is never sleeping is
because you can't actually count on it to run every 1ms but you mostly can.
Hence the whole philosophy on only making 1 thread run hot all the time to
ensure that the rest don't have to repeat the same algorithm. We focus on
high end performance this was the point of your experimentation because we
will need to use a compile time defines and other logic to make it more
efficient on your platform, a platform which we are not using. I am curious
what would happen if you install Kristian's astlinux on one of your devices,
i think you should also compare the kernel versions.
What OS are you running anyway?
Here are some more things to try (running plain trunk with no mods) do these
systematically each alone and all together with/without -hp or disable
monotonic etc to see what different combos create
comment out this line (line 10)
#define DISABLE_1MS_COND
rebuild, this tells it to run a conditional at 1ms in the same timer thread
which will make all the switch_cond_next share a 1ms conditional instead of
doing microsleeps
next
some kernels/devices work better using select(0) for sleep where others work
better using usleep.
comment out line 109
apr_sleep(t);
and try
usleep(t)
also mac works better using nanosleep so you could try changing it so it
uses the code starting at 101 instead.
also your claim about JS should be investigated because I do not think it
should be the case.
but you may want to move this to a jira http://jira.freeswitch.org
As for the asterisk comparison,
not sure how to answer you, that's your decision.
Post by eaf
Here is what I found...
I tried high-priority scheduling as per your suggestion, reniced the program
explicitly, rewrote timer thread to sleep on cond. variable and activate
only when there are timers and only when the timer actually had to be
clicked, turned off SQL thread and removed polling from sofia profile
thread.
That pretty much eliminated all idle 1ms sleepers that were there except for
three in sofia itself (su_epoll_port). And when I was about to be happy, I
found that two outgoing calls through my VOIP providers when bridged
together showed terrible distortions. I undid all my changes, tried 1.0.4,
trunk (noticed btw that when I bridge two calls via loopback in JS in the
trunk I must keep JS running, or the calls get terminated - NOT the same as
in 1.0.4 where exitting JS left calls running), got pretty much the same sad
results. At the same time calls bridged by freeswitch between LAN and any of
the VOIP providers behaved just fine. And calls bridged by Asterisk any way
were fine too. So that pretty much looked like the end of the freeswitch
trials for me.
But then I timed your code, mine and found that all those 1ms sleeps that
your timer thread was doing (and all those pollers were doing as well) were
actually 4ms sleeps because you know what unless kernel is configured with
HZ=1000, you can't sleep for less than 4ms (HZ=250) or perhaps even 10ms
(HZ=100). Mine was 250.
This actually meant that the original timer thread was firing once, sleeping
for 4ms, firing 3 more times back-to-back, sleeping for 4ms more, firing 4
times back-to-back, etc. It was still firing 20ms timers on time, but 30ms
ones of course were not, since 30ms doesn't divide by 4 evenly. Plus whoever
relied on runtime.reference or switch_micro_time_now() were kind of screwed
because both were running jumpy. Plus whoever assumed that
apr_sleep(1000)
or cond_yield() was sleeping for 1ms were also in for a surprise. It felt
satisfying to find that, however it didn't explain why the same distortions
were observed with rewritten timer thread and disabled RTP timers.
Anyway, I sighed (pretty much like you) and recompiled the kernel with
HZ=1000. Recompiling kernel on these ALIX boards is fun. If smth goes south,
you need to hook up serial console and see what the heck went wrong.
That eliminated distortions, ha! But made freeswitch more CPU hungry. Now
the remaining 1ms threads sitting in sofia epoll were really polling for
1ms, not 4, and freeswitch was consistently sitting in the first line of the
top chart showing 3% CPU utilization when idle.
Don't know whether it's because of the remaining epolls in sofia or whether
it's because there are still some threads left in freeswitch that I
neglected to change because they were sleeping with 100ms interval, so I
figured, who cares. Maybe when all things come together (sofia, 100ms*N)
freeswitch ends up spending 3% of CPU while doing pretty much nothing.
Btw, compared with Asterisk, the latter is not even visible on the first
top's screen and spends 1% CPU when bridging two G711 calls and recording
them to disk.
So, at this time I have both original Asterisk and FS setups running. One is
seemless but clumsy in configuration, the other one is neat and stylish but
too preoccupied with smth... Should I look into sofia epollers? That's kind
of deep in the code. Or should I just stick with Asterisk?
Post by Anthony Minessale
There is another user here with a 300mhz box. I am willing to
investigate
Post by Anthony Minessale
this improved performance for weak devices but I need to do it in a
sane
Post by Anthony Minessale
cross-platform way.
On Fri, Dec 4, 2009 at 1:32 PM, Yossi Neiman
<freeswitch at cartissolutions.com
Post by Yossi Neiman
A word to the wise to the general FreeSWITCH community: If Anthony
Minessale suggests that you try to do any number of things, it's a
very
Post by Anthony Minessale
Post by Yossi Neiman
good idea to try all those ideas before continuing on. I've known
him,
Post by Anthony Minessale
Post by Yossi Neiman
MikeJ, and bkw for several years, and they almost always have very
good
Post by Anthony Minessale
Post by Yossi Neiman
ideas as to troubleshoot a problem in FreeSWITCH. It's extremely
frustrating to try to help people out who won't try the provided
suggestions first.
And note directly to "eaf" - bogomips is quite possibly the least
significant bit of data about a cpu that you will get out of
/proc/cpuinfo... The name itself - bogo, means bogus.
http://en.wikipedia.org/wiki/Bogomips
-Yossi
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by Yossi Neiman
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com
<MSN%3Aanthony_minessale at hotmail.com><
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org
<sip%3A888 at conference.freeswitch.org><
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26678873.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26684048.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Kristian Kielhofner
2009-12-08 18:30:12 UTC
Permalink
For reference, here is the AstLinux kernel config for the ALIX:

http://astlinux.svn.sourceforge.net/viewvc/astlinux/trunk/target/device/alix/linux.config?view=markup

We've got what I consider to be excellent support for the ALIX - most
of the developers use them and they are very popular in the community.

On Mon, Dec 7, 2009 at 11:00 AM, Anthony Minessale
Post by Anthony Minessale
Did you do each thing alone too to tell the difference?
-hp alone, disable monotonic alone (i did not see you mention the disable
monotonic)
as for your 4ms thing, yes we require high resolution timing, if we ask to
sleep 1000 microseconds that is what we need it to sleep for or at least as
close as possible, and the main reason that thread is never sleeping is
because you can't actually count on it to run every 1ms but you mostly can.
Hence the whole philosophy on only making 1 thread run hot all the time to
ensure that the rest don't have to repeat the same algorithm.? We focus on
high end performance this was the point of your experimentation because we
will need to use a compile time defines and other logic to make it more
efficient on your platform, a platform which we are not using.? I am curious
what would happen if you install Kristian's astlinux on one of your devices,
i think you should also compare the kernel versions.
What OS are you running anyway?
Here are some more things to try (running plain trunk with no mods) do these
systematically each alone and all together with/without -hp or disable
monotonic etc to see what different combos create
comment out this line (line 10)
#define DISABLE_1MS_COND
rebuild, this tells it to run a conditional at 1ms in the same timer thread
which will make all the switch_cond_next share a 1ms conditional instead of
doing microsleeps
next
some kernels/devices work better using select(0) for sleep where others work
better using usleep.
comment out line 109
apr_sleep(t);
and try
usleep(t)
also mac works better using nanosleep so you could try changing it so it
uses the code starting at 101 instead.
also your claim about JS should be investigated because I do not think it
should be the case.
but you may want to move this to a jira http://jira.freeswitch.org
As for the asterisk comparison,
not sure how to answer you, that's your decision.
--
Kristian Kielhofner
http://www.astlinux.org
http://blog.krisk.org
http://www.star2star.com
http://www.submityoursip.com
http://www.voalte.com
Anthony Minessale
2009-12-07 16:00:17 UTC
Permalink
Did you do each thing alone too to tell the difference?
-hp alone, disable monotonic alone (i did not see you mention the disable
monotonic)

as for your 4ms thing, yes we require high resolution timing, if we ask to
sleep 1000 microseconds that is what we need it to sleep for or at least as
close as possible, and the main reason that thread is never sleeping is
because you can't actually count on it to run every 1ms but you mostly can.
Hence the whole philosophy on only making 1 thread run hot all the time to
ensure that the rest don't have to repeat the same algorithm. We focus on
high end performance this was the point of your experimentation because we
will need to use a compile time defines and other logic to make it more
efficient on your platform, a platform which we are not using. I am curious
what would happen if you install Kristian's astlinux on one of your devices,
i think you should also compare the kernel versions.


What OS are you running anyway?

Here are some more things to try (running plain trunk with no mods) do these
systematically each alone and all together with/without -hp or disable
monotonic etc to see what different combos create

comment out this line (line 10)
#define DISABLE_1MS_COND

rebuild, this tells it to run a conditional at 1ms in the same timer thread
which will make all the switch_cond_next share a 1ms conditional instead of
doing microsleeps

next

some kernels/devices work better using select(0) for sleep where others work
better using usleep.
comment out line 109
apr_sleep(t);

and try
usleep(t)

also mac works better using nanosleep so you could try changing it so it
uses the code starting at 101 instead.


also your claim about JS should be investigated because I do not think it
should be the case.
but you may want to move this to a jira http://jira.freeswitch.org

As for the asterisk comparison,
not sure how to answer you, that's your decision.
Post by eaf
Here is what I found...
I tried high-priority scheduling as per your suggestion, reniced the program
explicitly, rewrote timer thread to sleep on cond. variable and activate
only when there are timers and only when the timer actually had to be
clicked, turned off SQL thread and removed polling from sofia profile
thread.
That pretty much eliminated all idle 1ms sleepers that were there except for
three in sofia itself (su_epoll_port). And when I was about to be happy, I
found that two outgoing calls through my VOIP providers when bridged
together showed terrible distortions. I undid all my changes, tried 1.0.4,
trunk (noticed btw that when I bridge two calls via loopback in JS in the
trunk I must keep JS running, or the calls get terminated - NOT the same as
in 1.0.4 where exitting JS left calls running), got pretty much the same sad
results. At the same time calls bridged by freeswitch between LAN and any of
the VOIP providers behaved just fine. And calls bridged by Asterisk any way
were fine too. So that pretty much looked like the end of the freeswitch
trials for me.
But then I timed your code, mine and found that all those 1ms sleeps that
your timer thread was doing (and all those pollers were doing as well) were
actually 4ms sleeps because you know what unless kernel is configured with
HZ=1000, you can't sleep for less than 4ms (HZ=250) or perhaps even 10ms
(HZ=100). Mine was 250.
This actually meant that the original timer thread was firing once, sleeping
for 4ms, firing 3 more times back-to-back, sleeping for 4ms more, firing 4
times back-to-back, etc. It was still firing 20ms timers on time, but 30ms
ones of course were not, since 30ms doesn't divide by 4 evenly. Plus whoever
relied on runtime.reference or switch_micro_time_now() were kind of screwed
because both were running jumpy. Plus whoever assumed that apr_sleep(1000)
or cond_yield() was sleeping for 1ms were also in for a surprise. It felt
satisfying to find that, however it didn't explain why the same distortions
were observed with rewritten timer thread and disabled RTP timers.
Anyway, I sighed (pretty much like you) and recompiled the kernel with
HZ=1000. Recompiling kernel on these ALIX boards is fun. If smth goes south,
you need to hook up serial console and see what the heck went wrong.
That eliminated distortions, ha! But made freeswitch more CPU hungry. Now
the remaining 1ms threads sitting in sofia epoll were really polling for
1ms, not 4, and freeswitch was consistently sitting in the first line of the
top chart showing 3% CPU utilization when idle.
Don't know whether it's because of the remaining epolls in sofia or whether
it's because there are still some threads left in freeswitch that I
neglected to change because they were sleeping with 100ms interval, so I
figured, who cares. Maybe when all things come together (sofia, 100ms*N)
freeswitch ends up spending 3% of CPU while doing pretty much nothing.
Btw, compared with Asterisk, the latter is not even visible on the first
top's screen and spends 1% CPU when bridging two G711 calls and recording
them to disk.
So, at this time I have both original Asterisk and FS setups running. One is
seemless but clumsy in configuration, the other one is neat and stylish but
too preoccupied with smth... Should I look into sofia epollers? That's kind
of deep in the code. Or should I just stick with Asterisk?
Post by Anthony Minessale
There is another user here with a 300mhz box. I am willing to
investigate
Post by Anthony Minessale
this improved performance for weak devices but I need to do it in a sane
cross-platform way.
On Fri, Dec 4, 2009 at 1:32 PM, Yossi Neiman
<freeswitch at cartissolutions.com
Post by Yossi Neiman
A word to the wise to the general FreeSWITCH community: If Anthony
Minessale suggests that you try to do any number of things, it's a very
good idea to try all those ideas before continuing on. I've known him,
MikeJ, and bkw for several years, and they almost always have very good
ideas as to troubleshoot a problem in FreeSWITCH. It's extremely
frustrating to try to help people out who won't try the provided
suggestions first.
And note directly to "eaf" - bogomips is quite possibly the least
significant bit of data about a cpu that you will get out of
/proc/cpuinfo... The name itself - bogo, means bogus.
http://en.wikipedia.org/wiki/Bogomips
-Yossi
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by Yossi Neiman
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com><
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
Post by Anthony Minessale
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org><
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26678873.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091207/a83d6006/attachment-0002.html
eaf
2009-12-07 15:28:52 UTC
Permalink
Here is what I found...

I tried high-priority scheduling as per your suggestion, reniced the program
explicitly, rewrote timer thread to sleep on cond. variable and activate
only when there are timers and only when the timer actually had to be
clicked, turned off SQL thread and removed polling from sofia profile
thread.

That pretty much eliminated all idle 1ms sleepers that were there except for
three in sofia itself (su_epoll_port). And when I was about to be happy, I
found that two outgoing calls through my VOIP providers when bridged
together showed terrible distortions. I undid all my changes, tried 1.0.4,
trunk (noticed btw that when I bridge two calls via loopback in JS in the
trunk I must keep JS running, or the calls get terminated - NOT the same as
in 1.0.4 where exitting JS left calls running), got pretty much the same sad
results. At the same time calls bridged by freeswitch between LAN and any of
the VOIP providers behaved just fine. And calls bridged by Asterisk any way
were fine too. So that pretty much looked like the end of the freeswitch
trials for me.

But then I timed your code, mine and found that all those 1ms sleeps that
your timer thread was doing (and all those pollers were doing as well) were
actually 4ms sleeps because you know what unless kernel is configured with
HZ=1000, you can't sleep for less than 4ms (HZ=250) or perhaps even 10ms
(HZ=100). Mine was 250.

This actually meant that the original timer thread was firing once, sleeping
for 4ms, firing 3 more times back-to-back, sleeping for 4ms more, firing 4
times back-to-back, etc. It was still firing 20ms timers on time, but 30ms
ones of course were not, since 30ms doesn't divide by 4 evenly. Plus whoever
relied on runtime.reference or switch_micro_time_now() were kind of screwed
because both were running jumpy. Plus whoever assumed that apr_sleep(1000)
or cond_yield() was sleeping for 1ms were also in for a surprise. It felt
satisfying to find that, however it didn't explain why the same distortions
were observed with rewritten timer thread and disabled RTP timers.

Anyway, I sighed (pretty much like you) and recompiled the kernel with
HZ=1000. Recompiling kernel on these ALIX boards is fun. If smth goes south,
you need to hook up serial console and see what the heck went wrong.

That eliminated distortions, ha! But made freeswitch more CPU hungry. Now
the remaining 1ms threads sitting in sofia epoll were really polling for
1ms, not 4, and freeswitch was consistently sitting in the first line of the
top chart showing 3% CPU utilization when idle.

Don't know whether it's because of the remaining epolls in sofia or whether
it's because there are still some threads left in freeswitch that I
neglected to change because they were sleeping with 100ms interval, so I
figured, who cares. Maybe when all things come together (sofia, 100ms*N)
freeswitch ends up spending 3% of CPU while doing pretty much nothing.

Btw, compared with Asterisk, the latter is not even visible on the first
top's screen and spends 1% CPU when bridging two G711 calls and recording
them to disk.

So, at this time I have both original Asterisk and FS setups running. One is
seemless but clumsy in configuration, the other one is neat and stylish but
too preoccupied with smth... Should I look into sofia epollers? That's kind
of deep in the code. Or should I just stick with Asterisk?
Post by Anthony Minessale
There is another user here with a 300mhz box. I am willing to investigate
this improved performance for weak devices but I need to do it in a sane
cross-platform way.
On Fri, Dec 4, 2009 at 1:32 PM, Yossi Neiman
<freeswitch at cartissolutions.com
Post by Yossi Neiman
A word to the wise to the general FreeSWITCH community: If Anthony
Minessale suggests that you try to do any number of things, it's a very
good idea to try all those ideas before continuing on. I've known him,
MikeJ, and bkw for several years, and they almost always have very good
ideas as to troubleshoot a problem in FreeSWITCH. It's extremely
frustrating to try to help people out who won't try the provided
suggestions first.
And note directly to "eaf" - bogomips is quite possibly the least
significant bit of data about a cpu that you will get out of
/proc/cpuinfo... The name itself - bogo, means bogus.
http://en.wikipedia.org/wiki/Bogomips
-Yossi
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26678873.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Anthony Minessale
2009-12-04 19:48:08 UTC
Permalink
There is another user here with a 300mhz box. I am willing to investigate
this improved performance for weak devices but I need to do it in a sane
cross-platform way.


On Fri, Dec 4, 2009 at 1:32 PM, Yossi Neiman <freeswitch at cartissolutions.com
Post by Yossi Neiman
A word to the wise to the general FreeSWITCH community: If Anthony
Minessale suggests that you try to do any number of things, it's a very
good idea to try all those ideas before continuing on. I've known him,
MikeJ, and bkw for several years, and they almost always have very good
ideas as to troubleshoot a problem in FreeSWITCH. It's extremely
frustrating to try to help people out who won't try the provided
suggestions first.
And note directly to "eaf" - bogomips is quite possibly the least
significant bit of data about a cpu that you will get out of
/proc/cpuinfo... The name itself - bogo, means bogus.
http://en.wikipedia.org/wiki/Bogomips
-Yossi
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091204/1b13c721/attachment-0002.html
Kristian Kielhofner
2009-12-04 22:41:12 UTC
Permalink
A little more data from one of my (our) boxes:

starbox_352 ~ # uname -a
Linux starbox_352 2.6.26.8-astlinux #1 PREEMPT Tue Nov 24 16:20:52 EST
2009 i586 unknown
starbox_352 ~ #

starbox_352 ~ # cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 5
model : 10
model name : Geode(TM) Integrated Processor by AMD PCS
stepping : 2
cpu MHz : 498.053
cache size : 128 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu de pse tsc msr cx8 sep pge cmov clflush mmx mmxext 3dnowext 3dnow
bogomips : 997.21
clflush size : 32
power management:

starbox_352 ~ # cat /etc/astlinux-release
astlinux-s2s-3491
starbox_352 ~ #

I'll find one that has been in production for a while with some
active calls...

On Thu, Dec 3, 2009 at 6:49 PM, Anthony Minessale
Post by Anthony Minessale
Sigh,
You just took it up a notch in terms of disdain and sarcasm.
Why do people always only apologize sarcastically?
I asked you to try the -hp and turn off the monotonic clock just to gather
the results to help you.? You completely missed it and just went on about
the threads.?? Please save the "ok fine the code is perfect, blah blah" if
you would have just read the email and answered the question I might have
cared more about the status of your problem.
I told you both of those threads need to be on their toes because they try
to balance between a certian number of sql stmts or 500ms whatever comes
first.? When there are thousands of events per second being turned into SQL
statements which are in turn compiled into large sql transactions.
If you want to come up with a way that they can sleep longer until there is
a sign of activity and stay busy for a few seconds then slow down again,
that's probably possible but the process is already idle at 0% cpu so maybe
you can appreciate why we are not rushing to work on it.? Maybe I'll give it
a go just to show you it has nothing to do with your problem.
Please don't mock our comment about several years.? You have no idea how
hard this code was to develop and it's truly insulting.? Its clear to see
you are locked into assuming that the busy threads that are not all that
busy because they are constantly yielding to the scheduler is breaking the
timing code.? I begged you to understand me when i told you that the err is
not normal, most boxes do not see it doing nothing and there has to be a
specific problem on your box or configuration.? So instead of working with
us you want to escalate to snotty comments.? That's pretty normal on the
internet I guess.....? If you want to have a constructive conversation about
our core, install FS on a normal box, use it for a few weeks, figure out
everything about how it works then try.... There was pure speculation and
conjecture in your original emails and I never said a word about it until
you kept pushing.
Kristian mentioned he never sees that on that same hardware did you even
consider following up on why that is?
I don't have your device, but I assume if you get it working well it will
certainly help you more than it helps me so you could at least have the
decency to believe what we are trying to tell you.
Post by eaf
Oh, you mean giving FS higher priority? Yeah, as a last resort I'll do that.
At the moment, I hope it won't be necessary as I can make those "hyper"
threads behave, and will see how that goes first. I see where your
implementation could be coming from. There is a queue of SQL queries in
sofia.c processed by the worker thread. There are only two pop functions
available in APR: queue_pop() and queue_trypop(), so alas no option with a
timeout here. You don't want to block the thread in pop() indefinitely
because you chose that same worker needs to do ireg and gw processing once
in a while (separated by tens or hundreds of seconds, btw). You also want to
be able to detect shutdown condition so that the worker doesn't hold up
profile thread. So you chose to poll for events every millisecond instead of
just creating an apr_thread_cond_t for resource friendly signalling.
I agree that the timer thread philosophy is great and was the right choice
for scaling, but I just don't comprehend responses to things like these
other SQL or sofia worker threads. Did somebody even remotely acknowledge
that busy loops at least in those areas that I showed may probably be a bad
idea and could've been eliminated? I've heard suggestions to bump up
priority, I've heard that the code was perfect already, that it's the result
of 4-year effort, that I am arrogant, don't listen and don't understand
squat.
I'm sorry if I gave you impression that I was looking for the bad parts in
the software. I apologized for that already. All I wanted was to have
constructive conversation, perhaps I'm not too good at it. Code is already
perfect according to you? Fine with me.
Post by Anthony Minessale
no,
I mean the one after that that you must have completely skipped with a
command line option to try and a param to set in the config. It somewhat
annoys me for taking the time to compose it now. ?I wrote all of the
code
you are talking about myself and I was trying to give you some
suggestions....
Well, actually, ?you did answer my question about the platform so you
must
have seen it.....
The loops are not the cause of that migration message, something wrong with
the hardware or the kernel is.
Another guy just told you he does not see that problem on the same exact
hardware.
Even if you have a point about the sql threads, you could make a patch to
slow them down but you cant slow down too much or you will not be able to
handle 400 cps all asking to send updates to transactions in batches of
thousands of sql stmts. ?Every line of that code is carefully designed
so
I
don't know what else to tell you but to stop being so arrogant and re-read
this thread for all the advice you have totally ignored. ?I started out
trying to help you but I have a lot of work to do. ?I thoroughly
explained
it to you and you are choosing to ignore me so I guess I'm done.
You can do whatever you want with your working copy, i'll see you in 3
or
4
years when you get up to speed with the rest of us........
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26633739.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Kristian Kielhofner
http://www.astlinux.org
http://blog.krisk.org
http://www.star2star.com
http://www.submityoursip.com
http://www.voalte.com
Yossi Neiman
2009-12-04 19:32:56 UTC
Permalink
A word to the wise to the general FreeSWITCH community: If Anthony
Minessale suggests that you try to do any number of things, it's a very
good idea to try all those ideas before continuing on. I've known him,
MikeJ, and bkw for several years, and they almost always have very good
ideas as to troubleshoot a problem in FreeSWITCH. It's extremely
frustrating to try to help people out who won't try the provided
suggestions first.

And note directly to "eaf" - bogomips is quite possibly the least
significant bit of data about a cpu that you will get out of
/proc/cpuinfo... The name itself - bogo, means bogus.
http://en.wikipedia.org/wiki/Bogomips

-Yossi
Kristian Kielhofner
2009-12-04 22:41:12 UTC
Permalink
A little more data from one of my (our) boxes:

starbox_352 ~ # uname -a
Linux starbox_352 2.6.26.8-astlinux #1 PREEMPT Tue Nov 24 16:20:52 EST
2009 i586 unknown
starbox_352 ~ #

starbox_352 ~ # cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 5
model : 10
model name : Geode(TM) Integrated Processor by AMD PCS
stepping : 2
cpu MHz : 498.053
cache size : 128 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu de pse tsc msr cx8 sep pge cmov clflush mmx mmxext 3dnowext 3dnow
bogomips : 997.21
clflush size : 32
power management:

starbox_352 ~ # cat /etc/astlinux-release
astlinux-s2s-3491
starbox_352 ~ #

I'll find one that has been in production for a while with some
active calls...

On Thu, Dec 3, 2009 at 6:49 PM, Anthony Minessale
Post by Anthony Minessale
Sigh,
You just took it up a notch in terms of disdain and sarcasm.
Why do people always only apologize sarcastically?
I asked you to try the -hp and turn off the monotonic clock just to gather
the results to help you.? You completely missed it and just went on about
the threads.?? Please save the "ok fine the code is perfect, blah blah" if
you would have just read the email and answered the question I might have
cared more about the status of your problem.
I told you both of those threads need to be on their toes because they try
to balance between a certian number of sql stmts or 500ms whatever comes
first.? When there are thousands of events per second being turned into SQL
statements which are in turn compiled into large sql transactions.
If you want to come up with a way that they can sleep longer until there is
a sign of activity and stay busy for a few seconds then slow down again,
that's probably possible but the process is already idle at 0% cpu so maybe
you can appreciate why we are not rushing to work on it.? Maybe I'll give it
a go just to show you it has nothing to do with your problem.
Please don't mock our comment about several years.? You have no idea how
hard this code was to develop and it's truly insulting.? Its clear to see
you are locked into assuming that the busy threads that are not all that
busy because they are constantly yielding to the scheduler is breaking the
timing code.? I begged you to understand me when i told you that the err is
not normal, most boxes do not see it doing nothing and there has to be a
specific problem on your box or configuration.? So instead of working with
us you want to escalate to snotty comments.? That's pretty normal on the
internet I guess.....? If you want to have a constructive conversation about
our core, install FS on a normal box, use it for a few weeks, figure out
everything about how it works then try.... There was pure speculation and
conjecture in your original emails and I never said a word about it until
you kept pushing.
Kristian mentioned he never sees that on that same hardware did you even
consider following up on why that is?
I don't have your device, but I assume if you get it working well it will
certainly help you more than it helps me so you could at least have the
decency to believe what we are trying to tell you.
Post by eaf
Oh, you mean giving FS higher priority? Yeah, as a last resort I'll do that.
At the moment, I hope it won't be necessary as I can make those "hyper"
threads behave, and will see how that goes first. I see where your
implementation could be coming from. There is a queue of SQL queries in
sofia.c processed by the worker thread. There are only two pop functions
available in APR: queue_pop() and queue_trypop(), so alas no option with a
timeout here. You don't want to block the thread in pop() indefinitely
because you chose that same worker needs to do ireg and gw processing once
in a while (separated by tens or hundreds of seconds, btw). You also want to
be able to detect shutdown condition so that the worker doesn't hold up
profile thread. So you chose to poll for events every millisecond instead of
just creating an apr_thread_cond_t for resource friendly signalling.
I agree that the timer thread philosophy is great and was the right choice
for scaling, but I just don't comprehend responses to things like these
other SQL or sofia worker threads. Did somebody even remotely acknowledge
that busy loops at least in those areas that I showed may probably be a bad
idea and could've been eliminated? I've heard suggestions to bump up
priority, I've heard that the code was perfect already, that it's the result
of 4-year effort, that I am arrogant, don't listen and don't understand
squat.
I'm sorry if I gave you impression that I was looking for the bad parts in
the software. I apologized for that already. All I wanted was to have
constructive conversation, perhaps I'm not too good at it. Code is already
perfect according to you? Fine with me.
Post by Anthony Minessale
no,
I mean the one after that that you must have completely skipped with a
command line option to try and a param to set in the config. It somewhat
annoys me for taking the time to compose it now. ?I wrote all of the
code
you are talking about myself and I was trying to give you some
suggestions....
Well, actually, ?you did answer my question about the platform so you
must
have seen it.....
The loops are not the cause of that migration message, something wrong with
the hardware or the kernel is.
Another guy just told you he does not see that problem on the same exact
hardware.
Even if you have a point about the sql threads, you could make a patch to
slow them down but you cant slow down too much or you will not be able to
handle 400 cps all asking to send updates to transactions in batches of
thousands of sql stmts. ?Every line of that code is carefully designed
so
I
don't know what else to tell you but to stop being so arrogant and re-read
this thread for all the advice you have totally ignored. ?I started out
trying to help you but I have a lot of work to do. ?I thoroughly
explained
it to you and you are choosing to ignore me so I guess I'm done.
You can do whatever you want with your working copy, i'll see you in 3
or
4
years when you get up to speed with the rest of us........
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26633739.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Kristian Kielhofner
http://www.astlinux.org
http://blog.krisk.org
http://www.star2star.com
http://www.submityoursip.com
http://www.voalte.com
Anthony Minessale
2009-12-03 23:49:20 UTC
Permalink
Sigh,

You just took it up a notch in terms of disdain and sarcasm.
Why do people always only apologize sarcastically?

I asked you to try the -hp and turn off the monotonic clock just to gather
the results to help you. You completely missed it and just went on about
the threads. Please save the "ok fine the code is perfect, blah blah" if
you would have just read the email and answered the question I might have
cared more about the status of your problem.

I told you both of those threads need to be on their toes because they try
to balance between a certian number of sql stmts or 500ms whatever comes
first. When there are thousands of events per second being turned into SQL
statements which are in turn compiled into large sql transactions.

If you want to come up with a way that they can sleep longer until there is
a sign of activity and stay busy for a few seconds then slow down again,
that's probably possible but the process is already idle at 0% cpu so maybe
you can appreciate why we are not rushing to work on it. Maybe I'll give it
a go just to show you it has nothing to do with your problem.

Please don't mock our comment about several years. You have no idea how
hard this code was to develop and it's truly insulting. Its clear to see
you are locked into assuming that the busy threads that are not all that
busy because they are constantly yielding to the scheduler is breaking the
timing code. I begged you to understand me when i told you that the err is
not normal, most boxes do not see it doing nothing and there has to be a
specific problem on your box or configuration. So instead of working with
us you want to escalate to snotty comments. That's pretty normal on the
internet I guess..... If you want to have a constructive conversation about
our core, install FS on a normal box, use it for a few weeks, figure out
everything about how it works then try.... There was pure speculation and
conjecture in your original emails and I never said a word about it until
you kept pushing.

Kristian mentioned he never sees that on that same hardware did you even
consider following up on why that is?

I don't have your device, but I assume if you get it working well it will
certainly help you more than it helps me so you could at least have the
decency to believe what we are trying to tell you.
Post by eaf
Oh, you mean giving FS higher priority? Yeah, as a last resort I'll do that.
At the moment, I hope it won't be necessary as I can make those "hyper"
threads behave, and will see how that goes first. I see where your
implementation could be coming from. There is a queue of SQL queries in
sofia.c processed by the worker thread. There are only two pop functions
available in APR: queue_pop() and queue_trypop(), so alas no option with a
timeout here. You don't want to block the thread in pop() indefinitely
because you chose that same worker needs to do ireg and gw processing once
in a while (separated by tens or hundreds of seconds, btw). You also want to
be able to detect shutdown condition so that the worker doesn't hold up
profile thread. So you chose to poll for events every millisecond instead of
just creating an apr_thread_cond_t for resource friendly signalling.
I agree that the timer thread philosophy is great and was the right choice
for scaling, but I just don't comprehend responses to things like these
other SQL or sofia worker threads. Did somebody even remotely acknowledge
that busy loops at least in those areas that I showed may probably be a bad
idea and could've been eliminated? I've heard suggestions to bump up
priority, I've heard that the code was perfect already, that it's the result
of 4-year effort, that I am arrogant, don't listen and don't understand
squat.
I'm sorry if I gave you impression that I was looking for the bad parts in
the software. I apologized for that already. All I wanted was to have
constructive conversation, perhaps I'm not too good at it. Code is already
perfect according to you? Fine with me.
Post by Anthony Minessale
no,
I mean the one after that that you must have completely skipped with a
command line option to try and a param to set in the config. It somewhat
annoys me for taking the time to compose it now. I wrote all of the code
you are talking about myself and I was trying to give you some
suggestions....
Well, actually, you did answer my question about the platform so you
must
Post by Anthony Minessale
have seen it.....
The loops are not the cause of that migration message, something wrong with
the hardware or the kernel is.
Another guy just told you he does not see that problem on the same exact
hardware.
Even if you have a point about the sql threads, you could make a patch to
slow them down but you cant slow down too much or you will not be able to
handle 400 cps all asking to send updates to transactions in batches of
thousands of sql stmts. Every line of that code is carefully designed so I
don't know what else to tell you but to stop being so arrogant and
re-read
Post by Anthony Minessale
this thread for all the advice you have totally ignored. I started out
trying to help you but I have a lot of work to do. I thoroughly
explained
Post by Anthony Minessale
it to you and you are choosing to ignore me so I guess I'm done.
You can do whatever you want with your working copy, i'll see you in 3 or 4
years when you get up to speed with the rest of us........
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26633739.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091203/538b8cbc/attachment-0002.html
eaf
2009-12-03 21:44:15 UTC
Permalink
Oh, you mean giving FS higher priority? Yeah, as a last resort I'll do that.
At the moment, I hope it won't be necessary as I can make those "hyper"
threads behave, and will see how that goes first. I see where your
implementation could be coming from. There is a queue of SQL queries in
sofia.c processed by the worker thread. There are only two pop functions
available in APR: queue_pop() and queue_trypop(), so alas no option with a
timeout here. You don't want to block the thread in pop() indefinitely
because you chose that same worker needs to do ireg and gw processing once
in a while (separated by tens or hundreds of seconds, btw). You also want to
be able to detect shutdown condition so that the worker doesn't hold up
profile thread. So you chose to poll for events every millisecond instead of
just creating an apr_thread_cond_t for resource friendly signalling.

I agree that the timer thread philosophy is great and was the right choice
for scaling, but I just don't comprehend responses to things like these
other SQL or sofia worker threads. Did somebody even remotely acknowledge
that busy loops at least in those areas that I showed may probably be a bad
idea and could've been eliminated? I've heard suggestions to bump up
priority, I've heard that the code was perfect already, that it's the result
of 4-year effort, that I am arrogant, don't listen and don't understand
squat.

I'm sorry if I gave you impression that I was looking for the bad parts in
the software. I apologized for that already. All I wanted was to have
constructive conversation, perhaps I'm not too good at it. Code is already
perfect according to you? Fine with me.
Post by Anthony Minessale
no,
I mean the one after that that you must have completely skipped with a
command line option to try and a param to set in the config. It somewhat
annoys me for taking the time to compose it now. I wrote all of the code
you are talking about myself and I was trying to give you some
suggestions....
Well, actually, you did answer my question about the platform so you must
have seen it.....
The loops are not the cause of that migration message, something wrong with
the hardware or the kernel is.
Another guy just told you he does not see that problem on the same exact
hardware.
Even if you have a point about the sql threads, you could make a patch to
slow them down but you cant slow down too much or you will not be able to
handle 400 cps all asking to send updates to transactions in batches of
thousands of sql stmts. Every line of that code is carefully designed so I
don't know what else to tell you but to stop being so arrogant and re-read
this thread for all the advice you have totally ignored. I started out
trying to help you but I have a lot of work to do. I thoroughly explained
it to you and you are choosing to ignore me so I guess I'm done.
You can do whatever you want with your working copy, i'll see you in 3 or 4
years when you get up to speed with the rest of us........
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26633739.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Anthony Minessale
2009-12-03 20:40:59 UTC
Permalink
no,

I mean the one after that that you must have completely skipped with a
command line option to try and a param to set in the config. It somewhat
annoys me for taking the time to compose it now. I wrote all of the code
you are talking about myself and I was trying to give you some
suggestions....

Well, actually, you did answer my question about the platform so you must
have seen it.....

The loops are not the cause of that migration message, something wrong with
the hardware or the kernel is.
Another guy just told you he does not see that problem on the same exact
hardware.

Even if you have a point about the sql threads, you could make a patch to
slow them down but you cant slow down too much or you will not be able to
handle 400 cps all asking to send updates to transactions in batches of
thousands of sql stmts. Every line of that code is carefully designed so I
don't know what else to tell you but to stop being so arrogant and re-read
this thread for all the advice you have totally ignored. I started out
trying to help you but I have a lot of work to do. I thoroughly explained
it to you and you are choosing to ignore me so I guess I'm done.
You can do whatever you want with your working copy, i'll see you in 3 or 4
years when you get up to speed with the rest of us........
Post by eaf
You mean, upgrading to the trunk and disabling RTP timers? Yes, I did. I
thought I responded back. Perhaps it didn't make through though, as I just
emailed back to the list instead of using nabble.com...
Anyway, upgrading to the trunk didn't change much, forcing SPA to 30ms went
w/o any effect either, but disabling RTP timers did the trick. I don't have
the original "choppy sound with PCMU" problem any more, thanks a lot for the
quick turnaround on that question.
But your suggestions made me look, into logs, strace, code, etc, so now I'm
just checking on how to quiet down those busy loops a little and how to get
rid of periodic CRIT messages about Virtual Machine Migration.
Post by Anthony Minessale
What about the things I spent time suggesting in my last email?
Did you try them because I was actually curious if they made any impact.
Post by eaf
I'm sorry if I sounded that way. Did mean to. :)
Yes, it's an embedded platform. It's an ALIX board with AMD Geode LX800 chip
and 256MB of RAM. http://www.pcengines.ch/alix2d3.htm
Line offset difference is due to some minor logging changes I made to
see
Post by Anthony Minessale
Post by eaf
who's allocating timers and how often. This way I found MOH streaming
and
Post by Anthony Minessale
Post by eaf
that RTP still allocates timers even when it's set to none in the profile.
I feel that this platform turned out to be underpowered for FS because
it
Post by Anthony Minessale
Post by eaf
cannot meet its scheduling expectations. I guess, some degree of kernel
tweaking or setting priorities will fix that. Meanwhile I just got rid
of
Post by Anthony Minessale
Post by eaf
the SQLDB 1ms thread via -nosql command line option, split sofia worker 1ms
thread in two (one blocked and waiting for new commands in the SQL
queue,
Post by Anthony Minessale
Post by eaf
the other one checking registrations and gateways with 1sec interval), and
don't know yet what to do about the timer thread.
Again, I apologize for stupid or accusing questions, I'm just trying to see
how FS can be made friendlier to this board. Or the board be made friendlier
to FS ;)
Post by Anthony Minessale
If you see that message then your machine/os/combo is having some
problems
Post by Anthony Minessale
keeping up.
It's not the timer missing anything its the monotonic clock detecting
a
Post by Anthony Minessale
Post by eaf
1
Post by Anthony Minessale
second or more differential from what its next prediction for the time
should be. The best way to trigger this would be to suspend FS with
control-z or attach to it with gdb blocking the entire process, that
1ms
Post by Anthony Minessale
thread would have to miss 1000 iterations to trigger that warning.
Btw, that error message is at line 471 not 473 so you are using
modified
Post by Anthony Minessale
code.
Its possible your box has a bad monotonic timer, you can set
under <settings> in switch.conf.xml
We are now starting to guess you are using some small embedded type platform
perhaps?
I've run FS even on a nokia n810 and never caused that message to
fire.
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
if 1 call can interrupt the cpu enough to cause noticeable issues you might
want to consider running the process at a
greater priority by using the -hp command line arg or at least nice it
Why don't you tell us the whole story about what OS/platform you are
using
Post by Anthony Minessale
here rather that form conjectures about what is wrong with our code
that
Post by Anthony Minessale
thousands of people are happy with.
Post by eaf
2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314
(sofia/external/xxxxx at 4.68.250.148) Running State Change CS_HANGUP
2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration
Detected! Syncing Clock
In this case an incoming call rang to both FS and Asterisk, Asterisk picked
up, but the surge of activity made FS timer thread miss a beat or
two.
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Oh, it's not just one timer thread... Why, why is sql_thread keeps
on
Post by Anthony Minessale
Post by eaf
Post by eaf
checking for messages every millisecond? Couldn't there be some
signalling
Post by eaf
implemented that will make the thread suspend on condition variable
or
Post by Anthony Minessale
Post by eaf
a
Post by eaf
socket/pipe in between?
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8,
obj=0x0)
Post by Anthony Minessale
Post by eaf
at
Post by eaf
src/switch_core_sqldb.c:783
Why does this sofia_profile_worker_thread keeps on looping checking
for
Post by Anthony Minessale
Post by eaf
Post by eaf
the queue? Have a semaphore!
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb73a4701 in sofia_profile_worker_thread_run
(thread=0x80f3a30,
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
obj=0x80f2490) at sofia.c:978
Nothing's happening on the box, but there are three threads that
pretend
Post by eaf
to be actively busy with smth. Others at least sleep for hundreds
of
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
milliseconds, not for one.
And there is even infrastructure present to do blocking pops: i.e.
why
Post by Anthony Minessale
Post by eaf
Post by eaf
couldn't sqldb thread do queue_pop() instead of queue_trypop()
intermixed
Post by eaf
with 1ms sleeps? This looping is such a waste...
Post by eaf
As I see it, switch_cond_next() currently is just a
do_sleep(1000).
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Yes,
Post by eaf
Post by eaf
it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND"
overrides that.
Yeah, there is a global timestamp... It's easy to workaround that
for
Post by Anthony Minessale
Post by eaf
RTP
Post by eaf
Post by eaf
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that.
If
Post by Anthony Minessale
Post by eaf
only
Post by eaf
Post by eaf
this was C++...
I'll play around. Never liked polling too much. Never could've
guessed
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
that polling could be so useful for scalability ;) My naive
implementation would've pulled timestamp via system calls and
would've
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
done sleeping by passing exact interval to select() instead of
syncing
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
with a pacing thread. Which would be dead-quiet at idle time, but,
of
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
course, would stop scaling at some point due to excessive number
of
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic
is
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
used throughout the code even when there is not any calls up.
You
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
can
Post by eaf
Post by eaf
Post by Michael Jerris
grep for switch_cond_next if you would like to see where but it
is
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
required to keep our global timestamp and for pacing the
scheduler
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I
glanced
through the code, and see that among others (are there others?)
RTP
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
and IVR
set up their timers that are subsequently managed by this
thread.
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
RTP timers
should be eliminated by that setting you've suggested. IVR
timers
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
are set at
20ms... So, if the thread is set to wake up every 10ms instead
of
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains
accurate
timing and then broadcasts via condition variables to hundreds
of
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
other
threads events that they can register for. I'm sure it's one of
the
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
reasons
why FS scales so much better than Asterisk. But for poor low-end
setups that
sit in the closet, eat only 6W of power and hardly ever run more
than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do,
call
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
clock_gettime() right from the thread that wants and when it
wants
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
to know
current time?
Say, what if that thread is made to suspend on a condition
variable
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
in case
if there are no timers registered in TIMER_MATRIX? Then, if some
other
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
thread comes up and adds its timer into the matrix, it could
wake
Post by Anthony Minessale
Post by eaf
up
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
the
timer thread and enjoy accurate timing as needed, on demand? And
in-
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
between
the calls, when there is no RTP or IVR, it will all go silent? I
mean,
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
sitting on a wait queue in the kernel is way better than go back
and
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing
and
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Post by Anthony Minessale
expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet
len
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Post by Anthony Minessale
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Sent from the Freeswitch-users mailing list archive at
Nabble.com.
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by eaf
Post by eaf
Post by Michael Jerris
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
Post by eaf
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by eaf
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com<MSN%3Aanthony_minessale at hotmail.com>
<MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
<
Post by eaf
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
<MSN%253Aanthony_minessale at hotmail.com<MSN%25253Aanthony_minessale at hotmail.com>
Post by Anthony Minessale
Post by eaf
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
Post by eaf
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
<PAYPAL%253Aanthony.minessale at gmail.com<PAYPAL%25253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org<sip%3A888 at conference.freeswitch.org>
<sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
<
Post by eaf
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
<sip%253A888 at conference.freeswitch.org<sip%25253A888 at conference.freeswitch.org>
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by Anthony Minessale
Post by eaf
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
<googletalk%253Aconf%252B888 at conference.freeswitch.org<googletalk%25253Aconf%25252B888 at conference.freeswitch.org>
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by eaf
Post by Anthony Minessale
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26629856.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by eaf
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com><
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
Post by Anthony Minessale
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org><
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26630994.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091203/b872350e/attachment-0002.html
eaf
2009-12-03 18:43:40 UTC
Permalink
You mean, upgrading to the trunk and disabling RTP timers? Yes, I did. I
thought I responded back. Perhaps it didn't make through though, as I just
emailed back to the list instead of using nabble.com...

Anyway, upgrading to the trunk didn't change much, forcing SPA to 30ms went
w/o any effect either, but disabling RTP timers did the trick. I don't have
the original "choppy sound with PCMU" problem any more, thanks a lot for the
quick turnaround on that question.

But your suggestions made me look, into logs, strace, code, etc, so now I'm
just checking on how to quiet down those busy loops a little and how to get
rid of periodic CRIT messages about Virtual Machine Migration.
Post by Anthony Minessale
What about the things I spent time suggesting in my last email?
Did you try them because I was actually curious if they made any impact.
Post by eaf
I'm sorry if I sounded that way. Did mean to. :)
Yes, it's an embedded platform. It's an ALIX board with AMD Geode LX800 chip
and 256MB of RAM. http://www.pcengines.ch/alix2d3.htm
Line offset difference is due to some minor logging changes I made to see
who's allocating timers and how often. This way I found MOH streaming and
that RTP still allocates timers even when it's set to none in the profile.
I feel that this platform turned out to be underpowered for FS because it
cannot meet its scheduling expectations. I guess, some degree of kernel
tweaking or setting priorities will fix that. Meanwhile I just got rid of
the SQLDB 1ms thread via -nosql command line option, split sofia worker 1ms
thread in two (one blocked and waiting for new commands in the SQL queue,
the other one checking registrations and gateways with 1sec interval), and
don't know yet what to do about the timer thread.
Again, I apologize for stupid or accusing questions, I'm just trying to see
how FS can be made friendlier to this board. Or the board be made friendlier
to FS ;)
Post by Anthony Minessale
If you see that message then your machine/os/combo is having some
problems
Post by Anthony Minessale
keeping up.
It's not the timer missing anything its the monotonic clock detecting a
1
Post by Anthony Minessale
second or more differential from what its next prediction for the time
should be. The best way to trigger this would be to suspend FS with
control-z or attach to it with gdb blocking the entire process, that
1ms
Post by Anthony Minessale
thread would have to miss 1000 iterations to trigger that warning.
Btw, that error message is at line 471 not 473 so you are using
modified
Post by Anthony Minessale
code.
Its possible your box has a bad monotonic timer, you can set
under <settings> in switch.conf.xml
We are now starting to guess you are using some small embedded type platform
perhaps?
I've run FS even on a nokia n810 and never caused that message to fire.
if 1 call can interrupt the cpu enough to cause noticeable issues you might
want to consider running the process at a
greater priority by using the -hp command line arg or at least nice it
Why don't you tell us the whole story about what OS/platform you are
using
Post by Anthony Minessale
here rather that form conjectures about what is wrong with our code
that
Post by Anthony Minessale
thousands of people are happy with.
Post by eaf
2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314
(sofia/external/xxxxx at 4.68.250.148) Running State Change CS_HANGUP
2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration
Detected! Syncing Clock
In this case an incoming call rang to both FS and Asterisk, Asterisk picked
up, but the surge of activity made FS timer thread miss a beat or two.
Post by eaf
Oh, it's not just one timer thread... Why, why is sql_thread keeps
on
Post by Anthony Minessale
Post by eaf
Post by eaf
checking for messages every millisecond? Couldn't there be some
signalling
Post by eaf
implemented that will make the thread suspend on condition variable
or
Post by Anthony Minessale
Post by eaf
a
Post by eaf
socket/pipe in between?
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8,
obj=0x0)
Post by Anthony Minessale
Post by eaf
at
Post by eaf
src/switch_core_sqldb.c:783
Why does this sofia_profile_worker_thread keeps on looping checking
for
Post by Anthony Minessale
Post by eaf
Post by eaf
the queue? Have a semaphore!
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30,
obj=0x80f2490) at sofia.c:978
Nothing's happening on the box, but there are three threads that
pretend
Post by eaf
to be actively busy with smth. Others at least sleep for hundreds of
milliseconds, not for one.
And there is even infrastructure present to do blocking pops: i.e.
why
Post by Anthony Minessale
Post by eaf
Post by eaf
couldn't sqldb thread do queue_pop() instead of queue_trypop()
intermixed
Post by eaf
with 1ms sleeps? This looping is such a waste...
Post by eaf
As I see it, switch_cond_next() currently is just a do_sleep(1000).
Yes,
Post by eaf
Post by eaf
it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND"
overrides that.
Yeah, there is a global timestamp... It's easy to workaround that
for
Post by Anthony Minessale
Post by eaf
RTP
Post by eaf
Post by eaf
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that.
If
Post by Anthony Minessale
Post by eaf
only
Post by eaf
Post by eaf
this was C++...
I'll play around. Never liked polling too much. Never could've
guessed
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
that polling could be so useful for scalability ;) My naive
implementation would've pulled timestamp via system calls and
would've
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
done sleeping by passing exact interval to select() instead of
syncing
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
with a pacing thread. Which would be dead-quiet at idle time, but,
of
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
course, would stop scaling at some point due to excessive number of
system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You
can
Post by eaf
Post by eaf
Post by Michael Jerris
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?)
RTP
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
and IVR
set up their timers that are subsequently managed by this thread.
RTP timers
should be eliminated by that setting you've suggested. IVR timers
are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of
the
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
reasons
why FS scales so much better than Asterisk. But for poor low-end
setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do,
call
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
clock_gettime() right from the thread that wants and when it
wants
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
to know
current time?
Say, what if that thread is made to suspend on a condition
variable
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
in case
if there are no timers registered in TIMER_MATRIX? Then, if some
other
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
thread comes up and adds its timer into the matrix, it could wake
up
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
the
timer thread and enjoy accurate timing as needed, on demand? And
in-
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
between
the calls, when there is no RTP or IVR, it will all go silent? I
mean,
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
sitting on a wait queue in the kernel is way better than go back
and
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing
and
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Post by Anthony Minessale
expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet
len
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Post by Anthony Minessale
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Sent from the Freeswitch-users mailing list archive at
Nabble.com.
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by eaf
Post by eaf
Post by Michael Jerris
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by eaf
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com
<MSN%3Aanthony_minessale at hotmail.com><
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org
<sip%3A888 at conference.freeswitch.org><
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26629856.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26630994.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Kristian Kielhofner
2009-12-03 17:44:27 UTC
Permalink
I don't think it's the board itself...

We have extensively tested FreeSwitch (no modifications) on that exact
board with AstLinux and have it running at multiple customer
locations.

No timing errors, no warnings or errors of any kind. Pretty standard
really just don't expect too much from the LX800 (transcoding,
resampling, massive numbers of calls, etc).
Post by eaf
I'm sorry if I sounded that way. Did mean to. :)
Yes, it's an embedded platform. It's an ALIX board with AMD Geode LX800 chip
and 256MB of RAM. http://www.pcengines.ch/alix2d3.htm
Line offset difference is due to some minor logging changes I made to see
who's allocating timers and how often. This way I found MOH streaming and
that RTP still allocates timers even when it's set to none in the profile.
I feel that this platform turned out to be underpowered for FS because it
cannot meet its scheduling expectations. I guess, some degree of kernel
tweaking or setting priorities will fix that. Meanwhile I just got rid of
the SQLDB 1ms thread via -nosql command line option, split sofia worker 1ms
thread in two (one blocked and waiting for new commands in the SQL queue,
the other one checking registrations and gateways with 1sec interval), and
don't know yet what to do about the timer thread.
Again, I apologize for stupid or accusing questions, I'm just trying to see
how FS can be made friendlier to this board. Or the board be made friendlier
to FS ;)
--
Kristian Kielhofner
http://www.astlinux.org
http://blog.krisk.org
http://www.star2star.com
http://www.submityoursip.com
http://www.voalte.com
Michael Jerris
2009-12-03 17:50:26 UTC
Permalink
I know people with hardware out there in production based on arm11 and those are pretty small processors, not sure how they compare to this. In regards to the DISABLE_1MS_COND, try getting rid of that, it did increase performance on the high end but may be better for you on the low end with lower compute on idle busy loops.

Mike
Post by eaf
I'm sorry if I sounded that way. Did mean to. :)
Yes, it's an embedded platform. It's an ALIX board with AMD Geode LX800 chip
and 256MB of RAM. http://www.pcengines.ch/alix2d3.htm
Line offset difference is due to some minor logging changes I made to see
who's allocating timers and how often. This way I found MOH streaming and
that RTP still allocates timers even when it's set to none in the profile.
I feel that this platform turned out to be underpowered for FS because it
cannot meet its scheduling expectations. I guess, some degree of kernel
tweaking or setting priorities will fix that. Meanwhile I just got rid of
the SQLDB 1ms thread via -nosql command line option, split sofia worker 1ms
thread in two (one blocked and waiting for new commands in the SQL queue,
the other one checking registrations and gateways with 1sec interval), and
don't know yet what to do about the timer thread.
Again, I apologize for stupid or accusing questions, I'm just trying to see
how FS can be made friendlier to this board. Or the board be made friendlier
to FS ;)
Post by Anthony Minessale
If you see that message then your machine/os/combo is having some problems
keeping up.
It's not the timer missing anything its the monotonic clock detecting a 1
second or more differential from what its next prediction for the time
should be. The best way to trigger this would be to suspend FS with
control-z or attach to it with gdb blocking the entire process, that 1ms
thread would have to miss 1000 iterations to trigger that warning.
Btw, that error message is at line 471 not 473 so you are using modified
code.
Its possible your box has a bad monotonic timer, you can set
under <settings> in switch.conf.xml
We are now starting to guess you are using some small embedded type platform
perhaps?
I've run FS even on a nokia n810 and never caused that message to fire.
if 1 call can interrupt the cpu enough to cause noticeable issues you might
want to consider running the process at a
greater priority by using the -hp command line arg or at least nice it
Why don't you tell us the whole story about what OS/platform you are using
here rather that form conjectures about what is wrong with our code that
thousands of people are happy with.
Post by eaf
2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314
(sofia/external/xxxxx at 4.68.250.148) Running State Change CS_HANGUP
2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration
Detected! Syncing Clock
In this case an incoming call rang to both FS and Asterisk, Asterisk picked
up, but the surge of activity made FS timer thread miss a beat or two.
Post by eaf
Oh, it's not just one timer thread... Why, why is sql_thread keeps on
checking for messages every millisecond? Couldn't there be some
signalling
Post by eaf
implemented that will make the thread suspend on condition variable or
a
Post by eaf
socket/pipe in between?
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0)
at
Post by eaf
src/switch_core_sqldb.c:783
Why does this sofia_profile_worker_thread keeps on looping checking for
the queue? Have a semaphore!
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30,
obj=0x80f2490) at sofia.c:978
Nothing's happening on the box, but there are three threads that
pretend
Post by eaf
to be actively busy with smth. Others at least sleep for hundreds of
milliseconds, not for one.
And there is even infrastructure present to do blocking pops: i.e. why
couldn't sqldb thread do queue_pop() instead of queue_trypop()
intermixed
Post by eaf
with 1ms sleeps? This looping is such a waste...
Post by eaf
As I see it, switch_cond_next() currently is just a do_sleep(1000).
Yes,
Post by eaf
Post by eaf
it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND"
overrides that.
Yeah, there is a global timestamp... It's easy to workaround that for
RTP
Post by eaf
Post by eaf
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that. If
only
Post by eaf
Post by eaf
this was C++...
I'll play around. Never liked polling too much. Never could've guessed
that polling could be so useful for scalability ;) My naive
implementation would've pulled timestamp via system calls and would've
done sleeping by passing exact interval to select() instead of syncing
with a pacing thread. Which would be dead-quiet at idle time, but, of
course, would stop scaling at some point due to excessive number of
system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You
can
Post by eaf
Post by eaf
Post by Michael Jerris
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do,
call
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some
other
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-
between
the calls, when there is no RTP or IVR, it will all go silent? I
mean,
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by eaf
Post by eaf
Post by Michael Jerris
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26629856.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Anthony Minessale
2009-12-03 18:10:05 UTC
Permalink
What about the things I spent time suggesting in my last email?
Did you try them because I was actually curious if they made any impact.
Post by eaf
I'm sorry if I sounded that way. Did mean to. :)
Yes, it's an embedded platform. It's an ALIX board with AMD Geode LX800 chip
and 256MB of RAM. http://www.pcengines.ch/alix2d3.htm
Line offset difference is due to some minor logging changes I made to see
who's allocating timers and how often. This way I found MOH streaming and
that RTP still allocates timers even when it's set to none in the profile.
I feel that this platform turned out to be underpowered for FS because it
cannot meet its scheduling expectations. I guess, some degree of kernel
tweaking or setting priorities will fix that. Meanwhile I just got rid of
the SQLDB 1ms thread via -nosql command line option, split sofia worker 1ms
thread in two (one blocked and waiting for new commands in the SQL queue,
the other one checking registrations and gateways with 1sec interval), and
don't know yet what to do about the timer thread.
Again, I apologize for stupid or accusing questions, I'm just trying to see
how FS can be made friendlier to this board. Or the board be made friendlier
to FS ;)
Post by Anthony Minessale
If you see that message then your machine/os/combo is having some
problems
Post by Anthony Minessale
keeping up.
It's not the timer missing anything its the monotonic clock detecting a 1
second or more differential from what its next prediction for the time
should be. The best way to trigger this would be to suspend FS with
control-z or attach to it with gdb blocking the entire process, that 1ms
thread would have to miss 1000 iterations to trigger that warning.
Btw, that error message is at line 471 not 473 so you are using modified
code.
Its possible your box has a bad monotonic timer, you can set
under <settings> in switch.conf.xml
We are now starting to guess you are using some small embedded type platform
perhaps?
I've run FS even on a nokia n810 and never caused that message to fire.
if 1 call can interrupt the cpu enough to cause noticeable issues you might
want to consider running the process at a
greater priority by using the -hp command line arg or at least nice it
Why don't you tell us the whole story about what OS/platform you are
using
Post by Anthony Minessale
here rather that form conjectures about what is wrong with our code that
thousands of people are happy with.
Post by eaf
2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314
(sofia/external/xxxxx at 4.68.250.148) Running State Change CS_HANGUP
2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration
Detected! Syncing Clock
In this case an incoming call rang to both FS and Asterisk, Asterisk picked
up, but the surge of activity made FS timer thread miss a beat or two.
Post by eaf
Oh, it's not just one timer thread... Why, why is sql_thread keeps on
checking for messages every millisecond? Couldn't there be some
signalling
Post by eaf
implemented that will make the thread suspend on condition variable or
a
Post by eaf
socket/pipe in between?
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0)
at
Post by eaf
src/switch_core_sqldb.c:783
Why does this sofia_profile_worker_thread keeps on looping checking
for
Post by Anthony Minessale
Post by eaf
Post by eaf
the queue? Have a semaphore!
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30,
obj=0x80f2490) at sofia.c:978
Nothing's happening on the box, but there are three threads that
pretend
Post by eaf
to be actively busy with smth. Others at least sleep for hundreds of
milliseconds, not for one.
And there is even infrastructure present to do blocking pops: i.e. why
couldn't sqldb thread do queue_pop() instead of queue_trypop()
intermixed
Post by eaf
with 1ms sleeps? This looping is such a waste...
Post by eaf
As I see it, switch_cond_next() currently is just a do_sleep(1000).
Yes,
Post by eaf
Post by eaf
it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND"
overrides that.
Yeah, there is a global timestamp... It's easy to workaround that for
RTP
Post by eaf
Post by eaf
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that. If
only
Post by eaf
Post by eaf
this was C++...
I'll play around. Never liked polling too much. Never could've
guessed
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
that polling could be so useful for scalability ;) My naive
implementation would've pulled timestamp via system calls and
would've
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
done sleeping by passing exact interval to select() instead of
syncing
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
with a pacing thread. Which would be dead-quiet at idle time, but, of
course, would stop scaling at some point due to excessive number of
system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You
can
Post by eaf
Post by eaf
Post by Michael Jerris
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread.
RTP timers
should be eliminated by that setting you've suggested. IVR timers
are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end
setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do,
call
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some
other
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
thread comes up and adds its timer into the matrix, it could wake
up
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
the
timer thread and enjoy accurate timing as needed, on demand? And
in-
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
between
the calls, when there is no RTP or IVR, it will all go silent? I
mean,
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
sitting on a wait queue in the kernel is way better than go back
and
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing
and
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Post by Anthony Minessale
expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-
Post by Anthony Minessale
Post by eaf
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by eaf
Post by eaf
Post by Michael Jerris
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by Anthony Minessale
Post by eaf
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com><
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
Post by Anthony Minessale
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org><
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26629856.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091203/d9ff3e1b/attachment-0002.html
eaf
2009-12-03 17:29:46 UTC
Permalink
I'm sorry if I sounded that way. Did mean to. :)

Yes, it's an embedded platform. It's an ALIX board with AMD Geode LX800 chip
and 256MB of RAM. http://www.pcengines.ch/alix2d3.htm

Line offset difference is due to some minor logging changes I made to see
who's allocating timers and how often. This way I found MOH streaming and
that RTP still allocates timers even when it's set to none in the profile.

I feel that this platform turned out to be underpowered for FS because it
cannot meet its scheduling expectations. I guess, some degree of kernel
tweaking or setting priorities will fix that. Meanwhile I just got rid of
the SQLDB 1ms thread via -nosql command line option, split sofia worker 1ms
thread in two (one blocked and waiting for new commands in the SQL queue,
the other one checking registrations and gateways with 1sec interval), and
don't know yet what to do about the timer thread.

Again, I apologize for stupid or accusing questions, I'm just trying to see
how FS can be made friendlier to this board. Or the board be made friendlier
to FS ;)
Post by Anthony Minessale
If you see that message then your machine/os/combo is having some problems
keeping up.
It's not the timer missing anything its the monotonic clock detecting a 1
second or more differential from what its next prediction for the time
should be. The best way to trigger this would be to suspend FS with
control-z or attach to it with gdb blocking the entire process, that 1ms
thread would have to miss 1000 iterations to trigger that warning.
Btw, that error message is at line 471 not 473 so you are using modified
code.
Its possible your box has a bad monotonic timer, you can set
under <settings> in switch.conf.xml
We are now starting to guess you are using some small embedded type platform
perhaps?
I've run FS even on a nokia n810 and never caused that message to fire.
if 1 call can interrupt the cpu enough to cause noticeable issues you might
want to consider running the process at a
greater priority by using the -hp command line arg or at least nice it
Why don't you tell us the whole story about what OS/platform you are using
here rather that form conjectures about what is wrong with our code that
thousands of people are happy with.
Post by eaf
2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314
(sofia/external/xxxxx at 4.68.250.148) Running State Change CS_HANGUP
2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration
Detected! Syncing Clock
In this case an incoming call rang to both FS and Asterisk, Asterisk picked
up, but the surge of activity made FS timer thread miss a beat or two.
Post by eaf
Oh, it's not just one timer thread... Why, why is sql_thread keeps on
checking for messages every millisecond? Couldn't there be some
signalling
Post by eaf
implemented that will make the thread suspend on condition variable or
a
Post by eaf
socket/pipe in between?
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0)
at
Post by eaf
src/switch_core_sqldb.c:783
Why does this sofia_profile_worker_thread keeps on looping checking for
the queue? Have a semaphore!
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30,
obj=0x80f2490) at sofia.c:978
Nothing's happening on the box, but there are three threads that
pretend
Post by eaf
to be actively busy with smth. Others at least sleep for hundreds of
milliseconds, not for one.
And there is even infrastructure present to do blocking pops: i.e. why
couldn't sqldb thread do queue_pop() instead of queue_trypop()
intermixed
Post by eaf
with 1ms sleeps? This looping is such a waste...
Post by eaf
As I see it, switch_cond_next() currently is just a do_sleep(1000).
Yes,
Post by eaf
Post by eaf
it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND"
overrides that.
Yeah, there is a global timestamp... It's easy to workaround that for
RTP
Post by eaf
Post by eaf
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that. If
only
Post by eaf
Post by eaf
this was C++...
I'll play around. Never liked polling too much. Never could've guessed
that polling could be so useful for scalability ;) My naive
implementation would've pulled timestamp via system calls and would've
done sleeping by passing exact interval to select() instead of syncing
with a pacing thread. Which would be dead-quiet at idle time, but, of
course, would stop scaling at some point due to excessive number of
system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You
can
Post by eaf
Post by eaf
Post by Michael Jerris
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do,
call
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some
other
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-
between
the calls, when there is no RTP or IVR, it will all go silent? I
mean,
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Post by eaf
Post by eaf
Post by Michael Jerris
Post by eaf
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by eaf
Post by eaf
Post by Michael Jerris
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26629856.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Anthony Minessale
2009-12-03 16:16:26 UTC
Permalink
If you see that message then your machine/os/combo is having some problems
keeping up.
It's not the timer missing anything its the monotonic clock detecting a 1
second or more differential from what its next prediction for the time
should be. The best way to trigger this would be to suspend FS with
control-z or attach to it with gdb blocking the entire process, that 1ms
thread would have to miss 1000 iterations to trigger that warning.

Btw, that error message is at line 471 not 473 so you are using modified
code.

Its possible your box has a bad monotonic timer, you can set

<param name="disable-monotonic-timing" value="true"/>

under <settings> in switch.conf.xml

We are now starting to guess you are using some small embedded type platform
perhaps?
I've run FS even on a nokia n810 and never caused that message to fire.

if 1 call can interrupt the cpu enough to cause noticeable issues you might
want to consider running the process at a
greater priority by using the -hp command line arg or at least nice it

Why don't you tell us the whole story about what OS/platform you are using
here rather that form conjectures about what is wrong with our code that
thousands of people are happy with.
Post by eaf
2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314
(sofia/external/xxxxx at 4.68.250.148) Running State Change CS_HANGUP
2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration
Detected! Syncing Clock
In this case an incoming call rang to both FS and Asterisk, Asterisk picked
up, but the surge of activity made FS timer thread miss a beat or two.
Post by eaf
Oh, it's not just one timer thread... Why, why is sql_thread keeps on
checking for messages every millisecond? Couldn't there be some
signalling
Post by eaf
implemented that will make the thread suspend on condition variable or a
socket/pipe in between?
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0) at
src/switch_core_sqldb.c:783
Why does this sofia_profile_worker_thread keeps on looping checking for
the queue? Have a semaphore!
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30,
obj=0x80f2490) at sofia.c:978
Nothing's happening on the box, but there are three threads that pretend
to be actively busy with smth. Others at least sleep for hundreds of
milliseconds, not for one.
And there is even infrastructure present to do blocking pops: i.e. why
couldn't sqldb thread do queue_pop() instead of queue_trypop() intermixed
with 1ms sleeps? This looping is such a waste...
Post by eaf
As I see it, switch_cond_next() currently is just a do_sleep(1000). Yes,
it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND"
overrides that.
Yeah, there is a global timestamp... It's easy to workaround that for
RTP
Post by eaf
Post by eaf
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that. If
only
Post by eaf
Post by eaf
this was C++...
I'll play around. Never liked polling too much. Never could've guessed
that polling could be so useful for scalability ;) My naive
implementation would've pulled timestamp via system calls and would've
done sleeping by passing exact interval to select() instead of syncing
with a pacing thread. Which would be dead-quiet at idle time, but, of
course, would stop scaling at some point due to excessive number of
system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You can
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do, call
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some other
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-
between
the calls, when there is no RTP or IVR, it will all go silent? I mean,
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by eaf
Post by eaf
Post by Michael Jerris
http://www.freeswitch.org
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091203/061f041e/attachment-0002.html
eaf
2009-12-03 14:55:21 UTC
Permalink
Btw, I have these popping up in my logs from time to time:

2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314
(sofia/external/xxxxx at 4.68.250.148) Running State Change CS_HANGUP
2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration
Detected! Syncing Clock

In this case an incoming call rang to both FS and Asterisk, Asterisk picked
up, but the surge of activity made FS timer thread miss a beat or two.
Post by eaf
Oh, it's not just one timer thread... Why, why is sql_thread keeps on
checking for messages every millisecond? Couldn't there be some signalling
implemented that will make the thread suspend on condition variable or a
socket/pipe in between?
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0) at
src/switch_core_sqldb.c:783
Why does this sofia_profile_worker_thread keeps on looping checking for
the queue? Have a semaphore!
#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30,
obj=0x80f2490) at sofia.c:978
Nothing's happening on the box, but there are three threads that pretend
to be actively busy with smth. Others at least sleep for hundreds of
milliseconds, not for one.
And there is even infrastructure present to do blocking pops: i.e. why
couldn't sqldb thread do queue_pop() instead of queue_trypop() intermixed
with 1ms sleeps? This looping is such a waste...
Post by eaf
As I see it, switch_cond_next() currently is just a do_sleep(1000). Yes,
it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND"
overrides that.
Yeah, there is a global timestamp... It's easy to workaround that for RTP
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that. If only
this was C++...
I'll play around. Never liked polling too much. Never could've guessed
that polling could be so useful for scalability ;) My naive
implementation would've pulled timestamp via system calls and would've
done sleeping by passing exact interval to select() instead of syncing
with a pacing thread. Which would be dead-quiet at idle time, but, of
course, would stop scaling at some point due to excessive number of
system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You can
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do, call
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some other
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-
between
the calls, when there is no RTP or IVR, it will all go silent? I mean,
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Michael Jerris
2009-12-03 13:48:55 UTC
Permalink
First off, maybe this conversation is better suited to the dev list, and second off, the current setup of where we do timers, where we poll, polling frequency and architecture is the result of 4+ years of ongoing testing and optimization. We have tried all different methods throughout. Sometimes what we found to be most efficient is not what we thought at first would be, but testing showed otherwise. We have always optimized the general case as to if there are many calls, and no suggestion would be implemented that hurts this case. That being said, if you could really come up with a way for this to be more efficient in any case, without sacrificing performance int he other cases, you are able to prove this with extensive test results, and you are able to prove that it does not impact for example call quality in any of the hundreds of edge cases that have led us to the point we are now, then we may be interested in taking such a patch.

Mike
Post by eaf
As I see it, switch_cond_next() currently is just a do_sleep(1000). Yes, it
could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND" overrides
that.
Yeah, there is a global timestamp... It's easy to workaround that for RTP
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that. If only
this was C++...
I'll play around. Never liked polling too much. Never could've guessed that
polling could be so useful for scalability ;) My naive implementation
would've pulled timestamp via system calls and would've done sleeping by
passing exact interval to select() instead of syncing with a pacing thread.
Which would be dead-quiet at idle time, but, of course, would stop scaling
at some point due to excessive number of system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You can
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do, call
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some other
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-
between
the calls, when there is no RTP or IVR, it will all go silent? I mean,
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26621005.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
eaf
2009-12-03 14:17:54 UTC
Permalink
Oh, it's not just one timer thread... Why, why is sql_thread keeps on
checking for messages every millisecond? Couldn't there be some signalling
implemented that will make the thread suspend on condition variable or a
socket/pipe in between?

#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0) at
src/switch_core_sqldb.c:783

Why does this sofia_profile_worker_thread keeps on looping checking for the
queue? Have a semaphore!

#0 do_sleep (t=1000) at src/switch_time.c:109
#1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30,
obj=0x80f2490) at sofia.c:978

Nothing's happening on the box, but there are three threads that pretend to
be actively busy with smth. Others at least sleep for hundreds of
milliseconds, not for one.

And there is even infrastructure present to do blocking pops: i.e. why
couldn't sqldb thread do queue_pop() instead of queue_trypop() intermixed
with 1ms sleeps? This looping is such a waste...
Post by eaf
As I see it, switch_cond_next() currently is just a do_sleep(1000). Yes,
it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND"
overrides that.
Yeah, there is a global timestamp... It's easy to workaround that for RTP
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that. If only
this was C++...
I'll play around. Never liked polling too much. Never could've guessed
that polling could be so useful for scalability ;) My naive implementation
would've pulled timestamp via system calls and would've done sleeping by
passing exact interval to select() instead of syncing with a pacing
thread. Which would be dead-quiet at idle time, but, of course, would stop
scaling at some point due to excessive number of system calls.
Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You can
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do, call
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some other
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-
between
the calls, when there is no RTP or IVR, it will all go silent? I mean,
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26626634.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
eaf
2009-12-03 04:58:39 UTC
Permalink
As I see it, switch_cond_next() currently is just a do_sleep(1000). Yes, it
could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND" overrides
that.

Yeah, there is a global timestamp... It's easy to workaround that for RTP
who calls switch_micro_time_now()... But if somebody accesses
runtime.timestamp directly, it's gonna be tough to grep for that. If only
this was C++...

I'll play around. Never liked polling too much. Never could've guessed that
polling could be so useful for scalability ;) My naive implementation
would've pulled timestamp via system calls and would've done sleeping by
passing exact interval to select() instead of syncing with a pacing thread.
Which would be dead-quiet at idle time, but, of course, would stop scaling
at some point due to excessive number of system calls.

Thanks.
Post by Michael Jerris
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You can
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.
Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do, call
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some other
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-
between
the calls, when there is no RTP or IVR, it will all go silent? I mean,
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26621005.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
eaf
2009-12-03 03:35:39 UTC
Permalink
Oh, looks like the timers are also used for streaming local data in
read_stream_thread(). Due to this there is always one timer active with 20ms
interval.

But wait a sec, why is freeswitch periodically trying to stream
/opt/freeswitch/sounds/music/8000/ponce-preludio-in-e-major.wav somewhere?
Every minute or so? Did I misconfigure it?
Post by eaf
Say, what if that thread is made to suspend on a condition variable in
case if there are no timers registered in TIMER_MATRIX? Then, if some
other thread comes up and adds its timer into the matrix, it could wake up
the timer thread and enjoy accurate timing as needed, on demand? And
in-between the calls, when there is no RTP or IVR, it will all go silent?
I mean, sitting on a wait queue in the kernel is way better than go back
and forth incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len that it's
using and we set our timer
to the wrong speed.
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26620518.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Michael Jerris
2009-12-03 04:00:00 UTC
Permalink
In short. No, you can not for many reasons. The milisecond tic is
used throughout the code even when there is not any calls up. You can
grep for switch_cond_next if you would like to see where but it is
required to keep our global timestamp and for pacing the scheduler
among other services that run all the time.

Mike
Post by eaf
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?
That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-
friendly? I.e.
make it stuck in select() or recv() when there is nothing to do, call
clock_gettime() right from the thread that wants and when it wants to know
current time?
Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some other
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-
between
the calls, when there is no RTP or IVR, it will all go silent? I mean,
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len
that
it's
using and we set our timer
to the wrong speed.
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
users
http://www.freeswitch.org
eaf
2009-12-03 00:31:31 UTC
Permalink
Can I reduce resolution of that timer thread 10 times? I mean, I glanced
through the code, and see that among others (are there others?) RTP and IVR
set up their timers that are subsequently managed by this thread. RTP timers
should be eliminated by that setting you've suggested. IVR timers are set at
20ms... So, if the thread is set to wake up every 10ms instead of 1ms it
should be able to wake up those IVR timers just fine. Right?

That's a cool design to have one dedicated thread that maintains accurate
timing and then broadcasts via condition variables to hundreds of other
threads events that they can register for. I'm sure it's one of the reasons
why FS scales so much better than Asterisk. But for poor low-end setups that
sit in the closet, eat only 6W of power and hardly ever run more than two
calls at the same time, can I hack it somehow to be more UNIX-friendly? I.e.
make it stuck in select() or recv() when there is nothing to do, call
clock_gettime() right from the thread that wants and when it wants to know
current time?

Say, what if that thread is made to suspend on a condition variable in case
if there are no timers registered in TIMER_MATRIX? Then, if some other
thread comes up and adds its timer into the matrix, it could wake up the
timer thread and enjoy accurate timing as needed, on demand? And in-between
the calls, when there is no RTP or IVR, it will all go silent? I mean,
sitting on a wait queue in the kernel is way better than go back and forth
incrementing counters that nobody even needs at the moment?
Post by Anthony Minessale
idle is a 4 letter word to a realtime application.
The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.
Your choppy audio is caused by linksys lying about the packet len that it's
using and we set our timer
to the wrong speed.
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
Anthony Minessale
2009-12-02 18:32:21 UTC
Permalink
idle is a 4 letter word to a realtime application.

The core keeps a single high-priority thread to keep 1ms timing and expands
that broadcasting
to hundreds or thousand of threads who need accurate timing.

Your choppy audio is caused by linksys lying about the packet len that it's
using and we set our timer
to the wrong speed.
Post by erandr-junk
Wow... Thinking about this timer setting and about how it converted
send()/recv() from non-blocking to blocking, I straced freeswitch when it was
supposed to be idle. It never pauses! It keeps going in and out of select()
every millisecond! Why??
------ Original Message ------
Received: Tue, 01 Dec 2009 08:31:46 PM EST
From: erandr-junk at usa.net
To: <freeswitch-users at lists.freeswitch.org>
Subject: Re: [Freeswitch-users] Choppy sound with PCMU
Post by erandr-junk
Thanks. I tried that... Just forcing SPA to 20ms didn't change anything.
Just
Post by erandr-junk
installing SVN trunk didn't fix it either, but setting that option
afterwards
Post by erandr-junk
surely did the trick.
One thing I've noticed while staring at the console is that it *looks
like*
Post by erandr-junk
that w/o the new setting the stuttering happens when FS either
re-registers
Post by erandr-junk
itself with the provider or one of the SPA's port re-registers with FS.
------ Original Message ------
Received: Tue, 01 Dec 2009 05:33:26 PM EST
From: Anthony Minessale <anthony.minessale at gmail.com>
To: freeswitch-users at lists.freeswitch.org
Subject: Re: [Freeswitch-users] Choppy sound with PCMU
Post by Anthony Minessale
linksys has had a bug for eons that can be fixed by setting the ptime
(or
Post by erandr-junk
Post by Anthony Minessale
rtp packet size in their terms)
in it's firmware to .20 instead of .30
Asterisk does not use async RTP like we do so it's never a problem
you can disable the timer by setting the channel var
rtp_timer_name=none
or
Post by erandr-junk
Post by Anthony Minessale
sofia param rtp-timer-name to none in the sofia profile.
You should also test this on latest SVN trunk or wait for pre8
Post by eaf
I should also add, after browsing through some topics here, that my
SIP
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
provider sends 172-byte RTP frames, which is in accordance with
ptime:20
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
that it gives to FreeSWITCH.
Post by eaf
Hi,
I'm trying to migrate from Asterisk to FreeSWITCH (really like the
way
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
how
Post by eaf
it can be programmed), but ran into one issue with sound quality
that
I
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
just cannot workaround by myself. I would describe the sound
problem
as
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
being "choppy". From time to time small portions of the other
party's
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
voice are dropped, so the voice kind of stutters. This is not too
bad,
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
but
Post by eaf
is really noticeable, happens in every call and I don't experience
the
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
freeswitch.wav and asterisk.mp3 to illustrate my point.
Issue completely goes away, if I set inbound-proxy-media to true.
The way how I test is to connect SPA-2000 via 10mbps LAN to the box
directly exposed to internet, and then dial a toll-free via
FutureNine
Post by erandr-junk
(a
Post by Anthony Minessale
Post by eaf
Post by eaf
SIP provider).
The codec in use is PCMU. Can't really try PCMA or anything else
with
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
this
Post by eaf
provider. Only PCMU. Tried to match ptime of provider (30) with
ptime
Post by erandr-junk
of
Post by Anthony Minessale
Post by eaf
Post by eaf
the SPA, didn't get any improvement. Tried turning off recording,
no
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
change either.
What puzzles me is that even with greedy codec negotiations and
with
Post by erandr-junk
PCMU
Post by Anthony Minessale
Post by eaf
Post by eaf
on both sides of FreeSWITCH, it's still saying that
TRANSCODING_NECESSARY. I'm attaching relevant portion of
freeswitch.log
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
to
Post by eaf
illustrate.
The box isn't particularly fast: Linux (Debian 4), CPU - AMD Geode
LX800
Post by Anthony Minessale
Post by eaf
Post by eaf
with 997 bogomips. 256MB RAM. Only one call in progress, so I hope
that
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
it's not a performance issue.
http://old.nabble.com/file/p26594250/freeswitch.wavfreeswitch.wav
http://old.nabble.com/file/p26594250/asterisk.mp3 asterisk.mp3
http://old.nabble.com/file/p26594250/freeswitch.logfreeswitch.log
Tried both 1.0.4 and 1.0.5pre5. Same results.
What should I do next? Calls are consistently bad with FreeSWITCH,
and
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
consistently show no glitches with Asterisk.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26599565.html
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com<MSN%3Aanthony_minessale at hotmail.com><
MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
<PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
Post by erandr-junk
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org<sip%3A888 at conference.freeswitch.org><
sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
Post by erandr-junk
Post by Anthony Minessale
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
<googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
Post by erandr-junk
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by erandr-junk
Post by Anthony Minessale
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091202/a320b89e/attachment-0002.html
eaf
2009-12-01 21:51:42 UTC
Permalink
Hi,

I'm trying to migrate from Asterisk to FreeSWITCH (really like the way how
it can be programmed), but ran into one issue with sound quality that I just
cannot workaround by myself. I would describe the sound problem as being
"choppy". From time to time small portions of the other party's voice are
dropped, so the voice kind of stutters. This is not too bad, but is really
noticeable, happens in every call and I don't experience the same with
Asterisk running on the same box. I attached two files: freeswitch.wav and
asterisk.mp3 to illustrate my point.

Issue completely goes away, if I set inbound-proxy-media to true.

The way how I test is to connect SPA-2000 via 10mbps LAN to the box directly
exposed to internet, and then dial a toll-free via FutureNine (a SIP
provider).

The codec in use is PCMU. Can't really try PCMA or anything else with this
provider. Only PCMU. Tried to match ptime of provider (30) with ptime of the
SPA, didn't get any improvement. Tried turning off recording, no change
either.

What puzzles me is that even with greedy codec negotiations and with PCMU on
both sides of FreeSWITCH, it's still saying that TRANSCODING_NECESSARY. I'm
attaching relevant portion of freeswitch.log to illustrate.

The box isn't particularly fast: Linux (Debian 4), CPU - AMD Geode LX800
with 997 bogomips. 256MB RAM. Only one call in progress, so I hope that it's
not a performance issue.

http://old.nabble.com/file/p26594250/freeswitch.wav freeswitch.wav

http://old.nabble.com/file/p26594250/asterisk.mp3 asterisk.mp3

http://old.nabble.com/file/p26594250/freeswitch.log freeswitch.log

Tried both 1.0.4 and 1.0.5pre5. Same results.

What should I do next? Calls are consistently bad with FreeSWITCH, and
consistently show no glitches with Asterisk.
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26594250.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
eaf
2009-12-01 21:52:03 UTC
Permalink
I should also add, after browsing through some topics here, that my SIP
provider sends 172-byte RTP frames, which is in accordance with ptime:20
that it gives to FreeSWITCH.
Post by eaf
Hi,
I'm trying to migrate from Asterisk to FreeSWITCH (really like the way how
it can be programmed), but ran into one issue with sound quality that I
just cannot workaround by myself. I would describe the sound problem as
being "choppy". From time to time small portions of the other party's
voice are dropped, so the voice kind of stutters. This is not too bad, but
is really noticeable, happens in every call and I don't experience the
freeswitch.wav and asterisk.mp3 to illustrate my point.
Issue completely goes away, if I set inbound-proxy-media to true.
The way how I test is to connect SPA-2000 via 10mbps LAN to the box
directly exposed to internet, and then dial a toll-free via FutureNine (a
SIP provider).
The codec in use is PCMU. Can't really try PCMA or anything else with this
provider. Only PCMU. Tried to match ptime of provider (30) with ptime of
the SPA, didn't get any improvement. Tried turning off recording, no
change either.
What puzzles me is that even with greedy codec negotiations and with PCMU
on both sides of FreeSWITCH, it's still saying that
TRANSCODING_NECESSARY. I'm attaching relevant portion of freeswitch.log to
illustrate.
The box isn't particularly fast: Linux (Debian 4), CPU - AMD Geode LX800
with 997 bogomips. 256MB RAM. Only one call in progress, so I hope that
it's not a performance issue.
http://old.nabble.com/file/p26594250/freeswitch.wav freeswitch.wav
http://old.nabble.com/file/p26594250/asterisk.mp3 asterisk.mp3
http://old.nabble.com/file/p26594250/freeswitch.log freeswitch.log
Tried both 1.0.4 and 1.0.5pre5. Same results.
What should I do next? Calls are consistently bad with FreeSWITCH, and
consistently show no glitches with Asterisk.
--
View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26599565.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
erandr-junk
2009-12-02 01:26:58 UTC
Permalink
Thanks. I tried that... Just forcing SPA to 20ms didn't change anything. Just
installing SVN trunk didn't fix it either, but setting that option afterwards
surely did the trick.

One thing I've noticed while staring at the console is that it *looks like*
that w/o the new setting the stuttering happens when FS either re-registers
itself with the provider or one of the SPA's port re-registers with FS.

------ Original Message ------
Received: Tue, 01 Dec 2009 05:33:26 PM EST
From: Anthony Minessale <anthony.minessale at gmail.com>
To: freeswitch-users at lists.freeswitch.org
Subject: Re: [Freeswitch-users] Choppy sound with PCMU
Post by Anthony Minessale
linksys has had a bug for eons that can be fixed by setting the ptime (or
rtp packet size in their terms)
in it's firmware to .20 instead of .30
Asterisk does not use async RTP like we do so it's never a problem
you can disable the timer by setting the channel var rtp_timer_name=none or
sofia param rtp-timer-name to none in the sofia profile.
You should also test this on latest SVN trunk or wait for pre8
Post by eaf
I should also add, after browsing through some topics here, that my SIP
provider sends 172-byte RTP frames, which is in accordance with ptime:20
that it gives to FreeSWITCH.
Post by eaf
Hi,
I'm trying to migrate from Asterisk to FreeSWITCH (really like the way
how
Post by eaf
it can be programmed), but ran into one issue with sound quality that I
just cannot workaround by myself. I would describe the sound problem as
being "choppy". From time to time small portions of the other party's
voice are dropped, so the voice kind of stutters. This is not too bad,
but
Post by eaf
is really noticeable, happens in every call and I don't experience the
freeswitch.wav and asterisk.mp3 to illustrate my point.
Issue completely goes away, if I set inbound-proxy-media to true.
The way how I test is to connect SPA-2000 via 10mbps LAN to the box
directly exposed to internet, and then dial a toll-free via FutureNine
(a
Post by Anthony Minessale
Post by eaf
Post by eaf
SIP provider).
The codec in use is PCMU. Can't really try PCMA or anything else with
this
Post by eaf
provider. Only PCMU. Tried to match ptime of provider (30) with ptime
of
Post by Anthony Minessale
Post by eaf
Post by eaf
the SPA, didn't get any improvement. Tried turning off recording, no
change either.
What puzzles me is that even with greedy codec negotiations and with
PCMU
Post by Anthony Minessale
Post by eaf
Post by eaf
on both sides of FreeSWITCH, it's still saying that
TRANSCODING_NECESSARY. I'm attaching relevant portion of freeswitch.log
to
Post by eaf
illustrate.
The box isn't particularly fast: Linux (Debian 4), CPU - AMD Geode
LX800
Post by Anthony Minessale
Post by eaf
Post by eaf
with 997 bogomips. 256MB RAM. Only one call in progress, so I hope that
it's not a performance issue.
http://old.nabble.com/file/p26594250/freeswitch.wav freeswitch.wav
http://old.nabble.com/file/p26594250/asterisk.mp3 asterisk.mp3
http://old.nabble.com/file/p26594250/freeswitch.log freeswitch.log
Tried both 1.0.4 and 1.0.5pre5. Same results.
What should I do next? Calls are consistently bad with FreeSWITCH, and
consistently show no glitches with Asterisk.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26599565.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
erandr-junk
2009-12-02 03:19:39 UTC
Permalink
Wow... Thinking about this timer setting and about how it converted
send()/recv() from non-blocking to blocking, I straced freeswitch when it was
supposed to be idle. It never pauses! It keeps going in and out of select()
every millisecond! Why??

------ Original Message ------
Received: Tue, 01 Dec 2009 08:31:46 PM EST
From: erandr-junk at usa.net
To: <freeswitch-users at lists.freeswitch.org>
Subject: Re: [Freeswitch-users] Choppy sound with PCMU
Post by erandr-junk
Thanks. I tried that... Just forcing SPA to 20ms didn't change anything.
Just
Post by erandr-junk
installing SVN trunk didn't fix it either, but setting that option
afterwards
Post by erandr-junk
surely did the trick.
One thing I've noticed while staring at the console is that it *looks like*
that w/o the new setting the stuttering happens when FS either re-registers
itself with the provider or one of the SPA's port re-registers with FS.
------ Original Message ------
Received: Tue, 01 Dec 2009 05:33:26 PM EST
From: Anthony Minessale <anthony.minessale at gmail.com>
To: freeswitch-users at lists.freeswitch.org
Subject: Re: [Freeswitch-users] Choppy sound with PCMU
Post by Anthony Minessale
linksys has had a bug for eons that can be fixed by setting the ptime (or
rtp packet size in their terms)
in it's firmware to .20 instead of .30
Asterisk does not use async RTP like we do so it's never a problem
you can disable the timer by setting the channel var rtp_timer_name=none
or
Post by erandr-junk
Post by Anthony Minessale
sofia param rtp-timer-name to none in the sofia profile.
You should also test this on latest SVN trunk or wait for pre8
Post by eaf
I should also add, after browsing through some topics here, that my SIP
provider sends 172-byte RTP frames, which is in accordance with
ptime:20
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
that it gives to FreeSWITCH.
Post by eaf
Hi,
I'm trying to migrate from Asterisk to FreeSWITCH (really like the
way
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
how
Post by eaf
it can be programmed), but ran into one issue with sound quality that
I
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
just cannot workaround by myself. I would describe the sound problem
as
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
being "choppy". From time to time small portions of the other party's
voice are dropped, so the voice kind of stutters. This is not too
bad,
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
but
Post by eaf
is really noticeable, happens in every call and I don't experience
the
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
freeswitch.wav and asterisk.mp3 to illustrate my point.
Issue completely goes away, if I set inbound-proxy-media to true.
The way how I test is to connect SPA-2000 via 10mbps LAN to the box
directly exposed to internet, and then dial a toll-free via
FutureNine
Post by erandr-junk
(a
Post by Anthony Minessale
Post by eaf
Post by eaf
SIP provider).
The codec in use is PCMU. Can't really try PCMA or anything else with
this
Post by eaf
provider. Only PCMU. Tried to match ptime of provider (30) with ptime
of
Post by Anthony Minessale
Post by eaf
Post by eaf
the SPA, didn't get any improvement. Tried turning off recording, no
change either.
What puzzles me is that even with greedy codec negotiations and with
PCMU
Post by Anthony Minessale
Post by eaf
Post by eaf
on both sides of FreeSWITCH, it's still saying that
TRANSCODING_NECESSARY. I'm attaching relevant portion of
freeswitch.log
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
to
Post by eaf
illustrate.
The box isn't particularly fast: Linux (Debian 4), CPU - AMD Geode
LX800
Post by Anthony Minessale
Post by eaf
Post by eaf
with 997 bogomips. 256MB RAM. Only one call in progress, so I hope
that
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
it's not a performance issue.
http://old.nabble.com/file/p26594250/freeswitch.wav freeswitch.wav
http://old.nabble.com/file/p26594250/asterisk.mp3 asterisk.mp3
http://old.nabble.com/file/p26594250/freeswitch.log freeswitch.log
Tried both 1.0.4 and 1.0.5pre5. Same results.
What should I do next? Calls are consistently bad with FreeSWITCH,
and
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
Post by eaf
consistently show no glitches with Asterisk.
--
http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26599565.html
Sent from the Freeswitch-users mailing list archive at Nabble.com.
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
Post by erandr-junk
Post by Anthony Minessale
Post by eaf
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
Post by erandr-junk
Post by Anthony Minessale
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
Post by erandr-junk
Post by Anthony Minessale
pstn:213-799-1400
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
Continue reading on narkive:
Loading...