[xiph-rtp] linphone-1.5.0 mostly complies to theora over RTP draft

[Simon Morlat]

Hello,

Finally after all the discussions we had together here, and the clarifications
you made to me, I implemented a compliant theora-RTP usage into
linphone-1.5.0 (an audio/video SIP phone).
It's in file mediastreamer2/src/theora.c for those who are curious.
- it uses the packed conf technique (which I still found the easiest and more
flexible way in a SIP/SDP context )
- unfortunately I said "mostly compliant" because it won't accept several
theora frames grouped into one single RTP packet... Maybe a SDP parameters
suchs a=fmtp:XX maxframes=1 would be necessary to indicate this limitations
to the remote implementation.
- in order to dynamically limit bandwidth according to capabilities of both
sides, I used the b=AS SDP header, which is the most suitable technique whose
meaning is independant from any codec details.

No interroperability test yet, perhaps linphone is the first SIP videophone
using theora. Have you heard of any other ?
So there may be some bugs...

See http://www.linphone.org

Thanks to everybody here who made this possible,

Simon

[Luca Barbato]

Post by Simon Morlat
Hello,
Finally after all the discussions we had together here, and the clarifications
you made to me, I implemented a compliant theora-RTP usage into
linphone-1.5.0 (an audio/video SIP phone).

Great!

Post by Simon Morlat
It's in file mediastreamer2/src/theora.c for those who are curious.

I'll have a look now.

Post by Simon Morlat
- it uses the packed conf technique (which I still found the easiest and more
flexible way in a SIP/SDP context )

hm

Post by Simon Morlat
- unfortunately I said "mostly compliant" because it won't accept several
theora frames grouped into one single RTP packet... Maybe a SDP parameters
suchs a=fmtp:XX maxframes=1 would be necessary to indicate this limitations
to the remote implementation.

is it an usage limitation? If is the case I'll update the rfc to allow it.

Post by Simon Morlat
- in order to dynamically limit bandwidth according to capabilities of both
sides, I used the b=AS SDP header, which is the most suitable technique whose
meaning is independant from any codec details.

Sounds ok.

Post by Simon Morlat
No interroperability test yet, perhaps linphone is the first SIP videophone
using theora. Have you heard of any other ?

None told me anything.

Post by Simon Morlat
So there may be some bugs...

So they will be fixed =)

Post by Simon Morlat
Thanks to everybody here who made this possible,

Thanks for your hard work and the feedback =)

lu

[Simon Morlat]

Post by Luca Barbato
is it an usage limitation? If is the case I'll update the rfc to allow it.

It's a limitation because I did not want to write the code for that, but
actually for telephony it does not make sense to group frames since this
increases latency.
We can imagine also that a small embeded device won't have the memory capacity
to store several frames.
I know similar parameters exists for other codecs, for example for AMR it's
called maxptime.

Simon

[Luca Barbato]

Post by Simon Morlat

Post by Luca Barbato
is it an usage limitation? If is the case I'll update the rfc to allow it.

It's a limitation because I did not want to write the code for that

You may look at the code I wrote for nemesi (yes it isn't complete since
finish has lower priority than making the api and the foundation robust
and usable)

Post by Simon Morlat
We can imagine also that a small embedded device won't have the memory capacity
to store several frames.

Well, for fragments it could be a problem, not for collated frames (the
size is 1 mtu at most anyway).

Post by Simon Morlat
I know similar parameters exists for other codecs, for example for AMR it's
called maxptime.

Makes sense adding it, do you know if there is an rfc that generalizes it?

Ping me during the week if I don't update the rfc today.

lu

PS: the sdp provides the first configuration inline? It is compulsory.

[Luca Barbato]

Post by Simon Morlat
Hello,
Finally after all the discussions we had together here, and the clarifications
you made to me, I implemented a compliant theora-RTP usage into
linphone-1.5.0 (an audio/video SIP phone).

Great =)

Post by Simon Morlat
It's in file mediastreamer2/src/theora.c for those who are curious.
- it uses the packed conf technique (which I still found the easiest and more
flexible way in a SIP/SDP context )

Make sure you also inline the first conf in the sdp...

Post by Simon Morlat
- unfortunately I said "mostly compliant" because it won't accept several
theora frames grouped into one single RTP packet... Maybe a SDP parameters
suchs a=fmtp:XX maxframes=1 would be necessary to indicate this limitations
to the remote implementation.

I'm reading rfc3984 to pick parameters and parameters name in a way that
matches other rfcs, if you could have a look too and tell me which
parameters could be useful in your fields, I'll try to avoid ones that
don't make sense. The O/A section seems interesting too.

Let me know, hopefully I'll commit an update soon.

lu

[Simon Morlat]

Hello Luca,

Post by Luca Barbato

Post by Simon Morlat
It's in file mediastreamer2/src/theora.c for those who are curious.
- it uses the packed conf technique (which I still found the easiest and
more flexible way in a SIP/SDP context )

Make sure you also inline the first conf in the sdp...

Why ? Is this required ?

Post by Luca Barbato
I'm reading rfc3984 to pick parameters and parameters name in a way that
matches other rfcs, if you could have a look too and tell me which
parameters could be useful in your fields, I'll try to avoid ones that
don't make sense. The O/A section seems interesting too.

rfc2984 has an extensive set of parameter that I don't find very useful...
I don't think they will be used by many people around the earth.
I suggested a 'maxframes' parameter to handle the case of an application like
linphone that never sends multiple frames in a single RTP packet and thus
does not want to support it at the receiving side, however this is not a good
idea. There may be the case where linphone will call a SIP server that
broadcast TV or whatever, so the case of multiples frames into one packet can
produce on a SIP phone.
I'd rather implement the multiple frames decoding algorithm into linphone, and
keep the draft as simple as possible.
Parameters such as max_br are redundant with already existing b=AS or b=TI SDP
parameters.
Some other parameters indicate the maximum decoding capabilities of a receiver
(max frames per seconds, max size, max macroblocks per second and so on),
those ones might be interesting for small embedded devices that may be
limited processing or display capabilities. I would prefer having them common
to all video codecs, but it does not seem to be the way things are going.
So I think it's up to you to decide such paremeters to theora draft, but I at
least recommend you to keep them:
- optionnal
- as receiver options (to allow asymetric operations)

Simon

Post by Luca Barbato
Let me know, hopefully I'll commit an update soon.
lu

[Luca Barbato]

Post by Simon Morlat
Hello Luca,

Post by Luca Barbato

Post by Simon Morlat
It's in file mediastreamer2/src/theora.c for those who are curious.
- it uses the packed conf technique (which I still found the easiest and
more flexible way in a SIP/SDP context )

Make sure you also inline the first conf in the sdp...

Why ? Is this required ?

I didn't state it as strict requirement, would be a problem if I restate
it as a MUST? The reason is that this way you should cut some latency
(maybe it's pretty equal) and at least the first (and in many cases
only) stream will be decoded for sure (since sdp is delivered reliably).

Client anyway already MUST be able to understand it.

Post by Simon Morlat
rfc2984 has an extensive set of parameter that I don't find very useful...
I don't think they will be used by many people around the earth.

Yes, I wasn't thinking about adding all of them.

Post by Simon Morlat
I suggested a 'maxframes' parameter to handle the case of an application like
linphone that never sends multiple frames in a single RTP packet and thus
does not want to support it at the receiving side, however this is not a good
idea. There may be the case where linphone will call a SIP server that
broadcast TV or whatever, so the case of multiples frames into one packet can
produce on a SIP phone.
I'd rather implement the multiple frames decoding algorithm into linphone, and
keep the draft as simple as possible.

I don't have strong opinions about this and since the IANA section is
already pending a rewrite it won't make me write more =)

Post by Simon Morlat
Parameters such as max_br are redundant with already existing b=AS or b=TI SDP
parameters.

eh...

Post by Simon Morlat
Some other parameters indicate the maximum decoding capabilities of a receiver
(max frames per seconds, max size, max macroblocks per second and so on),
those ones might be interesting for small embedded devices that may be
limited processing or display capabilities.

Yup.

Post by Simon Morlat
I would prefer having them common
to all video codecs, but it does not seem to be the way things are going.

If I use the same names they'll be the same for 2 formats, if a third
person will use them for another codec we'll be 3, that's the design
decision I took when writing vorbis and theora: keep things as common as
possible.

Post by Simon Morlat
So I think it's up to you to decide such paremeters to theora draft, but I at

No, I'm polling people for comments because MY specific purpose (having
vorbis and theora over rtp supported in media players) isn't the only
one, I have no pretenses to know what's better for my application, let
alone other different ones =)

Post by Simon Morlat
- optional

Agree

Post by Simon Morlat
- as receiver options (to allow asymetric operations)

Sure

So far what we could add are parameters to describe some limits,
possibly ripped from other generic rfcs:

- max number of frames in a packet
- decoding limits (fps, size, complexity...)

Anybody thinks that is valuable or that we could just avoid them and
hopefully wait for (write?) a generic rfc that describes them?

lu

PS: I'm still trying to push the changes suggested by Colin in a
consistent way, help proofreading and/or suggesting changes (using diff)
would be quite well accepted =)

[Simon Morlat]

Post by Luca Barbato
I didn't state it as strict requirement, would be a problem if I restate
it as a MUST? The reason is that this way you should cut some latency
(maybe it's pretty equal) and at least the first (and in many cases
only) stream will be decoded for sure (since sdp is delivered reliably).

Actually it is; in order to provide a theora config string in the SDP, I must
before configure my theora encoder with a theora_info, and in order to do
that, I must have the requirements from the remote phone in terms of
bandwidth, to set a correct bitrate into the theora_info.
That 's the reason why I prefered sending the packed_conf within the RTP
stream.
Otherwise it would not be possible to do that with only 2 SDP messages.

Post by Luca Barbato
Client anyway already MUST be able to understand it.

I have no problem with that requirement. But I think it is powerfull that
clients may not provide config in the SDP, but only after the SDP exchange is
done, because this is the moment where all receive capabilities are known by
both ends, thus theora encoders can be configured with accurate parameters.

Simon