[xiph-rtp] Theora "setup idents" and codebook URLs in SDP

[David Barrett]

So it sounds like the final word is that each Theora data packet will
have a 16-bit "setup ident" field (ie, a codebook identifier) in the
following payload header:

From: http://lists.xiph.org/pipermail/xiph-rtp/2005-May/000229.html

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Setup Ident | Reserved |C|F|R| # pkts. |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Currently it sounds like the only requirement is that the "setup ident"
be unique within a given session. Ie, ident #1234 might indicate a
different codebook in two different sessions (though it always refers to
the same codebook within any given session). With all this in mind, I
have a couple questions:

1) What is the SDP syntax for associating a particular codebook URL with
a given setup ident?

2) Can I specify many codebooks (each with a different setup ident) in
the SDP such that the decoder can "prefetch" the codebooks before they
are used in the RTP?

3) What are the rules for caching codebooks in the decoder? If setup
idents are only unique and persistent within a given session, obviously
I can't use those. I can easily use the URL, but because it's
essentially arbitrary (ie, not derived from the codebook itself), it's
entirely possible that the codebook "at the other end" of the URL might
change without my knowledge (ie, my cached version can become dirty).
One proposal is to say:

"A codebook URL SHOULD include a CRC32 of the codebook itself, so as to
prevent the codebook referenced by URL from changing, and thereby
enabling decoding clients to use the URL as a persistent,
globally-unique identifier of the codebook itself, suitable for reliable
caching purposes."

-david

[Phil Kerr]

Hi David,

Post by David Barrett
So it sounds like the final word is that each Theora data packet will
have a 16-bit "setup ident" field (ie, a codebook identifier) in the
From: http://lists.xiph.org/pipermail/xiph-rtp/2005-May/000229.html

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Setup Ident | Reserved |C|F|R| # pkts. |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Currently it sounds like the only requirement is that the "setup
ident" be unique within a given session. Ie, ident #1234 might
indicate a different codebook in two different sessions (though it
always refers to the same codebook within any given session). With
1) What is the SDP syntax for associating a particular codebook URL
with a given setup ident?
2) Can I specify many codebooks (each with a different setup ident) in
the SDP such that the decoder can "prefetch" the codebooks before they
are used in the RTP?

I do not see any problem with this, apart from.....

Post by David Barrett
3) What are the rules for caching codebooks in the decoder? If setup
idents are only unique and persistent within a given session,
obviously I can't use those. I can easily use the URL, but because
it's essentially arbitrary (ie, not derived from the codebook itself),
it's entirely possible that the codebook "at the other end" of the URL
might change without my knowledge (ie, my cached version can become

Without a unique identifier it becomes problematical to cache codebook
outside a session, unless.....

Post by David Barrett
"A codebook URL SHOULD include a CRC32 of the codebook itself, so as
to prevent the codebook referenced by URL from changing, and thereby
enabling decoding clients to use the URL as a persistent,
globally-unique identifier of the codebook itself, suitable for
reliable caching purposes."

Yup. But how do we prevent clashes with the packet ident?

Regards

Phil

Post by David Barrett
-david
_______________________________________________
xiph-rtp mailing list
http://lists.xiph.org/mailman/listinfo/xiph-rtp

[David Barrett]

Post by Phil Kerr
Hi David,

Post by David Barrett
"A codebook URL SHOULD include a CRC32 of the codebook itself, so as
to prevent the codebook referenced by URL from changing, and thereby
enabling decoding clients to use the URL as a persistent,
globally-unique identifier of the codebook itself, suitable for
reliable caching purposes."

Yup. But how do we prevent clashes with the packet ident?

In this proposal, there is no global uniqueness requirement for the
"setup ident" field in each packet. Rather, there is a much easier
"session uniqueness" requirement.

Because the broadcaster writes the SDP *and* the packets, it is both the
sole author and user of the namespace. (i.e., it defines the "setup
ident" namespace by associating codebook URLs with "setup idents" in the
SDP, and then it stamps packets as it sends them to match.) Clashes are
therefore extremely easy to avoid.

-david

[Aaron Colwell]

Post by David Barrett

Post by Phil Kerr
Hi David,

Post by David Barrett
"A codebook URL SHOULD include a CRC32 of the codebook itself, so as
to prevent the codebook referenced by URL from changing, and thereby
enabling decoding clients to use the URL as a persistent,
globally-unique identifier of the codebook itself, suitable for
reliable caching purposes."

There are several constraints that need to be taken into account here.

1. SDP needs to be less than 1k in some scenarios like SAP. In other
environments it can be larger, but it would be nice to have a solution that
works in both cases.

2. If the codebook/ident info isn't known at SDP generation time there needs
to be a mechanism for getting this info.

3. If all codebook/ident info is known at SDP generation time you should
provide enough information so that all this info can be retrieved before
playback starts.


Constraint 1 implies compact representation of the information. Mandating CRCs
or MD5 in the URLs definitely does not achieve this. It can work for low
chain counts, but it can quickly cause the SDP to get too large. Allowing
looser URL requirements allow you to make URLs more compact.

Constrain 2 implies to me the need for some sort of baseURL that
the "setup ident" can be appended to so that it can retrieve the ident/codebook
URLs.

Constraint 3 could be solved by listing all the "setup idents" in the SDP and
then using the constraint 2 solution. You could also just provide a single
URL that contains a list of (setup ident, ident MD5, ident URL, codebook MDS5,
codebook URL) structures. This would reduce the amount of HTTP traffic needed.
Putting the URLs in the SDP directly causes problems with constraint 1.

A while ago I posted sample SDP solutions to the list. I don't know if they
have been accepted or integrated into the draft document yet.

Post by David Barrett

Post by Phil Kerr
Yup. But how do we prevent clashes with the packet ident?

In this proposal, there is no global uniqueness requirement for the
"setup ident" field in each packet. Rather, there is a much easier
"session uniqueness" requirement.
Because the broadcaster writes the SDP *and* the packets, it is both the
sole author and user of the namespace. (i.e., it defines the "setup
ident" namespace by associating codebook URLs with "setup idents" in the
SDP, and then it stamps packets as it sends them to match.) Clashes are
therefore extremely easy to avoid.

setup ident has always only needed to be unique within the session. I don't
see how this changes anything.

Aaron

Post by David Barrett
-david
_______________________________________________
xiph-rtp mailing list
http://lists.xiph.org/mailman/listinfo/xiph-rtp

[David Barrett]

Post by Aaron Colwell
There are several constraints that need to be taken into account here.
1. SDP needs to be less than 1k in some scenarios like SAP. In other
environments it can be larger, but it would be nice to have a solution that
works in both cases.
2. If the codebook/ident info isn't known at SDP generation time there needs
to be a mechanism for getting this info.
3. If all codebook/ident info is known at SDP generation time you should
provide enough information so that all this info can be retrieved before
playback starts.

All good points, thanks for laying them out clearly.

Post by Aaron Colwell
Constraint 1 implies compact representation of the information. Mandating CRCs
or MD5 in the URLs definitely does not achieve this. It can work for low
chain counts, but it can quickly cause the SDP to get too large. Allowing
looser URL requirements allow you to make URLs more compact.

You're right, I hadn't considered that.

Post by Aaron Colwell
Constrain 2 implies to me the need for some sort of baseURL that
the "setup ident" can be appended to so that it can retrieve the ident/codebook
URLs.

Yes, that'd work. (Though implies a session can only use codebooks on a
single server.)

Post by Aaron Colwell
Constraint 3 could be solved by listing all the "setup idents" in the SDP and
then using the constraint 2 solution. You could also just provide a single
URL that contains a list of (setup ident, ident MD5, ident URL, codebook MDS5,
codebook URL) structures. This would reduce the amount of HTTP traffic needed.
Putting the URLs in the SDP directly causes problems with constraint 1.

This's also work. And assuming this "codebook list URL" were itself
immutably tied to the "codebook list", then I could cache the whole set
of codebooks (and the mapping to setup idents) with one blow.

For my application I'd probably just publish a single "codebook list"
(with every conceivable codebook I might ever use) that all clients
would pick from as needed. Thus once you talked to any of my clients,
you'd already have the codeboks cached to talk to any other of my
clients. And if I need to add to it, simply publish a new list with a
new version of the app, while leaving the old in place for old
versions.

My only concern again is ensuring that the list of codebooks pointed to
by a URL never changes, else caching is difficult.

Post by Aaron Colwell
A while ago I posted sample SDP solutions to the list. I don't know if they
have been accepted or integrated into the draft document yet.

Great, I'll dig through the archives and try to find them. I didn't
mean to imply my proposal was the only possible solution.

Post by Aaron Colwell

Post by David Barrett

Post by Phil Kerr
Yup. But how do we prevent clashes with the packet ident?

In this proposal, there is no global uniqueness requirement for the
"setup ident" field in each packet. Rather, there is a much easier
"session uniqueness" requirement.

setup ident has always only needed to be unique within the session. I don't
see how this changes anything.

I didn't mean to imply it does. I was just restating it to address
Phil's concern.


Ideally we could have an SDP grammar sufficiently flexible to handle
them both, so I can either specify a list of ident/codebook URLs
directly in the SDP (if it fits, and possibly using a baseurl), or give
a link to a list of ident/codebook URLs on the server. Maybe even allow
linking to multiple lists (and specify more on the fly), with the
resulting set being a union of everything.


Anyway, thanks for reviewing the options with me. I'm going to start
with simple URLs direct in the SDP, and I'll drop the CRC with the blind
assumption that they are safe to cache. I'll assume a more robust
technique is in the works, possibly with baseurls, possibly with
serverside codebook lists, and possibly in a way that combines both.

-david