Should HLS live manifest refreshing consider http request delay?

[baiqindotfubotv]

I am using exoPlayer to play a live HLS stream. I noticed the exoPlayer doesn't take the network request delay into consideration when scheduling manifest refreshing. Here are some logs I printed in onTimelineUpdate method:

23:34:47.890  E  Manifest refreshed
23:34:52.020  E  Manifest refreshed
23:34:56.212  E  Manifest refreshed
23:35:00.507  E  Manifest refreshed
23:35:04.712  E  Manifest refreshed
23:35:08.881  E  Manifest refreshed
23:35:13.075  E  Manifest refreshed
23:35:17.258  E  Manifest refreshed

As you can see the gap between each refresh is several hundreds of ms greater than segment size (4 seconds). According to the description in 6.3.4 here: https://datatracker.ietf.org/doc/html/draft-pantos-http-live-streaming-16#section-6.3.4:

"When a client loads a Playlist file for the first time or reloads a Playlist file and finds that it has changed since the last time it was loaded, the client MUST wait for at least the target duration before attempting to reload the Playlist file again, measured from the last time the client began loading the Playlist file."

It says the next refresh time should be measured from the last time the client begin loading the Playlist file. However looks like exoPlayer is scheduling next refreshing based on the the timestamp when last loading was finished. I think this is where exoPlayer does the calculation: https://github.com/androidx/media/blob/1.1.1/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/playlist/DefaultHlsPlaylistTracker.java#L764.

When the default playback position is close to the end of live window end, this behavior will cause a lot of bufferings.

So is this expected behavior?

For comparison, DASH manifest refreshing does consider network delay:
https://github.com/androidx/media/blob/1.1.1/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashMediaSource.java#L931. And for this reason, when exoPlayer is playing DASH live manifest at a position close to live window end, bufferings seldom happen.

[tianyif]

Hi @baiqindotfubotv,

Thanks for the question! Sounds like using "the last time the client begin loading the Playlist file" should be more accurate.

But even with the current implementation, we still comply the standard the client MUST wait for at least the target duration before attempting to reload the Playlist file again as the interval between the two start loading time should be slightly longer than the target duration. So I'm wondering the buffering issue is coming from the other causes. Would it be possible for you to provide more details on:

1. If you are able to reproduce and debug with demo app by changing to use "the last time the client begin loading the Playlist file" instead of "the time when last loading was finished", does it solve your buffering issue?
2. How long is your live window? The reason why I'm asking is based on this blog post (you can search for the section "Use a long live window"), that being too close to the start of the window might cause the loading to fall behind, which will require a repreparation of the player.

We're looking forward to your feedback!

[baiqindotfubotv]

I tested a change at this line: DefaultHlsPlaylistTracker

I changed this line to

earliestNextLoadTimeMs = currentTimeMs + Util.usToMs(durationUntilNextLoadUs) - loadEventInfo.loadDurationMs;

It does fix 95% of the buffering in my case.

The live window size is 32 seconds in length with 8 segments but I don't think window size matters. What matters is starting playback position. The buffering issue only happens with HLS if the target playback position is close to the end of live window. For example, at 28 seconds. Let's assume each manifest refresh takes exactly 500 ms. Then the refresh sequence is something like this:

Refresh at 0: window start time: 1000, buffer size 4000 ms
Refresh at 4500: window start time 5000, buffer size 3500 ms
Refresh at 9000: window start time 9000, buffer size 3000 ms
Refresh at 13500: window start time 13000, buffer size 2500 ms
Refresh at 18000: window start time 17000, buffer size 2000 ms
Refresh at 22500: window start time 21000, buffer size 1500 ms
Refresh at 27000: window start time 25000, buffer size 1000 ms
Refresh at 31500: window start time 29000, buffer size 500 ms

At this time the buffer size is under 500 ms and the player needs to buffer. And the same buffering issue will happen again and again.

This won't be an issue if the playback position is not close to the end live window end. This is because in a later manifest refresh request, the live window will jump ahead 8 seconds instead of 4 seconds, thus the previous accumulated delay will be erased. If we set the playback position to 18 seconds instead of 28 seconds, it will be something like this:

Refresh at 0: window start time: 1000, buffer size 14000 ms
Refresh at 4500: window start time 5000, buffer size 13500 ms
Refresh at 9000: window start time 9000, buffer size 13000 ms
Refresh at 13500: window start time 13000, buffer size 12500 ms
Refresh at 18000: window start time 17000, buffer size 12000 ms
Refresh at 22500: window start time 21000, buffer size 11500 ms
Refresh at 27000: window start time 25000, buffer size 11000 ms
Refresh at 31500: window start time 29000, buffer size 10500 ms
Refresh at 36000: window start time 33000, buffer size 10000 ms
Refresh at 40500: window start time 41000, buffer size 14500 ms
Refresh at 45000: window start time 45000, buffer size 14000 ms

Note that the refresh at 40500 returns a manifest that jumps ahead 8 seconds, thus previous accumulated manifest refresh delay is erased. If playback position is close to end of live window, it is very likely to cause a lot of buffering issues.

The work around I am using right now is to wait for the first onTimelineUpdate event, after that I use the exoPlayer.seekTo method to seek to a target playback position, thus ensuring a good buffering size.

---

A note on the default playback position. Right now there is no media3 interface to configure initial playback position / buffer size for live playback. Both DASH and HLS. There are some DASH (minBufferTime) / HLS (ext-x-start) tags that could affect the initial playback position. But a lot of times the packaging is done by third parties so the client doesn't have control over it.

ExoPlayer does have a LiveConfiguration class, but it is not useful in a lot of cases because the live offset in it is the offset to unix now time. Given a streaming url, it is impossible to know the accurate manifest start and end until the manifest is fetched from server. So it is also impossible to calculate a target live offset ahead of time.

Previously DashMediaSource has a livePresentationDelayMs parameter, but even that was removed in this commit: livePresentationDelayMs. If exoPlayer has an api configure initial playback position it will be helpful.

[tianyif]

Thanks @baiqindotfubotv for the detailed explanation! After the internal discussion, we'll be making the changes for an accurate reload interval with considering the network delay.

Also, as for

ExoPlayer does have a LiveConfiguration class, but it is not useful in a lot of cases because the live offset in it is the offset to unix now time. Given a streaming url, it is impossible to know the accurate manifest start and end until the manifest is fetched from server. So it is also impossible to calculate a target live offset ahead of time.

please keep track on the existing feature request - #652.

[tianyif]

And would it be possible for you to provide a media sample for us to test the change?