Seeking into AC4 audio only content with FMP4 subfragments creates discontinuities

[ronak2121]

ExoPlayer Version

2.15.1

Devices that reproduce the issue

• Samsung S11 with Android 11

Devices that do not reproduce the issue

Google Pixels as they do not support decoding the Dolby AC-4 codec

Reproducible in the demo app?

Yes

Reproduction steps

1. Play attached media
2. Seek the player forward
3. Watch playhead snap to the beginning of the segment instead of seeking to specified point

Expected result

Player seeks to IFrame as close to specified point as possible, parsing subfragments within the segment accordingly.

Actual result

Player detects discontinuity and snaps to the closest Iframe at beginning of segment

Media

Will email clear AC4 media separately

Bug Report

• You will email the zip file produced by adb bugreport to dev.exoplayer@gmail.com after filing this issue.

[ronak2121]

Just to note that if we set the fragment sizes to be the same as the segments this doesn’t happen.

it’s only when fragments are smaller that Exoplayer doesn’t take subfragments into account when finding the appropriate IFrame.

[ronak2121]

Also to note, this works just fine for audio codecs that don’t use IFrames such as AAC, DD+JOC.

[ronak2121]

Here's what we've traced so far:

1. FragmentedMP4Extractor seems to only seek into the current fragment at the start of the segment regardless of codec. If the desired position is past that fragment, it doesn't do anything.
2. SampleQueue's seeking logic seems to kick in to seek into the extracted audio buffer. It seeks into the fragments themselves, looking for the closest keyFrame.
3. The logic to decide to commit the sample or not seems to be the key. If we force AC4 to behave like all frames are iframes here; it works fine.

[ronak2121]

FYI that we've emailed our media and ADB log dump as requested

[ronak2121]

We’re working on a PR that should resolve this issue. My coworker (@hcgil) will post it within the next day or so.

[hcgil]

Just put up a PR with a rough solution to the problem we're encountering. More info is given above as well as in the pr description - from the looks of it we're starting playback at the end of the first fragment because we cannot guarantee the presence of sync frames later on. This PR should resolve this by keeping a record of which samples we've encountered since the last sync frame, and committing them (and any successive frames) once we reach our start time, to hopefully decrease the magnitude of discrepancies when seeking.

[ronak2121]

Hi @rohitjoins we learned that xHE-AAC May also impacted by this issue. Have you had a chance to look at our PR? We're working with Dolby on this to ensure they are aligned with our suggested solution, but we're looking for your guidance as well.

[rohitjoins]

@ronak2121 I haven't had the time to look into this yet. You can expect a reply from me by next week. Thanks :)

[ronak2121]

@rohitjoins FYI that we’ve received confirmation from Dolby that this fix is the correct one. Please advise.

[ronak2121]

Hi @rohitjoins any update? This issue is blocking a critical upcoming launch for us. Can you please look into this?