Background

I've been participating in MPEG's DASH group, and currently a lot of work has been focused on reducing live streaming latency. The latency problem in DASH is that clients have to poll servers to check for new media segments. If they poll too slowly, it introduces latency, but if they poll too quickly, it increases server load. When MPD's are dynamic, a client needs to poll the server until it finds a new MPD, then request newly available segments, then only after the segment has downloaded enough to start, it can continue playback.

One solution to this is to use HTTP/2's PUSH_PROMISE feature to immediately push new MPD's and media segments, but that introduces several new problems:

  • How does the server know what to push (does it have to understand DASH MPD's)?
  • How do you tell the server to start and stop pushing?
  • How do you detect pushed content in JavaScript (XHR doesn't have push events)?
  • How do you ensure that the browser doesn't fully close the HTTP request stream, preventing pushed content?

These problems can be solved, but current solutions are fairly complex, involving DASH-aware HTTP/2 servers, new headers, and either new JavaScript API's or a second event stream using WebSocket or Web Push.

Timeout Header

I realized recently that our goal here is to request a resource that doesn't exist yet, so I figured, why not just add a request header to indicate that? My proposed header is "Timeout", with an integer value indicating how long to wait before returning a 404.

Here's an example of a request, telling the server to look for the file for 10 seconds before giving up:

GET /segment-1.mp4 HTTP/1.0
Timeout: 10000

Then the server just needs to use an API like inotify to watch for the file. If it appears, the server can immediately send it. If it doesn't appear within the timeout (or the server's maximum configured timeout), it can send a 404 like usual.

In the case of MPEG-DASH, a client would use MPD@availabilityStartTime, SegmentBase@availabilityTimeOffset and SegmentBase@duration or S@d to predict when a segment will appear, and request each one a short time before (at least one RTT early). If the server responds with a 404, the client can fallback to normal behavior (waiting until slightly after the segment availability time to request it). The only requirement for the MPD is that segment names be predictable, meaning SegmentTemplate's $Time$ probably shouldn't be used.

My example GitHub repo contains a more detailed description.

Example

Update: The server is now available as Express middleware. Using it looks like this:

var express = require("express");
var expressTimeout = require("express-timeout-header");

var staticFolder = "static";
var port = 8080;

var app = express;
app.use(expressTimeout(staticFolder));
app.use(express.static(staticFolder))
app.listen(port);

I created a GitHub repo with an example HTTP server supporting the Timeout header. It should work on at least Linux (tested on Fedora 21) and OS X, with Google Chrome or GPAC's MP4Client. This should work in any browser which supports DASH.js and h.264 (but I wasn't able to get DASH.js to work in Safari, and I don't have a machine with Internet Explorer 11 available to test on).

By default, it adds a timeout to all requests, to make it easier to demonstrate with unmodified clients (like DASH.js in Google Chrome -- which works right now!). If you want to see the proposed behavior, run http-server.coffee with --require-header.

You can also see a screencast demonstrating it on YouTube.

Notice that playback starts the exact moment the first segment appears.

With Caching

Update: This is now supported. Just set etag = true in the settings for express-timeout-header:

app.use(expressTimeout(staticFolder, {etag: true}));

One thing I'd like to add is to extend this to caching with the "ETag" header (using hashes as an etag): If a client sends a request with "If-None-Match" and "Timeout" headers, the server should watch the file until the timeout expires or until the ETag doesn't match before responding with a 304 - Not Modified. This would allow us to instantly push dynamic MPD updates too.

Other Use Cases

Something I'm wondering about is other use cases. This seems to solve DASH's problem, but to propose this in IETF, I'd like to see if this is useful for other cases. I have a vague idea that people could use this in REST API's, but that's not specific enough.

In the caching situation, this should be useful for HLS playlists, and possibly applications like Facebook or Twitter (or ATOM and RSS) with information "streams".

If you have other use-cases, please email me.