Design of new-streaming

Introduction

The streaming-control subsystem underwent a major revision in SqueezeCenter 7.3. This page documents the design and some of the significant APIs.

There were several goals of this work:

• To bring some structure back to control of streaming. Source.pm, in particular, had become incomprehensible and was very fragile.
• The increased emphasis on multi-player synchronization made the purely player-centric view of the world difficult to work with. Things like gapless synced play were impractical.
• The code had too close a knowledge of the interaction with the player control protocols which made it unnecessarily difficult to support different types of players (SliMP3, SB1, SB2/SB3/TP, SqueezePlay).
• There was a lot of duplication of effort in many of the music-services and internet-radio protocol handlers.
• There is a desire to make SC smaller and less resource-intensive and the opacity of the code was hindering that.

Here are some highlights, both functional and non-functional:

• Slim::Player::Source largely replaced by an explicit state-machine implementation, in Source::Player::StreamingController.
• Gapless synchronization (but not for SliMP3 or SB1 - there is no architectural limitation but I have not taken the trouble to look at what would be necessary for these cases, although there are a few places where there is an implicit assumption of this non-support).
• Cross-fade works with sync.
• Mid-track join, when a new player joins, by restarting all players in a sync-group at the current playing position (for stream sources that support this). I have not (yet) tackled non-interrupting mid-track join, which is much more difficult and will require specific firmware changes among other things.
• Sync-group management independent of whether a player is (logically) powered on. That is, you can add an off player to a sync-group without it turning on, and UIs show off players as part of sync-groups.
• Gapless play (well almost) on SliMP3 (but not when synced, see above).
• Harmonised Getting-Track-Info/Connecting/Bufffering/Rebuffering control and feedback.
• Refactoring of transcoding framework to include explicit declaration of capabilities and more-flexible argument substitution.

Design

Streaming-control logical class model

• Local files are now streamed using a protocol handler, just like remote streams. There is still lots of code throughout SC that is conditional on isRemote() but not much in the streaming control code. The current implementation may be marginally less efficient for local transcoded files under Windows but this is not an architectural issue.
• Streaming control - play, stop, pause, etc. - are now the responsibility of a StreamingController object. This is implemented as a state machine. It replaces most of Source.pm and Sync.pm. All the players in a sync-group are, in effect, owned by a StreamingController. Much of the associated state which was previously held in the Client object is now held by the StreamingController.
• Players which are off and in a sync-group are still associated with a StreamingController. So, for example, isSynced() will return TRUE even if a player is off.
• A Song is now a blessed object, not just a hash. It is responsible for getting a track ready to play (interacting with the protocol handler) and opening the stream. The open() returns a SongStreamController.
• A SongStreamController is a generalized handle for controlling an active stream, regardless of whether it is direct, remote (indirect, proxied), local or transcoded. It's functionality and use are not properly designed/developed at the moment.

The Song Object

This used to be a simple hash - now it is a first class object. Both for legacy reasons and some development sloppiness, there is still plenty of delving inside its (private) hash. This needs to be cleaned up.

A Song captures the details of a single instance of a Track being streamed. It is responsible for getting the track ready to play, for example by interacting with SqueezeNetwork, and opening the stream itself along with any necessary transcoding. For direct streaming, it obviously does not actually open the stream, but it still prepares all the groundwork. Song::open() returns a SongStreamController which is the actual handle to an instance of a stream.

Occasionally a Song instance is reused: specifically when seeking within a playing stream. The SongStreamController is not reused.

If the current Track from the player's (or sync-group's) Playlist is in fact a Playlist in itself, then the Song is used to hold knowledge of the current position in the playlist (currentTrack), When moving from one track to the next, the existing Song is cloned and advanced to the next track. This kind of playlist is used for Internet radio and the like, where multiple alternate sources are available, possibly preceded by some banner track. It does not properly cope with the type of playlist that can include both sequential tracks and alternative streams for the individual tracks.

When getting the next-track information, scanUrl() and getNextTrack() are the two primary mechanisms. Most services have been fully converted to the new model. Song still uses onJump() and onDecoderUnderrun() as a fallback (Slacker) for the moment.

StreamingController

The heart of the system is instances of the StreamingController class. Each instance represents a set of players in a so-called sync-group.

This class implements four logical interfaces:

• PlayControl - stop / play / pause / resume / flush / skip / jumpToTime
• SyncControl - Controls membership of the sync-group for both active and inactive players.
• PlayStatus - Provides information about the current status of streaming to other functions within Squeezecenter. Much of this information can also be obtained via proxy methods on Client instances.
• PlayerNotificationHandler - Handles state-change notifications from players.

Controller State Machine

The state diagram is divided into two parts, and the state is maintained separately for both parts.

PlayingState tracks what the player is doing from an end-user point of view: Stopped, Buffering, WaitingToSync, Playing and Paused.

StreamingState tracks what the streaming-management is doing: Idle, TrackWait, Streaming, and Streamout. TrackWait is where information is being fetched about the next track: for example, pre-scanning HTTP tracks or interacting with SqueezeNetwork.

Most incoming events effect only one of the two parts of the overall state. stop and some error cases are the obvious exceptions. Also Buffering and WaitingToSync can be given a kick by some stream-management events (not shown).

I have tried to move away from explicit knowledge of the inner workings of the player software, although there is obviously some correspondence. For example, we no longer talk about decoderUnderrun, rather playerReadyToStream.

An outstanding problem is how to deal with incoming events which might cause a state change but which are now obsolete - some other action has already overtaken the situation that is being reported.

I am not convinced that Streaming and Streamout need to be separate states. I think that it could be possible to combine them.

Some combinations of PlayingState and StreamingState are illegal: Streaming & Streamout while Stopped, and Idle & TrackWait while Buffering or WaitingToSync. Thus there are 14 distinct states, which could be reduced to 10 if Streaming and Streamout were combined.

A couple of other variables overlay the pure states: rebuffering augments Buffering; nextTrack can augment TrackWait even after the next track information is ready but it is not possible to start streaming yet.

Players signal playerReadyToStream when they might be ready to stream the next track. It may be that it is not possible for the next track to follow until after the current track has finished playing out (or some other player-specific condition), and so the Controller has to check with each player in the sync-group to see if it is really prepared to start streaming the next song once the details are known.

Protocol Handlers

Protocol Handlers, or simply Handlers, are used to manage audio streams. There are a few core handlers:

• File: for local files
• HTTP: for remote HTTP streams, such as Internet radio
• MMS: for remote Windows Media streams

Many more handlers are added by various plugins.

Note that local files are now streamed using a handler. If a format does not have an associated handler then it cannot be played. Nonetheless, there are many cases where SC tests to see whether a handler manages a remote stream source - effectively, everything except local files.

Handlers have four main areas of responsibility:

• Provide a byte-stream for forwarding to players. This is for local files and remote stream sources that are proxied via SC. Such indirect streaming is used when transcoding is required or multiple players are synchronized (and indirect streaming is possible). When transcoding is used, the transcoding pipeline may open the stream (local or remote) directly, rather than using a handler instance.
• Manage the details of a remote stream source so that a player can play it directly (direct streaming). For any specific instance of playing a stream, this option is mutually incompatible with the first. A handler can support both modes.
• Provide metatdata about the stream. Most of the metadata will generally come from the Track object and is mediated by Slim::Music::Info but handlers have the opportunity to override some of this.
• Interact with remote music service, nearly always via SqueezeNetwork, to perform stream or track management functions.

Not all handlers support both direct and indirect or local streaming. In general, the implementations mix up the business of providing an actual byte-stream, with their other functions; that is, they have both class (static) methods and instance methods.

There is close interaction with the Song class. In the past, handlers have stored their private metadata via the Client object of a player, generally the master player in a sync-group. As much as possible, this work attempts to move this to the Song object instance associated with playing a track or stream. Sometimes this is via a generic pluginData element in the Song, and sometimes as more specific elements. The Song schema needs quite a bit of cleanup.

The handler API functions with regard to streaming are:

• scanUrl - Preform prescanning of a track URL to discover up-to-date information. It may be that the result actually represents a playlist, in which case the scanning process is expect to perform any necessary scanning of the constituent tracks. This scanning is performed asynchronously, the results being notified by callbacks.
• getNextTrack - Perform any operations, such as interaction with music services, to obtain all the necessary details of the next track to be streamed. Like, scanUrl, the work of this method is performed asynchronously, the results being notified by callbacks. This may determine that the stream URL will be different from the Track URL, and so may also require a different handler.
• canDirectStream - Return the actual URL to be streamed, if direct streaming by the player is supported in the current circumstances. Some of the checks associated with sync-groups that the handler is expected to do could probably better be done by Song.
• new - Open and return a byte-stream to the stream. The resulting object will be a Filehandle of some sort.
• isRemote
• canDoAction - Is it permitted to do the proposed stream-control action. Current possibilities are:
  • 'rew' - restart the current track,
  • 'pause' - pause the stream without closing it,
  • 'stop' - a proxy for skip, meaning skip to the next track (provided by this handler).
• canSeek - Is it possible to seek to a specific time in the current track.
• getSeekInfo - Get a (mostly opaque) dataset that the handler could use to reopen the stream at a specifed time offset. The timeOffset element, if present, may be used to pass to seek-capable transcoding piplines, if appropriate.
• trackGain
• shouldLoop
• onStream - Called for each player in the sync-group immediately before instructing that player to start streaming.
• onStop - Called when playback is stopped, including skips and playlist jumps.
• onPlayout - Called when finishing streaming a track, generally with about 10s left to go and before (trying to) stream the next track, if any.
• overridePlayback - Used by pseudo-handlers which generally replace the current track in the playlist, or the entire playlist. This method, if implemented, is called instead of creating and opening a Song instance.

The following are planned:

• canIndirectStream