Managing a video archive with Membrane Software
Tuesday, May 23 2017 22:20 GMT
Posted by: MembraneTags:programmingmembrane control
As more people join the world of modern, high-speed Internet, we've seen the rise of video as the preferred medium for communicating information. Indeed, according to some reports, video will comprise a staggering 82% of internet traffic by 2020. Use of video in place of text certainly has its advantanges, not least of all the ability to capture nonverbal queues helpful for human interaction. However, video also comes with a rather heavy downside in its sheer bulk and general lack of convenient ways to work with it. Managing thousands of text files is a snap for a personal computer and its operator, and the whole file set can be transmitted across the Internet in moments. On the other hand, managing thousands of video files, each of which might have its own quality, encoding standard, and format, quickly becomes a tangled and time-consuming mess. At Membrane Software, we seek better lives through computing, and today we'll look at development screenshots from a software system designed to help those like us: video enthusiasts who crave a convenient archive and retrieval system.
A byte-size problem
Watching video logs sure can take a long time and a lot of effort compared to just reading words from a page. Images from TrekCore
Many sci-fi visions of the future seem to indicate the death of text as a medium, even going all the way down to personal journals such as a starship captain's log. As mentioned, there are compelling reasons to record information in video form: a deep-thinking haiku could be stored as a dry text file, or it could be recorded as a dramatic reading to capture its fullest essence and intent. However, the haiku text file might consume a mere hundreds of bytes worth of space, let's say 1024 to use a nice round number. The haiku video file, if using decent quality settings and a 10-second reading time, will easily reach into the megabyte range and give us a 1000-fold or more increase in storage requirements. And of course, a typical video collection contains far more than haiku: home videos, surveillance records, blockbuster action films, you name it.
The two main functions we ask of our video collection are storage, or the ability to accept any item we want to keep in the archive, and retrieval, or the ability to quickly find and present any desired item. These are, in fact, the same functions we ask of any file system, but when all of the files are huge, things become a great deal more difficult to manage. Every operation involving a large file has a proportional cost in time, and must therefore be carefully considered. Routine backups of a multi-terabyte video archive to, say, an external hard drive can involve hours of waiting, and most users should forget about sending the archive to someone over the Internet; even at speeds typical for today, it's probably faster to copy our files to a hard drive and take it there ourselves.
So, we see that the primary issue of working with video is one of scale. Video file sizes can be backbreaking at all stages of operation, from storage to copying to transmission and playback; any significant collection easily outstrips the capabilities of PCs to hold it and the Internet to forward it. As we design a system to handle videos, we must keep this constraint at the forefront of our thinking.
A system of agents
A naming scheme can be an effective system in managing files, but requires user thought and effort to maintain. Image from PCMag.com
A typical user with a video collection has few good options for management. A Google search for "manage video collection" brings up the PCMag.com article Get Organized: How to Manage Video Files, which does offer some effective practices, such as use of a date-based naming scheme for organization. They summarize the practices as follows:
To recap, organizing your non-professional videos really comes down [to] three steps: centralizing your video files (that is, getting them off your smartphone and video camera), naming them intelligently and tagging them so you know what they contain, and backing up and archiving your final edited files.
While plainly sensible, these practices are attached to what I'd consider a serious flaw in the requirement for user interaction. If it sounds like it takes a fair amount of effort to do all of this centralizing and intelligent naming, that's because it does. For businesses that work with large amounts of video, it can be helpful to hire employees whose entire job revolves around archival management, even just to have someone to hang around during copying so the boss can go have lunch. But wait: aren't computers supposed to make life easier by doing all of the work for us, instead of us hiring people to work for them?
The Membrane Software system for managing video, then, seeks to minimize the need for any human agent by providing a series of software agents, each with a specific set of roles to fulfill:
Media server: Store media files. Provide thumbnail images for browsing. Allow media files to be transferred on request.
Stream server: Store video-on-demand (VOD) stream data files. Implement streaming protocols such as HLS and DASH, allowing playback over a network connection by the display control agent (below), as well as devices such as tablets and PCs. Pull files from a media server agent and store them locally as VOD streams. Provide thumbnail images for browsing.
Display control: Provide complete command over content displayed by an attached monitor. Execute playback operations from stream servers on demand.
If we are able to implement each agent as an autonomous and fully reliable appliance, the user should be relieved from most responsibility over mundane file management. In the ideal outcome, the user's role is reduced to simple decision making such as "I want video X to play on screen Y right now", and the system takes care of the rest. And by defining the system in terms of role-playing agents, it becomes malleable according to the real-world needs of its users. For example, a user with petabytes upon petabytes of video might spread files across multiple media server agents, while another user fits his collection on just one. Or, a user running a space with digital signage might set up an array of many display control agents with continuous playback, while another user casually scrubs through his latest home video on a single display control agent that drives a living room TV. Even with just three agent types defined, we can see that our the system is able to adapt to a virtually endless number of use cases, and to do so far more effectively than it would if it depended on a single PC to run the whole show.
Browsing media in the control interface
The astute reader may notice one type of agent missing in the above list.We have already created software for working examples of media server, stream server, and display control agents described above, which have formed the basis for various technical demonstrations. In our system of agents, however, one of the most crucial elements to consider is user interface. After all, without demands from a user, the agents in our video management system sit by, largely idle and providing benefit to no one. We've conceived this "Membrane control" layer as a GUI application, which was briefly described in a previous blog post.
Membrane control: Allow the user to execute commands targeting other agents in the system
- When the interface starts, it searches for available agents and displays them on a desktop-style surface. Here, we've found two agents, localhost and pi-display1.
- Agent capabilities are indicated with icons. localhost is running a media server and a stream server, and is probably a PC with a powerful CPU and large storage space. pi-display1 is running just a display control, and is probably a Raspberry Pi device attached to a monitor.
- This screen lets the user drag items into any desired arrangement. Zooming in on the view shows fewer items in greater detail, while zooming out shows more items in lesser detail.
- Clicking the "Media" button on the start screen brings us to the interface for browsing video items. The top line shows our available media server and stream server agents, both on localhost.
- The interface automatically requests directory listings and thumbnail images from media servers. It offers three levels of detail for browsing, again with a tradeoff of item volume versus item detail. Here, we start in "Tiles" mode, the medium detail view.
- In "Thumbnails" mode, we can scroll through many items quickly but must do so without text queues.
- In "Details" mode, each item consumes its own chunk of vertical space and offers a full interface for executing applicable actions. In this case, the "Create stream" action executes one of the stream server agent operations described above: Pull files from a media server agent and store them locally as VOD streams.
- When the "Create stream" action is pressed, the interface collects required parameters for the operation and allows the user to confirm or cancel. In this case, we only need one parameter: the stream server that should prepare a VOD resource for this video, thereby allowing playback over a network.
- Once confirmed, the interface sets the create stream task in motion and reports on its progress.
- The create stream task persists on the system independently of the interface application. Therefore, we can quit the application and launch it again later to check on the task's progress. This is great for the user, because a stream server, even a very powerful one, will take hours to prepare VOD for a single high quality video of moderate duration.
Making real progress (dialogs)
That's as far as our interface goes for now. At this point, the create stream progress dialog is a mockup that doesn't actually execute any operations, and so the next bit of work picks up there: the interface application must now send the correct stream server command and monitor progress for the task. Piece by piece, as the interface continues along the development path, it comes to represent more and more of the media server, stream server, and display control operations making up our video system.