IPEN

Location: work / Software / IPEN

Vision and authorship: Daniel Apostolov

Summary

In the scope of this document the term Integrated Personal Electronics Network (IPEN) denotes the concept for a centrally and efficiently orchestrated streaming and redirection of multimedia data between all personal electronic devices with the provisioning of strong Internet and Cloud service connectivity.

Motivation

Similar innitiatives exist, such as the development of the Digital Living Network Alliance (DLNA) protocol and custom hybrid hardware/software solutions like Sonos, Google Chromecast . All of the known platforms (additional reasearch needed) exhibit a range of show-stopper limitations, though:

  • Closed code
  • Closed standards
  • Lack of documentation
  • Lack of community
  • Bugs
  • Functionality is limited to a narrow set of highly specific usecases
  • Driven by huge organizations, slow to progress

A unified, standartized solution is needed to allow seamless integration of all media-capable personal devices and services (mobile phones, tablets, TVs, set boxes, reveivers, dvd players, car audio, home central audio, youtube, soundcloud, spotify, netflix, sonos etc.), aiming to allow the end user or groups of users to purchase, cache, share and play music, video, movies and to use voice and video communication over any available network and on any available display and audio-speaker with the ability for fine-grained control over resource allocation.

Use Case Showcase

  • A friend shares a youtube link in a messanger. The user is currently at home. With a maximum of TWO actions they instruct the IPEN to play the song over the central home audio system. While the song is playing the user leaves the home. As per precofigured rulest the IPEN pauses the playback in the home when the user departure is detected.
  • The user listens to an audiobook on their headset while cycling in the wild. As they get back home the IPEN detects the arrival and offers the user to catch on with the audiobook playback over the central home audio system. A single click on the headset accept call button confirms the redirection.
  • A user has a playlist of selected songs on their mobile device. While visiting a VIP social club they wish to play the playlist on the central audio system of the club. With a maximum of TWO actions the user throws the playlist to the local IPEN. The IPEN scans it's local song cache, the registered Spotify accounts and youtube (possibly other media storage locations and multimedia services) and for each song from the playlist automatically selects the source with lowest latency and resource requirements. Songs that are not found are automatically cached in the local IPEN cache storage for smoother playback. Depending on a preconfgured strategy the IPEN may decide to temporaily rearrange the playlist so that the songs that are readily available are played first while slower caching ops are executed.
  • A couple is watching a movie in their bedroom on the bedroom TV. One of the partners dozes off. The other partner decides to limit the lighting and sound levels in the bedroom, while continuing to watch the movie. They put on a wireless headset, open their lapton and with a maximum of THREE actions redirect the video track of the movie to the laptop screen and the audio track to the headset. The IPEN automatically puts the TV to sleep.
  • A central home audio system is installed at the user's home. Every bedroom and the kitchen hahave their own 5+1 DSP system installed, the hallway and the bathrooms are connected in a single Hi-FI (stereo) setup with shared speakers. There are two laptops in the home, a dedicated media-center computer and also two mobile phones and one iPad. The user is capable with a maximum of TWO actions executed on either of the described devices to direct the audio output of any software running of either of the devices to any combination of the connected 5+1 DSP systems and/or to the shared Hi-Fi system. With a maximum of TWO more actions per target audio system the user is capable of adjusting the master volume level, separate speaker levels and other audio-playback related settings.
  • A user enters their car. Their mobile device is automatically connected to the car audio, redirecting both music and voice communication. If the user's phone is currently playing a song or a playlist the playback is continued over the car audio. The automation and streaming is performed by the IPEN. At a later time another passenger carrying a laptop can instruct IPEN to play youtube songs as they select them. The driver has authorative control over the IPEN behavior from the touch screen of the car's dahsboard.
  • A user installs on their mobile phone an app that uses ML to detect their physical state and/or mood based on the phone and smartwatch sensors. The user wants the app to continuously adjust their playlist with music that corresponds to their current mood (happy, sad, aggrevated) and physical state (asleep, at work, excersizing). The app provides new songs in the playlist, IPEN automatically locates the best sources of the songs and plays them from whatever media storage and services are available.
  • A user wants to synchronize their LED lighting with the music playing on the central audio system. IPEN-enabled LED controllers allow the IPEN to receive instructions from the user's mobile device to do so.

Key Points

  • IPEN must be developed into an industry standart for multimedia sourcing, streaming and redicrecting (Stage 1)
    • a logical next step would be to extend the standart to include general data sourcing, routing, dispatching and presenting over heterogenous networks and platforms (Stage 2)
  • When talking about multimedia (Stage 1), UX and general usability (configuration, predefined templates, scheduling, automation) is one of the most important points to address; this is the USP of the product - to make things for the end user as easy as it gets. This is the point of excellence
  • Implementations could be kept propriatary (it's a matter of discussion and adaptation); protocols and standards must be open and from some point on - community driven while moderated and formulated by the IPEN vendor. The expertise of the industry leading companies (audio electronics manufacturers, mobile device manufacturers, multimedia services, computer part manufacturers) is invaluable and must be harvested and orchestrated in order to acheive a durable solution actevly supported and endoresd by the giants.
  • The accent here is set on the data flow orchestration and UX rather than on stream encoding algos (codecs) or underlying topology; whenever possible existing codecs (open and propriatary) shall be preferred to custom tailored ones; the development of new algorithmically-complex solutions shall be outsourced.
  • An effort should be made to create and polish IPEN drivers and smart agents for each and any multimedia-capable device and multimedia service that exists on Earth. While the platform is still young this effort falls to the IPEN vendor. At later stages there will be possible to establish a certification process and outsource the driver development to the multimedia device and service vendors.
  • A very short and easy-to-remember label must be created for IPEN to make it recognizable by individuals at any age and background. I simple, unique and friendly icon must be provided along with the label.

Challenges

  • Avoid competition with existing efforts in the same direction; nurture an environment of cooperation instead, while establishing and maintaining a leading role
  • High level of expertise in the area of multimedia computing is required from day 1
  • The variety of multimedia-capable devices available on the market is overwhelming. Early-on, a Device Integration Lab must be set-up and clear and strict internal procedures must be defined in order to guarantie a certial level of... buglessness. The performance of such lab would be a crucial factor for business growth, as it presents itself as a bottleneck during market adoption.

Further Thinking

Multimedia streaming orchestration is a subset of computing resource distribution and orchestration. The emergence of a distributed OS utilizing-, spanning-over- and partitioning all of a person's/organization's electronc devices is plausible as a result of the effort to create IPEN and the hands-on experience gathered thereon.