NowPlaying needs further work before it's ready

[timwindsor]

1. Should use the NPconnection->acquire() call and wait for the signal before playing.
2. Should listen for Revoked signal and stop playback if received.
3. Do not hardcode any file paths, or default audio streams.
4. Should accept more metadata properties than the 3 given now.
5. Should take an icon for the media being played
6. Should use the Fancy OverlayStyle always (Plain is default and shows nothing valuable).

[parker-mar]

Some questions I have at the moment:
- Are we supposed to allow the user to specify actions wrt to a signal?
Story implies we hard-code, but https://developer.blackberry.com/native/documentation/graphics_multimedia/audio_video/accessing_media_notification_areas.html implies we don't.
- How do we specify signal handling normally then? Are Cordova slots/signals replacing internal npc handling?
- How about play, stop, pause - because we hard-coded behavior, no need signal/slot?
- How do WE specify what acquire does in C++ (not qml)? Signals and slots? Then user who uses our plugin won't? Shoulnd't user specify some kind of signals slots thing?

Initial story sizing and prioritization:
1. ~7 Should use the NPconnection->acquire() call and wait for the signal before playing.
- New function that just emits NPconnection->acquire() signal
- New slot it to acquire()
- mp->play()
npc->setMediaState(bb::multimedia::MediaState::Started);
npc->setMetaData(mp->metaData());
2. ~5 Should listen for Revoked signal and stop playback if received.
- New function that just emits NPconnection->revoke() signal
- New slot it to revoke()
- mp->stop()
3. ~1 Do not hardcode any file paths, or default audio streams.
- How is the NowPlaying plugin intended to provide access to stored media? Where would the media be? (https://developer.blackberry.com/native/documentation/graphics_multimedia/audio_video/accessing_media_notification_areas.html - assets... right? Or media player songs added by user? If the app accesses files there, how can I specify them anymore thru plugin?)
4. ~1 Should accept more metadata properties than the 3 given now.
- Create a test function to see json values set?
- Are we checking whether individual json values are set?
5. ~1 Should take an icon for the media being played
- Need to know how NPconnection works first? Get sample?
6. ~1 Should use the Fancy OverlayStyle always (Plain is default and shows nothing valuable).
- https://developer.blackberry.com/native/documentation/graphics_multimedia/audio_video/accessing_media_notification_areas.html
*- what is overlay, still? https://developer.blackberry.com/native/documentation/graphics_multimedia/audio_video/accessing_media_notification_areas.html
-> play with controls, music controls. That pops out with metadata.

[parker-mar]

I've been working to understand the functional requirements of NowPlaying, but it isn't yet clear to me. This is my current understanding:

QML Example:

• User can specify when npc.acquire(), npc.revoke(), mp.play(), mp.pause(), mp.stop() called (main.qml in https://developer.blackberry.com/native/documentation/graphics_multimedia/audio_video/accessing_media_notification_areas.html)
• Users specify handlers - they do so by using some (predefined somewhere?) signals: onAcquired, onRevoked, onPause, onPlay, onStop

C++ Example:

• User can specify when npc.acquire(), npc.revoke(), mp.play(), mp.pause(), mp.stop() called (main function)
• Don't see how users specify handlers.
  Q: How are users intended to interact with the plugin?

NowPlaying Cordova:

• User can specify when npc.acquire(), npc.revoke(), mp.play(), mp.pause(), mp.stop() called (called in application's index.js)
• Don't see how users specify handlers.
  Q: What exactly do we want? How are users intended to interact with the plugin?
  • The stories sound like we hard-code the signal handlers.
  • What is it that we really want? From my understanding, we don't need play, stop, pause, etc. (these belong to music player - do we have another cordova module for this), just native NowPlaying's acquire, revoke.
  • Why are we defining our own signals and slots in cordova NowPlaying? Shouldn't we use native NowPlaying's internal signals and slots? Or is defining our own signals and slots really how C++ is meant to interact with the native NowPlaying (By ignoring the internal signals and slots and instead using our own method of communication)?

[timwindsor]

It may be important to note that this Plugin's primary goal is not to play audio - there is already a standard HTML5 way to play audio. The goal here is to allow audio to be played while being a good citizen on the system, and to interact with the overlay feature.

So, we want to provide an API where the user (app developer) can "request" playback of their media, and when the service is acquired, playback starts with the source URL, metadata, and icon, that the user requested. Options to enable or disable Next/Previous buttons should be included.

The user should be able to receive callbacks on the various events that can happen: Acquire, Revoke, Play, Pause, Stop, Error, Next, Previous. The first six will be primarily notification in case the application wishes to update some UI to reflect that state. The Next and Previous events would be something the application would respond to by setting new media details.

Accordingly, there should be trackChange API, which doesn't do the Acquire again, but sets the new icon, metadata, url, and calls the trackChange slot to notify the Now Playing Connection. It should also include the options for enabling and disabling the Next/Previous buttons, as that would likely happen at this time.

Since the app will almost certainly have it's own UI for playback, we'll also need APIs for Play, Pause, Stop, which will pass through to the enclosed MediaPlayer object. Stop should probably release everything as well, so that it would need Acquiring to start playback again.

It may be useful for debugging to have methods to return the current media playback state, and if the connections is "acquired' or "preempted".

I would propose something like this for the API:

cordova.plugins.nowplaying {
    requestPlayback: function( 
        { url: "/filepath",
          icon: "/path",
          metadata: { artist: "Jane Doe", ... },
          nextEnabled: true,
          prevEnabled: true,
          callbacks: {
               acquired: function() { ... },
               ...
          }
        }) { ... },
     trackChange: function ({
          same as requestPlayback, but no callback piece
     }) { ... },
     stop: function() { ... },
     resume: function() { ... },
     pause: function() { ... },
     currentState: function() { returns { state: string value?, acquired: boolean, preempted: boolean }
}

The callbacks defined in the first method would be used for success/error when calling the other methods.

[timwindsor]

The previous developer thought it was useful to define their own signals and slots in the plugin. It worked for them, but it's not the only way to do this. I've tried to define above what the API should provide to a user of the plugin - the implementation details don't need to be apparent at that level. An app developer is unlikely to know or care about Signals and Slots or that the media player and now playing connection are separate objects for example.

[timwindsor]

Edited the API to use "resume" instead of "play" as it's really the opposite of "pause".

One other thing I was thinking is that we should only allow nextEnabled or prevEnabled to be set true if there is a callback defined for them.

[parker-mar]

(It is easier to read this post by copy-pasting the text to a text-editor that line-wraps.)

Native functionality (as I imagine it to be):

Underlying logic:
    - Each media has a source and a logical (not physical) media notification area associated with it. Events associated with a media are sent and handled in its media notification area. 
    - There is underlying logic for designating a "main" active media source between several. This is abstracted away from all media notification areas.
    - There is a volume overlay widget that is associated with the media notification area of the "main" active media source. The logic (JS), structure and presentation (HTML, CSS)... everything for the widget wrt the media notification area, the media source, etc. is handled by the OS.
    - There is underlying logic for playing/stopping/pausing media when an app is closed, sent to the background, or sent to the foreground. This is abstracted away from all media notification areas. It is done through connection priorities (https://developer.blackberry.com/native/documentation/graphics_multimedia/audio_video/accessing_media_notification_areas.html see that section.)
    - There is underlying logic for how playing/stopping/pausing media causes the hardware to do so. This is abstracted away from all media notification areas.
    - Has concept of signal and slots: http://developer.blackberry.com/native/documentation/dev/signals_slots/ Signals are handled by slots.

APIs: 
    MediaPlayer: An instance of this class can:
        - control a single media source (up to 8 instances can play simultanesouly.) by sending a play/stop/pause command to the hardware.
    NowPlayingConnection: An instance of this class can:
        - call acquire()/revoke() to gain exclusive system-wide access to the media notification area for the currently "main" active media source.
        - when access is acquired to the media notification area, it can display/hide and modify the data in its volume overlay from specified data.
        - enable/disable the next/previous buttons on the volume overlay of the media notification area.
        - handle native signals for play/pause/stop/acquired/revoked which can be attached to a slot
        - return the current media playback state (stopped, playing, paused), whether the media notification area is acquired, and whether the connection to the media notification area is preempted by a higher priority connection (https://developer.blackberry.com/native/documentation/graphics_multimedia/audio_video/accessing_media_notification_areas.html under connection priority). 
    NowPlayingController: An instance of this class can:
        - control the media source that is acquired by an instance of NowPlayingConnection by sending a play/stop/pause command to the hardware.

Desired functionality:

Plugin providing a API between JS and the native APIs, which will allow a
developer to:
1. Play audio "while being a good citizen on the system"
2. Interact with the overlay feature

User Stories: 
-------------
(Note: play is intitial "play", as opposed to resume)

- "As a developer, I want API to acquire(play)/revoke(stop) system-wide access to the media notification area for the currently "main" active media source."
    - NowPlayingConnection.acquire()/revoke().
    - Update the current media player state with NowPlaying for stop with setMediaState(Stopped)

- "As a developer, I want API to change the track being played."
    - NowPlayingConnection.trackChange().
    - Ensure you don't call NowPlayingConnection.acquire(), in case you intentded to reuse that API. Can create helper function instead.

- "As a developer, I want API to resume/pause the currently "main" active media (that was started with acquire)."
    - MediaPlayer.play()/pause().
    - Update the current media player state with NowPlaying.setMediaState(Stated/Paused)

- "As a developer, I want to be able to specify the music I want to play when I acquire the media notification area or change the track."
    - Basically, call MediaPlayer.play()/pause()/stop().
    - The user specifies music through a URL (MediaPlayer.setSourceUrl()) or through the device filesystem (need to find out how)

- "As a developer, I want to be able to specify metadata for the music I want to play when I acquire the media notification area or change the track."
    - Basically, call NowPlayingConnection.setMetaData().
    - Do more than just the three current ones (title, artist, album) http://developer.blackberry.com/native/reference/cascades/bb__multimedia__metadata.html 

- "As a developer, I want to be able to specify the icon for the music I want to play when I acquire the media notification area or change the track."
    - NowPlayingConnection->setIconUrl()

- "As a developer, when I acquire the media notification area or change the track, I want it to play the music specified."

- "As a developer, when I acquire the media notification area or change the track, I want the volume overlay to be displayed with the metadata and icon specified."
    - NowPlayingConnection->setOverlayStyle(OverlayStyle::Fancy)

- "As a developer, I want API to enable/disable the next/previous buttons of the volume overlay"
    - NowPlayingConnection->setNext/PreviousEnabled(bool)
    - Assert callback is specified when bool is true.

- "As a developer, I want API to receive callbacks from Acquire(play), Revoke(stop), Resume, Pause, Stop, Error, Next, Previous so that I can do actions like updating UI (first six) or set new media details (last two: Next, Previous) to reflect the state of the app after the function is done executing."
    - See https://github.com/blackberry/WebWorks-Community-APIs/tree/master/BB10/Memory for how SendPluginEvent works: "If you want the native code to be able to trigger an event on the JavaScript side then you'll need to call the SendPluginEvent function."
    - Example code path: setMetaButtonClick() 

- "As a developer, I want API to return the current media playback state and if the connection is acquired or preempted"
    - NowPlayingConnection.mediaState(),NowPlayingConnection.isAcquired(), NowPlayingConnection.isPreempted()

- "As a developer, I want API that will allow me to define callback handlers for when my app gets preempted or is no longer preempted."
    - "With no audioManagerHandle specified, the now playing service will automatically call [emit] play() when no longer preempted. If an audioManagerHandle is specified, the now playing service will only call [emit] play() if the audio routing has not changed to a more public device while preempted." (http://developer.blackberry.com/native/reference/cascades/bb__multimedia__nowplayingconnection.html#function-play). I think this play signal is the one for NowPlayingConnection (http://developer.blackberry.com/native/reference/cascades/bb__multimedia__nowplayingconnection.html#function-play), and can be tied to a slot. We just have to provide an API function which requires a callback function (that we should assert must be specified?) for this slot, which will execute when the play signal occurs, i.e. when the media is no longer preempted. 
        - I am not sure this is feasibly implementable though becauase idk if we can catch/handle/slot the emitted play() signal. One way to find out is to create an app and see how preempting behaves, and see if we can catch the signal and slot it.
    - I'm assuming pause() signal is emitted when we are preempted (I didn't find this specified in the native references though) and that we can do the same as for the play() signal.


To ask:
    - What do you mean by "Error" event? ("The user should be able to receive callbacks on the various events that can happen: Acquire, Revoke, Play, Pause, Stop, _Error_, Next, Previous.") 

    - "[The trackChange API] should also include the options for enabling and disabling the Next/Previous buttons, as that would likely happen at this time."? Do you mean because e.g. we might hit the last track or something, we should ensure next/previous button enabling/disabling can be specified as a callback?

    - Clarification on syntax for API for requestPlayback: Are you proposing to use a JSON parameter to specify how the JSON argument must be formatted (is that valid? I am unfamiliar with this syntax)? Are you are proposing to specify acquired as a named function callback paramter that must be specified in the function JSON argument (is that valid syntax, with the {...}?)? Is this intended to be a variable argument list of callbacks?

    - Clarification on syntax for API for stop, resume, pause, currentState: These will take callbacks as parameters too, right?

[parker-mar]

Storyboard of example app using the NowPlaying plugin.

1. Play -> Pause -> Resume -> Pause -> Resume -> Stop
2. Play -> trackChange -> Stop
3. Preempted -> ?

[parker-mar]

Here's some information on the file structure with respect to the architecture of this plugin.

The plugin is in "plugin".
"sample" is a sample app that uses the plugin.
nowplaying_build.sh automatically creates another app called "debug1" and adds the plugin to it. It then uses the "sample" app as boilerplate code to copy into the newly created "debug1" app. The script then adds blackberry10 as a platform, and does a cordova run. You can use this script to run and test changes, because the first thing it always does is remove the "debug1" app.

Below, you'll see the layers wrt the drawing Tim made on the whiteboard during the code sprint. The first file listed in each layer is what you need to change (I like to open the non-.cpp/hpp files in order side by side on my editor, then the .cpp/hpp files in order side-by-side on momentics). If I remember correctly, the second file (second "->") is where the file is copied to when you add the plugin or when it gets copied from the "sample" app to the newly created "debug1" app; the third file is where the file is copied to when you add blackberry10 as a platform or is what the file is compiled to (a library).

App "main" html: sample/www/index.html
-> debug1/www/index.html
-> debug1/platforms/blackberry10/www/index.html

App "main" js: sample/www/js/index.js
-> debug1/www/js/index.js
-> debug1/platforms/blackberry10/www/js/index.js

App: plugin/www/client.js
-> debug1/plugins/com.blackberry.community.nowplaying/www/client.js
-> debug1/platforms/blackberry10/www/plugins/com.blackberry.community.nowplaying/www/client.js

Cordova: debug1/platforms/blackberry10/platform_www/cordova.js

Controller: plugin/src/blackberry10/index.js
-> debug1/plugins/com.blackberry.community.nowplaying/src/blackberry10/index.js
-> debug1/platforms/blackberry10/native/device/chrome/plugin/com.blackberry.community.nowplaying/index.js

TemplateJS: plugin/src/blackberry10/native/src/NowPlaying_js.cpp
-> debug1/plugins/com.blackberry.community.nowplaying/src/blackberry10/native/src/NowPlaying_js.cpp
-> debug1/platforms/blackberry10/native/device/plugins/jnext/libNowPlaying.so

TemplateNDK: plugin/src/blackberry10/native/src/NowPlaying_ndk.cpp
-> debug1/plugins/com.blackberry.community.nowplaying/src/blackberry10/native/src/NowPlaying_ndk.cpp
-> debug1/platforms/blackberry10/native/device/plugins/jnext/libNowPlaying.so

[parker-mar]

We can divided the user stories into these 6 main items (tried to make them as mutually exclusive as possible):

1. Implement function to return state: media playback state and connection acquired or preempted.
2. Implement acquire, revoke, resume, pause, stop, error, next previous using native MediaPlayer API.
3. Implement callbacks into JS layer for acquire, revoke, resume, pause, stop, error, next previous.
4. Implement handlers for when the app gets premempted or is no longer preempted (needs further investigation).
5. Implement functions for specifying metadata, icon, and music to play.
6. Implement functions to acquire/revoke media notification area. Prepare volume overlay over media notification area.

• Populate it with metadata, icon, and music.
• Provide functionality to enable/disable the next/previous buttons of volume overlay.

[timwindsor]

Answers to your questions from earlier:

• What do you mean by "Error" event? ("The user should be able to receive callbacks on the various events that can happen: Acquire, Revoke, Play, Pause, Stop, Error, Next, Previous.")

In the course of operation, an error might be encountered - like media not being found, low level issues from the native API/hardware, attempts to play without setting media, etc. There should be a way to get those messages.

• "[The trackChange API] should also include the options for enabling and disabling the Next/Previous buttons, as that would likely happen at this time."? Do you mean because e.g. we might hit the last track or something, we should ensure next/previous button enabling/disabling can be specified as a callback?

I think that should be a property set on the input when calling trackChange. When you set the track, that would be when you know if you are able to handle a Next or Previous request. It could be that you've reached the end of the list you were playing from, or you are starting a new list, or perhaps you are playing a single track with no other tracks to switch to. It seems less likely that the value would change after starting to play a track.

• Clarification on syntax for API for requestPlayback: Are you proposing to use a JSON parameter to specify how the JSON argument must be formatted (is that valid? I am unfamiliar with this syntax)? Are you are proposing to specify acquired as a named function callback paramter that must be specified in the function JSON argument (is that valid syntax, with the {...}?)? Is this intended to be a variable argument list of callbacks?

With JavaScript you typically document the properties you are expecting, and you can check for them and return an error if something is missing or incorrectly formatted. What I'm suggesting is the format of that object and the property names that you would use. The { ... } parts are what a developer would supply. With JavaScript you can specify the name of a function or define it inline.

• Clarification on syntax for API for stop, resume, pause, currentState: These will take callbacks as parameters too, right?

I think stop, resume, and pause would likely be simple methods that take no parameters and return nothing. The output of anything that happens with them would likely fall into one of the callbacks defined earlier.

currentState could return a JSON object directly, or through a callback.

[parker-mar]

"I think that should be a property set on the input when calling trackChange. When you set the track, that would be when you know if you are able to handle a Next or Previous request. It could be that you've reached the end of the list you were playing from, or you are starting a new list, or perhaps you are playing a single track with no other tracks to switch to. It seems less likely that the value would change after starting to play a track."

• I just want to clarify what you mean by trackChange API. Stuff that have to do with changing tracks (next/previous actions) are just part of the overall NowPlaying API, right? "trackChange" is just a name associated with those set of actions?
• So the proposal is for next/previous API to have optional parameters to enable/disable next/previous buttons on the overlay? Just to clarify, couldn't the user could just call these right after calling the next/previous API? So this is just for convenience?

"I think stop, resume, and pause would likely be simple methods that take no parameters and return nothing. The output of anything that happens with them would likely fall into one of the callbacks defined earlier."

• I thought that they needed callbacks to update the UI?