Mike Boer's notes

[WyattBlue]

Welcome to the PyAV wiki!

This is mostly an area for the developers to work on the TODO lists, and design ideas.

• flags docs

• autoremaining directive to autodoc all non-documented things

• options

• class descriptors

• rename containers to formats?

• Defer the avformat_find_stream_info, and get basic stream info before setting options. An alternative is to provide a av.probe function or something.

• Container.decode_one(**demux_kwargs) -> Frame. There would be a Container._buffered_decoded_frames or something that would contain extras, just in case. Stream and CodecContext have the same.

• Frame.save(file_or_path)

• Frame.load(file_or_path) (classmethod); opens the file and decodes the first frame. Perhaps av.load_frame or something.

• isdeferredinstance loads the module via sys.modules, then getattr on it, and
  finally do an isinstance. isdeferredinstance(x, ('PIL.Image', 'Image'))

• Rename Buffer and ByteSource to represent they are opposite directions of
  the same operation.

• atexit.register something to clean up FFmpeg threads

• Directly use the Numpy C-API

  • see: http://docs.scipy.org/doc/numpy/reference/c-api.array.html
  • see: https://github.com/cython/cython/wiki/tutorials-numpy#c-api-initalization

• Move the logic regard initializing a VideoReformatter into the object
  itself instead of the VideoFrame.reformat method.

• Protocol for streams to pull packets from iterators. Then secondary streams
  get easier: container.add_stream('audio_codec', source=av.open('music.aiff').iter_packets(only_primary=True))

• Make av.utils.SmartPointer to wrap around a pointer? All this would do is
  hold a reference to said pointer and make sure it isn't garbage collected.

• Look at how stream alloc works; can we manually do this and register the
  stream on a format context later?

• Various AudioLayout/AudioFormat/VideoFormat attributes should be writable.

  • is_mutable flags on various objects (including formats, layouts, contexts,
    streams, etc.) could guard the set methods of properties.

• TestCase.rms_diff(one, two) -> Root-mean-square diff

• try to wrap API of testsrc filters

• Replicate av_frame_get_best_effort_timestamp

  • http://ffmpeg.org/pipermail/ffmpeg-devel/2011-February/104327.html
  • http://pastebin.com/Aq8eDZw3/
  • http://web.archiveorange.com/archive/v/yR2T4bybpYnYCUXmzAI5

TODO Filters

• Expose options on filter classes.

• Graph.add_buffer(template_frame)

• Graph.add_abuffer(template_frame)

• Graph.add_sink(template_frame)

• Graph.add_asink(template_frame)

• Graph.parse()

  • create FilterContext objects for every newly created filter
  • return inputs/outputs

• FilterContext.push(frame, input=None)

• FilterContext.pull(frame, output=None)

• Graph.push(frame) -> push the frame into the single input, if it exists.

• Graph.pull(frame) -> pull the frame from the single output, if it exists.

• Graph.process(frame) -> push, followed by pull

• FilterContext.send_command(cmd, arg)

• FilterContext.{inputs, outputs} -> tuple of FilterPad

TODO General

• Lazy caching tuple properties:

  • Codec.frame_rates
  • Codec.audio_rates
  • Codec.video_formats
  • Codec.audio_formats
  • ...

• Case insensitive enum names.

  • Reason 1: I could turn AVMediaType into an enum, and retain the old stream.type == 'video'. This could also be done by having that enum have multiple keys for each value.
  • Reason 2: I don't like the feeling of frame.side_data.get('MOTION_VECTORS'). Perhaps frame.side_data.motion_vectors is better, or just frame.motion_vectors. This speaks to how flags and such are exposed as well.

• Auto flushing/muxing of streams.
  Perhaps stream.encode_and_mux(frame) is a new API that would take responsibility. We would have to assert that you don't blend the two of them.

• Planes increase the buffer refcounts instead of holding onto frame reference (via av_frame_get_plane_buffer).

• build caching?

  • ides 0: https://github.com/actions/cache

  • idea 1: rsync to my server

  • idea 2: zip to my server via SSH

    • How to stuff a SSH key into a secret: https://www.webfactory.de/blog/use-ssh-key-for-private-repositories-in-github-actions

    • secrets: https://help.github.com/en/articles/virtual-environments-for-github-actions#creating-and-using-secrets-encrypted-variables

    • make pyav user on one of my servers that can only do this stuff; can it be chrooted?

    • try to download cache for the branch, and then develop

    • ssh pyav.mikeboers.com 'cat > cache/$BRANCH/docs.zip' < docs.zip

    • wget cache.pyav.mikeboers.com/$BRANCH.docs.zip

  • should I have one which does the sdist, and then the rest compile that?
    doesn't really seem like it would save much time TBH.

  cached-build (--get|--put) [--etag FOO] name directory