-
Notifications
You must be signed in to change notification settings - Fork 19
PoC: Plugins in separate environments #277
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
leszko
wants to merge
31
commits into
main
Choose a base branch
from
rafal/subprocess-3
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Pipeline proxy was added in PR #173 but is not actually used. The pipeline is now accessed directly in the worker process through WorkerFrameProcessor, so the proxy is unnecessary.
Signed-off-by: Yondon Fu <yondon.fu@gmail.com>
Signed-off-by: Yondon Fu <yondon.fu@gmail.com>
Signed-off-by: Yondon Fu <yondon.fu@gmail.com>
Signed-off-by: Yondon Fu <yondon.fu@gmail.com>
Signed-off-by: Yondon Fu <yondon.fu@gmail.com>
Signed-off-by: Yondon Fu <yondon.fu@gmail.com>
Signed-off-by: Yondon Fu <yondon.fu@gmail.com>
…for prompts box Signed-off-by: Yondon Fu <yondon.fu@gmail.com>
Signed-off-by: Rafał Leszko <rafal@livepeer.org> Signed-off-by: Yondon Fu <yondon.fu@gmail.com>
Signed-off-by: Yondon Fu <yondon.fu@gmail.com>
Signed-off-by: Yondon Fu <yondon.fu@gmail.com>
…ironments This merge combines: - rafal/subprocess-3: Running each pipeline in a separate process with its own environment - yf/plugins: Plugin support with pluggy-based plugin system Key changes for plugin isolation: - Plugins are installed to ~/.daydream-scope/plugins/<plugin_id>/ - Each plugin has its own pyproject.toml with dependencies - Worker process is started from the plugin's directory with uv run - Modified install/uninstall CLI commands to copy plugin code to plugins directory - Worker can dynamically load plugin pipelines from the current working directory This allows plugins to have their own isolated dependencies that don't conflict with the main scope environment or other plugins.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Based on #241 and #173
This PR extends the plugin support introduced in #241 by running each plugin:
This allows plugins to have conflicting dependencies without interfering with each other.
demo_plusing_separate_environment.mp4
Why this PR?
This PR is primarily a proof of concept to demonstrate that:
daydream-scopedependency needs to be changed, see:How does it work?
uv daydream-scope install <url>, the plugin code is copied to~/.daydream-scope/plugins/<plugin-name>pyproject.tomlthat depends on the base plugin dependencies defined inworker_env/pyproject.toml.uv run.Side notes
pyproject.tomlfor each core pipeline, though we may not want to do ithttps://github.com/daydreamlive/scope/blob/rafal/subprocess-3/src/scope/server/frame_processor_proxy.py