-
Notifications
You must be signed in to change notification settings - Fork 40
[Weave] New Weave projects page #1913
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
e6d9ba7
df4cd29
96995f0
c49ba6e
26cd864
8eb408c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Super minor, but the browser could be a bit narrower for this screenshot to improve the information density. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| --- | ||
| title: "Weave projects overview" | ||
| description: "Weave projects are collections of traces, evaluations, models, and other Weave assets" | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How about something like "Use Weave projects to organize related assets like traces, prompts, evaluations, models, and dashboards"? FYI the description also shows up at the top of the page content, so you may not need to duplicate it all in the first paragraph, depending on what you think about the preview. |
||
| --- | ||
|
|
||
| Weave projects are organizational containers that group related traces, evaluations, models, datasets, and other Weave assets together. Projects help you organize your work, collaborate with team members, and track the evolution of your LLM applications over time. | ||
|
|
||
| All Weave data is organized within projects, making it easy to find related work and maintain context across your development workflow. | ||
|
|
||
| Projects store the following information and resources: | ||
|
|
||
| - **[Traces](/weave/guides/tracking/tracing)**: Individual LLM calls, function executions, and application flows that capture inputs, outputs, latency, and token usage for debugging and analysis. | ||
| - **[Evaluations](/weave/guides/core-types/evaluations)**: Test results that compare model performance against datasets using scoring functions to measure quality, accuracy, and other metrics. | ||
| - **[Monitors](/weave/guides/evaluation/guardrails_and_monitors)**: Background processes that automatically score a subset of function calls using LLM-as-a-judge scorers to track quality metrics over time in production. | ||
| - **[Leaderboards](/weave/guides/core-types/leaderboards)**: Ranked comparisons of model performance across different evaluation runs, helping you identify the best-performing configurations. | ||
| - **[Threads](/weave/guides/tracking/threads)**: Multi-turn conversations and session-based workflows that group related calls under a shared thread ID for analyzing complete user interactions. | ||
| - **Assets**: The Assets tab in the Weave UI stores and organizes the following resources in your project: | ||
| - **[Models](/weave/guides/core-types/models)**: Versioned model definitions that capture prompts, parameters, and configuration, automatically tracking changes for reproducibility and comparison. | ||
| - **[Datasets](/weave/guides/core-types/datasets)**: Collections of test examples stored as Weave objects that can be downloaded, browsed, and used for running evaluations. | ||
| - **[Prompts](/weave/guides/core-types/prompts)**: Versioned system prompts that you can test and reuse in your model configurations. | ||
| - **[Scorers](/weave/guides/evaluation/scorers)**: Functions that evaluate model inputs and outputs to measure quality, safety, or custom metrics. | ||
| - **[Ops](/weave/guides/tracking/ops)**: Automatically versioned and tracked functions decorated with `@weave.op()` that produce calls and maintain a record of function modifications. | ||
| - **Other objects and metadata**: Additional Weave objects, custom types, and associated metadata stored within the project. | ||
|
|
||
| ## Creating and using projects | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Aha, so there are tasks here after all, so maybe "Projects" is the right title for the page. You could call this section "Work with projects", then you could have Create, View, Manage, Delete. Or you could have a section Projects with "Overview" and "Work with projects" or something like that. I'd prefer the non-gerund forms. |
||
|
|
||
| A Weave project is owned and managed by a [team](/platform/hosting/iam/org_team_struct#team) (also called an entity) in your W&B organization. The full project path follows this format: | ||
|
|
||
| ``` | ||
| team-name/project-name | ||
| ``` | ||
|
|
||
| For example, a team called `my-team` with a project called `llm-chatbot` has the project path `my-team/llm-chatbot`. | ||
|
|
||
| To associate traces, evaluations, and other resources with a project, pass the project's path as a string when you initialize Weave in your code: | ||
|
|
||
| <Tabs> | ||
| <Tab title="Python"> | ||
| ```python | ||
| import weave | ||
|
|
||
| weave.init('my-team/my-llm-app') | ||
| ``` | ||
| </Tab> | ||
| <Tab title="TypeScript"> | ||
| ```typescript | ||
| import * as weave from 'weave'; | ||
|
|
||
| await weave.init('my-team/my-llm-app'); | ||
| ``` | ||
| </Tab> | ||
| </Tabs> | ||
|
|
||
| If you don't specify a team name, Weave uses your [default entity](/platform/app/settings-page/user-settings#default-team) (your personal account or default team). | ||
|
|
||
| ## Viewing projects in the UI | ||
|
|
||
| Access a project in the Weave UI: | ||
|
|
||
| 1. Navigate to [wandb.ai](https://wandb.ai). | ||
| 2. Select your project from the **Projects** sidebar. This opens the project's overview page. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You could send them straight here in step 1. |
||
| 3. From the overview page, you can: | ||
| * Review your projects's statistics, such as the total number of traces logged to the project and how much storage the project is using. | ||
| * Navigate to traces, evaluations, and other resources using the sidebar. | ||
| * Navigate to recently logged traces, evaluations, models, and datasets. | ||
|
|
||
| <Frame> | ||
|  | ||
| </Frame> | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This new page seems either oddly placed and/or oddly titled. It's too bad there are not more pages about projects, but how about just "Weave projects" fore a title?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I completely agree. I really couldn't find a better place to shoe-horn this in using the current IA. The IA desprerately needs work for stuff like this. I'm kinda stuck between a rock and hard place, because I have tickets that require this coverage, but the IA needs a huge amount of rework that requires not just reorganizing pages, but breaking them out in places and consolidating in others. It's a mess.
I'll think of something to do here.