Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Some questions and known missing pieces or mistakes in these docs:
|
tylere
left a comment
There was a problem hiding this comment.
The product Memberships page does not appear to be described in the docs.
Ex: https://source.coop/edit/product/account/product-id/memberships
|
|
||
| **Cost-Effective at Scale**: Source hosts over 1 petabyte of data across 300+ data products. Whether you're publishing a few gigabytes or hundreds of terabytes, Source provides cost-effective hosting without requiring you to manage cloud infrastructure. | ||
|
|
||
| **Cloud-Native Access**: Data on Source is stored in S3-compatible object storage, enabling efficient programmatic access through standard tools like the AWS CLI, Python's boto3, and various other programming libraries. Access data via the web interface or bring your compute directly to the data in the cloud. |
There was a problem hiding this comment.
Docs should provide examples of these (with links), to clearly demonstrate how efficient it is.
- The AWS CLI is shown on the data-proxy page
- boto3 is not demonstrated
- "bring your compute directly to the data in the cloud" is not demonstrated
|
|
||
| Source is currently in beta. While all data hosted in Source is available to the public, publishing data requires applying to be a beta tester. To apply, visit [the beta tester application form](https://forms.gle/4weS1hkRjZhQLoPE9). | ||
|
|
||
| Source currently: |
There was a problem hiding this comment.
How recent are these stats? How easy is it to recalculate?
| - Logs an average of 126 million data requests per month | ||
| - Supports over 300 data products from 66+ organizations | ||
|
|
||
| Source is funded by Taylor Geospatial, with in-kind support from AWS and Azure for data hosting. No newline at end of file |
There was a problem hiding this comment.
This info is different than what is in index.md. Suggest consolidating into one place.
| sidebar_position: 2 | ||
| --- | ||
|
|
||
| Source is a data publishing utility designed to make data shared in object stores easier to find, explore, and share on the web. Understanding these fundamental concepts will help you navigate and use Source Cooperative effectively. |
There was a problem hiding this comment.
Start with full name and abbreviation...
| Source is a data publishing utility designed to make data shared in object stores easier to find, explore, and share on the web. Understanding these fundamental concepts will help you navigate and use Source Cooperative effectively. | |
| Source Cooperative ("Source") is a data publishing utility designed to make data shared in object stores easier to find, explore, and share on the web. Understanding these fundamental concepts will help you navigate and use Source effectively. |
There was a problem hiding this comment.
| Source is a data publishing utility designed to make data shared in object stores easier to find, explore, and share on the web. Understanding these fundamental concepts will help you navigate and use Source Cooperative effectively. | |
| For how to create and manage data products in the web interface, see [Create a Data Product](/create-a-data-product). |
|
|
||
| Data products are the primary organizational unit in Source. They serve as containers for related data files and provide a way to group and organize information logically. | ||
|
|
||
| A data product is a collection of related data files with associated metadata and documentation. Each data product consists of: |
There was a problem hiding this comment.
Suggest using consistent naming for fields in the docs vs. on the website. Examples of current differences below...
| sidebar_position: 2 | ||
| --- | ||
|
|
||
| To create a data product, you need an account and [beta access](/create-an-account). After approval, sign out and sign back in—the option to create a new data product will then appear in the dropdown at the top right of the navigation bar. |
There was a problem hiding this comment.
| To create a data product, you need an account and [beta access](/create-an-account). After approval, sign out and sign back in—the option to create a new data product will then appear in the dropdown at the top right of the navigation bar. | |
| Before creating a data product, you need to [create an account](/create-an-account) and apply for beta access. After approval, sign out and sign back in—the option to create a new data product will then appear in the dropdown at the top right of the navigation bar. |
|
|
||
| Once you have an account and beta access: | ||
|
|
||
| - **Publish data products**: Create data products to host your datasets |
There was a problem hiding this comment.
| - **Publish data products**: Create data products to host your datasets | |
| - **Publish data products**: [Create data products](/create-a-data-product) to host your datasets |
|
|
||
| Data products can be owned by an organization or an individual. You will see a dropdown option when creating the data product of who will be displayed as the data product host (you or one of your organizations). | ||
|
|
||
| ## When creating a data product |
There was a problem hiding this comment.
Similar to my comment on core-concepts.md, it woudld be beneficial to use consistent naming for data product fields in the docs vs. on the website
| - **Title**: Maximum 200 characters. | ||
| - **Description**: Optional; maximum 500 characters. Use it for a short overview; put detailed documentation in the README. | ||
| - **Tags**: Comma-separated, up to 20 tags. They help others discover your data. | ||
| - **Visibility**: New data products are created **Unlisted** (not shown in search). When ready to publish, open the data product page, click **Edit** in the sidebar, and set the state to **Listed**. |
There was a problem hiding this comment.
The Edit button now longer is part of the UI.
| - **Visibility**: New data products are created **Unlisted** (not shown in search). When ready to publish, open the data product page, click **Edit** in the sidebar, and set the state to **Listed**. | |
| - **Visibility**: New data products are created **Unlisted** (not shown in search). When ready to publish, open the data product page, click the gear icon, and set the state to **Listed**. |
|
|
||
| ## Editing a data product | ||
|
|
||
| To change the title, description, tags, or visibility later, open your data product page and click **Edit** in the sidebar. |
There was a problem hiding this comment.
| To change the title, description, tags, or visibility later, open your data product page and click **Edit** in the sidebar. | |
| To change the title, description, tags, or visibility later, open your data product page and click the gear icon. |
| This guide explains how to deliver your data to Source Cooperative in a secure and simple way. | ||
| It is written for data providers and does not require deep Amazon Web Service (AWS) knowledge. | ||
|
|
||
| If you do not see the option to upload (for example, Edit Mode or View Credentials on your product page), contact [hello@source.coop](mailto:hello@source.coop) to request upload access. |
There was a problem hiding this comment.
| If you do not see the option to upload (for example, Edit Mode or View Credentials on your product page), contact [hello@source.coop](mailto:hello@source.coop) to request upload access. | |
| If you do not see the option to upload (for example, Edit Mode or View Credentials on your product page under the lock icon), contact [hello@source.coop](mailto:hello@source.coop) to request upload access. |
There was a problem hiding this comment.
Actually, don't they need to first complete the beta tester application before contacting hello@ ?
There was a problem hiding this comment.
Although not changed in this PR, I noticed a few issues in later lines...
- "linke" -> "line"
- "selectory" -> "selector"
- "ilke" -> "like"
There was a problem hiding this comment.
Option 2 and Option 3 instructions may need to be updated to reflect an OpenID Connect (OIDC) authentication flow.
|
|
||
| # Accessing Data Through the Source Data Proxy | ||
|
|
||
| The Source Data Proxy provides S3-compatible access to all data hosted on Source Cooperative. You can access data through the proxy without authentication, making it easy to programmatically download datasets using standard AWS CLI commands. |
There was a problem hiding this comment.
This page may need to be updated to reflect alternative workflows that use OpenID Connect (OIDC).
I broke the documentation up into 3 sections:
I added a few files and got rid of the web-ui.md doc (information all in new files). Flow should be more intuitive for both users and those curious about Source. Surely some docs are missing, but hopefully heading in the right direction.
To add: visuals and graphics where helpful.
Please let me know your thoughts!