Skip to content

Use fsspec in everything #660

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 86 commits into
base: main
Choose a base branch
from
Open

Use fsspec in everything #660

wants to merge 86 commits into from

Conversation

@deep1401
Copy link
Member

@deep1401 deep1401 commented Nov 3, 2025 •
edited
Loading

deep1401 and others added 25 commits November 3, 2025 08:51
@deep1401
changes for using fsspec in non-sdk experiment functions
1ddf94f
@deep1401
bump versions
3123544
@deep1401
fix dataset bug for locally uploaded files and convert dataset to use…
ea34b24 
… fsspec
@deep1401
change everything in shared.py
2089faa
@deep1401
ruff
6495513
@deep1401
Merge branch 'main' into add/use-fsspec
970d1e5
@deep1401
fsspec for export
b767e25
@deep1401
Merge branch 'add/use-fsspec' of https://github.com/transformerlab/tr…
c010015 
…ansformerlab-api into add/use-fsspec
@deep1401
changes for diffusion
a2b227f
@deep1401
convert for train, tasks and serverinfo routers
ac76bcb
@deep1401
convert for tasks library
2567e1b
@deep1401
remove blockages limiting to a local filesystem and use posixpath
b72177b
@deep1401
More changes in shared and for the model router
8625df7
@deep1401
fix bugs
9848ac1
@deep1401
fix
ec87cb8
@deep1401
major changes for rag.py
94ab13a
@deep1401
add changes for conversations
0dfdab3
@deep1401
add changes for documents
da94868
@deep1401
add changes for experiment router
7e4de16
@deep1401
add changes for evals router
cf2a6c5
@deep1401
add changes for export router
2471579
@deep1401
add changes for generations router
0b577bf
@deep1401
add changes for jobs router and fix bugs
f3db4be
@deep1401
remove imports
b158367
@deep1401
make changes for local models
b2fc928
github-advanced-security[bot]
github-advanced-security bot found potential problems Nov 3, 2025
View reviewed changes
@deep1401 deep1401 marked this pull request as ready for review November 4, 2025 20:16
This was referenced Nov 4, 2025
Open
Open
dadmobile
dadmobile requested changes Nov 12, 2025
View reviewed changes
Copy link
Member

@dadmobile dadmobile left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sent to Deep but just recording so we can keep track:

I get AttributeError: 'float' object has no attribute 'timestamp' from line 104 in routers/experiment/documents.py line 104. Looks like timestamp might be getting returned as a float now instead of object or something?

@deep1401
Copy link
Member Author

deep1401 commented Nov 12, 2025

Sent to Deep but just recording so we can keep track:

I get AttributeError: 'float' object has no attribute 'timestamp' from line 104 in routers/experiment/documents.py line 104. Looks like timestamp might be getting returned as a float now instead of object or something?

should be fixed now

@dadmobile
Copy link
Member

dadmobile commented Nov 12, 2025

This all works for me. Will wait for us to release SDK tomorrow and verify all tests etc. look good and then we'll merge!

@aliasaria
Copy link
Member

aliasaria commented Nov 13, 2025

Waiting for for build on Nov 13 before merging

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dadmobile dadmobile dadmobile requested changes

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@deep1401 @dadmobile @aliasaria