Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 9, 2026

pelican object put now behaves like cp/scp when the destination is an existing directory - it infers the filename from the source. This feature also handles multiple file uploads intelligently.

Previously:

pelican object put ./file.txt osdf:///namespace
# Error: remote object already exists, upload aborted

Now:

pelican object put ./file.txt osdf:///namespace
# Uploads to osdf:///namespace/file.txt

pelican object put ./file1.txt ./file2.txt osdf:///namespace
# Uploads to osdf:///namespace/file1.txt and osdf:///namespace/file2.txt

Changes

  • Optimized directory check in cmd/object_put.go: For non-recursive uploads, uses client.DoStat once before the upload loop to check if the remote destination is a directory. Within the loop, only filename transformations are performed, avoiding redundant stat calls.

  • Multi-file upload support: When uploading multiple files to a destination that doesn't exist (stat fails), the code assumes the destination should be treated as a directory and infers filenames for each source file.

  • Implementation approach: Uses client.DoStat to check if destination is a collection, then modifies the destination URL for each file before the actual transfer begins. Logic is implemented at the CLI level rather than in the library.

  • Error handling: If URL parsing fails during filename inference, the operation terminates with a clear error message rather than silently continuing.

  • Integration test added: TestObjectPutToDirectoryInfersFilename in cmd/object_put_test.go verifies that uploading to a directory correctly infers the filename from the source.

This matches the existing DoGet behavior which already infers local filenames when downloading to a directory.

Testing

  • ✅ Integration test added to verify directory inference behavior
  • ✅ Code review and security scan completed (no vulnerabilities found)
  • ✅ Build verification completed
  • ⏳ Full federation testing requires xrootd environment (will be tested in CI)
Original prompt

This section details on the original issue you should resolve

<issue_title>Infer destination object name for pelican object put to "directory" location</issue_title>
<issue_description>Pelican Service:

  • Client
  • Plugin
  • Registry
  • Director
  • Origin
  • Cache
  • Other (please give the detail)

Is your feature request related to a problem? Please describe.
When uploading (put-ing) to an existing namespace, users have built up the intuition that the object name at the destination is inferred by the object name at the source. That is, users expect

pelican object put ./aowen-uwmad_test.txt osdf:///<existing_namespace>

to have the same outcome as

pelican object put ./aowen-uwmad_test.txt osdf:///<existing_namespace>/aowen-uwmad_test.txt

This arises from such behavior of common unix commands such as mv, cp, and scp when an existing directory is provided as the destination.

Instead, the user receives an error like this:

ERROR[2026-01-05T12:31:53-06:00] Error when attempting to transfer object osdf:///<existing_namespace> for client 019b8f6e-0f40-711d-a992-3ab2693ab191: remote object already exists, upload aborted

The intuition built up by the user further confuses the point - they interpret the error to mean that the object they are trying to upload (aowen-uwmad_test.txt) already exists in the namespace (osdf:///<existing_namespace>/aowen-uwmad_test.txt), but any subsequent queries (pelican object ls, pelican object get) against that supposed object will fail.

Describe the solution you'd like
The command

pelican object put ./aowen-uwmad_test.txt osdf:///<existing_namespace>

has the same outcome as

pelican object put ./aowen-uwmad_test.txt osdf:///<existing_namespace>/aowen-uwmad_test.txt

Additional context

  1. For downloads, the client already infers the object name if not provided at the destination. That is,

    pelican object get osdf:///<existing_namespace>/aowen-uwmad_test.txt ./
    

    behaves the same as

    pelican object get osdf:///<existing_namespace>/aowen-uwmad_test.txt ./aowen-uwmad_test.txt
    

    as expected.

  2. If the current behavior is a policy decision (e.g., "users should not upload objects without being explicit about the name"), I'll counter that implicit naming is already used for --recursive or sync uploads anyways, so implementing this feature will not break a fundamental premise.

  3. If there is no desire to implement this feature, then I recommend that, at the very least, the client handles the existing error and suggests to the user something like Did you mean to run "pelican object put aowen-uwmad_test.txt osdf:///<existing_namespace>/aowen-uwmad_test.txt"?.</issue_description>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI changed the title [WIP] Update pelican object put to infer destination object name Infer destination filename when uploading to a directory Jan 9, 2026
Copilot AI requested a review from bbockelm January 9, 2026 17:49
@aowen-uwmad
Copy link
Contributor

@copilot -- can you explain why the changes to web_ui/frontend/package-lock.json are necessary for this pull request?

Copy link
Contributor Author

Copilot AI commented Jan 9, 2026

@copilot -- can you explain why the changes to web_ui/frontend/package-lock.json are necessary for this pull request?

The package-lock.json changes were unintentional - they occurred when I ran make web-build to compile the Go binary (which requires the web UI to be built). These changes are not necessary for this PR and have been reverted in commit 6872052.

Copy link
Collaborator

@bbockelm bbockelm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Drop the package-lock.json changes and don't bring them back in future commits-
  • I don't like changing the semantics of DoPut at the library level - it feels like this is a CLI nicety. Can you use DoStat in the CLI instead and adjust the URLs to transfer there? You'd need to change cmd/object_put.go and cmd/object_sync.go. Do this in a helper function to simplify the addition of an integration test. Add an integration test.

Copy link
Contributor Author

Copilot AI commented Jan 9, 2026

  • Drop the package-lock.json changes and don't bring them back in future commits-
  • I don't like changing the semantics of DoPut at the library level - it feels like this is a CLI nicety. Can you...

I've addressed all requested changes:

  1. ✅ Package-lock.json changes were already reverted in commit 6872052
  2. ✅ Moved logic from library level (DoPut) to CLI level - commit 7ec8941
    • Reverted changes from client/main.go
    • Created inferDestinationPath helper function in cmd/cmd_utils.go
    • Updated cmd/object_put.go to use client.DoStat for directory checking
  3. ✅ Added integration test - commit 2cd3e80
    • Created TestObjectPutToDirectoryInfersFilename in cmd/object_put_test.go
    • Test verifies filename inference when uploading to a directory

Note: object_sync.go doesn't need changes as it always uses recursive mode (DoPut(..., true)), so directory inference doesn't apply there.

Copilot AI changed the title Infer destination filename when uploading to a directory Infer destination filename when uploading to a directory (CLI level) Jan 9, 2026
Copilot AI requested a review from bbockelm January 9, 2026 20:24
@turetske turetske added this to the v7.24 milestone Jan 13, 2026
Copilot AI requested a review from bbockelm January 19, 2026 20:21
@bbockelm bbockelm force-pushed the copilot/infer-destination-object-name branch from 03bb1dd to da08a35 Compare January 20, 2026 01:08
@bbockelm bbockelm added enhancement New feature or request client Issue affecting the OSDF client labels Jan 20, 2026
@bbockelm
Copy link
Collaborator

@brianaydemir, @aowen-uwmad - can I get you to do a quick test of this?

Particularly, I'm not sure how the behavior should interact with the --recursive flag (current version -- no changes when flag is present; just creates a directory instead of a file).

In this case:

pelican object put foo osdf://bar/baz

it creates osdf://bar/baz if osdf://bar/baz didn't previously exist and osdf://bar/baz/foo if it exists and is a directory.

In this case:

pelican object put foo bar osdf://qux/test

then it creates osdf://qux/test/foo and osdf://qux/test/bar in both cases.

@bbockelm bbockelm force-pushed the copilot/infer-destination-object-name branch from da08a35 to 5802a9f Compare January 20, 2026 01:40
@aowen-uwmad
Copy link
Contributor

@bbockelm, I'm not sure what you'd like me to "test". So I recreated the same movements using cp.

Case 1

$ tree
.
├── bar
└── foo
    ├── foo1
    └── foo2

2 directories, 2 files
$ cp ./foo ./bar/baz
cp: -r not specified; omitting directory './foo'
$ cp -r ./foo ./bar/baz
$ tree
.
├── bar
│   └── baz
│       ├── foo1
│       └── foo2
└── foo
    ├── foo1
    └── foo2

3 directories, 4 files

Case 2

$ tree
.
├── bar
│   ├── bar1
│   └── bar2
├── foo
│   ├── foo1
│   └── foo2
└── qux

3 directories, 4 files
$ cp -r ./bar ./foo ./qux/test
cp: target './qux/test' is not a directory
$ mkdir ./qux/test
$ cp -r ./bar ./foo ./qux/test
$ tree
.
├── bar
│   ├── bar1
│   └── bar2
├── foo
│   ├── foo1
│   └── foo2
└── qux
    └── test
        ├── bar
        │   ├── bar1
        │   └── bar2
        └── foo
            ├── foo1
            └── foo2

6 directories, 8 files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

client Issue affecting the OSDF client enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Infer destination object name for pelican object put to "directory" location

4 participants