can drs_cp be made to work with controlled-access buckets? #71
Description
I'm trying to use drs_cp on a file in a controlled-access bucket with requester pays enabled. drs_stat works, but drs_cp does not:
> f <- "drs://dg.4503:dg.4503/288ff0aa-a426-11ea-82d1-8bda0857af94"
> drs_stat(f)
# A tibble: 1 × 9
drs fileN…¹ size gsUri acces…² timeU…³ hashes bucket name
<chr> <chr> <int> <chr> <chr> <chr> <list> <chr> <chr>
1 drs://dg.4503:dg.4503/288ff0aa-a426-11ea-82d1-8bda085… phs000… 58278 gs:/… https:… 2020-0… <named list> nih-n… phs0…
# … with abbreviated variable names ¹fileName, ²accessUrl, ³timeUpdated
> drs_cp(f, ".")
Error: 'gsutil -m cp -n 'gs://nih-nhlbi-topmed-released-phs000964-c3/phs000964.v3.pht004839.v2.p1.TOPMed_WGS_JHS_Sample.MULTI.txt.gz' '/home/rstudio'' failed:
AccessDeniedException: 403 pet-107443797655395020525@terra-0c3bdde8.iam.gserviceaccount.com does not have storage.objects.list access to the Google Cloud Storage bucket. Permission 'storage.objects.list' denied on resource (or it may not exist).
CommandException: 1 file/object could not be transferred.
exit status: 1
In addition: Warning message:
'gsutil_requesterpays()' returned an error:
'gsutil -u terra-0c3bdde8 requesterpays get gs://nih-nhlbi-topmed-released-phs000964-c3' failed:
AccessDeniedException: 403 pet-107443797655395020525@terra-0c3bdde8.iam.gserviceaccount.com does not have storage.buckets.get access to the Google Cloud Storage bucket. Permission 'storage.buckets.get' denied on resource (or it may not exist).
exit status: 1
Digging into this further, I get an identical error when using gsutil -u <project_id> cp in the terminal. However, I can access the same file by using terra-notebook-utils in the terminal:
rstudio@f29b33636061:~$ /home/rstudio/.local/bin/tnu drs copy drs://dg.4503:dg.4503/288ff0aa-a426-11ea-82d1-8bda0857af94 .
2023-01-20 11:46:40::INFO Enabling requester pays for your workspace. This will only take a few seconds...
/home/rstudio/phs000964.v3.pht004839.v2. 100% [========================================] 56.9KiB 374.9KiB/s 0.15s
Can drs_cp use the same mechanism as terra-notebook-utils?