Feature Request: API to support Download Terms of Use and Guestbook

doc/release-notes/12001-api-support-termofuse-guestbook.md

@@ -0,0 +1,29 @@
+## Feature Request: API to support Download Terms of Use and Guestbook
+
+## New Endpoints to download a file or files that required a Guestbook Response: POST
+A post to these endpoints with the body containing a JSON Guestbook Response will save the response and 
+`?signed=true`:  return a signed URL to download the file(s) or
+`?signed=false` or missing: Write the Guestbook Responses and download the file(s)
+
+`/api/access/datafile/{fileId:.+}`
+`/api/access/datafiles/{fileIds}`
+`/api/access/dataset/{id}`
+`/api/access/dataset/{id}/versions/{versionId}`
+
+The matching GET APIs will also take the `?signed=true` parameter to also return the signed url instead of downloading immediately. Note: Signed urls are only for Authenticated Users. Guest users will receive an error if requesting with signed=true
+
+A post to these endpoints with the body containing a JSON Guestbook Response will save the response before continuing the download.
+No signed URL option exists.
+`/api/access/datafiles`
+`/api/access/datafile/bundle/{fileId}` POST returns BundleDownloadInstance after processing Guestbook Responses from body.
+
+## New CRUD Endpoints for Guestbook:
+Create a Guestbook: POST `/api/guestbooks/{dataverseIdentifier}`
+Get a Guestbook: GET `/api/guestbooks/{id}`
+Get a list of Guestbooks linked to a Dataverse Collection: GET `/api/guestbooks/{dataverseIdentifier}/list`
+Enable/Disable a Guestbook: PUT `/api/guestbooks/{dataverseIdentifier}/{id}/enabled` Body: `true` or `false`
+Note: There is no Update or Delete at this time. You can disable a Guestbook and create a new one.
+
+## For Guestbook At Request:
+When JVM setting -Ddataverse.files.guestbook-at-request=true is used a request for access may require a Guestbook Response.
+PUT `/api/access/datafile/{id}/requestAccess` will now take a JSON Guestbook Response in the body.

doc/sphinx-guides/source/api/dataaccess.rst

@@ -32,6 +32,8 @@ Basic Download By Dataset
 
 The basic form downloads files from the latest accessible version of the dataset. If you are not using an API token, this means the most recently published version. If you are using an API token with full access to the dataset, this means the draft version or the most recently published version if no draft exists.
 
+.. note:: Restricted files that require a Guestbook Response will require an additional step to supply the Guestbook Response. A POST to the same endpoint with the Guestbook Response in the body can return a signed url (with query parameter ``&signed=true``) that can be used to download the file(s) via a browser or download manager. Without the ``signed`` parameter the download will start immediately. For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide.
+
 A curl example using a DOI (no version):
 
 .. code-block:: bash
@@ -59,6 +61,8 @@ The second form of the "download by dataset" API allows you to specify which ver
 * ``x.y`` a specific version, where ``x`` is the major version number and ``y`` is the minor version number.
 * ``x`` same as ``x.0``
 
+.. note:: Restricted files that require a Guestbook Response will require an additional step to supply the Guestbook Response. A POST to the same endpoint with the Guestbook Response in the body can return a signed url (with query parameter ``&signed=true``) that can be used to download the file(s) via a browser or download manager. Without the ``signed`` parameter the download will start immediately. For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide.
+
 A curl example using a DOI (with version):
 
 .. code-block:: bash
@@ -91,6 +95,11 @@ Basic access URI:
 
     GET http://$SERVER/api/access/datafile/:persistentId?persistentId=doi:10.5072/FK2/J8SJZB
 
+.. note:: Restricted files that require a Guestbook Response will require an additional step to supply the Guestbook Response. A POST to the same endpoint with the Guestbook Response in the body can return a signed url (with query parameter ``&signed=true``) that can be used to download the file(s) via a browser or download manager. Without the ``signed`` parameter the download will start immediately. For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide. In the following JSON example please note that the `name`, `email`, `institution`, and `position` fields will default to the User's account information if not included in the response.
+
+  Example ::
+
+    POST http://$SERVER/api/access/datafile/:persistentId?persistentId=doi:10.5072/FK2/J8SJZB&signed=true -d '{"guestbookResponse": {"name": "My Name", "email": "myemail@example.com", "institution": "Harvard","position": "Staff", "answers": [{"id": 123,"value": "Good"},{"id": 124,"value": ["Multi","Line"]},{"id": 125,"value": "Yellow"}]}}'
 
 Parameters:
 ~~~~~~~~~~~
@@ -198,6 +207,8 @@ Returns the files listed, zipped. As of v6.7 the name of the zipped bundle will
 
 .. note:: If any of the datafiles have the ``DirectoryLabel`` attributes in the corresponding ``FileMetadata`` entries, these will be added as folders to the Zip archive, and the files will be placed in them accordingly. 
 
+.. note:: If Guestbook Responses are required they can be included in the body along with the file ids as JSON: ``{"fileIds" :[1,2,3], {"guestbookResponse": {"answers": []}}}``. For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide.
+
 Parameters: 
 ~~~~~~~~~~~
 
@@ -378,7 +389,9 @@ This method requests access to the datafile whose id is passed on the behalf of
 A curl example using an ``id``::
 
     curl -H "X-Dataverse-key:$API_TOKEN" -X PUT http://$SERVER/api/access/datafile/{id}/requestAccess
-
+
+.. note:: Some installations of Dataverse may require you to provide a Guestbook Response when requesting access to certain restricted files. The response can be passed in the body of this call. See "Get a Guestbook for a Dataverse Collection" in the :doc:`native-api`.
+
 Grant File Access:
 ~~~~~~~~~~~~~~~~~~ 
 

doc/sphinx-guides/source/api/native-api.rst

@@ -1168,6 +1168,100 @@ The fully expanded example above (without environment variables) looks like this
 
   curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/dataverses/root/guestbookResponses?guestbookId=1" -o myResponses.csv
 
+.. _guestbook-api:
+
+Guestbooks
+~~~~~~~~~~
+
+Create a Guestbook for a Dataverse Collection
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide.
+
+Create a Guestbook that can be selected for a Dataset.
+You must have "EditDataverse" permission on the Dataverse collection.
+
+.. code-block:: bash
+
+  export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+  export SERVER_URL=https://demo.dataverse.org
+  export ID=root
+  export JSON='{"name": "my test guestbook","enabled": true,"emailRequired": true,"nameRequired": true,"institutionRequired": false,"positionRequired": false,"customQuestions": [{"question": "how is your day","required": true,"displayOrder": 0,"type": "text","hidden": false}]}'
+
+  curl -POST -H  "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/guestbooks/{ID}" -d "$JSON"
+
+The fully expanded example above (without environment variables) looks like this:
+
+.. code-block:: bash
+
+  curl -POST -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/guestbooks/root" -d '{"name": "my test guestbook","enabled": true,"emailRequired": true,"nameRequired": true,"institutionRequired": false,"positionRequired": false}'
+
+Get a list of Guestbooks for a Dataverse Collection
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide.
+
+Get a list of Guestbooks for a Dataverse Collection
+You must have "EditDataverse" permission on the Dataverse collection.
+
+.. code-block:: bash
+
+  export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+  export SERVER_URL=https://demo.dataverse.org
+  export ID=root
+
+  curl -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/guestbooks/{ID}/list"`
+
+The fully expanded example above (without environment variables) looks like this:
+
+.. code-block:: bash
+
+  curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/guestbooks/root/list"
+
+Get a Guestbook for a Dataverse Collection
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide.
+
+Get a Guestbook by it's id
+
+.. code-block:: bash
+
+  export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+  export SERVER_URL=https://demo.dataverse.org
+  export ID=1234
+
+  curl -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/guestbooks/{ID}"`
+
+The fully expanded example above (without environment variables) looks like this:
+
+.. code-block:: bash
+
+  curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/guestbooks/1234"
+
+Enable or Disable a Guestbook for a Dataverse Collection
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide.
+
+Use this endpoint to enable or disable the Guestbook. A Guestbook can not be deleted or modified since there may be responses linked to it.
+You must have "EditDataverse" permission on the Dataverse collection.
+
+.. code-block:: bash
+
+  export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+  export SERVER_URL=https://demo.dataverse.org
+  export dataverseIdentifier=root
+  export ID=1234
+
+  curl -X PUT -d 'true' -H  "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/guestbooks/{dataverseIdentifier}/{ID}/enabled"
+
+The fully expanded example above (without environment variables) looks like this:
+
+.. code-block:: bash
+
+  curl -X PUT -d 'true' -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/guestbooks/root/1234"
+
 .. _collection-attributes-api:
 
 Change Collection Attributes

docker-compose-dev.yml

@@ -60,6 +60,7 @@ services:
         -Ddataverse.pid.fake.label=FakeDOIProvider
         -Ddataverse.pid.fake.authority=10.5072
         -Ddataverse.pid.fake.shoulder=FK2/
+        #-Ddataverse.files.guestbook-at-request=true
         #-Ddataverse.lang.directory=/dv/lang
     ports:
       - "8080:8080" # HTTP (Dataverse Application)

scripts/api/data/guestbook-test-response.json

@@ -0,0 +1,17 @@
+{"guestbookResponse": {
+    "answers": [
+        {
+            "id": @QID1,
+            "value": "Good"
+        },
+        {
+            "id": @QID2,
+            "value": ["Multi","Line"]
+        },
+        {
+            "id": @QID3,
+            "value": "Yellow"
+        }
+    ]
+  }
+}

scripts/api/data/guestbook-test.json

@@ -0,0 +1,49 @@
+{
+  "name": "my test guestbook",
+  "enabled": true,
+  "emailRequired": true,
+  "nameRequired": true,
+  "institutionRequired": false,
+  "positionRequired": false,
+  "customQuestions": [
+    {
+      "question": "how's your day",
+      "required": true,
+      "displayOrder": 0,
+      "type": "text",
+      "hidden": false
+    },
+    {
+      "question": "Describe yourself",
+      "required": false,
+      "displayOrder": 1,
+      "type": "textarea",
+      "hidden": false
+    },
+    {
+      "question": "What color car do you drive",
+      "required": true,
+      "displayOrder": 2,
+      "type": "options",
+      "hidden": false,
+      "optionValues": [
+        {
+          "value": "Red",
+          "displayOrder": 0
+        },
+        {
+          "value": "White",
+          "displayOrder": 1
+        },
+        {
+          "value": "Yellow",
+          "displayOrder": 2
+        },
+        {
+          "value": "Purple",
+          "displayOrder": 3
+        }
+      ]
+    }
+  ]
+}

src/main/java/edu/harvard/iq/dataverse/CustomQuestion.java

@@ -1,9 +1,12 @@
 package edu.harvard.iq.dataverse;
-import java.io.Serializable;
-import java.util.List;
+
 import jakarta.persistence.*;
 import jakarta.validation.constraints.NotBlank;
 
+import java.io.Serializable;
+import java.util.List;
+import java.util.stream.Collectors;
+
 /**
  *
  * @author skraffmiller
@@ -92,6 +95,12 @@ public void setQuestionString(String questionString) {
     public List<CustomQuestionValue> getCustomQuestionValues() {
         return customQuestionValues;
     }
+
+    public List<String> getCustomQuestionOptions() {
+        return customQuestionValues.stream()
+                .map(CustomQuestionValue::getValueString)
+                .collect(Collectors.toList());
+    }
 
     public String getCustomQuestionValueString(){
         String retString = "";

src/main/java/edu/harvard/iq/dataverse/CustomQuestionResponse.java

@@ -5,11 +5,12 @@
  */
 package edu.harvard.iq.dataverse;
 
-import java.io.Serializable;
-import java.util.List;
 import jakarta.faces.model.SelectItem;
 import jakarta.persistence.*;
 
+import java.io.Serializable;
+import java.util.List;
+
 /**
  *
  * @author skraffmiller

src/main/java/edu/harvard/iq/dataverse/Guestbook.java

@@ -2,35 +2,28 @@
 package edu.harvard.iq.dataverse;
 
 import edu.harvard.iq.dataverse.util.BundleUtil;
+import edu.harvard.iq.dataverse.util.DateUtil;
+import jakarta.persistence.*;
+import org.hibernate.validator.constraints.NotBlank;
+
 import java.io.Serializable;
 import java.util.ArrayList;
 import java.util.Date;
-import jakarta.persistence.CascadeType;
-import jakarta.persistence.Entity;
-import jakarta.persistence.GeneratedValue;
-import jakarta.persistence.GenerationType;
-import jakarta.persistence.Id;
-import jakarta.persistence.JoinColumn;
-import jakarta.persistence.OneToMany;
 import java.util.List;
 import java.util.Objects;
-import jakarta.persistence.Column;
-import jakarta.persistence.ManyToOne;
-import jakarta.persistence.OrderBy;
-import jakarta.persistence.Temporal;
-import jakarta.persistence.TemporalType;
-import jakarta.persistence.Transient;
-
-import edu.harvard.iq.dataverse.util.DateUtil;
-import org.hibernate.validator.constraints.NotBlank;
 
 /**
  *
  * @author skraffmiller
  */
 @Entity
+@NamedQueries(
+        @NamedQuery(name = "Guestbook.findByDataverse",
+                query = "SELECT gb FROM Guestbook gb WHERE gb.dataverse=:dataverse")
+)
+
 public class Guestbook implements Serializable {
-    
+
     @Id
     @GeneratedValue(strategy = GenerationType.IDENTITY)
     private Long id;

src/main/java/edu/harvard/iq/dataverse/GuestbookResponseServiceBean.java

@@ -9,18 +9,6 @@
 import edu.harvard.iq.dataverse.authorization.users.User;
 import edu.harvard.iq.dataverse.externaltools.ExternalTool;
 import edu.harvard.iq.dataverse.util.StringUtil;
-import java.io.IOException;
-import java.io.OutputStream;
-import java.text.SimpleDateFormat;
-import java.time.LocalDate;
-import java.util.ArrayList;
-import java.util.Calendar;
-import java.util.Date;
-import java.util.HashMap;
-import java.util.Iterator;
-import java.util.List;
-import java.util.Map;
-import java.util.logging.Logger;
 import jakarta.ejb.EJB;
 import jakarta.ejb.Stateless;
 import jakarta.ejb.TransactionAttribute;
@@ -30,9 +18,15 @@
 import jakarta.persistence.EntityManager;
 import jakarta.persistence.PersistenceContext;
 import jakarta.persistence.Query;
-import jakarta.persistence.StoredProcedureQuery;
 import jakarta.persistence.TypedQuery;
 import org.apache.commons.text.StringEscapeUtils;
+
+import java.io.IOException;
+import java.io.OutputStream;
+import java.text.SimpleDateFormat;
+import java.time.LocalDate;
+import java.util.*;
+import java.util.logging.Logger;
 /**
  *
  * @author skraffmiller
@@ -815,7 +809,7 @@ public GuestbookResponse initAPIGuestbookResponse(Dataset dataset, DataFile data
         }        
         guestbookResponse.setDataset(dataset);
         guestbookResponse.setResponseTime(new Date());
-        guestbookResponse.setSessionId(session.toString());
+        guestbookResponse.setSessionId(session != null ? session.toString() : "");
         guestbookResponse.setEventType(GuestbookResponse.DOWNLOAD);
         setUserDefaultResponses(guestbookResponse, session, user);
         return guestbookResponse;