diff --git a/doc/release-notes/12001-api-support-termofuse-guestbook.md b/doc/release-notes/12001-api-support-termofuse-guestbook.md new file mode 100644 index 00000000000..249aabd2902 --- /dev/null +++ b/doc/release-notes/12001-api-support-termofuse-guestbook.md @@ -0,0 +1,25 @@ +## Feature Request: API to support Download Terms of Use and Guestbook + +## New Endpoints to download a file or files that required a Guestbook Response: POST +A post to these endpoints with the body containing a JSON Guestbook Response will save the response and return a signed URL to download the file(s) + +`/api/access/datafile/{fileId:.+}` +`/api/access/datafiles/{fileIds}` +`/api/access/dataset/{id}` +`/api/access/dataset/{id}/versions/{versionId}` + +A post to these endpoints with the body containing a JSON Guestbook Response will save the response before continuing the download. +No signed URL option exists. +`/api/access/datafiles` +`/api/access/datafile/bundle/{fileId}` POST returns BundleDownloadInstance after processing Guestbook Responses from body. + +## New CRUD Endpoints for Guestbook: +Create a Guestbook: POST `/api/guestbooks/{dataverseIdentifier}` +Get a Guestbook: GET `/api/guestbooks/{id}` +Get a list of Guestbooks linked to a Dataverse Collection: GET `/api/guestbooks/{dataverseIdentifier}/list` +Enable/Disable a Guestbook: PUT `/api/guestbooks/{dataverseIdentifier}/{id}/enabled` Body: `true` or `false` +Note: There is no Update or Delete at this time. You can disable a Guestbook and create a new one. + +## For Guestbook At Request: +When JVM setting -Ddataverse.files.guestbook-at-request=true is used a request for access may require a Guestbook Response. +PUT `/api/access/datafile/{id}/requestAccess` will now take a JSON Guestbook Response in the body. diff --git a/doc/sphinx-guides/source/api/dataaccess.rst b/doc/sphinx-guides/source/api/dataaccess.rst index b8b103b76ed..01431a02345 100755 --- a/doc/sphinx-guides/source/api/dataaccess.rst +++ b/doc/sphinx-guides/source/api/dataaccess.rst @@ -32,6 +32,8 @@ Basic Download By Dataset The basic form downloads files from the latest accessible version of the dataset. If you are not using an API token, this means the most recently published version. If you are using an API token with full access to the dataset, this means the draft version or the most recently published version if no draft exists. +.. note:: Restricted files that require a Guestbook Response will require an additional step to supply the Guestbook Response. A POST to the same endpoint with the Guestbook Response in the body can return a signed url that can be used to download the file(s) via a browser or download manager. For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide. + A curl example using a DOI (no version): .. code-block:: bash @@ -59,6 +61,8 @@ The second form of the "download by dataset" API allows you to specify which ver * ``x.y`` a specific version, where ``x`` is the major version number and ``y`` is the minor version number. * ``x`` same as ``x.0`` +.. note:: Restricted files that require a Guestbook Response will require an additional step to supply the Guestbook Response. A POST to the same endpoint with the Guestbook Response in the body can return a signed url that can be used to download the file(s) via a browser or download manager. For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide. + A curl example using a DOI (with version): .. code-block:: bash @@ -91,6 +95,11 @@ Basic access URI: GET http://$SERVER/api/access/datafile/:persistentId?persistentId=doi:10.5072/FK2/J8SJZB +.. note:: Restricted files that require a Guestbook Response will require an additional step to supply the Guestbook Response. A POST to the same endpoint with the Guestbook Response in the body can return a signed url that can be used to download the file(s) via a browser or download manager. For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide. In the following JSON example please note that the `name`, `email`, `institution`, and `position` fields will default to the User's account information if not included in the response. + + Example :: + + POST http://$SERVER/api/access/datafile/:persistentId?persistentId=doi:10.5072/FK2/J8SJZB&signed=true -d '{"guestbookResponse": {"name": "My Name", "email": "myemail@example.com", "institution": "Harvard","position": "Staff", "answers": [{"id": 123,"value": "Good"},{"id": 124,"value": ["Multi","Line"]},{"id": 125,"value": "Yellow"}]}}' Parameters: ~~~~~~~~~~~ @@ -198,6 +207,8 @@ Returns the files listed, zipped. As of v6.7 the name of the zipped bundle will .. note:: If any of the datafiles have the ``DirectoryLabel`` attributes in the corresponding ``FileMetadata`` entries, these will be added as folders to the Zip archive, and the files will be placed in them accordingly. +.. note:: If Guestbook Responses are required they can be included in the body along with the file ids as JSON: ``{"fileIds" :[1,2,3], {"guestbookResponse": {"answers": []}}}``. For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide. + Parameters: ~~~~~~~~~~~ @@ -378,7 +389,9 @@ This method requests access to the datafile whose id is passed on the behalf of A curl example using an ``id``:: curl -H "X-Dataverse-key:$API_TOKEN" -X PUT http://$SERVER/api/access/datafile/{id}/requestAccess - + +.. note:: Some installations of Dataverse may require you to provide a Guestbook Response when requesting access to certain restricted files. The response can be passed in the body of this call. See "Get a Guestbook for a Dataverse Collection" in the :doc:`native-api`. + Grant File Access: ~~~~~~~~~~~~~~~~~~ diff --git a/doc/sphinx-guides/source/api/native-api.rst b/doc/sphinx-guides/source/api/native-api.rst index 2befaa56b0c..e6af67579fc 100644 --- a/doc/sphinx-guides/source/api/native-api.rst +++ b/doc/sphinx-guides/source/api/native-api.rst @@ -1168,6 +1168,100 @@ The fully expanded example above (without environment variables) looks like this curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/dataverses/root/guestbookResponses?guestbookId=1" -o myResponses.csv +.. _guestbook-api: + +Guestbooks +~~~~~~~~~~ + +Create a Guestbook for a Dataverse Collection +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide. + +Create a Guestbook that can be selected for a Dataset. +You must have "EditDataverse" permission on the Dataverse collection. + +.. code-block:: bash + + export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx + export SERVER_URL=https://demo.dataverse.org + export ID=root + export JSON='{"name": "my test guestbook","enabled": true,"emailRequired": true,"nameRequired": true,"institutionRequired": false,"positionRequired": false,"customQuestions": [{"question": "how is your day","required": true,"displayOrder": 0,"type": "text","hidden": false}]}' + + curl -POST -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/guestbooks/{ID}" -d "$JSON" + +The fully expanded example above (without environment variables) looks like this: + +.. code-block:: bash + + curl -POST -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/guestbooks/root" -d '{"name": "my test guestbook","enabled": true,"emailRequired": true,"nameRequired": true,"institutionRequired": false,"positionRequired": false}' + +Get a list of Guestbooks for a Dataverse Collection +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide. + +Get a list of Guestbooks for a Dataverse Collection +You must have "EditDataverse" permission on the Dataverse collection. + +.. code-block:: bash + + export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx + export SERVER_URL=https://demo.dataverse.org + export ID=root + + curl -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/guestbooks/{ID}/list"` + +The fully expanded example above (without environment variables) looks like this: + +.. code-block:: bash + + curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/guestbooks/root/list" + +Get a Guestbook for a Dataverse Collection +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide. + +Get a Guestbook by it's id + +.. code-block:: bash + + export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx + export SERVER_URL=https://demo.dataverse.org + export ID=1234 + + curl -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/guestbooks/{ID}"` + +The fully expanded example above (without environment variables) looks like this: + +.. code-block:: bash + + curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/guestbooks/1234" + +Enable or Disable a Guestbook for a Dataverse Collection +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide. + +Use this endpoint to enable or disable the Guestbook. A Guestbook can not be deleted or modified since there may be responses linked to it. +You must have "EditDataverse" permission on the Dataverse collection. + +.. code-block:: bash + + export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx + export SERVER_URL=https://demo.dataverse.org + export dataverseIdentifier=root + export ID=1234 + + curl -X PUT -d 'true' -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/guestbooks/{dataverseIdentifier}/{ID}/enabled" + +The fully expanded example above (without environment variables) looks like this: + +.. code-block:: bash + + curl -X PUT -d 'true' -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/guestbooks/root/1234" + .. _collection-attributes-api: Change Collection Attributes diff --git a/docker-compose-dev.yml b/docker-compose-dev.yml index 7f12de50b32..88b902dfc7f 100644 --- a/docker-compose-dev.yml +++ b/docker-compose-dev.yml @@ -60,6 +60,7 @@ services: -Ddataverse.pid.fake.label=FakeDOIProvider -Ddataverse.pid.fake.authority=10.5072 -Ddataverse.pid.fake.shoulder=FK2/ + #-Ddataverse.files.guestbook-at-request=true #-Ddataverse.lang.directory=/dv/lang ports: - "8080:8080" # HTTP (Dataverse Application) diff --git a/scripts/api/data/guestbook-test-response.json b/scripts/api/data/guestbook-test-response.json new file mode 100644 index 00000000000..df08b52ff6a --- /dev/null +++ b/scripts/api/data/guestbook-test-response.json @@ -0,0 +1,17 @@ +{"guestbookResponse": { + "answers": [ + { + "id": @QID1, + "value": "Good" + }, + { + "id": @QID2, + "value": ["Multi","Line"] + }, + { + "id": @QID3, + "value": "Yellow" + } + ] + } +} diff --git a/scripts/api/data/guestbook-test.json b/scripts/api/data/guestbook-test.json new file mode 100644 index 00000000000..710192b510a --- /dev/null +++ b/scripts/api/data/guestbook-test.json @@ -0,0 +1,49 @@ +{ + "name": "my test guestbook", + "enabled": true, + "emailRequired": true, + "nameRequired": true, + "institutionRequired": false, + "positionRequired": false, + "customQuestions": [ + { + "question": "how's your day", + "required": true, + "displayOrder": 0, + "type": "text", + "hidden": false + }, + { + "question": "Describe yourself", + "required": false, + "displayOrder": 1, + "type": "textarea", + "hidden": false + }, + { + "question": "What color car do you drive", + "required": true, + "displayOrder": 2, + "type": "options", + "hidden": false, + "optionValues": [ + { + "value": "Red", + "displayOrder": 0 + }, + { + "value": "White", + "displayOrder": 1 + }, + { + "value": "Yellow", + "displayOrder": 2 + }, + { + "value": "Purple", + "displayOrder": 3 + } + ] + } + ] +} diff --git a/src/main/java/edu/harvard/iq/dataverse/CustomQuestion.java b/src/main/java/edu/harvard/iq/dataverse/CustomQuestion.java index d880da5b4a8..a4f36b1bad0 100644 --- a/src/main/java/edu/harvard/iq/dataverse/CustomQuestion.java +++ b/src/main/java/edu/harvard/iq/dataverse/CustomQuestion.java @@ -1,9 +1,12 @@ package edu.harvard.iq.dataverse; -import java.io.Serializable; -import java.util.List; + import jakarta.persistence.*; import jakarta.validation.constraints.NotBlank; +import java.io.Serializable; +import java.util.List; +import java.util.stream.Collectors; + /** * * @author skraffmiller @@ -92,6 +95,12 @@ public void setQuestionString(String questionString) { public List getCustomQuestionValues() { return customQuestionValues; } + + public List getCustomQuestionOptions() { + return customQuestionValues.stream() + .map(CustomQuestionValue::getValueString) + .collect(Collectors.toList()); + } public String getCustomQuestionValueString(){ String retString = ""; diff --git a/src/main/java/edu/harvard/iq/dataverse/CustomQuestionResponse.java b/src/main/java/edu/harvard/iq/dataverse/CustomQuestionResponse.java index f19ee3c3fc7..aead81cd289 100644 --- a/src/main/java/edu/harvard/iq/dataverse/CustomQuestionResponse.java +++ b/src/main/java/edu/harvard/iq/dataverse/CustomQuestionResponse.java @@ -5,11 +5,12 @@ */ package edu.harvard.iq.dataverse; -import java.io.Serializable; -import java.util.List; import jakarta.faces.model.SelectItem; import jakarta.persistence.*; +import java.io.Serializable; +import java.util.List; + /** * * @author skraffmiller diff --git a/src/main/java/edu/harvard/iq/dataverse/Guestbook.java b/src/main/java/edu/harvard/iq/dataverse/Guestbook.java index 2ef23d1f925..12b81e58506 100644 --- a/src/main/java/edu/harvard/iq/dataverse/Guestbook.java +++ b/src/main/java/edu/harvard/iq/dataverse/Guestbook.java @@ -2,35 +2,28 @@ package edu.harvard.iq.dataverse; import edu.harvard.iq.dataverse.util.BundleUtil; +import edu.harvard.iq.dataverse.util.DateUtil; +import jakarta.persistence.*; +import org.hibernate.validator.constraints.NotBlank; + import java.io.Serializable; import java.util.ArrayList; import java.util.Date; -import jakarta.persistence.CascadeType; -import jakarta.persistence.Entity; -import jakarta.persistence.GeneratedValue; -import jakarta.persistence.GenerationType; -import jakarta.persistence.Id; -import jakarta.persistence.JoinColumn; -import jakarta.persistence.OneToMany; import java.util.List; import java.util.Objects; -import jakarta.persistence.Column; -import jakarta.persistence.ManyToOne; -import jakarta.persistence.OrderBy; -import jakarta.persistence.Temporal; -import jakarta.persistence.TemporalType; -import jakarta.persistence.Transient; - -import edu.harvard.iq.dataverse.util.DateUtil; -import org.hibernate.validator.constraints.NotBlank; /** * * @author skraffmiller */ @Entity +@NamedQueries( + @NamedQuery(name = "Guestbook.findByDataverse", + query = "SELECT gb FROM Guestbook gb WHERE gb.dataverse=:dataverse") +) + public class Guestbook implements Serializable { - + @Id @GeneratedValue(strategy = GenerationType.IDENTITY) private Long id; diff --git a/src/main/java/edu/harvard/iq/dataverse/GuestbookResponseServiceBean.java b/src/main/java/edu/harvard/iq/dataverse/GuestbookResponseServiceBean.java index 04ab044cf5e..754fe51714a 100644 --- a/src/main/java/edu/harvard/iq/dataverse/GuestbookResponseServiceBean.java +++ b/src/main/java/edu/harvard/iq/dataverse/GuestbookResponseServiceBean.java @@ -9,18 +9,6 @@ import edu.harvard.iq.dataverse.authorization.users.User; import edu.harvard.iq.dataverse.externaltools.ExternalTool; import edu.harvard.iq.dataverse.util.StringUtil; -import java.io.IOException; -import java.io.OutputStream; -import java.text.SimpleDateFormat; -import java.time.LocalDate; -import java.util.ArrayList; -import java.util.Calendar; -import java.util.Date; -import java.util.HashMap; -import java.util.Iterator; -import java.util.List; -import java.util.Map; -import java.util.logging.Logger; import jakarta.ejb.EJB; import jakarta.ejb.Stateless; import jakarta.ejb.TransactionAttribute; @@ -30,9 +18,15 @@ import jakarta.persistence.EntityManager; import jakarta.persistence.PersistenceContext; import jakarta.persistence.Query; -import jakarta.persistence.StoredProcedureQuery; import jakarta.persistence.TypedQuery; import org.apache.commons.text.StringEscapeUtils; + +import java.io.IOException; +import java.io.OutputStream; +import java.text.SimpleDateFormat; +import java.time.LocalDate; +import java.util.*; +import java.util.logging.Logger; /** * * @author skraffmiller @@ -815,7 +809,7 @@ public GuestbookResponse initAPIGuestbookResponse(Dataset dataset, DataFile data } guestbookResponse.setDataset(dataset); guestbookResponse.setResponseTime(new Date()); - guestbookResponse.setSessionId(session.toString()); + guestbookResponse.setSessionId(session != null ? session.toString() : ""); guestbookResponse.setEventType(GuestbookResponse.DOWNLOAD); setUserDefaultResponses(guestbookResponse, session, user); return guestbookResponse; diff --git a/src/main/java/edu/harvard/iq/dataverse/GuestbookServiceBean.java b/src/main/java/edu/harvard/iq/dataverse/GuestbookServiceBean.java index fcd4e91d455..fc7f361b8b6 100644 --- a/src/main/java/edu/harvard/iq/dataverse/GuestbookServiceBean.java +++ b/src/main/java/edu/harvard/iq/dataverse/GuestbookServiceBean.java @@ -11,6 +11,8 @@ import jakarta.persistence.PersistenceContext; import jakarta.persistence.Query; +import java.util.List; + /** * * @author skraffmiller @@ -21,8 +23,17 @@ public class GuestbookServiceBean implements java.io.Serializable { @PersistenceContext(unitName = "VDCNet-ejbPU") private EntityManager em; - - + + public List findGuestbooksForGivenDataverse(Dataverse dataverse) { + if (dataverse != null) { + Query query = em.createNamedQuery("Guestbook.findByDataverse"); + query.setParameter("dataverse", dataverse); + return query.getResultList(); + } else { + return List.of(); + } + } + public Long findCountUsages(Long guestbookId, Long dataverseId) { String queryString = ""; if (guestbookId != null && dataverseId != null) { diff --git a/src/main/java/edu/harvard/iq/dataverse/api/Access.java b/src/main/java/edu/harvard/iq/dataverse/api/Access.java index cadd758a3ac..d4ca8922dd5 100644 --- a/src/main/java/edu/harvard/iq/dataverse/api/Access.java +++ b/src/main/java/edu/harvard/iq/dataverse/api/Access.java @@ -7,23 +7,15 @@ package edu.harvard.iq.dataverse.api; import edu.harvard.iq.dataverse.*; - -import static edu.harvard.iq.dataverse.api.Datasets.handleVersion; - import edu.harvard.iq.dataverse.api.auth.AuthRequired; import edu.harvard.iq.dataverse.authorization.DataverseRole; import edu.harvard.iq.dataverse.authorization.Permission; import edu.harvard.iq.dataverse.authorization.RoleAssignee; +import edu.harvard.iq.dataverse.authorization.users.ApiToken; import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser; import edu.harvard.iq.dataverse.authorization.users.GuestUser; import edu.harvard.iq.dataverse.authorization.users.User; -import edu.harvard.iq.dataverse.dataaccess.DataAccess; -import edu.harvard.iq.dataverse.dataaccess.DataAccessRequest; -import edu.harvard.iq.dataverse.dataaccess.StorageIO; -import edu.harvard.iq.dataverse.dataaccess.DataFileZipper; -import edu.harvard.iq.dataverse.dataaccess.GlobusAccessibleStore; -import edu.harvard.iq.dataverse.dataaccess.OptionalAccessService; -import edu.harvard.iq.dataverse.dataaccess.ImageThumbConverter; +import edu.harvard.iq.dataverse.dataaccess.*; import edu.harvard.iq.dataverse.datavariable.DataVariable; import edu.harvard.iq.dataverse.datavariable.VariableServiceBean; import edu.harvard.iq.dataverse.dataverse.featured.DataverseFeaturedItem; @@ -35,69 +27,20 @@ import edu.harvard.iq.dataverse.export.DDIExportServiceBean; import edu.harvard.iq.dataverse.makedatacount.MakeDataCountLoggingServiceBean; import edu.harvard.iq.dataverse.makedatacount.MakeDataCountLoggingServiceBean.MakeDataCountEntry; +import edu.harvard.iq.dataverse.settings.JvmSettings; import edu.harvard.iq.dataverse.settings.SettingsServiceBean; -import edu.harvard.iq.dataverse.util.BundleUtil; -import edu.harvard.iq.dataverse.util.FileUtil; -import edu.harvard.iq.dataverse.util.StringUtil; -import edu.harvard.iq.dataverse.util.SystemConfig; +import edu.harvard.iq.dataverse.util.*; +import edu.harvard.iq.dataverse.util.json.JsonParseException; +import edu.harvard.iq.dataverse.util.json.JsonUtil; import edu.harvard.iq.dataverse.util.json.NullSafeJsonBuilder; - -import java.util.logging.Logger; import jakarta.ejb.EJB; -import java.io.InputStream; -import java.io.ByteArrayOutputStream; -import java.io.File; -import java.io.FileInputStream; -import java.io.FileNotFoundException; -import java.io.IOException; -import java.io.OutputStream; -import java.sql.Timestamp; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.Date; -import java.util.List; -import java.util.logging.Level; import jakarta.inject.Inject; -import jakarta.json.Json; -import java.net.URI; -import jakarta.json.JsonArrayBuilder; +import jakarta.json.*; import jakarta.persistence.TypedQuery; - -import jakarta.ws.rs.GET; -import jakarta.ws.rs.Path; -import jakarta.ws.rs.PathParam; -import jakarta.ws.rs.Produces; - -import jakarta.ws.rs.container.ContainerRequestContext; -import jakarta.ws.rs.core.Context; -import jakarta.ws.rs.core.HttpHeaders; -import jakarta.ws.rs.core.UriInfo; - - import jakarta.servlet.http.HttpServletResponse; -import jakarta.ws.rs.BadRequestException; -import jakarta.ws.rs.Consumes; -import jakarta.ws.rs.DELETE; -import jakarta.ws.rs.ForbiddenException; -import jakarta.ws.rs.NotFoundException; -import jakarta.ws.rs.POST; -import jakarta.ws.rs.PUT; -import jakarta.ws.rs.QueryParam; -import jakarta.ws.rs.ServiceUnavailableException; -import jakarta.ws.rs.WebApplicationException; -import jakarta.ws.rs.core.Response; -import static jakarta.ws.rs.core.Response.Status.BAD_REQUEST; -import jakarta.ws.rs.core.StreamingOutput; -import static edu.harvard.iq.dataverse.util.json.JsonPrinter.json; -import java.net.URISyntaxException; - -import jakarta.json.JsonObjectBuilder; -import jakarta.ws.rs.RedirectionException; -import jakarta.ws.rs.ServerErrorException; -import jakarta.ws.rs.core.MediaType; -import static jakarta.ws.rs.core.Response.Status.FORBIDDEN; -import static jakarta.ws.rs.core.Response.Status.UNAUTHORIZED; - +import jakarta.ws.rs.*; +import jakarta.ws.rs.container.ContainerRequestContext; +import jakarta.ws.rs.core.*; import org.eclipse.microprofile.openapi.annotations.Operation; import org.eclipse.microprofile.openapi.annotations.media.Content; import org.eclipse.microprofile.openapi.annotations.parameters.RequestBody; @@ -107,6 +50,21 @@ import org.glassfish.jersey.media.multipart.FormDataBodyPart; import org.glassfish.jersey.media.multipart.FormDataParam; +import java.io.*; +import java.net.URI; +import java.net.URISyntaxException; +import java.net.URLDecoder; +import java.nio.charset.StandardCharsets; +import java.sql.Timestamp; +import java.util.*; +import java.util.logging.Level; +import java.util.logging.Logger; +import java.util.stream.Collectors; + +import static edu.harvard.iq.dataverse.api.Datasets.handleVersion; +import static edu.harvard.iq.dataverse.util.json.JsonPrinter.json; +import static jakarta.ws.rs.core.Response.Status.*; + /* Custom API exceptions [NOT YET IMPLEMENTED] import edu.harvard.iq.dataverse.api.exceptions.NotFoundException; @@ -177,19 +135,22 @@ public class Access extends AbstractApiBean { @AuthRequired @Path("datafile/bundle/{fileId}") @Produces({"application/zip"}) - public BundleDownloadInstance datafileBundle(@Context ContainerRequestContext crc, @PathParam("fileId") String fileId, @QueryParam("fileMetadataId") Long fileMetadataId,@QueryParam("gbrecs") boolean gbrecs, @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response) /*throws NotFoundException, ServiceUnavailableException, PermissionDeniedException, AuthorizationRequiredException*/ { - + public BundleDownloadInstance datafileBundle(@Context ContainerRequestContext crc, @PathParam("fileId") String fileId, @QueryParam("fileMetadataId") Long fileMetadataId, + @QueryParam("gbrecs") boolean gbrecs, @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response) /*throws NotFoundException, ServiceUnavailableException, PermissionDeniedException, AuthorizationRequiredException*/ { - GuestbookResponse gbr = null; - DataFile df = findDataFileOrDieWrapper(fileId); // This will throw a ForbiddenException if access isn't authorized: checkAuthorization(crc, df); + + if (checkGuestbookRequiredResponse(crc, df)) { + throw new BadRequestException(BundleUtil.getStringFromBundle("access.api.download.failure.guestbookResponseMissing")); + } - if (gbrecs != true && df.isReleased()){ + if (gbrecs != true && df.isReleased()) { // Write Guestbook record if not done previously and file is released - gbr = guestbookResponseService.initAPIGuestbookResponse(df.getOwner(), df, session, getRequestor(crc)); + User requestor = getRequestor(crc); + GuestbookResponse gbr = guestbookResponseService.initAPIGuestbookResponse(df.getOwner(), df, session, requestor); guestbookResponseService.save(gbr); MakeDataCountEntry entry = new MakeDataCountEntry(uriInfo, headers, dvRequestService, df); mdcLogService.logEntry(entry); @@ -231,7 +192,19 @@ public BundleDownloadInstance datafileBundle(@Context ContainerRequestContext cr return downloadInstance; } - + + @POST + @AuthRequired + @Path("datafile/bundle/{fileId}") + @Produces({"application/zip"}) + public BundleDownloadInstance datafileBundleWithGuestbookResponse(@Context ContainerRequestContext crc, @PathParam("fileId") String fileId, @QueryParam("fileMetadataId") Long fileMetadataId, @QueryParam("gbrecs") boolean gbrecs, + @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response, String jsonBody) /*throws NotFoundException, ServiceUnavailableException, PermissionDeniedException, AuthorizationRequiredException*/ { + processDatafileWithGuestbookResponse(crc, headers, fileId, uriInfo, gbrecs, jsonBody); + // There is no get for this so we shouldn't return a signed url + // return the download instance + return datafileBundle(crc, fileId, fileMetadataId, gbrecs, uriInfo, headers, response); + } + //Added a wrapper method since the original method throws a wrapped response //the access methods return files instead of responses so we convert to a WebApplicationException @@ -254,22 +227,10 @@ private DataFile findDataFileOrDieWrapper(String fileId){ @AuthRequired @Path("datafile/{fileId:.+}") @Produces({"application/xml","*/*"}) - public Response datafile(@Context ContainerRequestContext crc, @PathParam("fileId") String fileId, @QueryParam("gbrecs") boolean gbrecs, @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response) /*throws NotFoundException, ServiceUnavailableException, PermissionDeniedException, AuthorizationRequiredException*/ { - - // check first if there's a trailing slash, and chop it: - while (fileId.lastIndexOf('/') == fileId.length() - 1) { - fileId = fileId.substring(0, fileId.length() - 1); - } - - if (fileId.indexOf('/') > -1) { - // This is for embedding folder names into the Access API URLs; - // something like /api/access/datafile/folder/subfolder/1234 - // instead of the normal /api/access/datafile/1234 notation. - // this is supported only for recreating folders during recursive downloads - - // i.e. they are embedded into the URL for the remote client like wget, - // but can be safely ignored here. - fileId = fileId.substring(fileId.lastIndexOf('/') + 1); - } + public Response datafile(@Context ContainerRequestContext crc, @PathParam("fileId") String fileId, @QueryParam("gbrecs") boolean gbrecs, + @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response) /*throws NotFoundException, ServiceUnavailableException, PermissionDeniedException, AuthorizationRequiredException*/ { + + fileId = normalizeFileId(fileId); DataFile df = findDataFileOrDieWrapper(fileId); GuestbookResponse gbr = null; @@ -282,6 +243,9 @@ public Response datafile(@Context ContainerRequestContext crc, @PathParam("fileI // This will throw a ForbiddenException if access isn't authorized: checkAuthorization(crc, df); + if (checkGuestbookRequiredResponse(crc, df)) { + return error(BAD_REQUEST, BundleUtil.getStringFromBundle("access.api.download.failure.guestbookResponseMissing")); + } if (gbrecs != true && df.isReleased()){ // Write Guestbook record if not done previously and file is released @@ -400,12 +364,130 @@ public Response datafile(@Context ContainerRequestContext crc, @PathParam("fileI * Provide some browser-friendly headers: (?) */ if (headers.getRequestHeaders().containsKey("Range")) { - return Response.status(Response.Status.PARTIAL_CONTENT).entity(downloadInstance).build(); + return Response.status(PARTIAL_CONTENT).entity(downloadInstance).build(); } return Response.ok(downloadInstance).build(); } - - + + @POST + @AuthRequired + @Path("datafile/{fileId:.+}") + @Produces({"application/json"}) + public Response datafileWithGuestbookResponse(@Context ContainerRequestContext crc, @PathParam("fileId") String fileId, + @QueryParam("gbrecs") boolean gbrecs, @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response, String jsonBody) { + + fileId = normalizeFileId(fileId); + return processDatafileWithGuestbookResponse(crc, headers, fileId, uriInfo, gbrecs, jsonBody); + } + + private String normalizeFileId(String fileId) { + String fId = fileId; + // check first if there's a trailing slash, and chop it: + while (fId.lastIndexOf('/') == fId.length() - 1) { + fId = fId.substring(0, fId.length() - 1); + } + + if (fId.indexOf('/') > -1) { + // This is for embedding folder names into the Access API URLs; + // something like /api/access/datafile/folder/subfolder/1234 + // instead of the normal /api/access/datafile/1234 notation. + // this is supported only for recreating folders during recursive downloads - + // i.e. they are embedded into the URL for the remote client like wget, + // but can be safely ignored here. + fId = fId.substring(fId.lastIndexOf('/') + 1); + } + return fId; + } + + // Process the guestbook response from JSON and return a signedUrl to the matching GET call + private Response processDatafileWithGuestbookResponse(ContainerRequestContext crc, HttpHeaders headers, String fileIds, UriInfo uriInfo, boolean gbrecs, String jsonBody) { + + User user = getRequestUser(crc); + + // Get and validate all the DataFiles first + Map datafilesMap = getDatafilesMap(crc, fileIds); + + // Handle Guestbook Responses + String displayName = ""; + try { + // since all files must be in the same Dataset we can generate a Guestbook Response once and just replace the DataFile for each file in the list + DataFile firstDatafile = datafilesMap.values().size() > 0 ? (DataFile) Arrays.stream(datafilesMap.values().toArray()).findFirst().get() : null; + GuestbookResponse gbr = getGuestbookResponseFromBody(firstDatafile, GuestbookResponse.DOWNLOAD, jsonBody, user); + for (DataFile df : datafilesMap.values()) { + displayName = df.getDisplayName(); + if (checkGuestbookRequiredResponse(crc, df)) { + if (gbr != null) { + gbr.setDataFile(df); + guestbookResponseService.save(gbr); + MakeDataCountEntry entry = new MakeDataCountEntry(uriInfo, headers, dvRequestService, df); + mdcLogService.logEntry(entry); + } else { + return error(BAD_REQUEST, BundleUtil.getStringFromBundle("access.api.download.failure.guestbookResponseMissing")); + } + } else if (gbrecs != true && df.isReleased()) { + // Write Guestbook record if not done previously and file is released + GuestbookResponse defaultResponse = guestbookResponseService.initAPIGuestbookResponse(df.getOwner(), df, session, user); + guestbookResponseService.save(defaultResponse); + MakeDataCountEntry entry = new MakeDataCountEntry(uriInfo, headers, dvRequestService, df); + mdcLogService.logEntry(entry); + } + } + } catch (JsonParseException ex) { + List args = Arrays.asList(displayName, ex.getLocalizedMessage()); + return error(BAD_REQUEST, BundleUtil.getStringFromBundle("access.api.download.failure.guestbook.commandError", args)); + } + return returnSignedUrl(uriInfo, user); + } + + private Map getDatafilesMap(ContainerRequestContext crc, String fileIds) { + String fileIdParams[] = getFileIdsCSV(fileIds); + Map datafilesMap = new HashMap<>(); + // Get and validate all the DataFiles first + if (fileIdParams != null && fileIdParams.length > 0) { + for (int i = 0; i < fileIdParams.length; i++) { + DataFile df = findDataFileOrDieWrapper(fileIdParams[i]); + + if (df.isHarvested()) { + String errorMessage = "Datafile " + df.getId() + " is a harvested file that cannot be accessed in this Dataverse"; + throw new NotFoundException(errorMessage); + // (nobody should ever be using this API on a harvested DataFile)! + } + + // This will throw a ForbiddenException if access isn't authorized: + checkAuthorization(crc, df); + + datafilesMap.put(df.getId(), df); + } + } + return datafilesMap; + } + + private Response returnSignedUrl(UriInfo uriInfo, User user) { + // Create the signed URL + String userIdentifier = null; + String key = null; + if (user != null && user instanceof AuthenticatedUser) { + AuthenticatedUser requestor = (AuthenticatedUser) user; + userIdentifier = requestor.getUserIdentifier(); + ApiToken apiToken = authSvc.findApiTokenByUser(requestor); + if (apiToken != null && !apiToken.isExpired() && !apiToken.isDisabled()) { + key = apiToken.getTokenString(); + } + } else { + // Guest + userIdentifier = "guest"; + key = uriInfo.getAbsolutePath().toASCIIString(); //TODO find a better one for here and in SignedUrlAuthMechanism.java + } + + UriBuilder builder = UriBuilder.fromUri(uriInfo.getRequestUri()); + builder.replaceQueryParam("gbrecs", true); + String baseUrlEncoded = builder.build().toString(); + String baseUrl = URLDecoder.decode(baseUrlEncoded, StandardCharsets.UTF_8); + key = JvmSettings.API_SIGNING_SECRET.lookupOptional().orElse("") + key; + String signedUrl = UrlSignerUtil.signUrl(baseUrl, 1, userIdentifier, "GET", key); + return ok(Json.createObjectBuilder().add(URLTokenUtil.SIGNED_URL, signedUrl)); + } + /* * Variants of the Access API calls for retrieving datafile-level * Metadata. @@ -629,7 +711,7 @@ public DownloadInstance downloadAuxiliaryFile(@Context ContainerRequestContext c /* * API method for downloading zipped bundles of multiple files. Uses POST to avoid long lists of file IDs that can make the URL longer than what's supported by browsers/servers */ - + // TODO: Rather than only supporting looking up files by their database IDs, // consider supporting persistent identifiers. @POST @@ -637,17 +719,20 @@ public DownloadInstance downloadAuxiliaryFile(@Context ContainerRequestContext c @Path("datafiles") @Consumes("text/plain") @Produces({ "application/zip" }) - public Response postDownloadDatafiles(@Context ContainerRequestContext crc, String fileIds, @QueryParam("gbrecs") boolean gbrecs, @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response) throws WebApplicationException { - + public Response postDownloadDatafiles(@Context ContainerRequestContext crc, String body, @QueryParam("gbrecs") boolean gbrecs, @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response) throws WebApplicationException { - return downloadDatafiles(crc, fileIds, gbrecs, uriInfo, headers, response, null); + processDatafileWithGuestbookResponse(crc, headers, body, uriInfo, gbrecs, body); + // There is no get for this so we shouldn't return a signed url + // initiate the download now + return downloadDatafiles(crc, body, gbrecs, uriInfo, headers, response, null); } @GET @AuthRequired @Path("dataset/{id}") @Produces({"application/zip"}) - public Response downloadAllFromLatest(@Context ContainerRequestContext crc, @PathParam("id") String datasetIdOrPersistentId, @QueryParam("gbrecs") boolean gbrecs, @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response) throws WebApplicationException { + public Response downloadAllFromLatest(@Context ContainerRequestContext crc, @PathParam("id") String datasetIdOrPersistentId, @QueryParam("gbrecs") boolean gbrecs, + @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response) throws WebApplicationException { try { User user = getRequestUser(crc); DataverseRequest req = createDataverseRequest(user); @@ -687,37 +772,44 @@ public Response downloadAllFromLatest(@Context ContainerRequestContext crc, @Pat return wr.getResponse(); } } + @POST + @AuthRequired + @Path("dataset/{id}") + @Produces({"application/zip"}) + public Response downloadAllFromLatestWithGuestbookResponse(@Context ContainerRequestContext crc, @PathParam("id") String datasetIdOrPersistentId, @QueryParam("gbrecs") boolean gbrecs, @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response, String jsonBody) throws WebApplicationException { + try { + User user = getRequestUser(crc); + DataverseRequest req = createDataverseRequest(user); + final Dataset retrieved = findDatasetOrDie(datasetIdOrPersistentId); + String fileIds = ""; + String version = null; + // If user can view the draft version download those files and don't count them + if (!(user instanceof GuestUser)) { + final DatasetVersion draft = versionService.getDatasetVersionById(retrieved.getId(), DatasetVersion.VersionState.DRAFT.toString()); + if (draft != null && permissionService.requestOn(req, retrieved).has(Permission.ViewUnpublishedDataset)) { + fileIds = getFileIdsAsCommaSeparated(draft.getFileMetadatas()); + gbrecs = true; + version = "draft"; + } + } + if (version == null) { + final DatasetVersion latest = versionService.getLatestReleasedVersionFast(retrieved.getId()); + fileIds = getFileIdsAsCommaSeparated(latest.getFileMetadatas()); + version = latest.getFriendlyVersionNumber(); + } + return processDatafileWithGuestbookResponse(crc, headers, fileIds, uriInfo, gbrecs, jsonBody); + } catch (WrappedResponse wr) { + return wr.getResponse(); + } + } @GET @AuthRequired @Path("dataset/{id}/versions/{versionId}") @Produces({"application/zip"}) - public Response downloadAllFromVersion(@Context ContainerRequestContext crc, @PathParam("id") String datasetIdOrPersistentId, @PathParam("versionId") String versionId, @QueryParam("gbrecs") boolean gbrecs, @QueryParam("key") String apiTokenParam, @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response) throws WebApplicationException { + public Response downloadAllFromVersion(@Context ContainerRequestContext crc, @PathParam("id") String datasetIdOrPersistentId, @PathParam("versionId") String versionId, @QueryParam("gbrecs") boolean gbrecs, @QueryParam("key") String apiTokenParam, @QueryParam("signed") boolean signed, @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response) throws WebApplicationException { try { - DataverseRequest req = createDataverseRequest(getRequestUser(crc)); - final Dataset ds = execCommand(new GetDatasetCommand(req, findDatasetOrDie(datasetIdOrPersistentId))); - DatasetVersion dsv = execCommand(handleVersion(versionId, new Datasets.DsVersionHandler>() { - - @Override - public Command handleLatest() { - return new GetLatestAccessibleDatasetVersionCommand(req, ds); - } - - @Override - public Command handleDraft() { - return new GetDraftDatasetVersionCommand(req, ds); - } - - @Override - public Command handleSpecific(long major, long minor) { - return new GetSpecificPublishedDatasetVersionCommand(req, ds, major, minor); - } - - @Override - public Command handleLatestPublished() { - return new GetLatestPublishedDatasetVersionCommand(req, ds); - } - })); + DatasetVersion dsv = getDatasetVersionFromVersion(crc, datasetIdOrPersistentId, versionId); if (dsv == null) { // (A "Not Found" would be more appropriate here, I believe, than a "Bad Request". // But we've been using the latter for a while, and it's a popular API... @@ -732,12 +824,56 @@ public Command handleLatestPublished() { if (dsv.isDraft()) { gbrecs = true; } + return downloadDatafiles(crc, fileIds, gbrecs, uriInfo, headers, response, dsv.getFriendlyVersionNumber().toLowerCase()); } catch (WrappedResponse wr) { return wr.getResponse(); } } + @POST + @AuthRequired + @Path("dataset/{id}/versions/{versionId}") + @Produces({"application/zip"}) + public Response downloadAllFromVersionWithGuestbookResponse(@Context ContainerRequestContext crc, @PathParam("id") String datasetIdOrPersistentId, @PathParam("versionId") String versionId, + @QueryParam("gbrecs") boolean gbrecs, @QueryParam("key") String apiTokenParam, String jsonBody, + @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response) throws WebApplicationException { + try { + DatasetVersion dsv = getDatasetVersionFromVersion(crc, datasetIdOrPersistentId, versionId); + String fileIds = getFileIdsAsCommaSeparated(dsv.getFileMetadatas()); + return processDatafileWithGuestbookResponse(crc, headers, fileIds, uriInfo, gbrecs, jsonBody); + } catch (WrappedResponse wr) { + return wr.getResponse(); + } + } + + private DatasetVersion getDatasetVersionFromVersion(ContainerRequestContext crc, String datasetIdOrPersistentId, String versionId) throws WrappedResponse { + DataverseRequest req = createDataverseRequest(getRequestUser(crc)); + final Dataset ds = execCommand(new GetDatasetCommand(req, findDatasetOrDie(datasetIdOrPersistentId))); + return execCommand(handleVersion(versionId, new Datasets.DsVersionHandler<>() { + + @Override + public Command handleLatest() { + return new GetLatestAccessibleDatasetVersionCommand(req, ds); + } + + @Override + public Command handleDraft() { + return new GetDraftDatasetVersionCommand(req, ds); + } + + @Override + public Command handleSpecific(long major, long minor) { + return new GetSpecificPublishedDatasetVersionCommand(req, ds, major, minor); + } + + @Override + public Command handleLatestPublished() { + return new GetLatestPublishedDatasetVersionCommand(req, ds); + } + })); + } + private static String getFileIdsAsCommaSeparated(List fileMetadatas) { List ids = new ArrayList<>(); for (FileMetadata fileMetadata : fileMetadatas) { @@ -772,192 +908,233 @@ private String generateMultiFileBundleName(Dataset dataset, String versionTag) { @AuthRequired @Path("datafiles/{fileIds}") @Produces({"application/zip"}) - public Response datafiles(@Context ContainerRequestContext crc, @PathParam("fileIds") String fileIds, @QueryParam("gbrecs") boolean gbrecs, @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response) throws WebApplicationException { + public Response datafiles(@Context ContainerRequestContext crc, @PathParam("fileIds") String fileIds, @QueryParam("gbrecs") boolean gbrecs, + @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response) throws WebApplicationException { + return downloadDatafiles(crc, fileIds, gbrecs, uriInfo, headers, response, null); } - private Response downloadDatafiles(ContainerRequestContext crc, String rawFileIds, boolean donotwriteGBResponse, UriInfo uriInfo, HttpHeaders headers, HttpServletResponse response, String versionTag) throws WebApplicationException /* throws NotFoundException, ServiceUnavailableException, PermissionDeniedException, AuthorizationRequiredException*/ { + @POST + @AuthRequired + @Path("datafiles/{fileIds}") + @Produces({"application/zip"}) + public Response datafilesWithGuestbookResponse(@Context ContainerRequestContext crc, @PathParam("fileIds") String fileIds, @QueryParam("gbrecs") boolean gbrecs, + @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response, String jsonBody) throws WebApplicationException { + + return processDatafileWithGuestbookResponse(crc, headers, fileIds, uriInfo, gbrecs, jsonBody); + } + + private String[] getFileIdsCSV(String body) { + /* BODY has 3 variations coming from path parameter of GET or body of POST: + "1,2,3," + "fileIds=1,2,3" + {fileIds:[1,2,3], "guestbookResponse":{}} + */ + if (body.startsWith("{")) { // assume json + // get fileIds from json. example: {fileIds:[1,2,3], "guestbookResponse":{}} + JsonObject jsonObject = JsonUtil.getJsonObject(body); + if (jsonObject.containsKey("fileIds")) { + JsonArray ids = jsonObject.getJsonArray("fileIds"); + List idList = ids.getValuesAs(JsonNumber.class); + return idList.stream().map(JsonNumber::toString).toArray(String[]::new); + } else { + return new String[0]; + } + } else { + // Trim string "fileIds=" from the front if exists + String csv = body.substring(body.startsWith("fileIds=") ? 8 : 0); + //String[] list = body.substring(body.startsWith("fileIds=") ? 8 : 0).replaceAll(",$", "").split(","); + return Arrays.asList(csv.split(",")).stream().map(String::trim) + .filter(s -> !s.isEmpty()).collect(Collectors.toList()).toArray(new String[0]); + } + } + + private Response downloadDatafiles(ContainerRequestContext crc, String body, boolean donotwriteGBResponse, UriInfo uriInfo, HttpHeaders headers, HttpServletResponse response, String versionTag) throws WebApplicationException /* throws NotFoundException, ServiceUnavailableException, PermissionDeniedException, AuthorizationRequiredException*/ { final long zipDownloadSizeLimit = systemConfig.getZipDownloadLimit(); - + logger.fine("setting zip download size limit to " + zipDownloadSizeLimit + " bytes."); - - if (rawFileIds == null || rawFileIds.equals("")) { + + if (body == null || body.equals("")) { throw new BadRequestException(); } - - final String fileIds; - if(rawFileIds.startsWith("fileIds=")) { - fileIds = rawFileIds.substring(8); // String "fileIds=" from the front - } else { - fileIds=rawFileIds; - } + + String[] fileIdParams = getFileIdsCSV(body); + /* Note - fileIds coming from the POST ends in '\n' and a ',' has been added after the last file id number and before a * final '\n' - this stops the last item from being parsed in the fileIds.split(","); line below. */ - + String customZipServiceUrl = settingsService.getValueForKey(SettingsServiceBean.Key.CustomZipDownloadServiceUrl); - boolean useCustomZipService = customZipServiceUrl != null; + boolean useCustomZipService = customZipServiceUrl != null; User user = getRequestor(crc); - + Boolean getOrig = false; for (String key : uriInfo.getQueryParameters().keySet()) { String value = uriInfo.getQueryParameters().getFirst(key); - if("format".equals(key) && "original".equals(value)) { + if ("format".equals(key) && "original".equals(value)) { getOrig = true; } } - + + Map datafilesMap = new HashMap<>(); + List authorizedDatafileIds = new ArrayList<>(); + + // Get DataFiles, check authorized access, check for multiple Datasets, and check for required guestbook response + Set datasetIds = new HashSet<>(); + for (int i = 0; i < fileIdParams.length; i++) { + DataFile df = findDataFileOrDieWrapper(fileIdParams[i]); + datafilesMap.put(df.getId(), df); + datasetIds.add(df.getOwner() != null ? df.getOwner().getId() : 0L); + if (isAccessAuthorized(user, df)) { + authorizedDatafileIds.add(df.getId()); + } + if (datasetIds.size() > 1) { + // All files must be from the same Dataset + return error(BAD_REQUEST, BundleUtil.getStringFromBundle("access.api.download.failure.multipleDatasets")); + } else if (authorizedDatafileIds.contains(df.getId()) && checkGuestbookRequiredResponse(crc, df)) { + try { + GuestbookResponse gbr = getGuestbookResponseFromBody(df, GuestbookResponse.DOWNLOAD, body, user); + if (gbr != null) { + engineSvc.submit(new CreateGuestbookResponseCommand(dvRequestService.getDataverseRequest(), gbr, gbr.getDataset())); + donotwriteGBResponse = true; + // Further down the actual download will also create a simple download response for every datafile listed based on the donotwriteGBResponse flag. + // Modifying donotwriteGBResponse will block that so we also need to log the MDC entry here + MakeDataCountEntry entry = new MakeDataCountEntry(uriInfo, headers, dvRequestService, df); + mdcLogService.logEntry(entry); + } else { + return error(BAD_REQUEST, BundleUtil.getStringFromBundle("access.api.download.failure.guestbookResponseMissing")); + } + } catch (JsonParseException | CommandException ex) { + List args = Arrays.asList(df.getDisplayName(), ex.getLocalizedMessage()); + return error(BAD_REQUEST, BundleUtil.getStringFromBundle("access.api.download.failure.guestbook.commandError", args)); + } + } + } + if (useCustomZipService) { - URI redirect_uri = null; + URI redirect_uri = null; try { - redirect_uri = handleCustomZipDownload(user, customZipServiceUrl, fileIds, uriInfo, headers, donotwriteGBResponse, true); + redirect_uri = handleCustomZipDownload(user, customZipServiceUrl, fileIdParams, uriInfo, headers, donotwriteGBResponse, true); } catch (WebApplicationException wae) { throw wae; } - + Response redirect = Response.seeOther(redirect_uri).build(); logger.fine("Issuing redirect to the file location on S3."); throw new RedirectionException(redirect); } - - // Not using the "custom service" - API will zip the file, + + // Not using the "custom service" - API will zip the file, // and stream the output, in the "normal" manner: - - final boolean getOriginal = getOrig; //to use via anon inner class - + + // to use via anon inner class + final boolean getOriginal = getOrig; + final boolean skipGBResponse = donotwriteGBResponse; // Response may have been written prior and donotwriteGBResponse may have been modified. + StreamingOutput stream = new StreamingOutput() { @Override public void write(OutputStream os) throws IOException, WebApplicationException { - String fileIdParams[] = fileIds.split(","); - DataFileZipper zipper = null; + DataFileZipper zipper = null; String fileManifest = ""; long sizeTotal = 0L; - - if (fileIdParams != null && fileIdParams.length > 0) { - logger.fine(fileIdParams.length + " tokens;"); - for (int i = 0; i < fileIdParams.length; i++) { - logger.fine("token: " + fileIdParams[i]); - Long fileId = null; - try { - fileId = Long.parseLong(fileIdParams[i]); - } catch (NumberFormatException nfe) { - fileId = null; + + for (DataFile file : datafilesMap.values()) { + if (authorizedDatafileIds.contains(file.getId())) { + logger.fine("adding datafile (id=" + file.getId() + ") to the download list of the ZippedDownloadInstance."); + //downloadInstance.addDataFile(file); + if (skipGBResponse != true && file.isReleased()) { + GuestbookResponse gbr = guestbookResponseService.initAPIGuestbookResponse(file.getOwner(), file, session, user); + guestbookResponseService.save(gbr); + MakeDataCountEntry entry = new MakeDataCountEntry(uriInfo, headers, dvRequestService, file); + mdcLogService.logEntry(entry); } - if (fileId != null) { - logger.fine("attempting to look up file id " + fileId); - DataFile file = dataFileService.find(fileId); - if (file != null) { - if (isAccessAuthorized(user, file)) { - - logger.fine("adding datafile (id=" + file.getId() + ") to the download list of the ZippedDownloadInstance."); - //downloadInstance.addDataFile(file); - if (donotwriteGBResponse != true && file.isReleased()){ - GuestbookResponse gbr = guestbookResponseService.initAPIGuestbookResponse(file.getOwner(), file, session, user); - guestbookResponseService.save(gbr); - MakeDataCountEntry entry = new MakeDataCountEntry(uriInfo, headers, dvRequestService, file); - mdcLogService.logEntry(entry); - } - - if (zipper == null) { - // This is the first file we can serve - so we now know that we are going to be able - // to produce some output. - zipper = new DataFileZipper(os); - zipper.setFileManifest(fileManifest); - String bundleName = generateMultiFileBundleName(file.getOwner(), versionTag); - response.setHeader("Content-disposition", "attachment; filename=\"" + bundleName + "\""); - response.setHeader("Content-Type", "application/zip; name=\"" + bundleName + "\""); - } - - long size = 0L; - // is the original format requested, and is this a tabular datafile, with a preserved original? - if (getOriginal - && file.isTabularData() - && !StringUtil.isEmpty(file.getDataTable().getOriginalFileFormat())) { - //This size check is probably fairly inefficient as we have to get all the AccessObjects - //We do this again inside the zipper. I don't think there is a better solution - //without doing a large deal of rewriting or architecture redo. - //The previous size checks for non-original download is still quick. - //-MAD 4.9.2 - // OK, here's the better solution: we now store the size of the original file in - // the database (in DataTable), so we get it for free. - // However, there may still be legacy datatables for which the size is not saved. - // so the "inefficient" code is kept, below, as a fallback solution. - // -- L.A., 4.10 - - if (file.getDataTable().getOriginalFileSize() != null) { - size = file.getDataTable().getOriginalFileSize(); - } else { - DataAccessRequest daReq = new DataAccessRequest(); - StorageIO storageIO = DataAccess.getStorageIO(file, daReq); - storageIO.open(); - size = storageIO.getAuxObjectSize(FileUtil.SAVED_ORIGINAL_FILENAME_EXTENSION); - - // save it permanently: - file.getDataTable().setOriginalFileSize(size); - fileService.saveDataTable(file.getDataTable()); - } - if (size == 0L){ - throw new IOException("Invalid file size or accessObject when checking limits of zip file"); - } - } else { - size = file.getFilesize(); - } - if (sizeTotal + size < zipDownloadSizeLimit) { - sizeTotal += zipper.addFileToZipStream(file, getOriginal); - } else { - String fileName = file.getFileMetadata().getLabel(); - String mimeType = file.getContentType(); - - zipper.addToManifest(fileName + " (" + mimeType + ") " + " skipped because the total size of the download bundle exceeded the limit of " + zipDownloadSizeLimit + " bytes.\r\n"); - } - } else { - boolean embargoed = FileUtil.isActivelyEmbargoed(file); - boolean retentionExpired = FileUtil.isRetentionExpired(file); - if (file.isRestricted() || embargoed || retentionExpired) { - if (zipper == null) { - fileManifest = fileManifest + file.getFileMetadata().getLabel() + " IS " - + (embargoed ? "EMBARGOED" : retentionExpired ? "RETENTIONEXPIRED" : "RESTRICTED") - + " AND CANNOT BE DOWNLOADED\r\n"; - } else { - zipper.addToManifest(file.getFileMetadata().getLabel() + " IS " - + (embargoed ? "EMBARGOED" : retentionExpired ? "RETENTIONEXPIRED" : "RESTRICTED") - + " AND CANNOT BE DOWNLOADED\r\n"); - } - } else { - fileId = null; - } - } - - } if (null == fileId) { - // As of now this errors out. - // This is bad because the user ends up with a broken zip and manifest - // This is good in that the zip ends early so the user does not wait for the results - String errorMessage = "Datafile " + fileId + ": no such object available"; - throw new NotFoundException(errorMessage); + + if (zipper == null) { + // This is the first file we can serve - so we now know that we are going to be able + // to produce some output. + zipper = new DataFileZipper(os); + zipper.setFileManifest(fileManifest); + String bundleName = generateMultiFileBundleName(file.getOwner(), versionTag); + response.setHeader("Content-disposition", "attachment; filename=\"" + bundleName + "\""); + response.setHeader("Content-Type", "application/zip; name=\"" + bundleName + "\""); + } + + long size = 0L; + // is the original format requested, and is this a tabular datafile, with a preserved original? + if (getOriginal + && file.isTabularData() + && !StringUtil.isEmpty(file.getDataTable().getOriginalFileFormat())) { + //This size check is probably fairly inefficient as we have to get all the AccessObjects + //We do this again inside the zipper. I don't think there is a better solution + //without doing a large deal of rewriting or architecture redo. + //The previous size checks for non-original download is still quick. + //-MAD 4.9.2 + // OK, here's the better solution: we now store the size of the original file in + // the database (in DataTable), so we get it for free. + // However, there may still be legacy datatables for which the size is not saved. + // so the "inefficient" code is kept, below, as a fallback solution. + // -- L.A., 4.10 + + if (file.getDataTable().getOriginalFileSize() != null) { + size = file.getDataTable().getOriginalFileSize(); + } else { + DataAccessRequest daReq = new DataAccessRequest(); + StorageIO storageIO = DataAccess.getStorageIO(file, daReq); + storageIO.open(); + size = storageIO.getAuxObjectSize(FileUtil.SAVED_ORIGINAL_FILENAME_EXTENSION); + + // save it permanently: + file.getDataTable().setOriginalFileSize(size); + fileService.saveDataTable(file.getDataTable()); + } + if (size == 0L) { + throw new IOException("Invalid file size or accessObject when checking limits of zip file"); } + } else { + size = file.getFilesize(); + } + if (sizeTotal + size < zipDownloadSizeLimit) { + sizeTotal += zipper.addFileToZipStream(file, getOriginal); + } else { + String fileName = file.getFileMetadata().getLabel(); + String mimeType = file.getContentType(); + + zipper.addToManifest(fileName + " (" + mimeType + ") " + " skipped because the total size of the download bundle exceeded the limit of " + zipDownloadSizeLimit + " bytes.\r\n"); + } + } else { + boolean embargoed = FileUtil.isActivelyEmbargoed(file); + boolean retentionExpired = FileUtil.isRetentionExpired(file); + String manifestEntry = file.getFileMetadata().getLabel() + " IS " + ( + embargoed ? "EMBARGOED" : + retentionExpired ? "RETENTIONEXPIRED" : + file.isRestricted() ? "RESTRICTED" : + "NOTAUTHORIZED") + + " AND CANNOT BE DOWNLOADED\r\n"; + if (zipper == null) { + fileManifest = fileManifest + manifestEntry; + } else { + zipper.addToManifest(manifestEntry); } } - } else { - throw new BadRequestException(); } if (zipper == null) { - // If the DataFileZipper object is still NULL, it means that - // there were file ids supplied - but none of the corresponding - // files were accessible for this user. - // In which casew we don't bother generating any output, and + // If the DataFileZipper object is still NULL, it means that + // there were file ids supplied - but none of the corresponding + // files were accessible for this user. + // In which case we don't bother generating any output, and // just give them a 403: throw new ForbiddenException(); } - // This will add the generated File Manifest to the zipped output, + // This will add the generated File Manifest to the zipped output, // then flush and close the stream: zipper.finalizeZipStream(); - + //os.flush(); //os.close(); } @@ -1336,7 +1513,7 @@ public Response deleteAuxiliaryFileWithVersion(@Context ContainerRequestContext } catch (FileNotFoundException e) { throw new NotFoundException(); } catch(IOException io) { - throw new ServerErrorException("IO Exception trying remove auxiliary file", Response.Status.INTERNAL_SERVER_ERROR, io); + throw new ServerErrorException("IO Exception trying remove auxiliary file", INTERNAL_SERVER_ERROR, io); } return ok("Auxiliary file deleted."); @@ -1394,17 +1571,16 @@ public Response allowAccessRequest(@Context ContainerRequestContext crc, @PathPa * * @param crc * @param fileToRequestAccessId - * @param headers * @return */ @PUT @AuthRequired @Path("/datafile/{id}/requestAccess") - public Response requestFileAccess(@Context ContainerRequestContext crc, @PathParam("id") String fileToRequestAccessId, @Context HttpHeaders headers) { - + public Response requestFileAccess(@Context ContainerRequestContext crc + ,@PathParam("id") String fileToRequestAccessId, String jsonBody) { + DataverseRequest dataverseRequest; DataFile dataFile; - try { dataFile = findDataFileOrDie(fileToRequestAccessId); } catch (WrappedResponse ex) { @@ -1439,8 +1615,21 @@ public Response requestFileAccess(@Context ContainerRequestContext crc, @PathPar } try { - engineSvc.submit(new RequestAccessCommand(dataverseRequest, dataFile, true)); - } catch (CommandException ex) { + // Is Guestbook response required? + // getEffectiveGuestbookEntryAtRequest response will be true (guestbook displays when making a request), false (guestbook displays at download), or will indicate that the dataset inherits one of these settings. + // Even if it is not required we will take it if it's included. Dataset must have a guestbook that is enabled + Dataset ds = dataFile.getOwner(); + GuestbookResponse guestbookResponse = getGuestbookResponseFromBody(dataFile, GuestbookResponse.ACCESS_REQUEST, jsonBody, getRequestUser(crc)); + if (ds.getGuestbook() != null && ds.getGuestbook().isEnabled()) { + if (ds.getEffectiveGuestbookEntryAtRequest() && guestbookResponse == null) { + return error(BAD_REQUEST, BundleUtil.getStringFromBundle("access.api.requestAccess.failure.guestbookAccessRequestResponseMissing")); + } else if (guestbookResponse != null) { + engineSvc.submit(new CreateGuestbookResponseCommand(dvRequestService.getDataverseRequest(), guestbookResponse, guestbookResponse.getDataset())); + } + } + + engineSvc.submit(new RequestAccessCommand(dataverseRequest, dataFile, guestbookResponse, true)); + } catch (CommandException | JsonParseException ex) { List args = Arrays.asList(dataFile.getDisplayName(), ex.getLocalizedMessage()); return error(BAD_REQUEST, BundleUtil.getStringFromBundle("access.api.requestAccess.failure.commandError", args)); } @@ -1489,7 +1678,7 @@ public Response listFileAccessRequests(@Context ContainerRequestContext crc, @Pa if (requests == null || requests.isEmpty()) { List args = Arrays.asList(dataFile.getDisplayName()); - return error(Response.Status.NOT_FOUND, BundleUtil.getStringFromBundle("access.api.requestList.noRequestsFound", args)); + return error(NOT_FOUND, BundleUtil.getStringFromBundle("access.api.requestList.noRequestsFound", args)); } JsonArrayBuilder userArray = Json.createArrayBuilder(); @@ -1730,6 +1919,29 @@ public Response getUserPermissionsOnFile(@Context ContainerRequestContext crc, @ return ok(jsonObjectBuilder); } + private boolean checkGuestbookRequiredResponse(ContainerRequestContext crc, DataFile df) { + // Check if guestbook response is required + // check if this was originally a signed url - since the property only exists if true then a null check should suffice + boolean notSigned = (crc.getProperty("wasSigned") == null); + boolean wasWrittenInPost = true; // TODO: not required if signed and has some flag stating that the guestbook response was already written + + return notSigned && !wasWrittenInPost && df.getOwner().hasEnabledGuestbook(); + } + + private GuestbookResponse getGuestbookResponseFromBody(DataFile dataFile, String type, String jsonBody, User requestor) throws JsonParseException { + GuestbookResponse guestbookResponse = null; + + if (dataFile != null && jsonBody != null && jsonBody.startsWith("{")) { + JsonObject guestbookResponseObj = JsonUtil.getJsonObject(jsonBody).getJsonObject("guestbookResponse"); + guestbookResponse = guestbookResponseService.initAPIGuestbookResponse(dataFile.getOwner(), dataFile, null, requestor); + guestbookResponse.setEventType(type); + // Parse custom question answers and validate them + jsonParser().parseGuestbookResponse(guestbookResponseObj, guestbookResponse); + } + + return guestbookResponse; + } + // checkAuthorization is a convenience method; it calls the boolean method // isAccessAuthorized(), the actual workhorse, and throws a 403 exception if not. private void checkAuthorization(ContainerRequestContext crc, DataFile df) throws WebApplicationException { @@ -1866,12 +2078,11 @@ private boolean isAccessAuthorized(User requestUser, DataFile df) { return false; } - private URI handleCustomZipDownload(User user, String customZipServiceUrl, String fileIds, UriInfo uriInfo, HttpHeaders headers, boolean donotwriteGBResponse, boolean orig) throws WebApplicationException { + private URI handleCustomZipDownload(User user, String customZipServiceUrl, String[] fileIdParams, UriInfo uriInfo, HttpHeaders headers, boolean donotwriteGBResponse, boolean orig) throws WebApplicationException { String zipServiceKey = null; Timestamp timestamp = null; - - String fileIdParams[] = fileIds.split(","); + int validIdCount = 0; int validFileCount = 0; int downloadAuthCount = 0; diff --git a/src/main/java/edu/harvard/iq/dataverse/api/Datasets.java b/src/main/java/edu/harvard/iq/dataverse/api/Datasets.java index b6688a8143b..192cf0849c9 100644 --- a/src/main/java/edu/harvard/iq/dataverse/api/Datasets.java +++ b/src/main/java/edu/harvard/iq/dataverse/api/Datasets.java @@ -18,17 +18,22 @@ import edu.harvard.iq.dataverse.batch.jobs.importer.ImportMode; import edu.harvard.iq.dataverse.dataaccess.*; import edu.harvard.iq.dataverse.datacapturemodule.DataCaptureModuleUtil; -import edu.harvard.iq.dataverse.datasetversionsummaries.DatasetVersionSummary; -import software.amazon.awssdk.services.s3.model.CompletedPart; import edu.harvard.iq.dataverse.datacapturemodule.ScriptRequestResponse; -import edu.harvard.iq.dataverse.dataset.*; +import edu.harvard.iq.dataverse.dataset.DatasetThumbnail; +import edu.harvard.iq.dataverse.dataset.DatasetType; +import edu.harvard.iq.dataverse.dataset.DatasetTypeServiceBean; +import edu.harvard.iq.dataverse.dataset.DatasetUtil; import edu.harvard.iq.dataverse.datasetutility.AddReplaceFileHelper; import edu.harvard.iq.dataverse.datasetutility.DataFileTagException; import edu.harvard.iq.dataverse.datasetutility.NoFilesException; import edu.harvard.iq.dataverse.datasetutility.OptionalFileParams; +import edu.harvard.iq.dataverse.datasetversionsummaries.DatasetVersionSummary; import edu.harvard.iq.dataverse.engine.command.Command; import edu.harvard.iq.dataverse.engine.command.DataverseRequest; -import edu.harvard.iq.dataverse.engine.command.exception.*; +import edu.harvard.iq.dataverse.engine.command.exception.CommandException; +import edu.harvard.iq.dataverse.engine.command.exception.IllegalCommandException; +import edu.harvard.iq.dataverse.engine.command.exception.PermissionException; +import edu.harvard.iq.dataverse.engine.command.exception.UnforcedCommandException; import edu.harvard.iq.dataverse.engine.command.impl.*; import edu.harvard.iq.dataverse.export.ExportService; import edu.harvard.iq.dataverse.externaltools.ExternalTool; @@ -37,6 +42,7 @@ import edu.harvard.iq.dataverse.globus.GlobusUtil; import edu.harvard.iq.dataverse.ingest.IngestServiceBean; import edu.harvard.iq.dataverse.ingest.IngestUtil; +import edu.harvard.iq.dataverse.license.License; import edu.harvard.iq.dataverse.makedatacount.*; import edu.harvard.iq.dataverse.makedatacount.MakeDataCountLoggingServiceBean.MakeDataCountEntry; import edu.harvard.iq.dataverse.metrics.MetricsUtil; @@ -67,8 +73,8 @@ import jakarta.ws.rs.container.ContainerRequestContext; import jakarta.ws.rs.core.*; import jakarta.ws.rs.core.Response.Status; -import org.apache.commons.lang3.exception.ExceptionUtils; import org.apache.commons.lang3.StringUtils; +import org.apache.commons.lang3.exception.ExceptionUtils; import org.apache.logging.log4j.util.Strings; import org.eclipse.microprofile.openapi.annotations.Operation; import org.eclipse.microprofile.openapi.annotations.media.Content; @@ -78,6 +84,7 @@ import org.glassfish.jersey.media.multipart.FormDataBodyPart; import org.glassfish.jersey.media.multipart.FormDataContentDisposition; import org.glassfish.jersey.media.multipart.FormDataParam; +import software.amazon.awssdk.services.s3.model.CompletedPart; import java.io.IOException; import java.io.InputStream; @@ -98,18 +105,11 @@ import java.util.logging.Logger; import java.util.regex.Pattern; import java.util.stream.Collectors; -import static edu.harvard.iq.dataverse.api.ApiConstants.*; - -import edu.harvard.iq.dataverse.dataset.DatasetType; -import edu.harvard.iq.dataverse.dataset.DatasetTypeServiceBean; -import edu.harvard.iq.dataverse.license.License; +import static edu.harvard.iq.dataverse.api.ApiConstants.*; import static edu.harvard.iq.dataverse.util.json.JsonPrinter.*; import static edu.harvard.iq.dataverse.util.json.NullSafeJsonBuilder.jsonObjectBuilder; - -import static jakarta.ws.rs.core.Response.Status.BAD_REQUEST; -import static jakarta.ws.rs.core.Response.Status.NOT_FOUND; -import static jakarta.ws.rs.core.Response.Status.FORBIDDEN; +import static jakarta.ws.rs.core.Response.Status.*; @Path("datasets") public class Datasets extends AbstractApiBean { @@ -128,6 +128,9 @@ public class Datasets extends AbstractApiBean { @EJB GuestbookResponseServiceBean guestbookResponseService; + @EJB + GuestbookServiceBean guestbookService; + @EJB GlobusServiceBean globusService; @@ -5991,6 +5994,54 @@ public Response updateDatasetTypeWithLicenses(@Context ContainerRequestContext c } } + @AuthRequired + @PUT + @Path("{identifier}/guestbook") + public Response updateDatasetGuestbook(@Context ContainerRequestContext crc, @PathParam("identifier") String identifier, String body) { + return response(req -> { + Dataset dataset = findDatasetOrDie(identifier); + Long guestbookId = null; + try { + guestbookId = Long.parseLong(body); + final Guestbook guestbook = guestbookService.find(guestbookId); + if (guestbook == null) { + return error(NOT_FOUND, "Could not find a guestbook with id " + guestbookId); + } + UpdateDatasetGuestbookCommand update_cmd = new UpdateDatasetGuestbookCommand(dataset, guestbook, req); + commandEngine.submit(update_cmd); + } catch (NumberFormatException nfe) { + return error(NOT_FOUND, "Could not find a guestbook with id " + guestbookId); + } catch (CommandException ex) { + logger.log(Level.WARNING, "Failed to update dataset guestbook for dataset " + dataset.getId(), ex); + return error(BAD_REQUEST, "Failed to update dataset guestbook."); + } + return ok("Guestbook " + guestbookId + " set"); + + }, getRequestUser(crc)); + } + + @AuthRequired + @DELETE + @Path("{identifier}/guestbook") + public Response deleteDatasetGuestbook(@Context ContainerRequestContext crc, @PathParam("identifier") String identifier) { + return response(req -> { + Dataset dataset = findDatasetOrDie(identifier); + if (dataset.getGuestbook() != null) { + Long guestbookId = dataset.getGuestbook().getId(); + try { + UpdateDatasetGuestbookCommand update_cmd = new UpdateDatasetGuestbookCommand(dataset, null, req); + commandEngine.submit(update_cmd); + } catch (CommandException ex) { + logger.log(Level.WARNING, "Failed to remove dataset guestbook from dataset " + dataset.getId(), ex); + return error(BAD_REQUEST, "Failed to remove dataset guestbook."); + } + return ok("Guestbook removed " + guestbookId); + } else { + return ok("No Guestbook to remove."); + } + }, getRequestUser(crc)); + } + @PUT @AuthRequired @Path("{id}/deleteFiles") diff --git a/src/main/java/edu/harvard/iq/dataverse/api/Guestbooks.java b/src/main/java/edu/harvard/iq/dataverse/api/Guestbooks.java new file mode 100644 index 00000000000..381f213f54b --- /dev/null +++ b/src/main/java/edu/harvard/iq/dataverse/api/Guestbooks.java @@ -0,0 +1,129 @@ +package edu.harvard.iq.dataverse.api; + +import edu.harvard.iq.dataverse.Dataverse; +import edu.harvard.iq.dataverse.Guestbook; +import edu.harvard.iq.dataverse.GuestbookServiceBean; +import edu.harvard.iq.dataverse.api.auth.AuthRequired; +import edu.harvard.iq.dataverse.authorization.Permission; +import edu.harvard.iq.dataverse.engine.command.impl.CreateGuestbookCommand; +import edu.harvard.iq.dataverse.engine.command.impl.UpdateGuestbookCommand; +import edu.harvard.iq.dataverse.util.BundleUtil; +import edu.harvard.iq.dataverse.util.json.JsonParseException; +import edu.harvard.iq.dataverse.util.json.JsonPrinter; +import edu.harvard.iq.dataverse.util.json.JsonUtil; +import jakarta.ejb.EJB; +import jakarta.json.*; +import jakarta.ws.rs.*; +import jakarta.ws.rs.container.ContainerRequestContext; +import jakarta.ws.rs.core.Context; +import jakarta.ws.rs.core.Response; + +import java.sql.Timestamp; +import java.time.Instant; +import java.util.List; +import java.util.logging.Level; +import java.util.logging.Logger; + +import static edu.harvard.iq.dataverse.util.json.JsonPrinter.json; + +@Path("guestbooks") +public class Guestbooks extends AbstractApiBean { + + private static final Logger logger = Logger.getLogger(Guestbooks.class.getCanonicalName()); + + @EJB + GuestbookServiceBean guestbookService; + + @GET + @AuthRequired + @Path("{id}") + public Response getGuestbook(@Context ContainerRequestContext crc, @PathParam("id") Long id) { + return response( req -> { + final Guestbook retrieved = guestbookService.find(id); + if (retrieved != null) { + final JsonObjectBuilder jsonbuilder = json(retrieved); + return ok(jsonbuilder); + } else { + return notFound(BundleUtil.getStringFromBundle("dataset.manageGuestbooks.message.notFound")); + } + }, getRequestUser(crc)); + } + + @GET + @AuthRequired + @Path("{identifier}/list") + public Response getGuestbooks(@Context ContainerRequestContext crc, @PathParam("identifier") String identifier) { + return response( req -> { + Dataverse dataverse = findDataverseOrDie(identifier); + if (!permissionSvc.request(req).on(dataverse).has(Permission.EditDataverse)) { + return error(Response.Status.FORBIDDEN, "Not authorized"); + } + List guestbooks = guestbookService.findGuestbooksForGivenDataverse(dataverse); + JsonArrayBuilder guestbookArray = Json.createArrayBuilder(); + JsonPrinter jsonPrinter = new JsonPrinter(); + for (Guestbook gb : guestbooks) { + guestbookArray.add(jsonPrinter.json(gb)); + } + return ok(guestbookArray); + }, getRequestUser(crc)); + } + + @POST + @AuthRequired + @Path("{identifier}") + public Response createGuestbook(@Context ContainerRequestContext crc, @PathParam("identifier") String identifier, String jsonBody) { + return response(req -> { + Dataverse dataverse = findDataverseOrDie(identifier); + if (!permissionSvc.request(req).on(dataverse).has(Permission.EditDataverse)) { + return error(Response.Status.FORBIDDEN, "Not authorized"); + } + + Guestbook guestbook = new Guestbook(); + guestbook.setDataverse(dataverse); + try { + JsonObject jsonObj = JsonUtil.getJsonObject(jsonBody); + jsonParser().parseGuestbook(jsonObj, guestbook); + } catch (JsonException | JsonParseException ex) { + logger.log(Level.WARNING, "Error parsing guestbook JSON", ex); + return badRequest("Error parsing guestbook JSON"); + } + guestbook.setCreateTime(Timestamp.from(Instant.now())); + execCommand(new CreateGuestbookCommand(guestbook, req, dataverse)); + return ok("Guestbook " + guestbook.getId() + " created"); + }, getRequestUser(crc)); + } + + @PUT + @AuthRequired + @Path("{identifier}/{id}/enabled") + public Response enableGuestbook(@Context ContainerRequestContext crc, @PathParam("identifier") String identifier, @PathParam("id") String id, String body) { + body = body.trim(); + if (!Util.isBoolean(body)) { + return badRequest("Illegal value '" + body + "'. Use 'true' or 'false'"); + } + Long guestbookId; + try { + guestbookId = Long.parseLong(id); + } catch (NumberFormatException nfe) { + return badRequest("Illegal id '" + id + "'"); + } + boolean enabled = Util.isTrue(body); + return response( req -> { + Dataverse dataverse = findDataverseOrDie(identifier); + if (!permissionSvc.request(req).on(dataverse).has(Permission.EditDataverse)) { + return error(Response.Status.FORBIDDEN, "Not authorized"); + } + List guestbooks = dataverse.getGuestbooks(); + if (guestbooks != null) { + for (Guestbook guestbook : guestbooks) { + if (guestbook.getId() == guestbookId) { // Ignore the fact the enable flag might not change. Just return ok + guestbook.setEnabled(enabled); + execCommand(new UpdateGuestbookCommand(guestbook, req, dataverse)); + return ok("Guestbook " + guestbookId + " enabled=" + enabled); + } + } + } + return notFound("Guestbook " + guestbookId + " not found."); + }, getRequestUser(crc)); + } +} diff --git a/src/main/java/edu/harvard/iq/dataverse/api/auth/SignedUrlAuthMechanism.java b/src/main/java/edu/harvard/iq/dataverse/api/auth/SignedUrlAuthMechanism.java index e701876d5ce..258770da0d4 100644 --- a/src/main/java/edu/harvard/iq/dataverse/api/auth/SignedUrlAuthMechanism.java +++ b/src/main/java/edu/harvard/iq/dataverse/api/auth/SignedUrlAuthMechanism.java @@ -1,10 +1,7 @@ package edu.harvard.iq.dataverse.api.auth; import edu.harvard.iq.dataverse.authorization.AuthenticationServiceBean; -import edu.harvard.iq.dataverse.authorization.users.ApiToken; -import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser; -import edu.harvard.iq.dataverse.authorization.users.PrivateUrlUser; -import edu.harvard.iq.dataverse.authorization.users.User; +import edu.harvard.iq.dataverse.authorization.users.*; import edu.harvard.iq.dataverse.privateurl.PrivateUrl; import edu.harvard.iq.dataverse.privateurl.PrivateUrlServiceBean; import edu.harvard.iq.dataverse.settings.JvmSettings; @@ -56,15 +53,20 @@ private String getSignedUrlRequestParameter(ContainerRequestContext containerReq private User getAuthenticatedUserFromSignedUrl(ContainerRequestContext containerRequestContext) { User user = null; // The signedUrl contains a param telling which user this is supposed to be for. - // We don't trust this. So we lookup that user, and get their API key, and use + // We don't trust this. So we look up that user, and get their API key, and use // that as a secret in validating the signedURL. If the signature can't be - // validated with their key, the user (or their API key) has been changed and + // validated with their key, the user (or their API key) has been changed, and // we reject the request. + // If User is Guest we can return a generic guest user with key made from URI UriInfo uriInfo = containerRequestContext.getUriInfo(); String userId = uriInfo.getQueryParameters().getFirst(SIGNED_URL_USER); - User targetUser = null; + User targetUser = null; ApiToken userApiToken = null; - if (!userId.startsWith(PrivateUrlUser.PREFIX)) { + if (userId.equalsIgnoreCase("guest")) { + targetUser = GuestUser.get(); + userApiToken = new ApiToken(); + userApiToken.setTokenString(uriInfo.getAbsolutePath().toASCIIString()); //TODO find a better one for here and in Access.java + } else if (!userId.startsWith(PrivateUrlUser.PREFIX)) { targetUser = authSvc.getAuthenticatedUser(userId); userApiToken = authSvc.findApiTokenByUser((AuthenticatedUser) targetUser); } else { @@ -92,6 +94,7 @@ private User getAuthenticatedUserFromSignedUrl(ContainerRequestContext container boolean isSignedUrlValid = UrlSignerUtil.isValidUrl(signedUrl, userId, requestMethod, signedUrlSigningKey); if (isSignedUrlValid) { user = targetUser; + containerRequestContext.setProperty("wasSigned", Boolean.TRUE); } } return user; diff --git a/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/UpdateDatasetGuestbookCommand.java b/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/UpdateDatasetGuestbookCommand.java new file mode 100644 index 00000000000..097c5312035 --- /dev/null +++ b/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/UpdateDatasetGuestbookCommand.java @@ -0,0 +1,50 @@ +package edu.harvard.iq.dataverse.engine.command.impl; + +import edu.harvard.iq.dataverse.Dataset; +import edu.harvard.iq.dataverse.Guestbook; +import edu.harvard.iq.dataverse.authorization.Permission; +import edu.harvard.iq.dataverse.engine.command.AbstractCommand; +import edu.harvard.iq.dataverse.engine.command.CommandContext; +import edu.harvard.iq.dataverse.engine.command.DataverseRequest; +import edu.harvard.iq.dataverse.engine.command.RequiredPermissions; +import edu.harvard.iq.dataverse.engine.command.exception.CommandException; +import edu.harvard.iq.dataverse.engine.command.exception.IllegalCommandException; + +import java.util.List; + +@RequiredPermissions(Permission.EditDataset) +public class UpdateDatasetGuestbookCommand extends AbstractCommand { + + private final Dataset dataset; + private final Guestbook guestbook; + + public UpdateDatasetGuestbookCommand(Dataset dataset, Guestbook guestbook, DataverseRequest aRequest) { + super(aRequest, dataset); + this.dataset = dataset; + this.guestbook = guestbook; + } + + @Override + public Dataset execute(CommandContext ctxt) throws CommandException { + Guestbook allowedGuestbook = null; + // if guestbook is null then we are removing it from the dataset + if (guestbook != null) { + // Make sure the requested guestbook is available via the dataset's ancestry + final List guestbooks = dataset.getOwner().getAvailableGuestbooks(); + for (Guestbook gb : guestbooks) { + if (gb.getId() == guestbook.getId()) { + allowedGuestbook = gb; + break; + } + } + + if (allowedGuestbook == null) { + throw new IllegalCommandException("Could not find an available guestbook with id " + guestbook.getId(), this); + } + } + dataset.setGuestbook(allowedGuestbook); + Dataset savedDataset = ctxt.em().merge(dataset); + ctxt.em().flush(); + return savedDataset; + } +} diff --git a/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/UpdateGuestbookCommand.java b/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/UpdateGuestbookCommand.java new file mode 100644 index 00000000000..2138657ee5b --- /dev/null +++ b/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/UpdateGuestbookCommand.java @@ -0,0 +1,26 @@ +package edu.harvard.iq.dataverse.engine.command.impl; + +import edu.harvard.iq.dataverse.Dataverse; +import edu.harvard.iq.dataverse.Guestbook; +import edu.harvard.iq.dataverse.authorization.Permission; +import edu.harvard.iq.dataverse.engine.command.AbstractCommand; +import edu.harvard.iq.dataverse.engine.command.CommandContext; +import edu.harvard.iq.dataverse.engine.command.DataverseRequest; +import edu.harvard.iq.dataverse.engine.command.RequiredPermissions; +import edu.harvard.iq.dataverse.engine.command.exception.CommandException; + +@RequiredPermissions( Permission.EditDataverse ) +public class UpdateGuestbookCommand extends AbstractCommand { + + private final Guestbook guestbook; + + public UpdateGuestbookCommand(Guestbook guestbook, DataverseRequest aRequest, Dataverse anAffectedDataverse) { + super(aRequest, anAffectedDataverse); + this.guestbook = guestbook; + } + + @Override + public Guestbook execute(CommandContext ctxt) throws CommandException { + return ctxt.guestbooks().save(guestbook); + } +} diff --git a/src/main/java/edu/harvard/iq/dataverse/util/UrlSignerUtil.java b/src/main/java/edu/harvard/iq/dataverse/util/UrlSignerUtil.java index 18ea3771301..8d81b9f57b6 100644 --- a/src/main/java/edu/harvard/iq/dataverse/util/UrlSignerUtil.java +++ b/src/main/java/edu/harvard/iq/dataverse/util/UrlSignerUtil.java @@ -1,20 +1,22 @@ package edu.harvard.iq.dataverse.util; +import org.apache.commons.codec.digest.DigestUtils; +import org.apache.http.NameValuePair; +import org.apache.http.client.utils.URIBuilder; +import org.apache.http.client.utils.URLEncodedUtils; +import org.joda.time.LocalDateTime; + import java.net.MalformedURLException; +import java.net.URISyntaxException; import java.net.URL; import java.nio.charset.StandardCharsets; import java.util.List; import java.util.logging.Level; import java.util.logging.Logger; -import org.apache.commons.codec.digest.DigestUtils; -import org.apache.http.NameValuePair; -import org.apache.http.client.utils.URLEncodedUtils; -import org.joda.time.LocalDateTime; - /** * Simple class to sign/validate URLs. - * + * */ public class UrlSignerUtil { @@ -24,8 +26,11 @@ public class UrlSignerUtil { public static final String SIGNED_URL_METHOD="method"; public static final String SIGNED_URL_USER="user"; public static final String SIGNED_URL_UNTIL="until"; + public static final String SIGNED_URL_KEY="key"; // do not propagate the key since it's a credential + public static final String SIGNED_URL_SIGNED="signed"; // we need to remove this when returning a singed url to prevent a loop of signing + public static final List reservedParameters = List.of(SIGNED_URL_UNTIL, SIGNED_URL_USER, SIGNED_URL_METHOD, SIGNED_URL_TOKEN, SIGNED_URL_KEY, SIGNED_URL_SIGNED); /** - * + * * @param baseUrl - the URL to sign - cannot contain query params * "until","user", "method", or "token" * @param timeout - how many minutes to make the URL valid for (note - time skew @@ -39,12 +44,23 @@ public class UrlSignerUtil { * @return - the signed URL */ public static String signUrl(String baseUrl, Integer timeout, String user, String method, String key) { - StringBuilder signedUrlBuilder = new StringBuilder(baseUrl); - boolean firstParam = true; - if (baseUrl.contains("?")) { - firstParam = false; + // check for reserved parameter names ("until","user", "method", or "token") + String[] urlQP = baseUrl.split("\\?"); + if (urlQP.length > 1) { + try { + URIBuilder uriBuilder = new URIBuilder(baseUrl); + List params = uriBuilder.getQueryParams(); + params.removeIf(pair -> reservedParameters.contains(pair.getName())); + uriBuilder.setParameters(params); + baseUrl = uriBuilder.build().toString(); + } catch (URISyntaxException e) { + logger.severe("Invalid URL for signing: " + baseUrl + " " + e.getMessage()); + } } + boolean firstParam = !baseUrl.contains("?"); + StringBuilder signedUrlBuilder = new StringBuilder(baseUrl); + if (timeout != null) { LocalDateTime validTime = LocalDateTime.now(); validTime = validTime.plusMinutes(timeout); diff --git a/src/main/java/edu/harvard/iq/dataverse/util/json/JsonParser.java b/src/main/java/edu/harvard/iq/dataverse/util/json/JsonParser.java index 15cb5d7febf..7de50ae0d0b 100644 --- a/src/main/java/edu/harvard/iq/dataverse/util/json/JsonParser.java +++ b/src/main/java/edu/harvard/iq/dataverse/util/json/JsonParser.java @@ -1,23 +1,7 @@ package edu.harvard.iq.dataverse.util.json; import com.google.gson.Gson; -import edu.harvard.iq.dataverse.ControlledVocabularyValue; -import edu.harvard.iq.dataverse.DataFile; -import edu.harvard.iq.dataverse.DataFileCategory; -import edu.harvard.iq.dataverse.Dataset; -import edu.harvard.iq.dataverse.DatasetField; -import edu.harvard.iq.dataverse.DatasetFieldConstant; -import edu.harvard.iq.dataverse.DatasetFieldCompoundValue; -import edu.harvard.iq.dataverse.DatasetFieldServiceBean; -import edu.harvard.iq.dataverse.DatasetFieldType; -import edu.harvard.iq.dataverse.DatasetFieldValue; -import edu.harvard.iq.dataverse.DatasetVersion; -import edu.harvard.iq.dataverse.Dataverse; -import edu.harvard.iq.dataverse.DataverseContact; -import edu.harvard.iq.dataverse.DataverseTheme; -import edu.harvard.iq.dataverse.FileMetadata; -import edu.harvard.iq.dataverse.MetadataBlockServiceBean; -import edu.harvard.iq.dataverse.TermsOfUseAndAccess; +import edu.harvard.iq.dataverse.*; import edu.harvard.iq.dataverse.api.Util; import edu.harvard.iq.dataverse.api.dto.DataverseDTO; import edu.harvard.iq.dataverse.api.dto.FieldDTO; @@ -35,34 +19,22 @@ import edu.harvard.iq.dataverse.settings.FeatureFlags; import edu.harvard.iq.dataverse.settings.SettingsServiceBean; import edu.harvard.iq.dataverse.util.BundleUtil; +import edu.harvard.iq.dataverse.util.StringUtil; +import edu.harvard.iq.dataverse.validation.EMailValidator; import edu.harvard.iq.dataverse.workflow.Workflow; import edu.harvard.iq.dataverse.workflow.step.WorkflowStepData; +import jakarta.json.*; +import jakarta.json.JsonValue.ValueType; import org.apache.commons.validator.routines.DomainValidator; import java.sql.Timestamp; import java.text.ParseException; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.Date; -import java.util.HashMap; -import java.util.HashSet; -import java.util.LinkedList; -import java.util.List; -import java.util.Map; -import java.util.Optional; -import java.util.Set; -import java.util.function.Function; +import java.util.*; import java.util.function.Consumer; +import java.util.function.Function; import java.util.logging.Logger; import java.util.stream.Collectors; -import jakarta.json.Json; -import jakarta.json.JsonArray; -import jakarta.json.JsonObject; -import jakarta.json.JsonString; -import jakarta.json.JsonValue; -import jakarta.json.JsonValue.ValueType; - /** * Parses JSON objects into domain objects. * @@ -579,6 +551,135 @@ public DatasetVersion parseDatasetVersion(JsonObject obj, DatasetVersion dsv) th } } + public Guestbook parseGuestbook(JsonObject obj, Guestbook gb) throws JsonParseException { + try { + gb.setName(obj.getString("name", null)); + gb.setEnabled(obj.getBoolean("enabled")); + gb.setEmailRequired(obj.getBoolean("emailRequired")); + gb.setNameRequired(obj.getBoolean("nameRequired")); + gb.setInstitutionRequired(obj.getBoolean("institutionRequired")); + gb.setPositionRequired(obj.getBoolean("positionRequired")); + gb.setCustomQuestions(parseCustomQuestions(obj.getJsonArray("customQuestions"), gb)); + + gb.setCreateTime(parseDate(obj.getString("createTime", null))); + } catch (ParseException ex) { + throw new JsonParseException(BundleUtil.getStringFromBundle("jsonparser.error.parsing.date", Arrays.asList(ex.getMessage())) , ex); + } + return gb; + } + private List parseCustomQuestions(JsonArray customQuestions, Guestbook gb) { + final List customQuestionList = (customQuestions != null && !customQuestions.isEmpty()) ? new ArrayList<>() : null; + if (customQuestionList != null) { + customQuestions.forEach(q -> { + JsonObject obj = q.asJsonObject(); + CustomQuestion cq = new CustomQuestion(); + cq.setQuestionString(obj.getString("question")); + cq.setRequired(obj.getBoolean("required")); + cq.setDisplayOrder(obj.getInt("displayOrder")); + cq.setQuestionType(obj.getString("type")); + cq.setHidden(obj.getBoolean("hidden")); + cq.setGuestbook(gb); + + JsonArray optionValues = obj.getJsonArray("optionValues"); + final List cqvList = (optionValues != null && !optionValues.isEmpty()) ? new ArrayList<>() : null; + if (cqvList != null) { + optionValues.forEach(v -> { + JsonObject ov = v.asJsonObject(); + CustomQuestionValue cqv = new CustomQuestionValue(); + cqv.setValueString(ov.getString("value")); + cqv.setDisplayOrder(ov.getInt("displayOrder")); + cqv.setCustomQuestion(cq); + cqvList.add(cqv); + }); + cq.setCustomQuestionValues(cqvList); + } + + customQuestionList.add(cq); + }); + } + return customQuestionList; + } + + public GuestbookResponse parseGuestbookResponse(JsonObject obj, GuestbookResponse guestbookResponse) throws JsonParseException { + + if (obj == null || guestbookResponse == null || guestbookResponse.getGuestbook() == null || guestbookResponse.getGuestbook().getCustomQuestions() == null) { + return null; + } + // overwrite name, email(if valid), institution and position. + guestbookResponse.setName(obj.getString("name", guestbookResponse.getName())); + String email = obj.getString("email", null); + if (email != null) { + email = email.trim(); + if (EMailValidator.isEmailValid(email)) { + guestbookResponse.setEmail(email); + } else { + logger.warning("Ignoring invalid email address in Guestbook response: " + email); + } + } + guestbookResponse.setInstitution(obj.getString("institution", guestbookResponse.getInstitution())); + guestbookResponse.setPosition(obj.getString("position", guestbookResponse.getPosition())); + Guestbook guestbook = guestbookResponse.getGuestbook(); + List missingResponses = new ArrayList<>(); + if (guestbook.isNameRequired() && StringUtil.isEmpty(guestbookResponse.getName())) { + missingResponses.add("Name"); + } + if (guestbook.isEmailRequired() && StringUtil.isEmpty(guestbookResponse.getEmail())) { + missingResponses.add("Email"); + } + if (guestbook.isInstitutionRequired() && StringUtil.isEmpty(guestbookResponse.getInstitution())) { + missingResponses.add("Institution"); + } + if (guestbook.isPositionRequired() && StringUtil.isEmpty(guestbookResponse.getPosition())) { + missingResponses.add("Position"); + } + + Map cqMap = new HashMap<>(); + guestbook.getCustomQuestions().stream().forEach(cq -> cqMap.put(cq.getId(),cq)); + JsonArray answers = obj.getJsonArray("answers"); + List customQuestionResponses = new ArrayList<>(); + for (JsonObject answer : answers.getValuesAs(JsonObject.class)) { + Long cqId = Long.valueOf(answer.getInt("id")); + // find the matching CustomQuestion + CustomQuestion cq = cqMap.get(cqId); + CustomQuestionResponse cqr = new CustomQuestionResponse(); + cqr.setGuestbookResponse(guestbookResponse); + cqr.setCustomQuestion(cq); + String response = null; + if (cq == null) { + throw new JsonParseException(BundleUtil.getStringFromBundle("access.api.requestAccess.failure.guestbookresponseQuestionIdNotFound",List.of(cqId.toString()))); + } else if (cq.getQuestionType().equalsIgnoreCase("textarea")) { + String lineFeed = String.valueOf((char) 10); + JsonArray jsonArray = answer.getJsonArray("value"); + List lines = jsonArray.getValuesAs(JsonString.class); + response = lines.stream().map(JsonString::getString).collect(Collectors.joining(lineFeed)); + } else if (cq.getQuestionType().equalsIgnoreCase("options")) { + String option = answer.getString("value"); + if (!cq.getCustomQuestionOptions().contains(option)) { + throw new JsonParseException(BundleUtil.getStringFromBundle("access.api.requestAccess.failure.guestbookresponseInvalidOption", List.of(option))); + } + response = option; + } else { + response = answer.getString("value"); + } + cqr.setResponse(response); + customQuestionResponses.add(cqr); + cqMap.remove(cqId); // remove so we can check the remaining for missing required questions + } + guestbookResponse.setCustomQuestionResponses(customQuestionResponses); + // verify each required question is in the response + for (Map.Entry e : cqMap.entrySet()) { + if (e.getValue().isRequired()) { + missingResponses.add(e.getValue().getQuestionString()); + } + } + if (!missingResponses.isEmpty()) { + String missing = String.join(",", missingResponses); + throw new JsonParseException(BundleUtil.getStringFromBundle("access.api.requestAccess.failure.guestbookresponseMissingRequired", List.of(missing))); + } + + return guestbookResponse; + } + private edu.harvard.iq.dataverse.license.License parseLicense(String licenseNameOrUri) throws JsonParseException { if (licenseNameOrUri == null){ boolean safeDefaultIfKeyNotFound = true; diff --git a/src/main/java/edu/harvard/iq/dataverse/util/json/JsonPrinter.java b/src/main/java/edu/harvard/iq/dataverse/util/json/JsonPrinter.java index 11a3e7b53d8..895be50b83a 100644 --- a/src/main/java/edu/harvard/iq/dataverse/util/json/JsonPrinter.java +++ b/src/main/java/edu/harvard/iq/dataverse/util/json/JsonPrinter.java @@ -1,18 +1,18 @@ package edu.harvard.iq.dataverse.util.json; import edu.harvard.iq.dataverse.*; -import edu.harvard.iq.dataverse.authorization.DataverseRole; -import edu.harvard.iq.dataverse.authorization.groups.impl.maildomain.MailDomainGroup; -import edu.harvard.iq.dataverse.authorization.providers.builtin.BuiltinUser; import edu.harvard.iq.dataverse.api.Util; +import edu.harvard.iq.dataverse.authorization.DataverseRole; import edu.harvard.iq.dataverse.authorization.Permission; import edu.harvard.iq.dataverse.authorization.RoleAssigneeDisplayInfo; import edu.harvard.iq.dataverse.authorization.groups.impl.explicit.ExplicitGroup; import edu.harvard.iq.dataverse.authorization.groups.impl.ipaddress.IpGroup; import edu.harvard.iq.dataverse.authorization.groups.impl.ipaddress.ip.IpAddress; import edu.harvard.iq.dataverse.authorization.groups.impl.ipaddress.ip.IpAddressRange; +import edu.harvard.iq.dataverse.authorization.groups.impl.maildomain.MailDomainGroup; import edu.harvard.iq.dataverse.authorization.groups.impl.shib.ShibGroup; import edu.harvard.iq.dataverse.authorization.providers.AuthenticationProviderRow; +import edu.harvard.iq.dataverse.authorization.providers.builtin.BuiltinUser; import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser; import edu.harvard.iq.dataverse.authorization.users.User; import edu.harvard.iq.dataverse.branding.BrandingUtil; @@ -21,36 +21,23 @@ import edu.harvard.iq.dataverse.dataset.DatasetType; import edu.harvard.iq.dataverse.dataset.DatasetUtil; import edu.harvard.iq.dataverse.datasetversionsummaries.*; -import edu.harvard.iq.dataverse.datavariable.CategoryMetadata; -import edu.harvard.iq.dataverse.datavariable.DataVariable; -import edu.harvard.iq.dataverse.datavariable.SummaryStatistic; -import edu.harvard.iq.dataverse.datavariable.VarGroup; -import edu.harvard.iq.dataverse.datavariable.VariableCategory; -import edu.harvard.iq.dataverse.datavariable.VariableMetadata; -import edu.harvard.iq.dataverse.datavariable.VariableRange; +import edu.harvard.iq.dataverse.datavariable.*; import edu.harvard.iq.dataverse.dataverse.featured.DataverseFeaturedItem; -import edu.harvard.iq.dataverse.license.License; import edu.harvard.iq.dataverse.globus.FileDetailsHolder; import edu.harvard.iq.dataverse.harvest.client.HarvestingClient; +import edu.harvard.iq.dataverse.license.License; import edu.harvard.iq.dataverse.privateurl.PrivateUrl; import edu.harvard.iq.dataverse.settings.SettingsServiceBean; import edu.harvard.iq.dataverse.util.BundleUtil; import edu.harvard.iq.dataverse.util.DatasetFieldWalker; - -import static edu.harvard.iq.dataverse.util.json.FileVersionDifferenceJsonPrinter.jsonFileVersionDifference; -import static edu.harvard.iq.dataverse.util.json.NullSafeJsonBuilder.jsonObjectBuilder; - import edu.harvard.iq.dataverse.util.MailUtil; import edu.harvard.iq.dataverse.workflow.Workflow; import edu.harvard.iq.dataverse.workflow.step.WorkflowStepData; +import jakarta.ejb.EJB; +import jakarta.ejb.Singleton; +import jakarta.json.*; -import java.io.IOException; import java.util.*; - -import jakarta.json.Json; -import jakarta.json.JsonArrayBuilder; -import jakarta.json.JsonObjectBuilder; - import java.util.function.BiConsumer; import java.util.function.BinaryOperator; import java.util.function.Function; @@ -58,12 +45,10 @@ import java.util.logging.Logger; import java.util.stream.Collector; import java.util.stream.Collectors; -import static java.util.stream.Collectors.toList; -import jakarta.ejb.EJB; -import jakarta.ejb.Singleton; -import jakarta.json.JsonArray; -import jakarta.json.JsonObject; +import static edu.harvard.iq.dataverse.util.json.FileVersionDifferenceJsonPrinter.jsonFileVersionDifference; +import static edu.harvard.iq.dataverse.util.json.NullSafeJsonBuilder.jsonObjectBuilder; +import static java.util.stream.Collectors.toList; /** * Convert objects to Json. @@ -392,6 +377,54 @@ public static JsonObjectBuilder getOwnersFromDvObject(DvObject dvObject){ return getOwnersFromDvObject(dvObject, null); } + public static JsonObjectBuilder json(Guestbook guestbook) { + JsonObjectBuilder guestbookObject = jsonObjectBuilder(); + if (guestbook != null) { + guestbookObject.add("id", guestbook.getId()); + guestbookObject.add("name", guestbook.getName()); + guestbookObject.add("enabled", guestbook.isEnabled()); + guestbookObject.add("emailRequired", guestbook.isEmailRequired()); + guestbookObject.add("nameRequired", guestbook.isNameRequired()); + guestbookObject.add("institutionRequired", guestbook.isInstitutionRequired()); + guestbookObject.add("positionRequired", guestbook.isPositionRequired()); + JsonArrayBuilder customQuestions = Json.createArrayBuilder(); + if (guestbook.getCustomQuestions() != null) { + for (CustomQuestion cq : guestbook.getCustomQuestions()) { + customQuestions.add(json(cq)); + } + } + guestbookObject.add("customQuestions", customQuestions); + if (guestbook.getCreateTime() != null) { + guestbookObject.add("createTime", guestbook.getCreateTime().toString()); + } + if (guestbook.getDataverse() != null) { + guestbookObject.add("dataverseId", guestbook.getDataverse().getId()); + } + } + return guestbookObject; + } + public static JsonObjectBuilder json(CustomQuestion customQuestion) { + JsonObjectBuilder customQuestionObject = jsonObjectBuilder(); + customQuestionObject.add("id", customQuestion.getId()); + customQuestionObject.add("question", customQuestion.getQuestionString()); + customQuestionObject.add("required", customQuestion.isRequired()); + customQuestionObject.add("displayOrder", customQuestion.getDisplayOrder()); + customQuestionObject.add("type", customQuestion.getQuestionType()); + customQuestionObject.add("hidden", customQuestion.isHidden()); + if (customQuestion.getCustomQuestionValues() != null && !customQuestion.getCustomQuestionValues().isEmpty()) { + JsonArrayBuilder customQuestionsValues = Json.createArrayBuilder(); + for (CustomQuestionValue value : customQuestion.getCustomQuestionValues()) { + JsonObjectBuilder customQuestionValueObject = jsonObjectBuilder(); + customQuestionValueObject.add("id", value.getId()); + customQuestionValueObject.add("value", value.getValueString()); + customQuestionValueObject.add("displayOrder", value.getDisplayOrder()); + customQuestionsValues.add(customQuestionValueObject); + } + customQuestionObject.add("optionValues", customQuestionsValues); + } + return customQuestionObject; + } + public static JsonObjectBuilder getOwnersFromDvObject(DvObject dvObject, DatasetVersion dsv) { List ownerList = new ArrayList(); dvObject = dvObject.getOwner(); // We're going to ignore the object itself @@ -477,6 +510,9 @@ public static JsonObjectBuilder json(Dataset ds, Boolean returnOwners) { .add("publisher", BrandingUtil.getInstallationBrandName()) .add("publicationDate", ds.getPublicationDateFormattedYYYYMMDD()) .add("storageIdentifier", ds.getStorageIdentifier()); + if (ds.getGuestbook() != null) { + bld.add("guestbookId", ds.getGuestbook().getId()); + } addDatasetFileCountLimit(ds, bld); if (DvObjectContainer.isMetadataLanguageSet(ds.getMetadataLanguage())) { @@ -554,6 +590,9 @@ public static JsonObjectBuilder json(DatasetVersion dsv, List anonymized .add("publicationDate", dataset.getPublicationDateFormattedYYYYMMDD()) .add("citationDate", dataset.getCitationDateFormattedYYYYMMDD()) .add("versionNote", dsv.getVersionNote()); + if (dataset.getGuestbook() != null) { + bld.add("guestbookId", dataset.getGuestbook().getId()); + } addDatasetFileCountLimit(dataset, bld); License license = DatasetUtil.getLicense(dsv); diff --git a/src/main/java/propertyFiles/Bundle.properties b/src/main/java/propertyFiles/Bundle.properties index 7f4518e65bd..08cb99b9901 100644 --- a/src/main/java/propertyFiles/Bundle.properties +++ b/src/main/java/propertyFiles/Bundle.properties @@ -1462,6 +1462,7 @@ dataset.manageGuestbooks.message.enableSuccess=The guestbook has been enabled. dataset.manageGuestbooks.message.enableFailure=The guestbook could not be enabled. dataset.manageGuestbooks.message.disableSuccess=The guestbook has been disabled. dataset.manageGuestbooks.message.disableFailure=The guestbook could not be disabled. +dataset.manageGuestbooks.message.notFound=The guestbook could not be found. dataset.manageGuestbooks.tip.title=Manage Dataset Guestbooks dataset.manageGuestbooks.tip.downloadascsv=Click \"Download All Responses\" to download all collected guestbook responses for this dataverse, as a CSV file. To navigate and analyze your collected responses, we recommend importing this CSV file into Excel, Google Sheets or similar software. dataset.guestbooksResponses.dataset=Dataset @@ -2734,6 +2735,8 @@ guestbook.save.fail=Guestbook Save Failed guestbook.option.msg= - An Option question requires multiple options. Please complete before saving. guestbook.create=The guestbook has been created. guestbook.save=The guestbook has been edited and saved. +#Guestbook API +guestbook.error.parsing=Error parsing Guestbook data. #Shib.java shib.invalidEmailAddress=The SAML assertion contained an invalid email address: "{0}". @@ -2923,6 +2926,10 @@ access.api.requestAccess.failure.commandError=Problem trying request access on { access.api.requestAccess.failure.requestExists=An access request for this file on your behalf already exists. access.api.requestAccess.failure.invalidRequest=You may not request access to this file. It may already be available to you. access.api.requestAccess.failure.retentionExpired=You may not request access to this file. It is not available because its retention period has ended. +access.api.requestAccess.failure.guestbookAccessRequestResponseMissing=You may not request access to this file without the required Guestbook response. +access.api.requestAccess.failure.guestbookresponseMissingRequired=Guestbook Response entry is required but not present ({0}). +access.api.requestAccess.failure.guestbookresponseInvalidOption=Guestbook Custom Question Answer not a valid option ({0}). +access.api.requestAccess.failure.guestbookresponseQuestionIdNotFound=Guestbook Custom Question ID {0} not found. access.api.requestAccess.noKey=You must provide a key to request access to a file. access.api.requestAccess.fileNotFound=Could not find datafile with id {0}. @@ -2944,6 +2951,9 @@ access.api.exception.metadata.not.available.for.nontabular.file=This type of met access.api.exception.metadata.restricted.no.permission=You do not have permission to download this file. access.api.exception.version.not.found=Could not find requested dataset version. access.api.exception.dataset.not.found=Could not find requested dataset. +access.api.download.failure.guestbookResponseMissing=You may not download this file without the required Guestbook response. +access.api.download.failure.guestbook.commandError=Problem trying download with guestbook response on {0} : {1} +access.api.download.failure.multipleDatasets=All files being downloaded must be from the same Dataset. #permission permission.AddDataverse.label=AddDataverse diff --git a/src/test/java/edu/harvard/iq/dataverse/api/AccessIT.java b/src/test/java/edu/harvard/iq/dataverse/api/AccessIT.java index dd8ddd2d315..f4cb2cc6b12 100644 --- a/src/test/java/edu/harvard/iq/dataverse/api/AccessIT.java +++ b/src/test/java/edu/harvard/iq/dataverse/api/AccessIT.java @@ -5,31 +5,32 @@ */ package edu.harvard.iq.dataverse.api; +import edu.harvard.iq.dataverse.DataFile; +import edu.harvard.iq.dataverse.Guestbook; +import edu.harvard.iq.dataverse.util.BundleUtil; +import edu.harvard.iq.dataverse.util.FileUtil; +import edu.harvard.iq.dataverse.util.json.JsonParseException; import io.restassured.RestAssured; import io.restassured.path.json.JsonPath; import io.restassured.response.Response; -import edu.harvard.iq.dataverse.DataFile; -import edu.harvard.iq.dataverse.util.FileUtil; -import java.io.IOException; -import java.util.zip.ZipInputStream; - -import jakarta.json.Json; +import org.hamcrest.collection.IsMapContaining; import org.junit.jupiter.api.AfterAll; import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Disabled; import org.junit.jupiter.api.Test; -import java.util.zip.ZipEntry; + import java.io.ByteArrayOutputStream; +import java.io.IOException; import java.io.InputStream; import java.util.HashMap; import java.util.regex.Matcher; import java.util.regex.Pattern; - -import org.hamcrest.collection.IsMapContaining; +import java.util.zip.ZipEntry; +import java.util.zip.ZipInputStream; import static jakarta.ws.rs.core.Response.Status.*; -import static org.hamcrest.MatcherAssert.*; -import static org.hamcrest.CoreMatchers.is; -import static org.hamcrest.CoreMatchers.not; +import static org.hamcrest.CoreMatchers.*; +import static org.hamcrest.MatcherAssert.assertThat; import static org.junit.jupiter.api.Assertions.*; /** @@ -158,6 +159,7 @@ public static void setUp() throws InterruptedException { tabFile4NameUnpublishedConvert = tabFile4NameUnpublished.substring(0, tabFile4NameUnpublished.indexOf(".dta")) + ".tab"; String tab4PathToFile = "scripts/search/data/tabular/" + tabFile4NameUnpublished; Response tab4AddResponse = UtilIT.uploadFileViaNative(datasetId.toString(), tab4PathToFile, apiToken); + tab4AddResponse.prettyPrint(); tabFile4IdUnpublished = JsonPath.from(tab4AddResponse.body().asString()).getInt("data.files[0].dataFile.id"); assertTrue(UtilIT.sleepForLock(datasetId.longValue(), "Ingest", apiToken, UtilIT.MAXIMUM_INGEST_LOCK_DURATION), "Failed test if Ingest Lock exceeds max duration " + tabFile2Name); @@ -411,18 +413,23 @@ public void testDownloadMultipleFiles_LoggedAndNot_Unpublished() throws IOExcept HashMap files2 = readZipResponse(authDownloadConvertedUnpublished.getBody().asInputStream()); assertEquals(4, files2.size()); //size +1 for manifest, we have access to unpublished + // Guest User can not access tabFile4IdUnpublished so only the first 2 files will be downloaded Response anonDownloadOriginalUnpublished = UtilIT.downloadFilesOriginal(new Integer[]{basicFileId,tabFile1Id,tabFile4IdUnpublished}); - assertEquals(404, anonDownloadOriginalUnpublished.getStatusCode()); + assertEquals(200, anonDownloadOriginalUnpublished.getStatusCode()); int origAnonSize = anonDownloadOriginalUnpublished.getBody().asByteArray().length; HashMap files3 = readZipResponse(anonDownloadOriginalUnpublished.getBody().asInputStream()); - assertEquals(0, files3.size()); //A size of 0 indicates the zip creation was interrupted. + // expect the zip to have 3 files: 2 downloaded files plus the manifest + assertEquals(3, files3.size()); + assertTrue(files3.containsKey("120745.dta")); assertTrue(origAnonSize < origAuthSize + margin); Response anonDownloadConvertedUnpublished = UtilIT.downloadFiles(new Integer[]{basicFileId,tabFile1Id,tabFile4IdUnpublished}); - assertEquals(404, anonDownloadConvertedUnpublished.getStatusCode()); + assertEquals(200, anonDownloadConvertedUnpublished.getStatusCode()); int convertAnonSize = anonDownloadConvertedUnpublished.getBody().asByteArray().length; HashMap files4 = readZipResponse(anonDownloadConvertedUnpublished.getBody().asInputStream()); - assertEquals(0, files4.size()); //A size of 0 indicates the zip creation was interrupted. + // expect the zip to have 3 files: 2 downloaded files plus the manifest + assertEquals(3, files4.size()); + assertTrue(files4.containsKey("120745.tab")); assertTrue(convertAnonSize < convertAuthSize + margin); } @@ -487,15 +494,15 @@ private HashMap readZipResponse(InputStream iStrea return fileStreams; } - + @Test public void testRequestAccess() throws InterruptedException { - + String pathToJsonFile = "scripts/api/data/dataset-create-new.json"; Response createDatasetResponse = UtilIT.createDatasetViaNativeApi(dataverseAlias, pathToJsonFile, apiToken); createDatasetResponse.prettyPrint(); Integer datasetIdNew = JsonPath.from(createDatasetResponse.body().asString()).getInt("data.id"); - + basicFileName = "004.txt"; String basicPathToFile = "scripts/search/data/replace_test/" + basicFileName; Response basicAddResponse = UtilIT.uploadFileViaNative(datasetIdNew.toString(), basicPathToFile, apiToken); @@ -507,7 +514,7 @@ public void testRequestAccess() throws InterruptedException { Integer tabFile3IdRestrictedNew = JsonPath.from(tab3AddResponse.body().asString()).getInt("data.files[0].dataFile.id"); assertTrue(UtilIT.sleepForLock(datasetIdNew.longValue(), "Ingest", apiToken, UtilIT.MAXIMUM_INGEST_LOCK_DURATION), "Failed test if Ingest Lock exceeds max duration " + tab3PathToFile); - + Response restrictResponse = UtilIT.restrictFile(tabFile3IdRestrictedNew.toString(), true, apiToken); restrictResponse.prettyPrint(); restrictResponse.then().assertThat() @@ -551,21 +558,127 @@ public void testRequestAccess() throws InterruptedException { //grant file access Response grantFileAccessResponse = UtilIT.grantFileAccess(tabFile3IdRestrictedNew.toString(), "@" + apiIdentifierRando, apiToken); assertEquals(200, grantFileAccessResponse.getStatusCode()); - + //if you make a request while you have been granted access you should get a command exception requestFileAccessResponse = UtilIT.requestFileAccess(tabFile3IdRestrictedNew.toString(), apiTokenRando); assertEquals(400, requestFileAccessResponse.getStatusCode()); - + + //if you make a request of a public file you should also get a command exception + requestFileAccessResponse = UtilIT.requestFileAccess(basicFileIdNew.toString(), apiTokenRando); + assertEquals(400, requestFileAccessResponse.getStatusCode()); + + + //Now should be able to download + randoDownload = UtilIT.downloadFile(tabFile3IdRestrictedNew, apiTokenRando); + assertEquals(OK.getStatusCode(), randoDownload.getStatusCode()); + + //revokeFileAccess + Response revokeFileAccessResponse = UtilIT.revokeFileAccess(tabFile3IdRestrictedNew.toString(), "@" + apiIdentifierRando, apiToken); + assertEquals(200, revokeFileAccessResponse.getStatusCode()); + + listAccessRequestResponse = UtilIT.getAccessRequestList(tabFile3IdRestrictedNew.toString(), apiToken); + assertEquals(404, listAccessRequestResponse.getStatusCode()); + } + + @Test + @Disabled // Only run manually after setting JVM setting -Ddataverse.files.guestbook-at-request=true + public void testRequestAccessWithGuestbook() throws IOException, JsonParseException { + + String pathToJsonFile = "scripts/api/data/dataset-create-new.json"; + Response createDatasetResponse = UtilIT.createDatasetViaNativeApi(dataverseAlias, pathToJsonFile, apiToken); + createDatasetResponse.prettyPrint(); + Integer datasetIdNew = JsonPath.from(createDatasetResponse.body().asString()).getInt("data.id"); + String persistentIdNew = JsonPath.from(createDatasetResponse.body().asString()).getString("data.persistentId"); + + // Create a Guestbook + Guestbook guestbook = UtilIT.createRandomGuestbook(dataverseAlias, persistentId, apiToken); + String guestbookResponseJson = UtilIT.generateGuestbookResponse(guestbook); + + basicFileName = "004.txt"; + String basicPathToFile = "scripts/search/data/replace_test/" + basicFileName; + Response basicAddResponse = UtilIT.uploadFileViaNative(datasetIdNew.toString(), basicPathToFile, apiToken); + Integer basicFileIdNew = JsonPath.from(basicAddResponse.body().asString()).getInt("data.files[0].dataFile.id"); + + String tabFile3NameRestrictedNew = "stata13-auto-withstrls.dta"; + String tab3PathToFile = "scripts/search/data/tabular/" + tabFile3NameRestrictedNew; + Response tab3AddResponse = UtilIT.uploadFileViaNative(datasetIdNew.toString(), tab3PathToFile, apiToken); + Integer tabFile3IdRestrictedNew = JsonPath.from(tab3AddResponse.body().asString()).getInt("data.files[0].dataFile.id"); + + assertTrue(UtilIT.sleepForLock(datasetIdNew.longValue(), "Ingest", apiToken, UtilIT.MAXIMUM_INGEST_LOCK_DURATION), "Failed test if Ingest Lock exceeds max duration " + tab3PathToFile); + + Response restrictResponse = UtilIT.restrictFile(tabFile3IdRestrictedNew.toString(), true, apiToken); + restrictResponse.prettyPrint(); + restrictResponse.then().assertThat() + .statusCode(OK.getStatusCode()); + + Response createUser = UtilIT.createRandomUser(); + createUser.prettyPrint(); + assertEquals(200, createUser.getStatusCode()); + String apiTokenRando = UtilIT.getApiTokenFromResponse(createUser); + String apiIdentifierRando = UtilIT.getUsernameFromResponse(createUser); + + Response randoDownload = UtilIT.downloadFile(tabFile3IdRestrictedNew, apiTokenRando); + assertEquals(403, randoDownload.getStatusCode()); + + Response requestFileAccessResponse = UtilIT.requestFileAccess(tabFile3IdRestrictedNew.toString(), apiTokenRando); + //Cannot request until we set the dataset to allow requests + assertEquals(400, requestFileAccessResponse.getStatusCode()); + //Update Dataset to allow requests + Response allowAccessRequestsResponse = UtilIT.allowAccessRequests(datasetIdNew.toString(), true, apiToken); + assertEquals(200, allowAccessRequestsResponse.getStatusCode()); + //Must republish to get it to work + Response publishDataset = UtilIT.publishDatasetViaNativeApi(datasetIdNew, "major", apiToken); + assertEquals(200, publishDataset.getStatusCode()); + + // Set the guestbook on the Dataset + UtilIT.updateDatasetGuestbook(persistentIdNew, guestbook.getId(), apiToken).prettyPrint(); + // Set the response required on the Access Request as apposed to being on Download + UtilIT.setGuestbookEntryOnRequest(datasetId.toString(), apiToken, Boolean.TRUE).prettyPrint(); + // Request file access WITHOUT the required Guestbook Response (getEffectiveGuestbookEntryAtRequest) + requestFileAccessResponse = UtilIT.requestFileAccess(tabFile3IdRestrictedNew.toString(), apiTokenRando); + requestFileAccessResponse.prettyPrint(); + assertEquals(400, requestFileAccessResponse.getStatusCode()); + // Request file access with the required Guestbook Response (getEffectiveGuestbookEntryAtRequest) + requestFileAccessResponse = UtilIT.requestFileAccess(tabFile3IdRestrictedNew.toString(), apiTokenRando, guestbookResponseJson); + requestFileAccessResponse.prettyPrint(); + assertEquals(200, requestFileAccessResponse.getStatusCode()); + // Request a second time should fail since the request was already made + requestFileAccessResponse = UtilIT.requestFileAccess(tabFile3IdRestrictedNew.toString(), apiTokenRando, guestbookResponseJson); + requestFileAccessResponse.prettyPrint(); + requestFileAccessResponse.then().assertThat() + .statusCode(BAD_REQUEST.getStatusCode()) + .body("message", equalTo(BundleUtil.getStringFromBundle("access.api.requestAccess.failure.requestExists"))); + + Response listAccessRequestResponse = UtilIT.getAccessRequestList(tabFile3IdRestrictedNew.toString(), apiToken); + listAccessRequestResponse.prettyPrint(); + assertEquals(200, listAccessRequestResponse.getStatusCode()); + System.out.println("List Access Request: " + listAccessRequestResponse.prettyPrint()); + + listAccessRequestResponse = UtilIT.getAccessRequestList(tabFile3IdRestrictedNew.toString(), apiTokenRando); + listAccessRequestResponse.prettyPrint(); + assertEquals(403, listAccessRequestResponse.getStatusCode()); + + Response rejectFileAccessResponse = UtilIT.rejectFileAccessRequest(tabFile3IdRestrictedNew.toString(), "@" + apiIdentifierRando, apiToken); + assertEquals(200, rejectFileAccessResponse.getStatusCode()); + + requestFileAccessResponse = UtilIT.requestFileAccess(tabFile3IdRestrictedNew.toString(), apiTokenRando); + //grant file access + Response grantFileAccessResponse = UtilIT.grantFileAccess(tabFile3IdRestrictedNew.toString(), "@" + apiIdentifierRando, apiToken); + assertEquals(200, grantFileAccessResponse.getStatusCode()); + + //if you make a request while you have been granted access you should get a command exception + requestFileAccessResponse = UtilIT.requestFileAccess(tabFile3IdRestrictedNew.toString(), apiTokenRando); + assertEquals(400, requestFileAccessResponse.getStatusCode()); + //if you make a request of a public file you should also get a command exception requestFileAccessResponse = UtilIT.requestFileAccess(basicFileIdNew.toString(), apiTokenRando); assertEquals(400, requestFileAccessResponse.getStatusCode()); - //Now should be able to download randoDownload = UtilIT.downloadFile(tabFile3IdRestrictedNew, apiTokenRando); assertEquals(OK.getStatusCode(), randoDownload.getStatusCode()); - //revokeFileAccess + //revokeFileAccess Response revokeFileAccessResponse = UtilIT.revokeFileAccess(tabFile3IdRestrictedNew.toString(), "@" + apiIdentifierRando, apiToken); assertEquals(200, revokeFileAccessResponse.getStatusCode()); diff --git a/src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java b/src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java index 0e45e38b1b5..78fe28ebc30 100644 --- a/src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java +++ b/src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java @@ -12,7 +12,10 @@ import edu.harvard.iq.dataverse.settings.SettingsServiceBean; import edu.harvard.iq.dataverse.util.BundleUtil; import edu.harvard.iq.dataverse.util.SystemConfig; -import edu.harvard.iq.dataverse.util.json.*; +import edu.harvard.iq.dataverse.util.json.JSONLDUtil; +import edu.harvard.iq.dataverse.util.json.JsonParseException; +import edu.harvard.iq.dataverse.util.json.JsonParser; +import edu.harvard.iq.dataverse.util.json.JsonUtil; import edu.harvard.iq.dataverse.util.xml.XmlUtil; import io.restassured.RestAssured; import io.restassured.http.ContentType; @@ -20,12 +23,7 @@ import io.restassured.path.json.JsonPath; import io.restassured.path.xml.XmlPath; import io.restassured.response.Response; -import jakarta.json.Json; -import jakarta.json.JsonArray; -import jakarta.json.JsonObject; -import jakarta.json.JsonObjectBuilder; -import jakarta.json.JsonValue; -import jakarta.json.JsonArrayBuilder; +import jakarta.json.*; import jakarta.ws.rs.core.Response.Status; import org.apache.commons.lang3.RandomStringUtils; import org.apache.commons.lang3.StringUtils; @@ -45,6 +43,7 @@ import java.nio.file.Path; import java.nio.file.Paths; import java.time.LocalDate; +import java.time.Year; import java.time.format.DateTimeFormatter; import java.util.*; import java.util.logging.Logger; @@ -57,7 +56,6 @@ import static io.restassured.path.json.JsonPath.with; import static jakarta.ws.rs.core.Response.Status.*; import static java.lang.Thread.sleep; -import java.time.Year; import static org.hamcrest.CoreMatchers.*; import static org.hamcrest.Matchers.contains; import static org.junit.jupiter.api.Assertions.*; @@ -7427,6 +7425,109 @@ public void testExcludeEmailOverride() { assertTrue(!json.contains("datasetContactEmail")); } + @Test + public void testGetDatasetWithTermsOfUseAndGuestbook() throws IOException, JsonParseException { + String apiToken = getSuperuserToken(); + Response createDataverseResponse = UtilIT.createRandomDataverse(apiToken); + String ownerAlias = UtilIT.getAliasFromResponse(createDataverseResponse); + + Response createDatasetResponse = UtilIT.createRandomDatasetViaNativeApi(ownerAlias, apiToken); + createDatasetResponse.prettyPrint(); + String persistentId = UtilIT.getDatasetPersistentIdFromResponse(createDatasetResponse); + Integer datasetId = UtilIT.getDatasetIdFromResponse(createDatasetResponse); + + // Create a Guestbook + Guestbook guestbook = UtilIT.createRandomGuestbook(ownerAlias, persistentId, apiToken); + + // Create a license for Terms of Use + String jsonString = """ + { + "customTerms": { + "termsOfUse": "testTermsOfUse" + } + } + """; + Response updateLicenseResponse = UtilIT.updateLicense(datasetId.toString(), jsonString, apiToken); + updateLicenseResponse.then().assertThat() + .statusCode(OK.getStatusCode()) + .body("data.message", equalTo(BundleUtil.getStringFromBundle("datasets.api.updateLicense.success"))); + + // Enable the Guestbook with invalid enable flag + Response guestbookEnableResponse = UtilIT.enableGuestbook(ownerAlias, guestbook.getId(), apiToken, "x"); + guestbookEnableResponse.prettyPrint(); + guestbookEnableResponse.then().assertThat() + .statusCode(BAD_REQUEST.getStatusCode()) + .body("message", startsWith("Illegal value")); + + Response getDataset = UtilIT.getDatasetVersions(persistentId, apiToken); + getDataset.prettyPrint(); + getDataset.then().assertThat() + .statusCode(OK.getStatusCode()) + .body("data[0].termsOfUse", equalTo("testTermsOfUse")) + .body("data[0].guestbookId", equalTo(guestbook.getId().intValue())); + + getDataset = UtilIT.nativeGet(datasetId, apiToken); + getDataset.prettyPrint(); + getDataset.then().assertThat() + .statusCode(OK.getStatusCode()) + .body("data.guestbookId", equalTo(guestbook.getId().intValue())); + + Response getGuestbook = UtilIT.getGuestbook(guestbook.getId(), apiToken); + getGuestbook.prettyPrint(); + getGuestbook.then().assertThat() + .statusCode(OK.getStatusCode()) + .body("data.id", equalTo(guestbook.getId().intValue())); + + getGuestbook = UtilIT.getGuestbook(-1L, apiToken); + getGuestbook.prettyPrint(); + getGuestbook.then().assertThat().statusCode(NOT_FOUND.getStatusCode()); + + // remove the guestbook from the dataset + Response removeGuestbook = UtilIT.updateDatasetGuestbook(persistentId, null, apiToken); + removeGuestbook.prettyPrint(); + removeGuestbook.then().assertThat() + .statusCode(OK.getStatusCode()) + .body("data.message", startsWith("Guestbook removed")); + // remove the already removed guestbook from the dataset + removeGuestbook = UtilIT.updateDatasetGuestbook(persistentId, null, apiToken); + removeGuestbook.prettyPrint(); + removeGuestbook.then().assertThat() + .statusCode(OK.getStatusCode()) + .body("data.message", startsWith("No Guestbook to remove")); + + // Get the dataset to show that the guestbook was removed + getDataset = UtilIT.nativeGet(datasetId, apiToken); + getDataset.prettyPrint(); + getDataset.then().assertThat() + .statusCode(OK.getStatusCode()) + .body("data.guestbookId", equalTo(null)); + + // Disable the Guestbook + guestbookEnableResponse = UtilIT.enableGuestbook(ownerAlias, guestbook.getId(), apiToken, Boolean.FALSE.toString()); + guestbookEnableResponse.prettyPrint(); + guestbookEnableResponse.then().assertThat() + .statusCode(OK.getStatusCode()) + .body("data.message", startsWith("Guestbook")); + + // Fail to add a disabled Guestbook to the Dataset + Response setGuestbook = UtilIT.updateDatasetGuestbook(persistentId, guestbook.getId(), apiToken); + setGuestbook.prettyPrint(); + setGuestbook.then().assertThat() + .statusCode(BAD_REQUEST.getStatusCode()) + .body("message", startsWith("Failed to update dataset guestbook")); + + // Enable the Guestbook. Add it to the Dataset. Then disable it. + // Show that the guestbook is still returned in the dataset Json even if it's disabled + UtilIT.enableGuestbook(ownerAlias, guestbook.getId(), apiToken, Boolean.TRUE.toString()).prettyPrint(); + UtilIT.updateDatasetGuestbook(persistentId, guestbook.getId(), apiToken).prettyPrint(); + UtilIT.enableGuestbook(ownerAlias, guestbook.getId(), apiToken, Boolean.FALSE.toString()).prettyPrint(); + getDataset = UtilIT.nativeGet(datasetId, apiToken); + getDataset.prettyPrint(); + getDataset.then().assertThat() + .statusCode(OK.getStatusCode()) + .body("data.guestbookId", equalTo(guestbook.getId().intValue())); + } + private String getSuperuserToken() { Response createResponse = UtilIT.createRandomUser(); String adminApiToken = UtilIT.getApiTokenFromResponse(createResponse); diff --git a/src/test/java/edu/harvard/iq/dataverse/api/FilesIT.java b/src/test/java/edu/harvard/iq/dataverse/api/FilesIT.java index 262f3252f9d..26ee900e91b 100644 --- a/src/test/java/edu/harvard/iq/dataverse/api/FilesIT.java +++ b/src/test/java/edu/harvard/iq/dataverse/api/FilesIT.java @@ -1,60 +1,59 @@ package edu.harvard.iq.dataverse.api; import edu.harvard.iq.dataverse.Dataverse; +import edu.harvard.iq.dataverse.Guestbook; +import edu.harvard.iq.dataverse.api.auth.ApiKeyAuthMechanism; import edu.harvard.iq.dataverse.authorization.DataverseRole; import edu.harvard.iq.dataverse.datasetutility.OptionalFileParams; +import edu.harvard.iq.dataverse.settings.SettingsServiceBean; +import edu.harvard.iq.dataverse.util.BundleUtil; +import edu.harvard.iq.dataverse.util.FileUtil; +import edu.harvard.iq.dataverse.util.SystemConfig; import edu.harvard.iq.dataverse.util.json.JsonParseException; import edu.harvard.iq.dataverse.util.json.JsonParser; import edu.harvard.iq.dataverse.util.json.JsonUtil; import io.restassured.RestAssured; +import io.restassured.path.json.JsonPath; +import io.restassured.path.xml.XmlPath; import io.restassured.response.Response; - -import java.nio.charset.StandardCharsets; -import java.util.*; -import java.util.logging.Logger; - -import edu.harvard.iq.dataverse.api.auth.ApiKeyAuthMechanism; +import jakarta.json.Json; import jakarta.json.JsonObject; +import jakarta.json.JsonObjectBuilder; import jakarta.ws.rs.core.Response.Status; import org.assertj.core.util.Lists; +import org.hamcrest.CoreMatchers; import org.hamcrest.Matcher; +import org.hamcrest.Matchers; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Nested; import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.BeforeAll; -import io.restassured.path.json.JsonPath; +import org.junit.jupiter.api.io.TempDir; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.Arguments; +import org.junit.jupiter.params.provider.MethodSource; -import static edu.harvard.iq.dataverse.api.ApiConstants.*; -import static edu.harvard.iq.dataverse.settings.SettingsServiceBean.Key; -import static io.restassured.path.json.JsonPath.with; -import io.restassured.path.xml.XmlPath; -import edu.harvard.iq.dataverse.settings.SettingsServiceBean; -import edu.harvard.iq.dataverse.util.BundleUtil; -import edu.harvard.iq.dataverse.util.FileUtil; -import edu.harvard.iq.dataverse.util.SystemConfig; import java.io.File; import java.io.IOException; - -import static java.lang.Thread.sleep; - +import java.nio.charset.StandardCharsets; import java.nio.file.Path; import java.nio.file.Paths; -import java.text.MessageFormat; - -import jakarta.json.Json; -import jakarta.json.JsonObjectBuilder; - -import static jakarta.ws.rs.core.Response.Status.*; import java.security.MessageDigest; import java.security.NoSuchAlgorithmException; +import java.text.MessageFormat; import java.time.Year; -import org.hamcrest.CoreMatchers; -import org.hamcrest.Matchers; -import org.junit.jupiter.api.AfterAll; -import org.junit.jupiter.api.io.TempDir; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.Arguments; -import org.junit.jupiter.params.provider.MethodSource; +import java.util.Arrays; +import java.util.Collections; +import java.util.List; +import java.util.Map; +import java.util.logging.Logger; +import static edu.harvard.iq.dataverse.api.ApiConstants.*; +import static edu.harvard.iq.dataverse.settings.SettingsServiceBean.Key; +import static io.restassured.RestAssured.get; +import static io.restassured.path.json.JsonPath.with; +import static jakarta.ws.rs.core.Response.Status.*; +import static java.lang.Thread.sleep; import static org.hamcrest.CoreMatchers.*; import static org.junit.jupiter.api.Assertions.*; @@ -3870,4 +3869,236 @@ public void testUpdateWithEmptyFieldsAndVersionCheck() throws InterruptedExcepti .body("message", equalTo(BundleUtil.getStringFromBundle("jsonparser.error.parsing.date",Collections.singletonList("bad-date")))) .statusCode(BAD_REQUEST.getStatusCode()); } + + @Test + public void testDownloadFileWithGuestbookResponse() throws IOException, JsonParseException { + msgt("testDownloadFileWithGuestbookResponse"); + // Create superuser + Response createUserResponse = UtilIT.createRandomUser(); + String ownerApiToken = UtilIT.getApiTokenFromResponse(createUserResponse); + String superusername = UtilIT.getUsernameFromResponse(createUserResponse); + UtilIT.makeSuperUser(superusername).then().assertThat().statusCode(200); + + // Create Dataverse + String dataverseAlias = createDataverseGetAlias(ownerApiToken); + + // Create user with no permission + createUserResponse = UtilIT.createRandomUser(); + assertEquals(200, createUserResponse.getStatusCode()); + String apiToken = UtilIT.getApiTokenFromResponse(createUserResponse); + String username = UtilIT.getUsernameFromResponse(createUserResponse); + + // Create second user with no permission + createUserResponse = UtilIT.createRandomUser(); + createUserResponse.prettyPrint(); + assertEquals(200, createUserResponse.getStatusCode()); + String apiToken2 = UtilIT.getApiTokenFromResponse(createUserResponse); + String username2 = UtilIT.getUsernameFromResponse(createUserResponse); + String user2Email = JsonPath.from(createUserResponse.body().asString()).getString("data.authenticatedUser.email"); + + // Create Dataset + Response createDatasetResponse = UtilIT.createRandomDatasetViaNativeApi(dataverseAlias, ownerApiToken); + createDatasetResponse.then().assertThat().statusCode(CREATED.getStatusCode()); + Integer datasetId = JsonPath.from(createDatasetResponse.body().asString()).getInt("data.id"); + String persistentId = JsonPath.from(createDatasetResponse.body().asString()).getString("data.persistentId"); + Response getDatasetMetadata = UtilIT.nativeGet(datasetId, ownerApiToken); + getDatasetMetadata.then().assertThat().statusCode(200); + + Response getGuestbooksResponse = UtilIT.getGuestbooks(dataverseAlias, ownerApiToken); + getGuestbooksResponse.then().assertThat().statusCode(200); + assertTrue(getGuestbooksResponse.getBody().jsonPath().getList("data").isEmpty()); + + // Create a Guestbook + Guestbook guestbook = UtilIT.createRandomGuestbook(dataverseAlias, persistentId, ownerApiToken); + + // Get the list of Guestbooks + getGuestbooksResponse = UtilIT.getGuestbooks(dataverseAlias, ownerApiToken); + getGuestbooksResponse.then().assertThat().statusCode(200); + assertEquals(1, getGuestbooksResponse.getBody().jsonPath().getList("data").size()); + + // Upload files + JsonObjectBuilder json1 = Json.createObjectBuilder().add("description", "my description1").add("directoryLabel", "data/subdir1").add("categories", Json.createArrayBuilder().add("Data")); + Response uploadResponse = UtilIT.uploadFileViaNative(datasetId.toString(), "src/main/webapp/resources/images/dataverseproject.png", json1.build(), ownerApiToken); + uploadResponse.then().assertThat().statusCode(OK.getStatusCode()); + Integer fileId1 = JsonPath.from(uploadResponse.body().asString()).getInt("data.files[0].dataFile.id"); + JsonObjectBuilder json2 = Json.createObjectBuilder().add("description", "my description2").add("directoryLabel", "data/subdir1").add("categories", Json.createArrayBuilder().add("Data")); + uploadResponse = UtilIT.uploadFileViaNative(datasetId.toString(), "src/main/webapp/resources/images/orcid_16x16.png", json1.build(), ownerApiToken); + uploadResponse.then().assertThat().statusCode(OK.getStatusCode()); + Integer fileId2 = JsonPath.from(uploadResponse.body().asString()).getInt("data.files[0].dataFile.id"); + JsonObjectBuilder json3 = Json.createObjectBuilder().add("description", "my description3").add("directoryLabel", "data/subdir1").add("categories", Json.createArrayBuilder().add("Data")); + uploadResponse = UtilIT.uploadFileViaNative(datasetId.toString(), "src/main/webapp/resources/images/cc0.png", json1.build(), ownerApiToken); + uploadResponse.then().assertThat().statusCode(OK.getStatusCode()); + Integer fileId3 = JsonPath.from(uploadResponse.body().asString()).getInt("data.files[0].dataFile.id"); + JsonObjectBuilder json4 = Json.createObjectBuilder().add("description", "my description4").add("directoryLabel", "data/subdir1").add("categories", Json.createArrayBuilder().add("Data")); + uploadResponse = UtilIT.uploadFileViaNative(datasetId.toString(), "src/main/webapp/resources/images/Robot-Icon_2.png", json1.build(), ownerApiToken); + uploadResponse.then().assertThat().statusCode(OK.getStatusCode()); + Integer fileId4 = JsonPath.from(uploadResponse.body().asString()).getInt("data.files[0].dataFile.id"); + + // Restrict files + Response restrictResponse = UtilIT.restrictFile(fileId1.toString(), true, ownerApiToken); + restrictResponse.then().assertThat().statusCode(OK.getStatusCode()); + restrictResponse = UtilIT.restrictFile(fileId2.toString(), true, ownerApiToken); + restrictResponse.then().assertThat().statusCode(OK.getStatusCode()); + restrictResponse = UtilIT.restrictFile(fileId3.toString(), true, ownerApiToken); + restrictResponse.then().assertThat().statusCode(OK.getStatusCode()); + // do not restrict fileId4 + + // Update Dataset to allow requests + Response allowAccessRequestsResponse = UtilIT.allowAccessRequests(datasetId.toString(), true, ownerApiToken); + assertEquals(200, allowAccessRequestsResponse.getStatusCode()); + // Publish dataverse and dataset + Response publishDataverse = UtilIT.publishDataverseViaNativeApi(dataverseAlias, ownerApiToken); + assertEquals(200, publishDataverse.getStatusCode()); + Response publishDataset = UtilIT.publishDatasetViaNativeApi(datasetId, "major", ownerApiToken); + assertEquals(200, publishDataset.getStatusCode()); + + // Request access + Response requestFileAccessResponse = UtilIT.requestFileAccess(fileId1.toString(), apiToken, null); + assertEquals(200, requestFileAccessResponse.getStatusCode()); + requestFileAccessResponse = UtilIT.requestFileAccess(fileId2.toString(), apiToken, null); + assertEquals(200, requestFileAccessResponse.getStatusCode()); + requestFileAccessResponse = UtilIT.requestFileAccess(fileId3.toString(), apiToken, null); + assertEquals(200, requestFileAccessResponse.getStatusCode()); + + // Grant file access + Response grantFileAccessResponse = UtilIT.grantFileAccess(fileId1.toString(), "@" + username, ownerApiToken); + assertEquals(200, grantFileAccessResponse.getStatusCode()); + grantFileAccessResponse = UtilIT.grantFileAccess(fileId2.toString(), "@" + username, ownerApiToken); + assertEquals(200, grantFileAccessResponse.getStatusCode()); + grantFileAccessResponse = UtilIT.grantFileAccess(fileId3.toString(), "@" + username, ownerApiToken); + assertEquals(200, grantFileAccessResponse.getStatusCode()); + + String guestbookResponse = UtilIT.generateGuestbookResponse(guestbook); + + // Download unrestricted file by guest user fails without GuestbookResponse + Response downloadResponse = UtilIT.downloadFile(fileId4); + downloadResponse.prettyPrint(); + downloadResponse.then().assertThat() + .body("status", equalTo(ApiConstants.STATUS_ERROR)) + .body("message", equalTo(BundleUtil.getStringFromBundle("access.api.download.failure.guestbookResponseMissing"))) + .statusCode(BAD_REQUEST.getStatusCode()); + // With GuestbookResponse. Guest user doesn't have the required Name and Email. so this will still fail + downloadResponse = UtilIT.postDownloadFile(fileId4, guestbookResponse); + downloadResponse.prettyPrint(); + downloadResponse.then().assertThat() + .body("status", equalTo(ApiConstants.STATUS_ERROR)) + .body("message", containsString("(Name,Email)")) + .statusCode(BAD_REQUEST.getStatusCode()); + String guestbookResponseForGuest = guestbookResponse.replace("\"guestbookResponse\": {", + "\"guestbookResponse\": { \"name\":\"My Name\", \"email\":\"myemail@example.com\", \"position\":\"My Position\", \"institution\":\"My Institution\","); + // With GuestbookResponse. Guest user doesn't have the required Name, etc. So we will add those to the Guestbook Response + downloadResponse = UtilIT.postDownloadFile(fileId4, guestbookResponseForGuest); + downloadResponse.prettyPrint(); + downloadResponse.then().assertThat() + .statusCode(OK.getStatusCode()); + String signedUrl = UtilIT.getSignedUrlFromResponse(downloadResponse); + // Download the file using the signed url + Response signedUrlResponse = get(signedUrl); + signedUrlResponse.prettyPrint(); + assertEquals(OK.getStatusCode(), signedUrlResponse.getStatusCode()); + + // Get Download Url attempt - Guestbook Response is required but not found + downloadResponse = UtilIT.getDownloadFileUrlWithGuestbookResponse(fileId1, apiToken, null); + downloadResponse.prettyPrint(); + downloadResponse.then().assertThat() + .body("status", equalTo(ApiConstants.STATUS_ERROR)) + .body("message", equalTo(BundleUtil.getStringFromBundle("access.api.download.failure.guestbookResponseMissing"))) + .statusCode(BAD_REQUEST.getStatusCode()); + + // Get Signed Download Url with guestbook response + downloadResponse = UtilIT.getDownloadFileUrlWithGuestbookResponse(fileId1, apiToken, guestbookResponse); + downloadResponse.prettyPrint(); + downloadResponse.then().assertThat() + .statusCode(OK.getStatusCode()); + signedUrl = UtilIT.getSignedUrlFromResponse(downloadResponse); + + // Download the file using the signed url + signedUrlResponse = get(signedUrl); + signedUrlResponse.prettyPrint(); + assertEquals(OK.getStatusCode(), signedUrlResponse.getStatusCode()); + + // Download multiple files - Guestbook Response is required but not found for file2 and file3 + downloadResponse = UtilIT.postDownloadDatafiles(fileId1 + "," + fileId2+ "," + fileId3, apiToken); + downloadResponse.prettyPrint(); + downloadResponse.then().assertThat() + .body("status", equalTo(ApiConstants.STATUS_ERROR)) + .body("message", equalTo(BundleUtil.getStringFromBundle("access.api.download.failure.guestbookResponseMissing"))) + .statusCode(BAD_REQUEST.getStatusCode()); + + // Download multiple files with guestbook response and fileIds in json + String jsonBody = "{\"fileIds\":[" + fileId1 + "," + fileId2+ "," + fileId3 +"], " + guestbookResponse.substring(1); + downloadResponse = UtilIT.postDownloadDatafiles(jsonBody, apiToken); + downloadResponse.prettyPrint(); + assertEquals(OK.getStatusCode(), downloadResponse.getStatusCode()); + + downloadResponse = UtilIT.downloadFilesUrlWithGuestbookResponse(new Integer[]{fileId1, fileId2, fileId3}, apiToken, guestbookResponse); + downloadResponse.prettyPrint(); + signedUrl = UtilIT.getSignedUrlFromResponse(downloadResponse); + signedUrlResponse = get(signedUrl); + signedUrlResponse.prettyPrint(); + assertEquals(OK.getStatusCode(), signedUrlResponse.getStatusCode()); + + // TEST Overwrite name, email, institution and position in guestbook Response. Using user2 + requestFileAccessResponse = UtilIT.requestFileAccess(fileId1.toString(), apiToken2, null); + assertEquals(200, requestFileAccessResponse.getStatusCode()); + grantFileAccessResponse = UtilIT.grantFileAccess(fileId1.toString(), "@" + username2, ownerApiToken); + assertEquals(200, grantFileAccessResponse.getStatusCode()); + // Modify guestbookResponse excluding email to show that the email remains unchanged + guestbookResponse = guestbookResponse.replace("\"guestbookResponse\": {", + "\"guestbookResponse\": { \"name\":\"My Name\", \"position\":\"My Position\", \"institution\":\"My Institution\","); + downloadResponse = UtilIT.getDownloadFileUrlWithGuestbookResponse(fileId1, apiToken2, guestbookResponse); + downloadResponse.prettyPrint(); + downloadResponse.then().assertThat() + .statusCode(OK.getStatusCode()); + Response guestbookResponses = UtilIT.getGuestbookResponses(dataverseAlias, guestbook.getId(), ownerApiToken); + assertTrue(guestbookResponses.prettyPrint().contains("My Name," + user2Email + ",My Institution,My Position")); + } + + @Test + public void testDownloadFileWithSignedUrl() throws IOException, JsonParseException { + msgt("testDownloadFileWithSignedUrl"); + // Create superuser + Response createUserResponse = UtilIT.createRandomUser(); + String ownerApiToken = UtilIT.getApiTokenFromResponse(createUserResponse); + String superusername = UtilIT.getUsernameFromResponse(createUserResponse); + UtilIT.makeSuperUser(superusername).then().assertThat().statusCode(200); + + // Create Dataverse + String dataverseAlias = createDataverseGetAlias(ownerApiToken); + + // Create user with no permission + createUserResponse = UtilIT.createRandomUser(); + assertEquals(200, createUserResponse.getStatusCode()); + String apiToken = UtilIT.getApiTokenFromResponse(createUserResponse); + String username = UtilIT.getUsernameFromResponse(createUserResponse); + + // Create Dataset + Response createDatasetResponse = UtilIT.createRandomDatasetViaNativeApi(dataverseAlias, ownerApiToken); + createDatasetResponse.then().assertThat().statusCode(CREATED.getStatusCode()); + Integer datasetId = JsonPath.from(createDatasetResponse.body().asString()).getInt("data.id"); + String persistentId = JsonPath.from(createDatasetResponse.body().asString()).getString("data.persistentId"); + Response getDatasetMetadata = UtilIT.nativeGet(datasetId, ownerApiToken); + getDatasetMetadata.then().assertThat().statusCode(200); + + // Upload files + JsonObjectBuilder json1 = Json.createObjectBuilder().add("description", "my description1").add("directoryLabel", "data/subdir1").add("categories", Json.createArrayBuilder().add("Data")); + Response uploadResponse = UtilIT.uploadFileViaNative(datasetId.toString(), "src/main/webapp/resources/images/dataverseproject.png", json1.build(), ownerApiToken); + uploadResponse.then().assertThat().statusCode(OK.getStatusCode()); + Integer fileId1 = JsonPath.from(uploadResponse.body().asString()).getInt("data.files[0].dataFile.id"); + JsonObjectBuilder json2 = Json.createObjectBuilder().add("description", "my description2").add("directoryLabel", "data/subdir1").add("categories", Json.createArrayBuilder().add("Data")); + uploadResponse = UtilIT.uploadFileViaNative(datasetId.toString(), "src/main/webapp/resources/images/orcid_16x16.png", json1.build(), ownerApiToken); + uploadResponse.then().assertThat().statusCode(OK.getStatusCode()); + Integer fileId2 = JsonPath.from(uploadResponse.body().asString()).getInt("data.files[0].dataFile.id"); + JsonObjectBuilder json3 = Json.createObjectBuilder().add("description", "my description3").add("directoryLabel", "data/subdir1").add("categories", Json.createArrayBuilder().add("Data")); + uploadResponse = UtilIT.uploadFileViaNative(datasetId.toString(), "src/main/webapp/resources/images/cc0.png", json1.build(), ownerApiToken); + uploadResponse.then().assertThat().statusCode(OK.getStatusCode()); + Integer fileId3 = JsonPath.from(uploadResponse.body().asString()).getInt("data.files[0].dataFile.id"); + + // Get Signed Download Url with guestbook response + // downloadFile(Integer fileId, String byteRange, String format, String imageThumb, String apiToken) + Response downloadResponse = UtilIT.downloadFile(fileId1, "&signed=true&format=original" , ownerApiToken); + downloadResponse.prettyPrint(); + downloadResponse.then().assertThat() + .statusCode(OK.getStatusCode()); + } } diff --git a/src/test/java/edu/harvard/iq/dataverse/api/UtilIT.java b/src/test/java/edu/harvard/iq/dataverse/api/UtilIT.java index 24ab8b56eff..5577daaed27 100644 --- a/src/test/java/edu/harvard/iq/dataverse/api/UtilIT.java +++ b/src/test/java/edu/harvard/iq/dataverse/api/UtilIT.java @@ -1,56 +1,55 @@ package edu.harvard.iq.dataverse.api; +import com.mashape.unirest.http.Unirest; +import com.mashape.unirest.http.exceptions.UnirestException; +import com.mashape.unirest.request.GetRequest; import edu.harvard.iq.dataverse.*; +import edu.harvard.iq.dataverse.api.datadeposit.SwordConfigurationImpl; +import edu.harvard.iq.dataverse.mydata.MyDataFilterParams; +import edu.harvard.iq.dataverse.settings.FeatureFlags; +import edu.harvard.iq.dataverse.settings.SettingsServiceBean; +import edu.harvard.iq.dataverse.util.FileUtil; +import edu.harvard.iq.dataverse.util.StringUtil; +import edu.harvard.iq.dataverse.util.json.JsonParseException; +import edu.harvard.iq.dataverse.util.json.JsonParser; +import edu.harvard.iq.dataverse.util.json.JsonUtil; import edu.harvard.iq.dataverse.util.json.NullSafeJsonBuilder; import io.restassured.http.ContentType; import io.restassured.path.json.JsonPath; -import io.restassured.response.Response; - -import java.io.*; -import java.util.*; -import java.util.logging.Logger; -import jakarta.json.Json; -import jakarta.json.JsonArray; -import jakarta.json.JsonObjectBuilder; -import jakarta.json.JsonArrayBuilder; -import jakarta.json.JsonObject; - -import static edu.harvard.iq.dataverse.util.json.NullSafeJsonBuilder.jsonObjectBuilder; -import static jakarta.ws.rs.core.Response.Status.CREATED; - -import java.nio.charset.StandardCharsets; -import java.nio.file.Files; -import java.nio.file.Paths; -import java.time.LocalDateTime; -import java.util.logging.Level; -import edu.harvard.iq.dataverse.api.datadeposit.SwordConfigurationImpl; import io.restassured.path.xml.XmlPath; -import edu.harvard.iq.dataverse.mydata.MyDataFilterParams; -import jakarta.ws.rs.core.HttpHeaders; -import org.apache.commons.lang3.StringUtils; -import org.assertj.core.util.Lists; -import org.junit.jupiter.api.Test; -import edu.harvard.iq.dataverse.settings.SettingsServiceBean; +import io.restassured.response.Response; import io.restassured.specification.RequestSpecification; -import com.mashape.unirest.http.Unirest; -import com.mashape.unirest.http.exceptions.UnirestException; -import com.mashape.unirest.request.GetRequest; -import edu.harvard.iq.dataverse.util.FileUtil; +import jakarta.json.*; +import jakarta.ws.rs.core.HttpHeaders; import org.apache.commons.io.IOUtils; -import java.nio.file.Path; - +import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.math.NumberUtils; +import org.assertj.core.util.Lists; import org.hamcrest.BaseMatcher; import org.hamcrest.Description; import org.hamcrest.Matcher; +import org.junit.jupiter.api.Test; + +import java.io.File; +import java.io.FileOutputStream; +import java.io.IOException; +import java.io.InputStream; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.time.LocalDateTime; +import java.util.*; +import java.util.logging.Level; +import java.util.logging.Logger; import static edu.harvard.iq.dataverse.api.ApiConstants.*; -import static io.restassured.path.xml.XmlPath.from; +import static edu.harvard.iq.dataverse.util.json.NullSafeJsonBuilder.jsonObjectBuilder; import static io.restassured.RestAssured.given; - -import edu.harvard.iq.dataverse.settings.FeatureFlags; -import edu.harvard.iq.dataverse.util.StringUtil; - +import static io.restassured.path.xml.XmlPath.from; +import static jakarta.ws.rs.core.Response.Status.CREATED; +import static jakarta.ws.rs.core.Response.Status.OK; +import static org.hamcrest.CoreMatchers.startsWith; import static org.junit.jupiter.api.Assertions.*; public class UtilIT { @@ -595,6 +594,36 @@ static Response getGuestbookResponses(String dataverseAlias, Long guestbookId, S return requestSpec.get("/api/dataverses/" + dataverseAlias + "/guestbookResponses/"); } + public static Response createGuestbook(String dataverseAlias, String guestbookAsJson, String apiToken) { + Response createGuestbookResponse = given() + .header(API_TOKEN_HTTP_HEADER, apiToken) + .body(guestbookAsJson) + .contentType("application/json") + .post("/api/guestbooks/" + dataverseAlias); + return createGuestbookResponse; + } + + static Response getGuestbook(Long guestbookId, String apiToken) { + RequestSpecification requestSpec = given() + .header(API_TOKEN_HTTP_HEADER, apiToken); + return requestSpec.get("/api/guestbooks/" + guestbookId ); + } + + static Response getGuestbooks(String dataverseAlias, String apiToken) { + RequestSpecification requestSpec = given() + .header(API_TOKEN_HTTP_HEADER, apiToken); + return requestSpec.get("/api/guestbooks/" + dataverseAlias + "/list" ); + } + + static Response enableGuestbook(String dataverseAlias, Long guestbookId, String apiToken, String enable) { + Response createGuestbookResponse = given() + .header(API_TOKEN_HTTP_HEADER, apiToken) + .body(enable) + .contentType("application/json") + .put("/api/guestbooks/" + dataverseAlias + "/" + guestbookId + "/enabled"); + return createGuestbookResponse; + } + static Response getCollectionSchema(String dataverseAlias, String apiToken) { Response getCollectionSchemaResponse = given() .header(API_TOKEN_HTTP_HEADER, apiToken) @@ -808,6 +837,21 @@ static Response updateFieldLevelDatasetMetadataViaNative(String persistentId, St return editVersionMetadataFromJsonStr(persistentId, jsonIn, apiToken, null); } + static Response updateDatasetGuestbook(String persistentId, Long guestbookId, String apiToken) { + RequestSpecification requestSpecification = given() + .header(API_TOKEN_HTTP_HEADER, apiToken) + .contentType("application/json"); + String path = "/api/datasets/:persistentId/guestbook/?persistentId=" + persistentId; + if (guestbookId != null) { + return requestSpecification + .body(guestbookId) + .put(path); + } else { + return requestSpecification + .delete(path); + } + } + static Response editVersionMetadataFromJsonStr(String persistentId, String jsonString, String apiToken) { return editVersionMetadataFromJsonStr(persistentId, jsonString, apiToken, null); } @@ -1157,7 +1201,11 @@ static Response downloadFile(Integer fileId) { // .header(API_TOKEN_HTTP_HEADER, apiToken) .get("/api/access/datafile/" + fileId); } - + static Response postDownloadFile(Integer fileId, String jsonBody) { + return given() + .body(jsonBody) + .post("/api/access/datafile/" + fileId); + } static Response downloadFile(Integer fileId, String apiToken) { String nullByteRange = null; String nullFormat = null; @@ -1187,6 +1235,12 @@ static Response downloadFile(Integer fileId, String byteRange, String format, St //.header(API_TOKEN_HTTP_HEADER, apiToken) return requestSpecification.get("/api/access/datafile/" + fileId + "?key=" + apiToken + optionalFormat + optionalImageThumb); } + + static Response downloadFile(Integer fileId, String queryParams, String apiToken) { + RequestSpecification requestSpecification = given(); + + return requestSpecification.get("/api/access/datafile/" + fileId + "?key=" + apiToken + queryParams); + } static Response downloadTabularFile(Integer fileId) { return given() @@ -1208,7 +1262,39 @@ static Response downloadFileOriginal(Integer fileId, String apiToken) { return given() .get("/api/access/datafile/" + fileId + "?format=original&key=" + apiToken); } - + + static Response getDownloadFileUrlWithGuestbookResponse(Integer fileId, String apiToken, String body) { + RequestSpecification requestSpecification = given(); + requestSpecification.header(API_TOKEN_HTTP_HEADER, apiToken); + if (body != null) { + requestSpecification.body(body); + } + return requestSpecification.post("/api/access/datafile/" + fileId); + } + + static Response downloadFilesUrlWithGuestbookResponse(Integer[] fileIds, String apiToken, String body) { + RequestSpecification requestSpecification = given(); + requestSpecification.header(API_TOKEN_HTTP_HEADER, apiToken); + if (body != null) { + requestSpecification.body(body); + } + String getString = "/api/access/datafiles/"; + for (Integer fileId : fileIds) { + getString += fileId + ","; + } + return requestSpecification.post(getString); + } + + static Response postDownloadDatafiles(String body, String apiToken) { + String getString = "/api/access/datafiles"; + RequestSpecification requestSpecification = given(); + requestSpecification.header(API_TOKEN_HTTP_HEADER, apiToken); + if (body != null) { // body contains list of data file ids + requestSpecification.body(body); + } + return requestSpecification.post(getString); + } + static Response downloadFiles(Integer[] fileIds) { String getString = "/api/access/datafiles/"; for(Integer fileId : fileIds) { @@ -2075,8 +2161,11 @@ static Response allowAccessRequests(String datasetIdOrPersistentId, boolean allo } static Response requestFileAccess(String fileIdOrPersistentId, String apiToken) { - System.out.print ("Reuest file acceess + fileIdOrPersistentId: " + fileIdOrPersistentId); - System.out.print ("Reuest file acceess + apiToken: " + apiToken); + return requestFileAccess(fileIdOrPersistentId, apiToken, null); + } + static Response requestFileAccess(String fileIdOrPersistentId, String apiToken, String body) { + System.out.print ("Request file access + fileIdOrPersistentId: " + fileIdOrPersistentId); + System.out.print ("Request file access + apiToken: " + apiToken); String idInPath = fileIdOrPersistentId; // Assume it's a number. String optionalQueryParam = ""; // If idOrPersistentId is a number we'll just put it in the path. if (!NumberUtils.isCreatable(fileIdOrPersistentId)) { @@ -2088,10 +2177,16 @@ static Response requestFileAccess(String fileIdOrPersistentId, String apiToken) if (optionalQueryParam.isEmpty()) { keySeparator = "?"; } - System.out.print ("URL: " + "/api/access/datafile/" + idInPath + "/requestAccess" + optionalQueryParam + keySeparator + "key=" + apiToken); - Response response = given() - .put("/api/access/datafile/" + idInPath + "/requestAccess" + optionalQueryParam + keySeparator + "key=" + apiToken); - return response; + String path = "/api/access/datafile/" + idInPath + "/requestAccess" + optionalQueryParam + keySeparator + "key=" + apiToken; + System.out.print ("URL: " + path); + RequestSpecification requestSpecification = given() + .header(UtilIT.API_TOKEN_HTTP_HEADER, apiToken) + .contentType("application/json"); + if (body != null) { + requestSpecification.body(body); + } + + return requestSpecification.put(path); } static Response grantFileAccess(String fileIdOrPersistentId, String identifier, String apiToken) { @@ -5320,4 +5415,56 @@ public static Response sendMessageToLDNInbox(String message) { .when() .post("/api/inbox/"); } + + public static Response setGuestbookEntryOnRequest(String datasetId, String apiToken, Boolean enabled) { + return given() + .body(enabled) + .contentType(ContentType.JSON) + .header(API_TOKEN_HTTP_HEADER, apiToken) + .put("/api/datasets/" + datasetId + "/guestbookEntryAtRequest"); + } + + public static Guestbook createRandomGuestbook(String ownerAlias, String persistentId, String apiToken) throws IOException, JsonParseException { + Guestbook gb = new Guestbook(); + File guestbookJson = new File("scripts/api/data/guestbook-test.json"); + String guestbookAsJson = new String(Files.readAllBytes(Paths.get(guestbookJson.getAbsolutePath()))); + JsonObject jsonObj = JsonUtil.getJsonObject(guestbookAsJson); + JsonParser jsonParsor = new JsonParser(); + jsonParsor.parseGuestbook(jsonObj, gb); + + Response createGuestbookResponse = UtilIT.createGuestbook(ownerAlias, guestbookAsJson, apiToken); + createGuestbookResponse.prettyPrint(); + JsonPath createdGuestbook = JsonPath.from(createGuestbookResponse.body().asString()); + Long guestbookId = Long.parseLong(createdGuestbook.getString("data.message").split(" ")[1]); + Response guestbookEnableResponse = UtilIT.enableGuestbook(ownerAlias, guestbookId, apiToken, Boolean.TRUE.toString()); + guestbookEnableResponse.prettyPrint(); + guestbookEnableResponse.then().assertThat() + .statusCode(OK.getStatusCode()) + .body("data.message", startsWith("Guestbook")); + Response getGuestbookResponse = UtilIT.getGuestbook(guestbookId, apiToken); + getGuestbookResponse.prettyPrint(); + JsonPath jsonPath = JsonPath.from(getGuestbookResponse.body().asString()); + gb.setId(guestbookId); + gb.getCustomQuestions().get(0).setId(jsonPath.getLong("data.customQuestions[0].id")); + gb.getCustomQuestions().get(1).setId(jsonPath.getLong("data.customQuestions[1].id")); + gb.getCustomQuestions().get(2).setId(jsonPath.getLong("data.customQuestions[2].id")); + + // Add the Guestbook to the Dataset + Response setGuestbook = UtilIT.updateDatasetGuestbook(persistentId, guestbookId, apiToken); + setGuestbook.prettyPrint(); + return gb; + } + + public static String generateGuestbookResponse(Guestbook gb) throws IOException { + File guestbookJson = new File("scripts/api/data/guestbook-test-response.json"); + String guestbookAsJson = new String(Files.readAllBytes(Paths.get(guestbookJson.getAbsolutePath()))); + + List cqIDs = new ArrayList<>(); + gb.getCustomQuestions().stream().forEach(cq -> cqIDs.add(cq.getId())); + + return guestbookAsJson.replace("@ID", gb.getId().toString()) + .replace("@QID1", cqIDs.get(0).toString()) + .replace("@QID2", cqIDs.get(1).toString()) + .replace("@QID3", cqIDs.get(2).toString()); + } } diff --git a/src/test/java/edu/harvard/iq/dataverse/util/UrlSignerUtilTest.java b/src/test/java/edu/harvard/iq/dataverse/util/UrlSignerUtilTest.java index 09739b67023..d92f8822e59 100644 --- a/src/test/java/edu/harvard/iq/dataverse/util/UrlSignerUtilTest.java +++ b/src/test/java/edu/harvard/iq/dataverse/util/UrlSignerUtilTest.java @@ -1,12 +1,15 @@ package edu.harvard.iq.dataverse.util; -import static org.junit.jupiter.api.Assertions.*; -import static org.junit.jupiter.api.Assertions.assertTrue; +import jakarta.ws.rs.core.MultivaluedHashMap; +import jakarta.ws.rs.core.MultivaluedMap; +import org.junit.jupiter.api.Test; +import java.util.List; import java.util.logging.Level; import java.util.logging.Logger; -import org.junit.jupiter.api.Test; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; public class UrlSignerUtilTest { @@ -47,4 +50,40 @@ public void testSignAndValidate() { assertFalse(UrlSignerUtil.isValidUrl(signedUrl3, user1, get, key)); } + + @Test + public void testSignAndValidateWithParams() { + final String url1 = "http://localhost:8080/api/test1?p1=true&p2=test"; + final String url2 = "http://localhost:8080/api/test1?p1=true&p2=test&until=2999-01-01&user=Fred&method=POST&token=abracadabara&signed=true"; + final String url3 = "localhost:8080/api/test1?p1=true&p2&until=2099-01-01"; + final int longTimeout = 1000; + final String user1 = "Alice"; + final String key = "abracadabara open sesame"; + MultivaluedMap queryParameters = new MultivaluedHashMap<>(); + queryParameters.put("p1", List.of("true")); + queryParameters.put("p2", List.of("test")); + queryParameters.put("until", List.of("2099-01-01")); + + String signedUrl1 = UrlSignerUtil.signUrl(url1, longTimeout, user1, "GET", key); + assertTrue(signedUrl1.contains("test1?p1=true&p2=test")); + System.out.println(signedUrl1); + + String signedUrl2 = UrlSignerUtil.signUrl(url2, longTimeout, user1, "GET", key); + assertTrue(signedUrl2.contains("&until=")); // contains the until param but not the bogus one passed in + assertFalse(signedUrl2.contains("&until=2099-01-01")); + assertTrue(signedUrl2.contains("&user=Alice")); // contains the user param but not the bogus one passed in + assertFalse(signedUrl2.contains("&user=Fred")); + assertTrue(signedUrl2.contains("&method=GET")); // contains the method param but not the bogus one passed in + assertFalse(signedUrl2.contains("&method=POST")); + assertTrue(signedUrl2.contains("&token=")); // contains the signed token param but not the bogus one passed in + assertFalse(signedUrl2.contains("&token=abracadabara")); + assertFalse(signedUrl2.contains("&signed")); // make sure we don't propagate the "signed" param + System.out.println(signedUrl2); + + // This will log an error but will still return the signed url even if it's now a valid url + // All callers of this method don't handle errors being returned, and it's highly unlikely that the url would be bad + String signedUrl3 = UrlSignerUtil.signUrl(url3, longTimeout, user1, "GET", key); + System.out.println(signedUrl3); + assertTrue(signedUrl3.contains("&p2&")); // Show that this works with params that have no value + } } diff --git a/src/test/java/edu/harvard/iq/dataverse/util/json/JsonParserTest.java b/src/test/java/edu/harvard/iq/dataverse/util/json/JsonParserTest.java index d1cb30e2bc3..e1bfbefe8d5 100644 --- a/src/test/java/edu/harvard/iq/dataverse/util/json/JsonParserTest.java +++ b/src/test/java/edu/harvard/iq/dataverse/util/json/JsonParserTest.java @@ -65,6 +65,58 @@ public class JsonParserTest { DatasetFieldType pubIdType; DatasetFieldType compoundSingleType; JsonParser sut; + + final static String guestbookJson = """ + { + "name": "my test guestbook", + "enabled": true, + "emailRequired": true, + "nameRequired": true, + "institutionRequired": false, + "positionRequired": false, + "customQuestions": [ + { + "question": "how's your day", + "required": true, + "displayOrder": 0, + "type": "text", + "hidden": false + }, + { + "question": "Describe yourself", + "required": false, + "displayOrder": 1, + "type": "textarea", + "hidden": false + }, + { + "question": "What color car do you drive", + "required": true, + "displayOrder": 2, + "type": "options", + "hidden": false, + "optionValues": [ + { + "value": "Red", + "displayOrder": 0 + }, + { + "value": "White", + "displayOrder": 1 + }, + { + "value": "Yellow", + "displayOrder": 2 + }, + { + "value": "Purple", + "displayOrder": 3 + } + ] + } + ] + } + """; public JsonParserTest() { } @@ -733,4 +785,120 @@ public void testEnum() throws JsonParseException { assertTrue(typesSet.contains(Type.REVOKEROLE), "Set contains REVOKEROLE"); assertTrue(typesSet.contains(Type.ASSIGNROLE), "Set contains ASSIGNROLE"); } + + @Test + public void testGuestbook() throws JsonParseException { + JsonObject jsonObj = JsonUtil.getJsonObject(guestbookJson); + Guestbook gb = new Guestbook(); + gb = sut.parseGuestbook(jsonObj, gb); + assertEquals(true, gb.isEnabled()); + assertEquals(3, gb.getCustomQuestions().size()); + assertEquals(4, gb.getCustomQuestions().get(2).getCustomQuestionValues().size()); + assertEquals("Purple", gb.getCustomQuestions().get(2).getCustomQuestionValues().get(3).getValueString()); + assertEquals(3, gb.getCustomQuestions().get(2).getCustomQuestionValues().get(3).getDisplayOrder()); + } + + @Test + public void testGuestbookResponse() throws JsonParseException { + JsonObject jsonObj = JsonUtil.getJsonObject(guestbookJson); + Guestbook gb = new Guestbook(); + gb = sut.parseGuestbook(jsonObj, gb); + Long i = 1L; + for (CustomQuestion cq : gb.getCustomQuestions()) { + cq.setId(i++); + cq.setRequired(true); + } + + final String guestbookResponseJson = """ + { + "name": "My Name", + "email": "my.email@example.com", + "institution": "Harvard", + "position": "Upright", + "answers": [ + { + "id": 1, + "value": "Good" + }, + { + "id": 2, + "value": ["Multi","Line"] + }, + { + "id": 3, + "value": "Yellow" + } + ] + } + """; + final String guestbookResponseJsonMissing3 = """ + { + "answers": [ + { + "id": 1, + "value": "Good" + }, + { + "id": 2, + "value": ["Multi","Line"] + } + ] + } + """; + + GuestbookResponse guestbookResponse = new GuestbookResponse(); + guestbookResponse.setGuestbook(gb); + jsonObj = JsonUtil.getJsonObject(guestbookResponseJson); + GuestbookResponse gbr = sut.parseGuestbookResponse(jsonObj, guestbookResponse); + assertTrue(gbr.getCustomQuestionResponses().size() == 3); + + // Test missing required question response + try { + jsonObj = JsonUtil.getJsonObject(guestbookResponseJsonMissing3); + gbr = sut.parseGuestbookResponse(jsonObj, guestbookResponse); + } catch (JsonParseException e) { + System.out.println(e.getMessage()); + assertTrue(e.getMessage().contains("What color car do you drive")); + } + // Test invalid option in question response + try { + jsonObj = JsonUtil.getJsonObject(guestbookResponseJson.replace("Yellow", "Green")); + gbr = sut.parseGuestbookResponse(jsonObj, guestbookResponse); + } catch (JsonParseException e) { + System.out.println(e.getMessage()); + assertTrue(e.getMessage().contains("not a valid option (Green)")); + } + // Test invalid Custom Question ID in question response + try { + jsonObj = JsonUtil.getJsonObject(guestbookResponseJson.replace("3", "4")); + gbr = sut.parseGuestbookResponse(jsonObj, guestbookResponse); + } catch (JsonParseException e) { + System.out.println(e.getMessage()); + assertTrue(e.getMessage().contains("ID 4 not found")); + } + + // Test overwrite name, email, institution and position. + try { + jsonObj = JsonUtil.getJsonObject(guestbookResponseJson); + gbr = sut.parseGuestbookResponse(jsonObj, guestbookResponse); + assertEquals("My Name", gbr.getName()); + // Removing name from the JSON defaults it to the original value in guestbook response + gbr.setName("My Original Name"); + jsonObj = JsonUtil.getJsonObject(guestbookResponseJson.replace("\"name\": \"My Name\",", "")); + gbr = sut.parseGuestbookResponse(jsonObj, guestbookResponse); + assertEquals("My Original Name", gbr.getName()); + // test invalid email (does not change original) + gbr.setEmail("original@example.com"); + jsonObj = JsonUtil.getJsonObject(guestbookResponseJson.replace("my.email@example.com", "badEmail.com")); + gbr = sut.parseGuestbookResponse(jsonObj, guestbookResponse); + assertEquals("original@example.com", gbr.getEmail()); + // test valid email (overwrite email) + jsonObj = JsonUtil.getJsonObject(guestbookResponseJson.replace("my.email@example.com", "new@example.com")); + gbr = sut.parseGuestbookResponse(jsonObj, guestbookResponse); + assertEquals("new@example.com", gbr.getEmail()); + } catch (JsonParseException e) { + System.out.println(e.getMessage()); + assertTrue(e.getMessage().contains("ID 4 not found")); + } + } } diff --git a/src/test/java/edu/harvard/iq/dataverse/util/json/JsonPrinterTest.java b/src/test/java/edu/harvard/iq/dataverse/util/json/JsonPrinterTest.java index 2f4fda068d4..34676335857 100644 --- a/src/test/java/edu/harvard/iq/dataverse/util/json/JsonPrinterTest.java +++ b/src/test/java/edu/harvard/iq/dataverse/util/json/JsonPrinterTest.java @@ -2,6 +2,7 @@ import edu.harvard.iq.dataverse.*; import edu.harvard.iq.dataverse.DatasetFieldType.FieldType; +import edu.harvard.iq.dataverse.UserNotification.Type; import edu.harvard.iq.dataverse.authorization.DataverseRole; import edu.harvard.iq.dataverse.authorization.RoleAssignee; import edu.harvard.iq.dataverse.authorization.users.PrivateUrlUser; @@ -12,7 +13,12 @@ import edu.harvard.iq.dataverse.pidproviders.doi.AbstractDOIProvider; import edu.harvard.iq.dataverse.privateurl.PrivateUrl; import edu.harvard.iq.dataverse.settings.SettingsServiceBean; -import edu.harvard.iq.dataverse.UserNotification.Type; +import edu.harvard.iq.dataverse.util.BundleUtil; +import edu.harvard.iq.dataverse.util.template.TemplateBuilder; +import jakarta.json.*; +import org.assertj.core.util.Lists; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; import java.sql.Timestamp; import java.time.Instant; @@ -20,19 +26,7 @@ import java.util.*; import java.util.stream.Collectors; -import edu.harvard.iq.dataverse.util.template.TemplateBuilder; - -import jakarta.json.*; - -import edu.harvard.iq.dataverse.util.BundleUtil; -import org.assertj.core.util.Lists; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.BeforeEach; - -import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.*; -import static org.junit.jupiter.api.Assertions.assertTrue; -import static org.junit.jupiter.api.Assertions.assertFalse; public class JsonPrinterTest { @@ -480,6 +474,99 @@ public void testDatasetWithNondefaultType() { assertEquals(sut, result); } + @Test + public void testDatasetWithGuestbook() { + String sut = "foobar"; + DatasetType foobar = new DatasetType(); + foobar.setName(sut); + + Guestbook guestbook = new Guestbook(); + guestbook.setId(1L); + guestbook.setEnabled(true); + guestbook.setName("Test Guestbook"); + guestbook.setEmailRequired(true); + guestbook.setCreateTime(Timestamp.from(Instant.now())); + + int cqOrder = 0; + CustomQuestion cq1 = new CustomQuestion(); + cq1.setDisplayOrder(cqOrder); + cq1.setId(Long.valueOf(++cqOrder)); + cq1.setGuestbook(guestbook); + cq1.setRequired(true); + cq1.setQuestionString("My first question"); + cq1.setQuestionType("text"); // options, textarea, text + + CustomQuestion cq2 = new CustomQuestion(); + cq2.setDisplayOrder(cqOrder); + cq2.setId(Long.valueOf(++cqOrder)); + cq2.setGuestbook(guestbook); + cq2.setRequired(false); + cq2.setQuestionString("My second question"); + cq2.setQuestionType("textarea"); + + CustomQuestion cq3 = new CustomQuestion(); + cq3.setDisplayOrder(cqOrder); + cq3.setId(Long.valueOf(++cqOrder)); + cq3.setGuestbook(guestbook); + cq3.setRequired(false); + cq3.setQuestionString("My third question"); + cq3.setQuestionType("options"); + List values = new ArrayList<>(); + int cqvOrder = 0; + CustomQuestionValue cqv1 = new CustomQuestionValue(); + cqv1.setValueString("Red"); + cqv1.setDisplayOrder(cqvOrder); + cqv1.setId(Long.valueOf(++cqvOrder)); + values.add(cqv1); + CustomQuestionValue cqv2 = new CustomQuestionValue(); + cqv2.setValueString("White"); + cqv2.setDisplayOrder(cqvOrder); + cqv2.setId(Long.valueOf(++cqvOrder)); + values.add(cqv2); + CustomQuestionValue cqv3 = new CustomQuestionValue(); + cqv3.setValueString("Blue"); + cqv3.setDisplayOrder(cqvOrder); + cqv3.setId(Long.valueOf(++cqvOrder)); + values.add(cqv3); + cq3.setCustomQuestionValues(values); + List customQuestions = new ArrayList<>(); + customQuestions.add(cq1); + customQuestions.add(cq2); + customQuestions.add(cq3); + guestbook.setCustomQuestions(customQuestions); + + Dataverse dv = new Dataverse(); + dv.setId(41L); + Dataset dataset = createDataset(42); + dataset.setDatasetType(foobar); + dataset.setOwner(dv); + guestbook.setDataverse(dataset.getOwner()); + dataset.setGuestbook(guestbook); + + // verify that the guestbook id is in the dataset response + var jsob = JsonPrinter.json(dataset.getLatestVersion(), null, false, false, false, false).build(); + System.out.println(jsob); + var gbID = jsob.getInt("guestbookId"); + assertEquals(1, gbID); + + var gb = JsonPrinter.json(guestbook).build(); + System.out.println(gb); + + // verify guestbook values + assertEquals("Test Guestbook", gb.getString("name")); + assertEquals(true, gb.getBoolean("emailRequired")); + assertEquals(false, gb.getBoolean("nameRequired")); + assertEquals(3, gb.getJsonArray("customQuestions").size()); + // verify multiple choice question + var result_cq3 = gb.getJsonArray("customQuestions"); + System.out.println(result_cq3); + var result_cq3_options = result_cq3.getJsonObject(2).getJsonArray("optionValues"); // question 3 is index 2 + System.out.println(result_cq3_options); + assertEquals(3, result_cq3_options.size()); + var result_cq3_options2 = result_cq3_options.getJsonObject(1); // option 2 is index 1 + assertEquals("White", result_cq3_options2.getString("value")); + } + @Test public void testJsonArrayDataverseCollections() { List collections = new ArrayList<>();