TC-3071 add upload advisory and delete advisory scale test #77

bxf12315 · 2025-10-30T03:48:30Z

I have added a new environment variable called UPLOAD_FILE_PATH, which is used to represent the path for all uploaded files.(default value is "./test-data").
I have made corresponding changes based on Jens's comment (#74 (comment)). However, there is still an issue: if the upload has not fully completed, the delete operation may return a 404 error.
First, we can tolerate a certain number of errors. Second, we can add a thread sleep between the two REST requests. However, adding thread sleep may affect the results.

The test results are as shown below:

since they distinguish between different requests, they can also be considered acceptable.

src/restapi.rs

src/main.rs

src/scenario/mod.rs

test-data/advisories/rhba-2024_11505.json

ctron · 2025-11-03T08:10:27Z

.cargo/config.toml

@@ -0,0 +1,2 @@
+[env]
+CARGO_WORKSPACE_ROOT = { value = "", relative = true }


Why is it not possible to use CARGO_MANIFEST_DIR?

CARGO_WORKSPACE_ROOT and CARGO_MANIFEST_DIR serve the same purpose, and I think CARGO_WORKSPACE_ROOT is more semantically appropriate. Could you please explain why CARGO_MANIFEST_DIR would be more reasonable?

src/scenario/mod.rs

ctron · 2025-11-03T13:36:10Z

.cargo/config.toml

@@ -0,0 +1,2 @@
+[env]
+CARGO_MANIFEST_DIR = { value = "", relative = true }


This variable is provided by the cargo build. It does not need to be provided.

ctron · 2025-11-03T13:36:47Z

src/scenario/mod.rs

+    fn load_advisory_files(&self) -> anyhow::Result<Vec<PathBuf>> {
+        // Try UPLOAD_FILE_PATH first, then fall back to CARGO_MANIFEST_DIR
+        let base_path = std::env::var("UPLOAD_FILE_PATH")
+            .or_else(|_| std::env::var("CARGO_MANIFEST_DIR"))


You need to use env! in this case. As it is a build time variable.

Alternatively, why not use the current working directory?

I just want it to be as flexibly configurable as SCENARIO_FILE.

If you use env to access CARGO_MANIFEST_DIR, you will get an absolute path. Isn’t this potentially unreasonable?

Why would it be?

Ok please check again.

ctron · 2025-11-11T14:15:56Z

src/restapi.rs

+    advisory_file: String,
+    user: &mut GooseUser,
+) -> Result<String, Box<TransactionError>> {
+    let file_bytes = tokio::fs::read(&advisory_file).await.map_err(|e| {


Part of this test now is the loading of the file. Which won't change. Can we cache that?

Having a hard-coded list of a number of files will run into the same situation. Just a bit later.

I think the appropriate solution for this would be to:

load an example file on startup, hard-coded, only one

create a function returning a new file. in-memory

implement that function in a way that it generates a new version of the file by modifying certain element with random values (e.g. adding a random suffix to all package names)

use this function to generate the content each time it's required during the test

I’ve randomly modified the csaf.document.tracking.id and also added support for xz.

ctron · 2025-11-11T14:16:57Z

src/restapi.rs


-    Ok(())
-}
+    let response = user.post("/api/v2/advisory", file_bytes).await?;


This will upload the exact same file. Having multiple concurrent runners, the first one will be a real upload. The next one will be a no-op.

Wouldn't it make sense to insert some randomness into the content?

ctron · 2025-11-27T07:19:00Z

test-data/advisories/.DS_Store

Please remove this file

ctron · 2025-11-27T07:20:05Z

src/restapi.rs


 /// Upload file and return advisory ID
 pub async fn upload_advisory_and_get_id(
    advisory_file: String,


It looks like you're only using a reference to this. If that's the case, please use &str and don't clone.

ctron · 2025-11-27T07:21:04Z

src/main.rs

            if let Some(file) = &scenario.upload_advisory_files {
                // Use sequential transaction execution to ensure immediate deletion after upload
-                s = s.register_transaction(tx!(upload_and_immediately_delete(file.clone(), advisory_files_counter.clone()),name: format!("upload_and_immediately_delete")));
+                s = s.register_transaction(tx!(upload_and_immediately_delete(file.clone()),name: format!("upload_and_immediately_delete")));


Why is format! needed here?

ctron · 2025-11-27T07:25:40Z

src/restapi.rs

+    let (cached_path, content) = result;
+
+    // Verify that the cached path matches the requested path
+    if cached_path != file_path {


Maybe it makes more sense to have a "map" instead of failing here. Or maybe, instead of using a global static with a dynamic capture, load the file in the setup of the tests, and provide the loaded content directly.

ctron · 2025-11-27T07:27:00Z

Thank you, I think we are on the right track now. Just a few small things to iron out.

bxf12315 · 2025-11-28T00:25:42Z

Thank you, I think we are on the right track now. Just a few small things to iron out.

Updated, please check again.

ctron · 2025-11-28T09:54:29Z

src/scenario/mod.rs

        .await
    }
+    /// Load advisory files from UPLOAD_FILE_PATH directory, or fall back to CARGO_MANIFEST_DIR
+    fn load_advisory_files(&self) -> anyhow::Result<Vec<PathBuf>> {


This supposed to be a single file now, right? Why is this still a Vec?

This method keeps a certain level of generality and, in here, simply takes the first file encountered.