Pattern-Based Log Generator for Realistic Load

CHANGELOG.md

@@ -13,6 +13,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## Added
 - Stable throttle now has a 'timeout' configuration parameter to model IO done
   with timeout.
+- Added a `patterned` variant to the file generator that replays deterministic
+  log sequences with realistic request/job/database patterns.
 
 ## [0.28.0]
 ## Added

README.md

@@ -69,6 +69,35 @@ throughput, although the target might not be able to cope with that load and
 `lading` will consume 256 MiB of RAM to accommodate pre-build payloads. The
 blackhole in this configuration responds with an empty body 200 OK.
 
+### Patterned file generator
+
+For scenarios where the target benefits from realistic log lifecycles `lading`
+now exposes a patterned variant of the file generator. This option precomputes a
+deterministic cycle of common sequences (HTTP successes, transient API
+failures, background jobs, and slow database queries) to avoid purely random log
+lines while remaining repeatable. A minimal configuration looks like:
+
+```yaml
+generator:
+  - file_gen:
+      patterned:
+        seed: [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53,
+               59, 61, 67, 71, 73, 79, 83, 89, 97, 101, 103, 107, 109, 113, 127, 131]
+        path_template: "/tmp/lading-patterned/log-%NNN%.log"
+        duplicates: 2
+        maximum_bytes_per_file: 256MiB
+        maximum_prebuild_cache_size_bytes: 64MiB
+        rotate: true
+        distribution:
+          http_success: 70
+          http_error: 15
+          background_job: 10
+          database_slow: 5
+```
+
+See [`examples/lading-patterned-filegen.yaml`](examples/lading-patterned-filegen.yaml)
+for a complete example including throttling.
+
 `lading` supports three types of targets, binary launch mode, PID watch mode,
 and container watch mode.
 

examples/lading-patterned-filegen.yaml

@@ -0,0 +1,21 @@
+generator:
+  - file_gen:
+      patterned:
+        seed: [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53,
+               59, 61, 67, 71, 73, 79, 83, 89, 97, 101, 103, 107, 109, 113, 127, 131]
+        path_template: "/tmp/lading-patterned/log-%NNN%.log"
+        duplicates: 2
+        maximum_bytes_per_file: 256MiB
+        bytes_per_second: 12MiB
+        maximum_prebuild_cache_size_bytes: 64MiB
+        maximum_line_size: 8KiB
+        rotate: true
+        distribution:
+          http_success: 70
+          http_error: 15
+          background_job: 10
+          database_slow: 5
+
+blackhole:
+  - tcp:
+      binding_addr: "0.0.0.0:10514"

lading/src/generator/file_gen.rs

@@ -15,6 +15,7 @@
 pub mod logrotate;
 #[cfg(feature = "logrotate_fs")]
 pub mod logrotate_fs;
+pub mod patterned;
 pub mod traditional;
 
 use std::str;
@@ -36,6 +37,9 @@ pub enum Error {
     /// Wrapper around [`logrotate_fs::Error`].
     #[error(transparent)]
     LogrotateFs(#[from] logrotate_fs::Error),
+    /// Wrapper around [`patterned::Error`].
+    #[error(transparent)]
+    Patterned(#[from] patterned::Error),
 }
 
 /// Configuration of [`FileGen`]
@@ -50,6 +54,8 @@ pub enum Config {
     #[cfg(feature = "logrotate_fs")]
     /// See [`logrotate_fs::Config`].
     LogrotateFs(logrotate_fs::Config),
+    /// See [`patterned::Config`].
+    Patterned(patterned::Config),
 }
 
 #[derive(Debug)]
@@ -65,6 +71,8 @@ pub enum FileGen {
     #[cfg(feature = "logrotate_fs")]
     /// See [`logrotate_fs::Server`] for details.
     LogrotateFs(logrotate_fs::Server),
+    /// See [`patterned::Server`] for details.
+    Patterned(patterned::Server),
 }
 
 impl FileGen {
@@ -93,6 +101,7 @@ impl FileGen {
             Config::LogrotateFs(c) => {
                 Self::LogrotateFs(logrotate_fs::Server::new(general, c, shutdown)?)
             }
+            Config::Patterned(c) => Self::Patterned(patterned::Server::new(general, c, shutdown)?),
         };
         Ok(srv)
     }
@@ -115,6 +124,7 @@ impl FileGen {
             Self::Logrotate(inner) => inner.spin().await?,
             #[cfg(feature = "logrotate_fs")]
             Self::LogrotateFs(inner) => inner.spin().await?,
+            Self::Patterned(inner) => inner.spin().await?,
         }
 
         Ok(())