-
Notifications
You must be signed in to change notification settings - Fork 105
fix: code changes for resolving Critical and High vulnerabilities in the security tab of DKM repo. #519
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This pull request addresses Critical and High security vulnerabilities identified in the DKM repository's security tab. The changes focus on preventing log injection attacks and improving secure random number generation.
Key Changes:
- Implements log injection prevention across C# logging statements by sanitizing carriage return (
\r) and newline (\n) characters from user-controlled input before logging - Replaces insecure
Math.random()-based UUID generation with cryptographically securecrypto.getRandomValues()implementation in the frontend - Adds
X-Frame-Options: DENYsecurity header to prevent clickjacking attacks
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 27 comments.
Show a summary per file
| File | Description |
|---|---|
| App/kernel-memory/service/Service.AspNetCore/WebAPIEndpoints.cs | Sanitizes index names, document IDs, and filenames in log statements to prevent log injection |
| App/kernel-memory/service/Core/Pipeline/InProcessPipelineOrchestrator.cs | Sanitizes pipeline index and document ID in error and info logs |
| App/kernel-memory/service/Core/Pipeline/BaseOrchestrator.cs | Sanitizes pipeline identifiers in debug logs |
| App/kernel-memory/service/Core/Handlers/TextPartitioningHandler.cs | Sanitizes pipeline identifiers in debug and warning logs |
| App/kernel-memory/service/Core/Handlers/TextExtractionHandler.cs | Sanitizes pipeline identifiers in debug logs |
| App/kernel-memory/service/Core/Handlers/SummarizationParallelHandler.cs | Sanitizes pipeline identifiers in debug logs |
| App/kernel-memory/service/Core/Handlers/SummarizationHandler.cs | Sanitizes pipeline identifiers in debug logs |
| App/kernel-memory/service/Core/Handlers/SaveRecordsHandler.cs | Sanitizes pipeline identifiers, record IDs, and index names across multiple log statements |
| App/kernel-memory/service/Core/Handlers/KeywordExtractingHandler.cs | Sanitizes pipeline identifiers in debug logs |
| App/kernel-memory/service/Core/Handlers/GenerateEmbeddingsParallelHandler.cs | Sanitizes pipeline identifiers and generator type names in trace logs |
| App/kernel-memory/service/Core/Handlers/GenerateEmbeddingsHandlerBase.cs | Sanitizes pipeline identifiers and sub-step names in trace logs |
| App/kernel-memory/service/Core/Handlers/GenerateEmbeddingsHandler.cs | Sanitizes pipeline identifiers and generator type names in trace logs |
| App/kernel-memory/service/Core/Handlers/DeleteIndexHandler.cs | Sanitizes pipeline identifiers in debug logs |
| App/kernel-memory/service/Core/Handlers/DeleteGeneratedFilesHandler.cs | Sanitizes pipeline identifiers in debug logs |
| App/kernel-memory/service/Core/Handlers/DeleteDocumentHandler.cs | Sanitizes pipeline identifiers in debug logs |
| App/kernel-memory/service/Core/FileSystem/DevTools/VolatileFileSystem.cs | Sanitizes file paths in error logs |
| App/kernel-memory/service/Core/FileSystem/DevTools/DiskFileSystem.cs | Sanitizes file paths in warning, trace, error, and debug logs |
| App/kernel-memory/service/Core/DocumentStorage/DevTools/SimpleFileStorage.cs | Sanitizes index, document ID, and filename in error logs |
| App/kernel-memory/extensions/Elasticsearch/Elasticsearch/ElasticsearchMemory.cs | Sanitizes search text, index names, and filter debug strings in trace logs |
| App/kernel-memory/extensions/AzureBlobs/AzureBlobsStorage.cs | Sanitizes blob names, file paths, and prefixes in various log levels |
| App/kernel-memory/extensions/AzureAISearch/AzureAISearch/AzureAISearchMemory.cs | Sanitizes record IDs and index names in debug and trace logs |
| App/kernel-memory/extensions/AWS/S3/AWSS3Storage.cs | Sanitizes index names, object keys, and prefixes in trace, warning, and info logs |
| App/frontend-app/src/components/documentViewer/pageNumberTab.tsx | Implements URL sanitization to prevent CRLF injection in storage URLs |
| App/frontend-app/src/components/chat/chatRoom.tsx | Replaces insecure UUID generation with cryptographically secure implementation using Web Crypto API |
| App/frontend-app/public/web.config | Adds X-Frame-Options header to prevent clickjacking attacks |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| DataPipeline pipeline, CancellationToken cancellationToken = default) | ||
| { | ||
| this._log.LogDebug("Extracting text, pipeline '{0}/{1}'", pipeline.Index, pipeline.DocumentId); | ||
| this._log.LogDebug("Extracting text, pipeline '{Index}/{DocumentId}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The log template format string uses single quotes around placeholders (e.g., 'Extracting text, pipeline '{Index}/{DocumentId}''), which is inconsistent with the structured logging pattern. Structured logging placeholders should not be enclosed in quotes within the message template - the entire string portion can have quotes, but placeholders should remain bare like {Index}/{DocumentId}.
| this._log.LogDebug("Extracting text, pipeline '{Index}/{DocumentId}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty)); | |
| this._log.LogDebug("Extracting text, pipeline {Index}/{DocumentId}", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
| DataPipeline pipeline, CancellationToken cancellationToken = default) | ||
| { | ||
| this._log.LogDebug("Generating summary, pipeline '{0}/{1}'", pipeline.Index, pipeline.DocumentId); | ||
| this._log.LogDebug("Generating summary, pipeline '{Index}/{DocumentId}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The log template format string uses single quotes around placeholders (e.g., 'Generating summary, pipeline '{Index}/{DocumentId}''), which is inconsistent with the structured logging pattern. Structured logging placeholders should not be enclosed in quotes within the message template - the entire string portion can have quotes, but placeholders should remain bare like {Index}/{DocumentId}.
| this._log.LogDebug("Generating summary, pipeline '{Index}/{DocumentId}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty)); | |
| this._log.LogDebug("Generating summary, pipeline {Index}/{DocumentId}", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
| DataPipeline pipeline, CancellationToken cancellationToken = default) | ||
| { | ||
| this._log.LogDebug("Deleting document, pipeline '{0}/{1}'", pipeline.Index, pipeline.DocumentId); | ||
| this._log.LogDebug("Deleting document, pipeline '{Index}/{DocumentId}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The log template format string uses single quotes around placeholders (e.g., 'Deleting document, pipeline '{Index}/{DocumentId}''), which is inconsistent with the structured logging pattern. Structured logging placeholders should not be enclosed in quotes within the message template - the entire string portion can have quotes, but placeholders should remain bare like {Index}/{DocumentId}.
| this._log.LogDebug("Deleting document, pipeline '{Index}/{DocumentId}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty)); | |
| this._log.LogDebug("Deleting document, pipeline {Index}/{DocumentId}", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
| try | ||
| { | ||
| this._log.LogTrace("Saving batch of {0} records in index '{1}'", records.Count, pipeline.Index); | ||
| this._log.LogTrace("Saving batch of {RecordCount} records in index '{Index}'", records.Count, pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The log template format string uses single quotes around placeholders (e.g., "Saving batch of {RecordCount} records in index '{Index}'"), which is inconsistent. Either all placeholders should have quotes or none should. For structured logging best practices, placeholders should remain bare without quotes in the template.
| this._log.LogTrace("Saving batch of {RecordCount} records in index '{Index}'", records.Count, pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty)); | |
| this._log.LogTrace("Saving batch of {RecordCount} records in index {Index}", records.Count, pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
| if (!this._embeddingGenerationEnabled) | ||
| { | ||
| this._log.LogTrace("Embedding generation is disabled, skipping - pipeline '{0}/{1}'", pipeline.Index, pipeline.DocumentId); | ||
| this._log.LogTrace("Embedding generation is disabled, skipping - pipeline '{Index}/{DocumentId}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The log template format string uses single quotes around placeholders (e.g., "Embedding generation is disabled, skipping - pipeline '{Index}/{DocumentId}'"), which is inconsistent with the structured logging pattern. Structured logging placeholders should not be enclosed in quotes within the message template - the entire string portion can have quotes, but placeholders should remain bare like {Index}/{DocumentId}.
| this._log.LogTrace("Embedding generation is disabled, skipping - pipeline '{Index}/{DocumentId}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty)); | |
| this._log.LogTrace("Embedding generation is disabled, skipping - pipeline {Index}/{DocumentId}", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
| this._log.LogTrace("Generating embedding, pipeline '{Index}/{DocumentId}', generator '{GeneratorType}', content size {TokenCount} tokens", | ||
| pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), | ||
| pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty), | ||
| generator.GetType().FullName?.Replace("\r", string.Empty).Replace("\n", string.Empty), | ||
| generator.CountTokens(partitionInfo.PartitionContent)); |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The log template format string uses single quotes around placeholders inconsistently (e.g., "Generating embedding, pipeline '{Index}/{DocumentId}', generator '{GeneratorType}'..."). For structured logging best practices, placeholders should remain bare without quotes in the template.
| this._log.LogTrace("Deleting objects with prefix '{0}'", prefix); | ||
| this._log.LogTrace( | ||
| "Deleting objects with prefix {Prefix}", | ||
| prefix?.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Condition is always not null because of call to method IsNullOrWhiteSpace.
| prefix?.Replace("\r", string.Empty).Replace("\n", string.Empty)); | |
| prefix.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
| } | ||
|
|
||
| this._log.LogInformation("Deleting blobs at {0}", prefix); | ||
| this._log.LogInformation("Deleting blobs at {Prefix}", prefix?.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Condition is always not null because of call to method IsNullOrWhiteSpace.
| this._log.LogInformation("Deleting blobs at {Prefix}", prefix?.Replace("\r", string.Empty).Replace("\n", string.Empty)); | |
| this._log.LogInformation("Deleting blobs at {Prefix}", prefix.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
| } | ||
|
|
||
| this._log.LogDebug("Creating directory {0}", path); | ||
| this._log.LogDebug("Creating directory {Path}", path?.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Condition is always not null because of call to method IsNullOrEmpty.
| this._log.LogDebug("Creating directory {Path}", path?.Replace("\r", string.Empty).Replace("\n", string.Empty)); | |
| this._log.LogDebug("Creating directory {Path}", path.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
|
|
||
| log.LogTrace("Downloading file '{0}', size '{1}', type '{2}'", filename, file.FileSize, file.FileType); | ||
| log.LogTrace("Downloading file {FileName}, size {FileSize}, type {FileType}", | ||
| filename?.Replace("\r", string.Empty).Replace("\n", string.Empty), |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Condition is always not null because of access to local variable isValid.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 25 out of 25 changed files in this pull request and generated 10 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const base = window.ENV.STORAGE_URL.replace(/\r|\n/g, "").replace(/\/+$/,""); | ||
| const path = new URL(selectedPageMetadata.document_url, base).pathname.replace(/^\/+/, ""); | ||
| const imageUrl = `${base}/${path}/`; |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The URL construction may throw an error if window.ENV.STORAGE_URL is not a valid base URL or if selectedPageMetadata.document_url is malformed. The new URL constructor requires a valid base URL and will throw a TypeError if the base is invalid. Consider adding error handling or validation to ensure window.ENV.STORAGE_URL is a valid URL before constructing the URL object.
| private async Task UploadFormFilesAsync(DataPipeline pipeline, CancellationToken cancellationToken) | ||
| { | ||
| this.Log.LogDebug("Uploading {0} files, pipeline '{1}/{2}'", pipeline.FilesToUpload.Count, pipeline.Index, pipeline.DocumentId); | ||
| this.Log.LogDebug("Uploading {FileCount} files, pipeline '{Index}/{DocumentId}'", pipeline.FilesToUpload.Count, pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The log message template contains literal single quotes around placeholder names, which is inconsistent with structured logging best practices. Change 'pipeline '{Index}/{DocumentId}'' to 'pipeline {Index}/{DocumentId}' to ensure proper structured logging. The quotes interfere with log parsing and make the placeholders appear as literal text rather than structured properties.
| this._log.LogTrace("Saving record {RecordId} in index '{Index}'", record.Id?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty)); | ||
| await db.UpsertAsync(pipeline.Index, record, cancellationToken).ConfigureAwait(false); | ||
| } | ||
| catch (IndexNotFoundException e) | ||
| { | ||
| this._log.LogWarning(e, "Index {0} not found, attempting to create it", pipeline.Index); | ||
| this._log.LogWarning( | ||
| e, | ||
| "Index {Index} not found, attempting to create it", | ||
| pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty)); | ||
| await this.CreateIndexOnceAsync(db, createdIndexes, pipeline.Index, record.Vector.Length, cancellationToken, true).ConfigureAwait(false); | ||
|
|
||
| this._log.LogTrace("Retry: saving record {0} in index '{1}'", record.Id, pipeline.Index); | ||
| this._log.LogTrace( | ||
| "Retry: saving record {RecordId} in index {Index}", | ||
| record.Id?.Replace("\r", string.Empty).Replace("\n", string.Empty), | ||
| pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The log message templates contain literal single quotes around placeholder names in multiple locations. Remove the single quotes from 'Saving record {RecordId} in index '{Index}'' and 'Index {Index}' to ensure proper structured logging consistency. The single quotes interfere with structured logging frameworks and make the placeholders appear as literal text.
| this._log.LogTrace("Saving batch of {RecordCount} records in index '{Index}'", records.Count, pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty)); | ||
| await dbBatch.UpsertBatchAsync(pipeline.Index, records, cancellationToken).ToListAsync(cancellationToken).ConfigureAwait(false); | ||
| } | ||
| catch (IndexNotFoundException e) | ||
| { | ||
| this._log.LogWarning(e, "Index {0} not found, attempting to create it", pipeline.Index); | ||
| this._log.LogWarning(e, "Index {Index} not found, attempting to create it", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty)); | ||
| await this.CreateIndexOnceAsync(db, createdIndexes, pipeline.Index, records[0].Vector.Length, cancellationToken, true).ConfigureAwait(false); | ||
|
|
||
| this._log.LogTrace("Retry: Saving batch of {0} records in index '{1}'", records.Count, pipeline.Index); | ||
| this._log.LogTrace( | ||
| "Retry: Saving batch of {RecordCount} records in index {Index}", | ||
| records.Count, | ||
| pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty)); |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The log message templates contain literal single quotes around placeholder names. Change 'Saving batch of {RecordCount} records in index '{Index}'' and 'Retry: Saving batch of {RecordCount} records in index {Index}' to remove the single quotes around '{Index}' for consistency. The single quotes interfere with structured logging frameworks and make the placeholders appear as literal text.
| protected async Task UpdatePipelineStatusAsync(DataPipeline pipeline, CancellationToken cancellationToken) | ||
| { | ||
| this.Log.LogDebug("Saving pipeline status to '{0}/{1}/{2}'", pipeline.Index, pipeline.DocumentId, Constants.PipelineStatusFilename); | ||
| this.Log.LogDebug("Saving pipeline status to '{Index}/{DocumentId}/{FileName}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty), Constants.PipelineStatusFilename); |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The log message template contains literal single quotes around placeholder names, which is inconsistent with structured logging best practices. The template string '{Index}/{DocumentId}/{FileName}' should be 'Saving pipeline status to {Index}/{DocumentId}/{FileName}' without the surrounding quotes. The quotes make the placeholders appear as literal text rather than structured log properties. Additionally, Constants.PipelineStatusFilename should not be sanitized since it's a constant from the codebase, not user input.
| this.Log.LogDebug("Saving pipeline status to '{Index}/{DocumentId}/{FileName}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty), Constants.PipelineStatusFilename); | |
| this.Log.LogDebug("Saving pipeline status to {Index}/{DocumentId}/{FileName}", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty), Constants.PipelineStatusFilename); |
| ); | ||
| } | ||
|
|
||
| // Avoid insecure fallback (Math.random); fail fast if unsupported. |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing space after the comment marker. The comment should be '// Avoid insecure fallback...' with a space after '//' following standard JavaScript comment formatting conventions.
|
|
||
| log.LogTrace("Downloading file '{0}', size '{1}', type '{2}'", filename, file.FileSize, file.FileType); | ||
| log.LogTrace("Downloading file {FileName}, size {FileSize}, type {FileType}", | ||
| filename?.Replace("\r", string.Empty).Replace("\n", string.Empty), |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Condition is always not null because of access to local variable isValid.
| filename?.Replace("\r", string.Empty).Replace("\n", string.Empty), | |
| filename.Replace("\r", string.Empty).Replace("\n", string.Empty), |
Purpose
Does this introduce a breaking change?
Golden Path Validation
Deployment Validation
What to Check
Verify that the following are valid
Other Information