Skip to content

Conversation

@Akhileswara-Microsoft
Copy link

Purpose

  • ...

Does this introduce a breaking change?

  • Yes
  • No

Golden Path Validation

  • I have tested the primary workflows (the "golden path") to ensure they function correctly without errors.

Deployment Validation

  • I have validated the deployment process successfully and all services are running as expected with this change.

What to Check

Verify that the following are valid

  • ...

Other Information

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request addresses Critical and High security vulnerabilities identified in the DKM repository's security tab. The changes focus on preventing log injection attacks and improving secure random number generation.

Key Changes:

  • Implements log injection prevention across C# logging statements by sanitizing carriage return (\r) and newline (\n) characters from user-controlled input before logging
  • Replaces insecure Math.random()-based UUID generation with cryptographically secure crypto.getRandomValues() implementation in the frontend
  • Adds X-Frame-Options: DENY security header to prevent clickjacking attacks

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 27 comments.

Show a summary per file
File Description
App/kernel-memory/service/Service.AspNetCore/WebAPIEndpoints.cs Sanitizes index names, document IDs, and filenames in log statements to prevent log injection
App/kernel-memory/service/Core/Pipeline/InProcessPipelineOrchestrator.cs Sanitizes pipeline index and document ID in error and info logs
App/kernel-memory/service/Core/Pipeline/BaseOrchestrator.cs Sanitizes pipeline identifiers in debug logs
App/kernel-memory/service/Core/Handlers/TextPartitioningHandler.cs Sanitizes pipeline identifiers in debug and warning logs
App/kernel-memory/service/Core/Handlers/TextExtractionHandler.cs Sanitizes pipeline identifiers in debug logs
App/kernel-memory/service/Core/Handlers/SummarizationParallelHandler.cs Sanitizes pipeline identifiers in debug logs
App/kernel-memory/service/Core/Handlers/SummarizationHandler.cs Sanitizes pipeline identifiers in debug logs
App/kernel-memory/service/Core/Handlers/SaveRecordsHandler.cs Sanitizes pipeline identifiers, record IDs, and index names across multiple log statements
App/kernel-memory/service/Core/Handlers/KeywordExtractingHandler.cs Sanitizes pipeline identifiers in debug logs
App/kernel-memory/service/Core/Handlers/GenerateEmbeddingsParallelHandler.cs Sanitizes pipeline identifiers and generator type names in trace logs
App/kernel-memory/service/Core/Handlers/GenerateEmbeddingsHandlerBase.cs Sanitizes pipeline identifiers and sub-step names in trace logs
App/kernel-memory/service/Core/Handlers/GenerateEmbeddingsHandler.cs Sanitizes pipeline identifiers and generator type names in trace logs
App/kernel-memory/service/Core/Handlers/DeleteIndexHandler.cs Sanitizes pipeline identifiers in debug logs
App/kernel-memory/service/Core/Handlers/DeleteGeneratedFilesHandler.cs Sanitizes pipeline identifiers in debug logs
App/kernel-memory/service/Core/Handlers/DeleteDocumentHandler.cs Sanitizes pipeline identifiers in debug logs
App/kernel-memory/service/Core/FileSystem/DevTools/VolatileFileSystem.cs Sanitizes file paths in error logs
App/kernel-memory/service/Core/FileSystem/DevTools/DiskFileSystem.cs Sanitizes file paths in warning, trace, error, and debug logs
App/kernel-memory/service/Core/DocumentStorage/DevTools/SimpleFileStorage.cs Sanitizes index, document ID, and filename in error logs
App/kernel-memory/extensions/Elasticsearch/Elasticsearch/ElasticsearchMemory.cs Sanitizes search text, index names, and filter debug strings in trace logs
App/kernel-memory/extensions/AzureBlobs/AzureBlobsStorage.cs Sanitizes blob names, file paths, and prefixes in various log levels
App/kernel-memory/extensions/AzureAISearch/AzureAISearch/AzureAISearchMemory.cs Sanitizes record IDs and index names in debug and trace logs
App/kernel-memory/extensions/AWS/S3/AWSS3Storage.cs Sanitizes index names, object keys, and prefixes in trace, warning, and info logs
App/frontend-app/src/components/documentViewer/pageNumberTab.tsx Implements URL sanitization to prevent CRLF injection in storage URLs
App/frontend-app/src/components/chat/chatRoom.tsx Replaces insecure UUID generation with cryptographically secure implementation using Web Crypto API
App/frontend-app/public/web.config Adds X-Frame-Options header to prevent clickjacking attacks

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

DataPipeline pipeline, CancellationToken cancellationToken = default)
{
this._log.LogDebug("Extracting text, pipeline '{0}/{1}'", pipeline.Index, pipeline.DocumentId);
this._log.LogDebug("Extracting text, pipeline '{Index}/{DocumentId}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty));
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The log template format string uses single quotes around placeholders (e.g., 'Extracting text, pipeline '{Index}/{DocumentId}''), which is inconsistent with the structured logging pattern. Structured logging placeholders should not be enclosed in quotes within the message template - the entire string portion can have quotes, but placeholders should remain bare like {Index}/{DocumentId}.

Suggested change
this._log.LogDebug("Extracting text, pipeline '{Index}/{DocumentId}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty));
this._log.LogDebug("Extracting text, pipeline {Index}/{DocumentId}", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty));

Copilot uses AI. Check for mistakes.
DataPipeline pipeline, CancellationToken cancellationToken = default)
{
this._log.LogDebug("Generating summary, pipeline '{0}/{1}'", pipeline.Index, pipeline.DocumentId);
this._log.LogDebug("Generating summary, pipeline '{Index}/{DocumentId}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty));
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The log template format string uses single quotes around placeholders (e.g., 'Generating summary, pipeline '{Index}/{DocumentId}''), which is inconsistent with the structured logging pattern. Structured logging placeholders should not be enclosed in quotes within the message template - the entire string portion can have quotes, but placeholders should remain bare like {Index}/{DocumentId}.

Suggested change
this._log.LogDebug("Generating summary, pipeline '{Index}/{DocumentId}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty));
this._log.LogDebug("Generating summary, pipeline {Index}/{DocumentId}", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty));

Copilot uses AI. Check for mistakes.
DataPipeline pipeline, CancellationToken cancellationToken = default)
{
this._log.LogDebug("Deleting document, pipeline '{0}/{1}'", pipeline.Index, pipeline.DocumentId);
this._log.LogDebug("Deleting document, pipeline '{Index}/{DocumentId}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty));
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The log template format string uses single quotes around placeholders (e.g., 'Deleting document, pipeline '{Index}/{DocumentId}''), which is inconsistent with the structured logging pattern. Structured logging placeholders should not be enclosed in quotes within the message template - the entire string portion can have quotes, but placeholders should remain bare like {Index}/{DocumentId}.

Suggested change
this._log.LogDebug("Deleting document, pipeline '{Index}/{DocumentId}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty));
this._log.LogDebug("Deleting document, pipeline {Index}/{DocumentId}", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty));

Copilot uses AI. Check for mistakes.
try
{
this._log.LogTrace("Saving batch of {0} records in index '{1}'", records.Count, pipeline.Index);
this._log.LogTrace("Saving batch of {RecordCount} records in index '{Index}'", records.Count, pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty));
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The log template format string uses single quotes around placeholders (e.g., "Saving batch of {RecordCount} records in index '{Index}'"), which is inconsistent. Either all placeholders should have quotes or none should. For structured logging best practices, placeholders should remain bare without quotes in the template.

Suggested change
this._log.LogTrace("Saving batch of {RecordCount} records in index '{Index}'", records.Count, pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty));
this._log.LogTrace("Saving batch of {RecordCount} records in index {Index}", records.Count, pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty));

Copilot uses AI. Check for mistakes.
if (!this._embeddingGenerationEnabled)
{
this._log.LogTrace("Embedding generation is disabled, skipping - pipeline '{0}/{1}'", pipeline.Index, pipeline.DocumentId);
this._log.LogTrace("Embedding generation is disabled, skipping - pipeline '{Index}/{DocumentId}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty));
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The log template format string uses single quotes around placeholders (e.g., "Embedding generation is disabled, skipping - pipeline '{Index}/{DocumentId}'"), which is inconsistent with the structured logging pattern. Structured logging placeholders should not be enclosed in quotes within the message template - the entire string portion can have quotes, but placeholders should remain bare like {Index}/{DocumentId}.

Suggested change
this._log.LogTrace("Embedding generation is disabled, skipping - pipeline '{Index}/{DocumentId}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty));
this._log.LogTrace("Embedding generation is disabled, skipping - pipeline {Index}/{DocumentId}", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty));

Copilot uses AI. Check for mistakes.
Comment on lines +144 to +148
this._log.LogTrace("Generating embedding, pipeline '{Index}/{DocumentId}', generator '{GeneratorType}', content size {TokenCount} tokens",
pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty),
pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty),
generator.GetType().FullName?.Replace("\r", string.Empty).Replace("\n", string.Empty),
generator.CountTokens(partitionInfo.PartitionContent));
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The log template format string uses single quotes around placeholders inconsistently (e.g., "Generating embedding, pipeline '{Index}/{DocumentId}', generator '{GeneratorType}'..."). For structured logging best practices, placeholders should remain bare without quotes in the template.

Copilot uses AI. Check for mistakes.
this._log.LogTrace("Deleting objects with prefix '{0}'", prefix);
this._log.LogTrace(
"Deleting objects with prefix {Prefix}",
prefix?.Replace("\r", string.Empty).Replace("\n", string.Empty));
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Condition is always not null because of call to method IsNullOrWhiteSpace.

Suggested change
prefix?.Replace("\r", string.Empty).Replace("\n", string.Empty));
prefix.Replace("\r", string.Empty).Replace("\n", string.Empty));

Copilot uses AI. Check for mistakes.
}

this._log.LogInformation("Deleting blobs at {0}", prefix);
this._log.LogInformation("Deleting blobs at {Prefix}", prefix?.Replace("\r", string.Empty).Replace("\n", string.Empty));
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Condition is always not null because of call to method IsNullOrWhiteSpace.

Suggested change
this._log.LogInformation("Deleting blobs at {Prefix}", prefix?.Replace("\r", string.Empty).Replace("\n", string.Empty));
this._log.LogInformation("Deleting blobs at {Prefix}", prefix.Replace("\r", string.Empty).Replace("\n", string.Empty));

Copilot uses AI. Check for mistakes.
}

this._log.LogDebug("Creating directory {0}", path);
this._log.LogDebug("Creating directory {Path}", path?.Replace("\r", string.Empty).Replace("\n", string.Empty));
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Condition is always not null because of call to method IsNullOrEmpty.

Suggested change
this._log.LogDebug("Creating directory {Path}", path?.Replace("\r", string.Empty).Replace("\n", string.Empty));
this._log.LogDebug("Creating directory {Path}", path.Replace("\r", string.Empty).Replace("\n", string.Empty));

Copilot uses AI. Check for mistakes.

log.LogTrace("Downloading file '{0}', size '{1}', type '{2}'", filename, file.FileSize, file.FileType);
log.LogTrace("Downloading file {FileName}, size {FileSize}, type {FileType}",
filename?.Replace("\r", string.Empty).Replace("\n", string.Empty),
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Condition is always not null because of access to local variable isValid.

Copilot uses AI. Check for mistakes.
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated 10 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +16 to +18
const base = window.ENV.STORAGE_URL.replace(/\r|\n/g, "").replace(/\/+$/,"");
const path = new URL(selectedPageMetadata.document_url, base).pathname.replace(/^\/+/, "");
const imageUrl = `${base}/${path}/`;
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The URL construction may throw an error if window.ENV.STORAGE_URL is not a valid base URL or if selectedPageMetadata.document_url is malformed. The new URL constructor requires a valid base URL and will throw a TypeError if the base is invalid. Consider adding error handling or validation to ensure window.ENV.STORAGE_URL is a valid URL before constructing the URL object.

Copilot uses AI. Check for mistakes.
private async Task UploadFormFilesAsync(DataPipeline pipeline, CancellationToken cancellationToken)
{
this.Log.LogDebug("Uploading {0} files, pipeline '{1}/{2}'", pipeline.FilesToUpload.Count, pipeline.Index, pipeline.DocumentId);
this.Log.LogDebug("Uploading {FileCount} files, pipeline '{Index}/{DocumentId}'", pipeline.FilesToUpload.Count, pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty));
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The log message template contains literal single quotes around placeholder names, which is inconsistent with structured logging best practices. Change 'pipeline '{Index}/{DocumentId}'' to 'pipeline {Index}/{DocumentId}' to ensure proper structured logging. The quotes interfere with log parsing and make the placeholders appear as literal text rather than structured properties.

Copilot uses AI. Check for mistakes.
Comment on lines +272 to +286
this._log.LogTrace("Saving record {RecordId} in index '{Index}'", record.Id?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty));
await db.UpsertAsync(pipeline.Index, record, cancellationToken).ConfigureAwait(false);
}
catch (IndexNotFoundException e)
{
this._log.LogWarning(e, "Index {0} not found, attempting to create it", pipeline.Index);
this._log.LogWarning(
e,
"Index {Index} not found, attempting to create it",
pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty));
await this.CreateIndexOnceAsync(db, createdIndexes, pipeline.Index, record.Vector.Length, cancellationToken, true).ConfigureAwait(false);

this._log.LogTrace("Retry: saving record {0} in index '{1}'", record.Id, pipeline.Index);
this._log.LogTrace(
"Retry: saving record {RecordId} in index {Index}",
record.Id?.Replace("\r", string.Empty).Replace("\n", string.Empty),
pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty));
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The log message templates contain literal single quotes around placeholder names in multiple locations. Remove the single quotes from 'Saving record {RecordId} in index '{Index}'' and 'Index {Index}' to ensure proper structured logging consistency. The single quotes interfere with structured logging frameworks and make the placeholders appear as literal text.

Copilot uses AI. Check for mistakes.
Comment on lines +297 to +308
this._log.LogTrace("Saving batch of {RecordCount} records in index '{Index}'", records.Count, pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty));
await dbBatch.UpsertBatchAsync(pipeline.Index, records, cancellationToken).ToListAsync(cancellationToken).ConfigureAwait(false);
}
catch (IndexNotFoundException e)
{
this._log.LogWarning(e, "Index {0} not found, attempting to create it", pipeline.Index);
this._log.LogWarning(e, "Index {Index} not found, attempting to create it", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty));
await this.CreateIndexOnceAsync(db, createdIndexes, pipeline.Index, records[0].Vector.Length, cancellationToken, true).ConfigureAwait(false);

this._log.LogTrace("Retry: Saving batch of {0} records in index '{1}'", records.Count, pipeline.Index);
this._log.LogTrace(
"Retry: Saving batch of {RecordCount} records in index {Index}",
records.Count,
pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty));
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The log message templates contain literal single quotes around placeholder names. Change 'Saving batch of {RecordCount} records in index '{Index}'' and 'Retry: Saving batch of {RecordCount} records in index {Index}' to remove the single quotes around '{Index}' for consistency. The single quotes interfere with structured logging frameworks and make the placeholders appear as literal text.

Copilot uses AI. Check for mistakes.
protected async Task UpdatePipelineStatusAsync(DataPipeline pipeline, CancellationToken cancellationToken)
{
this.Log.LogDebug("Saving pipeline status to '{0}/{1}/{2}'", pipeline.Index, pipeline.DocumentId, Constants.PipelineStatusFilename);
this.Log.LogDebug("Saving pipeline status to '{Index}/{DocumentId}/{FileName}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty), Constants.PipelineStatusFilename);
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The log message template contains literal single quotes around placeholder names, which is inconsistent with structured logging best practices. The template string '{Index}/{DocumentId}/{FileName}' should be 'Saving pipeline status to {Index}/{DocumentId}/{FileName}' without the surrounding quotes. The quotes make the placeholders appear as literal text rather than structured log properties. Additionally, Constants.PipelineStatusFilename should not be sanitized since it's a constant from the codebase, not user input.

Suggested change
this.Log.LogDebug("Saving pipeline status to '{Index}/{DocumentId}/{FileName}'", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty), Constants.PipelineStatusFilename);
this.Log.LogDebug("Saving pipeline status to {Index}/{DocumentId}/{FileName}", pipeline.Index?.Replace("\r", string.Empty).Replace("\n", string.Empty), pipeline.DocumentId?.Replace("\r", string.Empty).Replace("\n", string.Empty), Constants.PipelineStatusFilename);

Copilot uses AI. Check for mistakes.
);
}

// Avoid insecure fallback (Math.random); fail fast if unsupported.
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing space after the comment marker. The comment should be '// Avoid insecure fallback...' with a space after '//' following standard JavaScript comment formatting conventions.

Copilot uses AI. Check for mistakes.

log.LogTrace("Downloading file '{0}', size '{1}', type '{2}'", filename, file.FileSize, file.FileType);
log.LogTrace("Downloading file {FileName}, size {FileSize}, type {FileType}",
filename?.Replace("\r", string.Empty).Replace("\n", string.Empty),
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Condition is always not null because of access to local variable isValid.

Suggested change
filename?.Replace("\r", string.Empty).Replace("\n", string.Empty),
filename.Replace("\r", string.Empty).Replace("\n", string.Empty),

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant