Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
.git
.github
.confluence-search-index
conf
dist
workspace
test-output
docs/test-logs
9 changes: 7 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ MAIN := ./cmd/conf
GO := go
GOFLAGS :=

.PHONY: build install test test-unit test-e2e release-check coverage-check fmt fmt-check lint clean
.PHONY: build install test test-unit test-e2e ci-ubuntu release-check coverage-check fmt fmt-check lint clean

## build: compile the conf binary
build:
Expand All @@ -26,7 +26,12 @@ coverage-check:

## test-e2e: run all end-to-end tests (requires CONF_E2E_DOMAIN, CONF_E2E_EMAIL, CONF_E2E_API_TOKEN, CONF_E2E_PRIMARY_SPACE_KEY, CONF_E2E_SECONDARY_SPACE_KEY)
test-e2e: build
$(GO) test -v -tags=e2e ./cmd -run '^TestWorkflow_'
$(GO) test -timeout 20m -v -tags=e2e ./cmd -run '^TestWorkflow_'

## ci-ubuntu: run the GitHub Actions ubuntu test job inside Docker
ci-ubuntu:
docker build -f docker/ubuntu-ci.Dockerfile -t conf-ci-ubuntu .
docker run --rm conf-ci-ubuntu

## release-check: run the release gate, including live sandbox E2E coverage
release-check: fmt-check lint test-unit test-e2e
Expand Down
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Write docs like code. Publish to Confluence with confidence. ✍️
## Why teams use `conf` ✨
- 📝 Markdown-first authoring with Confluence as the destination.
- 🛡️ Safe sync model with validation before remote writes.
- 👀 Clear preview step via `conf diff` for tracked pages and `conf push --preflight` for brand-new files.
- 👀 Clear preview step via `conf diff` for tracked pages and brand-new files, plus `conf push --preflight` for full push planning.
- 🔎 Local full-text search across synced Markdown with SQLite or Bleve backends.
- 🤖 Works in local repos and automation pipelines.

Expand Down Expand Up @@ -38,7 +38,7 @@ conf init
`conf init` prepares Git metadata, `.gitignore`, and `.env` scaffolding, and creates an initial commit when it initializes a new Git repository.
If `ATLASSIAN_*` or legacy `CONFLUENCE_*` credentials are already set in the environment, `conf init` writes `.env` from them without prompting.

`conf pull` mirrors Confluence hierarchy locally by placing folders and child pages in nested directories. Pages with children use `<Page>/<Page>.md` so they are distinct from pure folders. Incremental pulls reconcile remote creates, updates, and deletes without requiring `--force`. Leaf-page title renames can keep the existing Markdown path when the effective parent directory is unchanged, but pages that own subtree directories move when their self-owned directory segment changes. Hierarchy moves and ancestor/path-segment sanitization changes are surfaced as `PAGE_PATH_MOVED` notes in `conf pull`/`conf diff`, and `conf status` previews tracked moves before the next pull.
`conf pull` mirrors Confluence hierarchy locally by placing folders and child pages in nested directories. Pages with children use `<Page>/<Page>.md` so they are distinct from pure folders. Incremental pulls reconcile remote creates, updates, and deletes without requiring `--force`. Canonical pull paths always win, so older authored slugs are reconciled into the same path shape a fresh workspace would get. Hierarchy moves and ancestor/path-segment sanitization changes are surfaced as `PAGE_PATH_MOVED` notes in `conf pull`/`conf diff`, and `conf status` previews tracked moves before the next pull.

## Quick flow 🔄
> ⚠️ **IMPORTANT**: If you are developing `conf` itself, NEVER run sync commands against real Confluence spaces in the repository root. This prevents accidental commits of synced documentation. Use a separate sandbox folder.
Expand All @@ -57,7 +57,7 @@ conf validate ENG
conf diff ENG

# Preview a brand-new file before its first push
conf push .\ENG\New-Page.md --preflight
conf diff .\ENG\New-Page.md

# 4) Push local changes
conf push ENG --on-conflict=cancel
Expand All @@ -73,7 +73,9 @@ conf push ENG --on-conflict=cancel
- Removing tracked Markdown pages archives the corresponding remote page; follow-up pull removes the archived page from tracked local state
- `pull` and `push` are serialized per repository with a workspace lock, so concurrent mutating runs fail fast with a clear lock message
- `push` failures retain recovery refs and print exact `conf recover`, `git switch`, and cleanup commands for the retained run
- Status scope: `conf status` reports Markdown page drift only; use `git status` for local asset changes or `conf diff` for attachment-aware remote inspection. There is no attachment-aware `conf status` mode yet
- Status scope: `conf status` reports Markdown page drift by default; add `--attachments` to inspect attachment-only drift and orphaned local assets from the same command
- Non-interactive `push --on-conflict=pull-merge` requires `--merge-resolution=fail|keep-local|keep-remote|keep-both`
- Push never silently converts a directory-backed folder into a page; interactive runs require explicit confirmation before any folder-to-page downgrade
- Label rules: labels are trimmed, lowercased, deduplicated, and sorted; empty labels and labels containing whitespace are rejected
- Search filters: `--space`, repeatable `--label`, `--heading`, `--created-by`, `--updated-by`, date bounds, and `--result-detail`
- Git remote is optional (local Git is enough)
Expand Down
151 changes: 151 additions & 0 deletions cmd/create_preview.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
package cmd

import (
"context"
"encoding/json"
"fmt"
"path/filepath"
"strings"

"github.com/rgonek/confluence-markdown-sync/internal/converter"
"github.com/rgonek/confluence-markdown-sync/internal/fs"
syncflow "github.com/rgonek/confluence-markdown-sync/internal/sync"
)

type localCreatePreview struct {
RelPath string
Title string
ResolvedParent string
CanonicalTargetPath string
AttachmentUploads []string
ADFBytes int
ADFTopLevelNodes int
}

func buildLocalCreatePreview(
ctx context.Context,
spaceDir string,
relPath string,
domain string,
attachmentIndex map[string]string,
globalIndex syncflow.GlobalPageIndex,
) (localCreatePreview, error) {
relPath = normalizeRepoRelPath(relPath)
if relPath == "" {
return localCreatePreview{}, fmt.Errorf("relative path is required")
}

absPath := filepath.Join(spaceDir, filepath.FromSlash(relPath))
doc, err := fs.ReadMarkdownDocument(absPath)
if err != nil {
return localCreatePreview{}, err
}

title := strings.TrimSpace(doc.Frontmatter.Title)
if title == "" {
title = strings.TrimSuffix(filepath.Base(relPath), filepath.Ext(relPath))
}

pageIndex, err := syncflow.BuildPageIndex(spaceDir)
if err != nil {
return localCreatePreview{}, err
}
if err := syncflow.SeedPendingPageIDsForFiles(spaceDir, pageIndex, []string{absPath}); err != nil {
return localCreatePreview{}, err
}

referencedAssets, err := syncflow.CollectReferencedAssetPaths(spaceDir, absPath, doc.Body)
if err != nil {
return localCreatePreview{}, err
}

strictAttachmentIndex, _, err := syncflow.BuildStrictAttachmentIndex(spaceDir, absPath, doc.Body, attachmentIndex)
if err != nil {
return localCreatePreview{}, err
}
preparedBody, err := syncflow.PrepareMarkdownForAttachmentConversion(spaceDir, absPath, doc.Body, strictAttachmentIndex)
if err != nil {
return localCreatePreview{}, err
}
linkHook := syncflow.NewReverseLinkHookWithGlobalIndex(spaceDir, pageIndex, globalIndex, domain)
mediaHook := syncflow.NewReverseMediaHook(spaceDir, strictAttachmentIndex)
reverseResult, err := converter.Reverse(ctx, []byte(preparedBody), converter.ReverseConfig{
LinkHook: linkHook,
MediaHook: mediaHook,
Strict: true,
}, absPath)
if err != nil {
return localCreatePreview{}, err
}

return localCreatePreview{
RelPath: relPath,
Title: title,
ResolvedParent: resolvePreviewParent(relPath, doc.Frontmatter.ConfluenceParentPageID, pageIndex),
CanonicalTargetPath: canonicalCreatePreviewPath(relPath, title),
AttachmentUploads: referencedAssets,
ADFBytes: len(reverseResult.ADF),
ADFTopLevelNodes: adfTopLevelNodeCount(reverseResult.ADF),
}, nil
}

func resolvePreviewParent(relPath, fallbackParentID string, pageIndex syncflow.PageIndex) string {
dirPath := normalizeRepoRelPath(filepath.Dir(relPath))
if dirPath == "" || dirPath == "." {
if parentID := strings.TrimSpace(fallbackParentID); parentID != "" {
return "page " + parentID
}
return "space root"
}

for currentDir := dirPath; currentDir != "" && currentDir != "."; {
indexPath := previewIndexPathForDir(currentDir)
if indexPath != "" && normalizeRepoRelPath(indexPath) != normalizeRepoRelPath(relPath) {
if pageID := strings.TrimSpace(pageIndex[indexPath]); pageID != "" {
return fmt.Sprintf("page %s (%s)", pageID, indexPath)
}
}

nextDir := normalizeRepoRelPath(filepath.ToSlash(filepath.Dir(filepath.FromSlash(currentDir))))
if nextDir == currentDir {
break
}
currentDir = nextDir
}

if parentID := strings.TrimSpace(fallbackParentID); parentID != "" {
return "page " + parentID
}
return "space root"
}

func previewIndexPathForDir(dirPath string) string {
dirPath = normalizeRepoRelPath(dirPath)
if dirPath == "" || dirPath == "." {
return ""
}
dirBase := strings.TrimSpace(filepath.Base(filepath.FromSlash(dirPath)))
if dirBase == "" || dirBase == "." {
return ""
}
return normalizeRepoRelPath(filepath.ToSlash(filepath.Join(dirPath, dirBase+".md")))
}

func canonicalCreatePreviewPath(relPath, title string) string {
dirPath := normalizeRepoRelPath(filepath.Dir(relPath))
fileName := fs.SanitizeMarkdownFilename(title)
if dirPath == "" || dirPath == "." {
return fileName
}
return normalizeRepoRelPath(filepath.ToSlash(filepath.Join(dirPath, fileName)))
}

func adfTopLevelNodeCount(adf []byte) int {
var parsed struct {
Content []json.RawMessage `json:"content"`
}
if err := json.Unmarshal(adf, &parsed); err != nil {
return 0
}
return len(parsed.Content)
}
84 changes: 68 additions & 16 deletions cmd/diff.go
Original file line number Diff line number Diff line change
Expand Up @@ -92,10 +92,7 @@ func runDiff(cmd *cobra.Command, target config.Target) (runErr error) {
if err := ensureWorkspaceSyncReady("diff"); err != nil {
return err
}
if err := ensureDiffTargetSupportsRemoteComparison(target); err != nil {
return err
}
initialCtx, err := resolveInitialPullContext(target)
initialCtx, err := resolveInitialDiffContext(target)
if err != nil {
return err
}
Expand Down Expand Up @@ -143,6 +140,35 @@ func runDiff(cmd *cobra.Command, target config.Target) (runErr error) {
return fmt.Errorf("load state: %w", err)
}

if target.IsFile() {
absPath, err := filepath.Abs(target.Value)
if err != nil {
return err
}
doc, err := fs.ReadMarkdownDocument(absPath)
if err != nil {
return fmt.Errorf("read target file %s: %w", target.Value, err)
}
if strings.TrimSpace(doc.Frontmatter.ID) == "" {
globalPageIndex, buildErr := buildWorkspaceGlobalPageIndex(diffCtx.spaceDir)
if buildErr != nil {
return fmt.Errorf("build global page index: %w", buildErr)
}
preview, previewErr := buildLocalCreatePreview(ctx, diffCtx.spaceDir, diffDisplayRelPath(diffCtx.spaceDir, absPath), cfg.Domain, state.AttachmentIndex, globalPageIndex)
if previewErr != nil {
return fmt.Errorf("build create preview for %s: %w", target.Value, previewErr)
}
printDiffCreatePreview(out, preview)
report.MutatedFiles = append(report.MutatedFiles, preview.RelPath)
report.MutatedPages = append(report.MutatedPages, commandRunReportPage{
Operation: "create-preview",
Path: preview.RelPath,
Title: preview.Title,
})
return nil
}
}

pages, err := listAllDiffPages(ctx, remote, confluence.PageListOptions{
SpaceID: space.ID,
SpaceKey: space.Key,
Expand Down Expand Up @@ -212,29 +238,38 @@ func runDiff(cmd *cobra.Command, target config.Target) (runErr error) {
return err
}

func ensureDiffTargetSupportsRemoteComparison(target config.Target) error {
func resolveInitialDiffContext(target config.Target) (initialPullContext, error) {
if !target.IsFile() {
return nil
return resolveInitialPullContext(target)
}

absPath, err := filepath.Abs(target.Value)
if err != nil {
return err
return initialPullContext{}, err
}

doc, err := fs.ReadMarkdownDocument(absPath)
if err != nil {
return fmt.Errorf("read target file %s: %w", target.Value, err)
return initialPullContext{}, fmt.Errorf("read target file %s: %w", target.Value, err)
}

spaceDir := findSpaceDirFromFile(absPath, "")
spaceKey := ""
if state, stateErr := fs.LoadState(spaceDir); stateErr == nil {
spaceKey = strings.TrimSpace(state.SpaceKey)
}
if spaceKey == "" {
spaceKey = inferSpaceKeyFromDirName(spaceDir)
}
if strings.TrimSpace(doc.Frontmatter.ID) != "" {
return nil
if spaceKey == "" {
return initialPullContext{}, fmt.Errorf("target file %s missing tracked space context; run pull with a space target first", target.Value)
}

return fmt.Errorf(
"target file %s has no id, so diff cannot compare it to an existing remote page; for a brand-new page preview, run `conf push --preflight %s`",
target.Value,
target.Value,
)
return initialPullContext{
spaceKey: spaceKey,
spaceDir: spaceDir,
targetPageID: strings.TrimSpace(doc.Frontmatter.ID),
fixedDir: true,
}, nil
}

func runDiffFileMode(
Expand Down Expand Up @@ -349,6 +384,23 @@ func runDiffFileMode(
return result, err
}

func printDiffCreatePreview(out io.Writer, preview localCreatePreview) {
_, _ = fmt.Fprintln(out, "new page preview:")
_, _ = fmt.Fprintf(out, " operation: create page %q\n", preview.Title)
_, _ = fmt.Fprintf(out, " source file: %s\n", preview.RelPath)
_, _ = fmt.Fprintf(out, " resolved parent: %s\n", preview.ResolvedParent)
_, _ = fmt.Fprintf(out, " canonical target path: %s\n", preview.CanonicalTargetPath)
if len(preview.AttachmentUploads) == 0 {
_, _ = fmt.Fprintln(out, " attachment operations: none")
} else {
_, _ = fmt.Fprintf(out, " attachment operations: upload %d asset(s)\n", len(preview.AttachmentUploads))
for _, asset := range preview.AttachmentUploads {
_, _ = fmt.Fprintf(out, " - %s\n", asset)
}
}
_, _ = fmt.Fprintf(out, " ADF summary: %d byte(s), %d top-level node(s)\n", preview.ADFBytes, preview.ADFTopLevelNodes)
}

func runDiffSpaceMode(
ctx context.Context,
out io.Writer,
Expand Down
6 changes: 3 additions & 3 deletions cmd/diff_metadata_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -270,7 +270,7 @@ func TestRunDiff_SpaceModeShowsMetadataSummaryForRemoteMetadataOnlyChanges(t *te
t.Fatalf("mkdir space: %v", err)
}

writeMarkdown(t, filepath.Join(spaceDir, "root.md"), fs.MarkdownDocument{
writeMarkdown(t, filepath.Join(spaceDir, "Root.md"), fs.MarkdownDocument{
Frontmatter: fs.Frontmatter{
Title: "Root",
ID: "1",
Expand All @@ -281,7 +281,7 @@ func TestRunDiff_SpaceModeShowsMetadataSummaryForRemoteMetadataOnlyChanges(t *te

if err := fs.SaveState(spaceDir, fs.SpaceState{
PagePathIndex: map[string]string{
"root.md": "1",
"Root.md": "1",
},
AttachmentIndex: map[string]string{},
}); err != nil {
Expand Down Expand Up @@ -328,7 +328,7 @@ func TestRunDiff_SpaceModeShowsMetadataSummaryForRemoteMetadataOnlyChanges(t *te
if !strings.Contains(got, "metadata drift summary") {
t.Fatalf("expected metadata summary, got:\n%s", got)
}
if !strings.Contains(got, "root.md") {
if !strings.Contains(got, "Root.md") {
t.Fatalf("expected metadata summary to include path, got:\n%s", got)
}
if !strings.Contains(got, "state: current -> draft") {
Expand Down
10 changes: 9 additions & 1 deletion cmd/diff_pages.go
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,15 @@ func shouldIgnoreFolderHierarchyError(err error) bool {
return true
}
var apiErr *confluence.APIError
return errors.As(err, &apiErr)
if errors.As(err, &apiErr) {
switch apiErr.StatusCode {
case 400, 409:
return false
default:
return true
}
}
return false
}

func diffDisplayRelPath(spaceDir, path string) string {
Expand Down
Loading
Loading