Segmata - it's like uncrease a papyrus sheet...

---

This software is in link with Vesuvius project scrollprize.org.

Segmata is a program that allows users to optimize segmentation from Khartes, https://github.com/KhartesViewer/khartes. It is mandatory to install the segmata branch from https://github.com/sgoutteb/khartes_segmata.git

For rendering layers it uses: vesuvius-render from https://github.com/jrudolph/vesuvius-gui

It is written in Python.

The main emphasis of Segmata is to automatically perform modifications on .obj file in order to improve the segmentation in terms of:

• Visual rendering of layers
• Inference result on segment (in a future step)

The .obj modifications are done on individual vertex displacements along normals (loop over all vertex). This displacement is limited to 2 pixels (adjustable).

Several passes are possible.

The cost function for deciding is the modified point is better can be based on:

• Maximize bright pixels count (the papyrus layer is brighter than "holes")
• Minimize dark pixel count or dark contours area (decrease "holes" or dark zones in papyrus)

Tha actual version uses:

• Difference with previous image if np.mean(image_diff)>0 we keep the changes

In a second step you can render more layers and apply a composite filter to several layers. This can improve the ink detection/labelling thanks to a better visualization of papyrus surface.

Workflow

INPUT: .obj file (exported from Khartes)

OUTPUT: .obj file (optimized)

Example on a small segment (only 74 vertex)

1. Create a simple fragment in Khartes

See example for files.

2. Launch segmata

Result pass after pass:

A log file is created in order to follow the optimization steps, hereafter are the results: segmata_log file

3. Import the new created obj file and compare the difference using Khartes.

Difference between red (optimized version) and blue (original file) line is cleary visible !

The red line is following the papyrus sheet closer.

Example 2 on bigger segment 20241207134906 (4119 vertex)

Area= 0.75cm²

Optimization result:

Example 3 on segment 20241108120730 (official segment)

Area= 3.2cm²

Optimization result: See how "holes" are well eliminated and filled with papyrus sheet.

But also how ink is appearing more clearly.

Limits

The method used implies that rendering must be the same size before and after a vertex modification. Is not the comparison will be false.

This point is not too restrictive because we change only one vertex at a time.

Installation

Required installation:

• Khartes

• vesuvius-render

• Seems to have some issues with subprocess.run command when using an environment like Spyder (working on Thonny, IDLE Python,...)

Improvments on user-friendly usage. Only one command line (see below)

Usage

See code file

#----------------------------------------------
# Segmata
#----------------------------------------------
#
# main function
#
# version 17.04.2025
# S.Gouttebroze
#----------------------------------------------

import os
import utils

###########################################################

# Exemple d'utilisation
objfile=r"C:\\Vesuvius\\20241207134906\\20241207134906.obj"
renderer_path = r"C:\\Vesuvius\\vesuvius-render-v34-x86_64-pc-windows-msvc.exe"

utils.segmata(objfile,renderer_path,30,True)

The script needs a new specific file which is created by the segmata branch of Khartes.

This file is points.csv will be created when refining segment in Khartes and saved in the main Khartes directory, it has to be copied after on the specific directory where obj file is exported.

No parameters adjustments are needed.

Except that the script is working on my environment (windows) and you have probably to change directories names.

Especially the temp dir line 492 of utils.py

            "--data-directory", "F:\\Vesuvius"]

The script is valid only for Scroll5, if you want to change "-v", "20241024131838", must be adjusted accordingly to scroll volume (the example is for Scroll5 PHerc172)

Khartes integration

• it improves treatment speed
• interactivly view the results

Link to a demo video: demo khartes integration

Further developments

• Improve speed of treatment

  • work in memory, but how to deal with vesuvius-render ?
  • try villa//scroll-renderer/
  • work one vertex out of x instead of all --> done but in another way

• Automatic detection of width and height

  • Reading json file exported by Khartes --> done.

• Implementation of inference

• Add user interface for choosing folders and files