Skip to content

Commit 2280752

Browse files
ascibiszascibisz
authored
Mem3DG converter (#195)
* mem3dg converter * add link to source of input file data * default scale_factor to 1.0 * add unit tests * lint fixes * update justfile to install mem3dg dependencies * .obj file url should just be the file name, since it will be uploaded alongside the simularium file * fix typo
1 parent 8be0100 commit 2280752

8 files changed

Lines changed: 760 additions & 1 deletion

File tree

‎Justfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ clean:
2323
# install with all deps (and setup conda env with readdy)
2424
install:
2525
conda env update --file environment.yml
26-
pip install -e .[lint,test,docs,dev,mcell,physicell,md,cellpack]
26+
pip install -e .[lint,test,docs,dev,mcell,physicell,md,cellpack,mem3dg]
2727

2828
# lint, format, and check all files
2929
lint:

‎examples/Tutorial_mem3dg.ipynb‎

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# Simularium Conversion Tutorial : Mem3DG .nc Data"
8+
]
9+
},
10+
{
11+
"cell_type": "code",
12+
"execution_count": null,
13+
"metadata": {},
14+
"outputs": [],
15+
"source": [
16+
"from IPython.display import Image\n",
17+
"\n",
18+
"import numpy as np\n",
19+
"\n",
20+
"from simulariumio.mem3dg import Mem3dgConverter, Mem3dgData\n",
21+
"from simulariumio import MetaData, CameraData, UnitData"
22+
]
23+
},
24+
{
25+
"cell_type": "markdown",
26+
"metadata": {},
27+
"source": [
28+
"This notebook provides example python code for converting your own simulation trajectories into the format consumed by the Simularium Viewer. It creates a .simularium file which you can drag and drop onto the viewer like this:"
29+
]
30+
},
31+
{
32+
"cell_type": "markdown",
33+
"metadata": {},
34+
"source": [
35+
"![title](img/drag_drop.gif)"
36+
]
37+
},
38+
{
39+
"cell_type": "markdown",
40+
"metadata": {},
41+
"source": [
42+
"# _Note:_\n",
43+
"To install simulariumio with all depencies needed for Mem3DG conversion, use `pip install simulariumio[mem3dg]`"
44+
]
45+
},
46+
{
47+
"cell_type": "markdown",
48+
"metadata": {},
49+
"source": [
50+
"***\n",
51+
"## Prepare your spatial data"
52+
]
53+
},
54+
{
55+
"cell_type": "markdown",
56+
"metadata": {},
57+
"source": [
58+
"The Simularium `Mem3dgConverter` consumes spatiotemporal data from Mem3DG .nc output files using the netCDF4 Python package. \n",
59+
"\n",
60+
"The converter requires a `Mem3dgData` object as a parameter.\n",
61+
"\n",
62+
"Unlike other simulariumio converters, the `Mem3dgConverter` will generate several `.obj` files (one per frame) in addition to the `.simularium` file. Use `output_obj_file_path` to specify where the generated obj files should be saved.\n",
63+
"\n",
64+
"The test input .nc data for this example was created using [Mem3DG's Jupyter Notebook tutorials](https://github.com/RangamaniLabUCSD/Mem3DG/blob/main/tests/python/tutorial/tutorial1.ipynb)"
65+
]
66+
},
67+
{
68+
"cell_type": "code",
69+
"execution_count": null,
70+
"metadata": {},
71+
"outputs": [],
72+
"source": [
73+
"box_size = 5.\n",
74+
"scale_factor = 10\n",
75+
"\n",
76+
"example_data = Mem3dgData(\n",
77+
" input_file_path=\"../simulariumio/tests/data/mem3dg/traj.nc\",\n",
78+
" output_obj_file_path=\".\",\n",
79+
" meta_data=MetaData(\n",
80+
" box_size=np.array([box_size, box_size, box_size]),\n",
81+
" trajectory_title=\"Some parameter set\",\n",
82+
" camera_defaults=CameraData(position=np.array([0, 0, 200])),\n",
83+
" scale_factor=scale_factor\n",
84+
" ),\n",
85+
" agent_color=\"#a38fba\",\n",
86+
" agent_name=\"my-object\",\n",
87+
" time_units=UnitData(\"us\", 0.2),\n",
88+
")"
89+
]
90+
},
91+
{
92+
"cell_type": "markdown",
93+
"metadata": {},
94+
"source": [
95+
"## Convert and save as .simularium file"
96+
]
97+
},
98+
{
99+
"cell_type": "markdown",
100+
"metadata": {},
101+
"source": [
102+
"Once your data is shaped like in the `example_data` object, you can use the converter to generate the file at the given path"
103+
]
104+
},
105+
{
106+
"cell_type": "code",
107+
"execution_count": 3,
108+
"metadata": {},
109+
"outputs": [
110+
{
111+
"name": "stdout",
112+
"output_type": "stream",
113+
"text": [
114+
"Reading Mem3DG Data -------------\n",
115+
"Converting Trajectory Data to JSON -------------\n",
116+
"Writing JSON -------------\n",
117+
"saved to example_mem3dg.simularium\n"
118+
]
119+
}
120+
],
121+
"source": [
122+
"converter = Mem3dgConverter(example_data).save(\"example_mem3dg\", binary=False)"
123+
]
124+
},
125+
{
126+
"cell_type": "markdown",
127+
"metadata": {},
128+
"source": [
129+
"## Visualize in the Simularium viewer"
130+
]
131+
},
132+
{
133+
"cell_type": "markdown",
134+
"metadata": {},
135+
"source": [
136+
"In a supported web-browser (Firefox or Chrome), navigate to https://simularium.allencell.org/ and import your file into the view.\n",
137+
"\n",
138+
"**Note:** In order to view your data in simularium, you must import the generated .simularium file _and_ all of the generated .obj files!"
139+
]
140+
},
141+
{
142+
"cell_type": "code",
143+
"execution_count": null,
144+
"metadata": {},
145+
"outputs": [],
146+
"source": []
147+
}
148+
],
149+
"metadata": {
150+
"kernelspec": {
151+
"display_name": "Python 3 (ipykernel)",
152+
"language": "python",
153+
"name": "python3"
154+
},
155+
"language_info": {
156+
"codemirror_mode": {
157+
"name": "ipython",
158+
"version": 3
159+
},
160+
"file_extension": ".py",
161+
"mimetype": "text/x-python",
162+
"name": "python",
163+
"nbconvert_exporter": "python",
164+
"pygments_lexer": "ipython3",
165+
"version": "3.9.16"
166+
}
167+
},
168+
"nbformat": 4,
169+
"nbformat_minor": 4
170+
}

‎pyproject.toml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,9 @@ cellpack = [
5757
nerdss = [
5858
"MDAnalysis>=2.0.0",
5959
]
60+
mem3dg = [
61+
"netCDF4",
62+
]
6063
tutorial = [
6164
"jupyter",
6265
"scipy>=1.5.2",

‎simulariumio/mem3dg/__init__.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
#!/usr/bin/env python
2+
# -*- coding: utf-8 -*-
3+
4+
from .mem3dg_converter import Mem3dgConverter # noqa: F401
5+
from .mem3dg_data import Mem3dgData # noqa: F401

‎simulariumio/mem3dg/mem3dg_converter.py‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
from netCDF4 import Dataset
2+
import numpy as np
3+
from pathlib import Path
4+
5+
from ..trajectory_converter import TrajectoryConverter
6+
from ..data_objects import (
7+
AgentData,
8+
TrajectoryData,
9+
DimensionData,
10+
DisplayData,
11+
)
12+
from ..constants import DISPLAY_TYPE
13+
from ..exceptions import InputDataError
14+
from .mem3dg_data import Mem3dgData
15+
16+
17+
class Mem3dgConverter(TrajectoryConverter):
18+
def __init__(
19+
self,
20+
input_data: Mem3dgData,
21+
):
22+
"""
23+
Parameters
24+
----------
25+
input_data : Mem3dgData
26+
An object containing info for reading
27+
Mem3DG simulation trajectory output
28+
"""
29+
self._data = self._read(input_data)
30+
31+
def write_to_obj(self, filepath, data, frame):
32+
# Extract XYZ coordinates for vertices
33+
coordinates = np.array(
34+
data.groups["Trajectory"].variables["coordinates"][frame]
35+
)
36+
coordinates = np.reshape(coordinates, (-1, 3))
37+
38+
# Extract indices of vertices to make faces (all triangles)
39+
topology = np.array(data.groups["Trajectory"].variables["topology"][frame])
40+
topology = np.reshape(topology, (-1, 3))
41+
# change indices to be 1 indexed instead of 0 indexed for .obj files
42+
topology += 1
43+
44+
# Generate one .obj file per frame
45+
with open(filepath, "w") as file:
46+
file.write(f"# Frame {frame}\n")
47+
for v in coordinates:
48+
file.write(f"v {v[0]} {v[1]} {v[2]}\n")
49+
for t in topology:
50+
file.write(f"f {t[0]} {t[1]} {t[2]}\n")
51+
52+
def _read_traj_data(self, input_data: Mem3dgData) -> AgentData:
53+
try:
54+
data = Dataset(input_data.input_file_path, "r")
55+
n_frames = np.size(data.groups["Trajectory"].variables["time"])
56+
except Exception as e:
57+
raise InputDataError(f"Error reading input Mem3DG data: {e}")
58+
59+
# for now, we are representing converted Mem3DG trajectories as one
60+
# unique mesh agent per frame
61+
dimensions = DimensionData(total_steps=n_frames, max_agents=1)
62+
agent_data = AgentData.from_dimensions(dimensions)
63+
agent_data.n_timesteps = n_frames
64+
65+
base_agent_name = input_data.agent_name or "object"
66+
for frame in range(n_frames):
67+
agent_data.times[frame] = data.groups["Trajectory"].variables["time"][frame]
68+
agent_data.n_agents[frame] = 1
69+
70+
output_file_path = Path(input_data.output_obj_file_path) / f"{frame}.obj"
71+
self.write_to_obj(output_file_path, data, frame)
72+
73+
agent_data.radii[frame][0] = input_data.meta_data.scale_factor
74+
agent_data.unique_ids[frame][0] = frame
75+
76+
name = str(frame)
77+
agent_data.types[frame].append(name)
78+
object_display_data = DisplayData(
79+
name=f"{base_agent_name}#frame{frame}",
80+
display_type=DISPLAY_TYPE.OBJ,
81+
url=f"{frame}.obj",
82+
color=input_data.agent_color,
83+
)
84+
agent_data.display_data[name] = object_display_data
85+
return agent_data
86+
87+
def _read(self, input_data: Mem3dgData) -> TrajectoryData:
88+
"""
89+
Return a TrajectoryData object containing the Mem3DG data
90+
"""
91+
print("Reading Mem3DG Data -------------")
92+
if input_data.meta_data.scale_factor is None:
93+
input_data.meta_data.scale_factor = 1.0
94+
agent_data = self._read_traj_data(input_data)
95+
input_data.spatial_units.multiply(1.0 / input_data.meta_data.scale_factor)
96+
input_data.meta_data._set_box_size()
97+
result = TrajectoryData(
98+
meta_data=input_data.meta_data,
99+
agent_data=agent_data,
100+
time_units=input_data.time_units,
101+
spatial_units=input_data.spatial_units,
102+
plots=input_data.plots,
103+
)
104+
return result

‎simulariumio/mem3dg/mem3dg_data.py‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
from ..data_objects import MetaData, UnitData
2+
from typing import List, Dict, Any
3+
4+
5+
class Mem3dgData:
6+
input_file_path: str
7+
output_obj_file_path: str
8+
meta_data: MetaData
9+
agent_name: str
10+
agent_color: str
11+
time_units: UnitData
12+
spatial_units: UnitData
13+
plots: List[Dict[str, Any]]
14+
15+
def __init__(
16+
self,
17+
input_file_path: str,
18+
output_obj_file_path: str = None,
19+
meta_data: MetaData = None,
20+
agent_name: str = None,
21+
agent_color: str = None,
22+
time_units: UnitData = None,
23+
spatial_units: UnitData = None,
24+
plots: List[Dict[str, Any]] = None,
25+
):
26+
"""
27+
Parameters
28+
----------
29+
input_file_path : str
30+
The path to the .nc file output by Mem3DG for this trajectory.
31+
output_obj_file_path : str (optional)
32+
The path to the directory where output .obj files will be saved
33+
to. If nothing is provided, the output .obj files will be saved
34+
to the current directory.
35+
meta_data : MetaData
36+
An object containing metadata for the trajectory
37+
including box size, scale factor, and camera defaults
38+
agent_name: str (optional)
39+
This converter generates it's own DisplayData, but the agent name
40+
can optionally be overridden here. This will change the agent
41+
name that is displayed on the side column in simularium
42+
agent_color: string (optional)
43+
This converter generates it's own DisplayData, but the agent color
44+
can optionally be overridden here with a hex value for the color to
45+
display, e.g "#FFFFFF"
46+
Default: Use default colors from Simularium Viewer
47+
time_units: UnitData (optional)
48+
multiplier and unit name for time values
49+
Default: 1.0 second
50+
spatial_units: UnitData (optional)
51+
multiplier and unit name for spatial values
52+
(including positions, radii, and box size)
53+
Default: 1.0 meter
54+
plots : List[Dict[str, Any]] (optional)
55+
An object containing plot data already
56+
in Simularium format
57+
"""
58+
self.input_file_path = input_file_path
59+
self.output_obj_file_path = output_obj_file_path or "."
60+
self.meta_data = meta_data or MetaData()
61+
self.agent_name = agent_name
62+
self.agent_color = agent_color
63+
self.time_units = time_units or UnitData("s")
64+
self.spatial_units = spatial_units or UnitData("m")
65+
self.plots = plots or []

0 commit comments

Comments
 (0)