new open header erroring

[satra]

i was testing the open header with andre today at mit. and was running into this error. didn't have time to debug.

received info
processing 616 header data bytes
header received: TR=10
Requires: 294912 bytes
ExternalImageHeader(magic='ERTI', headerVersion=4, seriesUID='1.3.12.2.1107.5.2.32.35115.2013081322103941796402114', scanType='EPI', imageType='3D', note='Initial OpenHeader Implementation. repetitionDelayMS is currently hard-coded to 0', dataType='int16_t', isLittleEndian=True, isMosaic=True, pixelSpacingReadMM=3.28125, pixelSpacingPhaseMM=3.28125, pixelSpacingSliceMM=4.0, sliceGapMM=1.0, numPixelsRead=64, numPixelsPhase=64, numSlices=30, voxelToWorldMatrix=[-3.28125, 0.0, 0.0, 105.0, 0.0, -3.28125, 0.0, 105.0, 0.0, 0.0, 5.0, -72.5, 0.0, 0.0, 0.0, 1.0], repetitionTimeMS=2000, repetitionDelayMS=0, currentTR=10, totalTR=10, isMotionCorrected=True, mcOrder='TZYX', mcTranslationXMM=-0.0094293402949011, mcTranslationYMM=0.0008725363597534, mcTranslationZMM=0.014861163666259001, mcRotationXRAD=-0.0048493300566949, mcRotationYRAD=0.0055316874022655, mcRotationZRAD=-0.000439865096032)
----------------------------------------
Exception happened during processing of request from ('192.168.2.3', 54797)
Traceback (most recent call last):
  File "/software/anaconda/envs/devpype/lib/python2.7/SocketServer.py", line 593, in process_request_thread
    self.finish_request(request, client_address)
  File "/software/anaconda/envs/devpype/lib/python2.7/SocketServer.py", line 334, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "receive_nii.py", line 35, in createHandler
    return ThreadedTCPRequestHandler(callback, infoclient,  *args, **keys)
  File "receive_nii.py", line 23, in __init__
    SocketServer.BaseRequestHandler.__init__(self, *args, **keys)
  File "/software/anaconda/envs/devpype/lib/python2.7/SocketServer.py", line 649, in __init__
    self.handle()
  File "receive_nii.py", line 26, in handle
    self.callback(self.infoclient, self.request)
  File "receive_nii.py", line 40, in process_data_callback
    infoclient.process_data(sock)
  File "receive_nii.py", line 126, in process_data
    (self.ei.get_image_size(), len(img_data))
ValueError: Image data wrong size: expected 294912 bytes, got 8688

also for some reason hdr.seriesUID is changing in subsequent TRs

ExternalImageHeader(magic='ERTI', headerVersion=4, seriesUID='1.3.12.2.1107.5.2.32.35115.2013081322103941796402114', scanType='EPI', imageType='3D', note='Initial OpenHeader Implementation. repetitionDelayMS is currently hard-coded to 0', dataType='int16_t', isLittleEndian=True, isMosaic=True, pixelSpacingReadMM=3.28125, pixelSpacingPhaseMM=3.28125, pixelSpacingSliceMM=4.0, sliceGapMM=1.0, numPixelsRead=64, numPixelsPhase=64, numSlices=30, voxelToWorldMatrix=[-3.28125, 0.0, 0.0, 105.0, 0.0, -3.28125, 0.0, 105.0, 0.0, 0.0, 5.0, -72.5, 0.0, 0.0, 0.0, 1.0], repetitionTimeMS=2000, repetitionDelayMS=0, currentTR=7, totalTR=10, isMotionCorrected=True, mcOrder='TZYX', mcTranslationXMM=0.0042315309944431, mcTranslationYMM=-0.0025443566414593, mcTranslationZMM=0.0071906872163449, mcRotationXRAD=-0.0025820459857879, mcRotationYRAD=-0.0049068081238989, mcRotationZRAD=0.0008466105930182)
----------------------------------------
received info
processing 616 header data bytes
header received: TR=8
Requires: 294912 bytes
ExternalImageHeader(magic='ERTI', headerVersion=4, seriesUID='1.3.12.2.1107.5.2.32.35115.2013081322103945048702115', scanType='EPI', imageType='3D', note='Initial OpenHeader Implementation. repetitionDelayMS is currently hard-coded to 0', dataType='int16_t', isLittleEndian=True, isMosaic=True, pixelSpacingReadMM=3.28125, pixelSpacingPhaseMM=3.28125, pixelSpacingSliceMM=4.0, sliceGapMM=1.0, numPixelsRead=64, numPixelsPhase=64, numSlices=30, voxelToWorldMatrix=[-3.28125, 0.0, 0.0, 105.0, 0.0, -3.28125, 0.0, 105.0, 0.0, 0.0, 5.0, -72.5, 0.0, 0.0, 0.0, 1.0], repetitionTimeMS=2000, repetitionDelayMS=0, currentTR=8, totalTR=10, isMotionCorrected=False, mcOrder='TZYX', mcTranslationXMM=0.0, mcTranslationYMM=0.0, mcTranslationZMM=0.0, mcRotationXRAD=0.0, mcRotationYRAD=0.0, mcRotationZRAD=0.0)

so this breaks the assert.

[satra]

i think this if clause isn't necessary in receive_nii.py

https://github.com/gablab/murfi2/blob/master/util/python/receive_nii.py#L118

            if len(in_bytes) < 4096:
                break

[ohinds]

Looks like either the functor isn't sending all of the last image, or there is a bug in the python. Did you try receiving that series with murfi, too? That will tell you.

[ohinds]

Haha, yeah, that line is useless.

[satra]

got a chance to test this again. @pwighton, @ohinds - it would be good to finalize this and make sure the new murfi works with this.

summary: vsend is sending 8 extra bytes with the data. based on numPixelsRead=64, numPixelsPhase=64, numSlices=30 and dataType='int16_t', the python routine is calculating 294912 bytes, but the socket is receiving 294920 bytes. i forced discarded the last 8 bytes and the receiver saves a proper nifti image series, which visually looks fine.

so two questions:

• where is the extra 8 bytes coming from?
• can we test this with the vm?

i'll send me changes in a PR.

received info
processing 616 header data bytes
header received: TR=10
Requires: 294912 bytes
ExternalImageHeader(magic='ERTI', headerVersion=4, seriesUID='1.3.12.2.1107.5.2.32.35115.2013090410382357749936054', scanType='EPI', imageType='3D', note='Initial OpenHeader Implementation. repetitionDelayMS is currently hard-coded to 0', dataType='int16_t', isLittleEndian=True, isMosaic=True, pixelSpacingReadMM=3.28125, pixelSpacingPhaseMM=3.28125, pixelSpacingSliceMM=4.0, sliceGapMM=1.0, numPixelsRead=64, numPixelsPhase=64, numSlices=30, voxelToWorldMatrix=[-3.28125, 0.0, 0.0, 105.0, 0.0, -3.28125, 0.0, 105.0, 0.0, 0.0, 5.0, -72.5, 0.0, 0.0, 0.0, 1.0], repetitionTimeMS=2000, repetitionDelayMS=0, currentTR=10, totalTR=10, isMotionCorrected=True, mcOrder='TZYX', mcTranslationXMM=-0.00046807884778729997, mcTranslationYMM=-0.0003313311381045, mcTranslationZMM=0.008108871950879, mcRotationXRAD=-0.0033013666351501, mcRotationYRAD=-0.0027133978332463, mcRotationZRAD=-0.0053163561623943995)
----------------------------------------
Exception happened during processing of request from ('192.168.2.3', 43323)
Traceback (most recent call last):
  File "/software/anaconda/envs/devpype/lib/python2.7/SocketServer.py", line 593, in process_request_thread
    self.finish_request(request, client_address)
  File "/software/anaconda/envs/devpype/lib/python2.7/SocketServer.py", line 334, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "receive_nii.py", line 35, in createHandler
    return ThreadedTCPRequestHandler(callback, infoclient,  *args, **keys)
  File "receive_nii.py", line 23, in __init__
    SocketServer.BaseRequestHandler.__init__(self, *args, **keys)
  File "/software/anaconda/envs/devpype/lib/python2.7/SocketServer.py", line 649, in __init__
    self.handle()
  File "receive_nii.py", line 26, in handle
    self.callback(self.infoclient, self.request)
  File "receive_nii.py", line 40, in process_data_callback
    infoclient.process_data(sock)
  File "receive_nii.py", line 124, in process_data
    (self.ei.get_image_size(), len(img_data))
ValueError: Image data wrong size: expected 294912 bytes, got 294920

[ohinds]

But did you test with murfi? I still don't trust the python receiver as authoritative...

[satra]

is there a quick test with murfi that doesn't involve setting up a full config?

[ohinds]

You should be able to use neurofeedback.xml now that image parms are gotten from the header.