Skip to content

fermipy v1.4.1 (and possibly dev version) results are not stable #668

@HealthyPear

Description

@HealthyPear

Describe the bug

This is a bug I found while testing issue #566 after the new release.

Same exact dataset, same notebook run multiple times, once I see SED points as they should be
next time (or 3rd time) all flux errors are gone and sed["ts"] is nan in all bins

To Reproduce

Same dataset as per PG 1553 tutorial but 4FGL-DR4 version, namely:

Query Parameters (L2603111134438994BE7968)
Equatorial coordinates (degrees) | RA = 238.929, Dec = 11.1901
Time range (MET) | (239557417, 256970880)
Time range (Gregorian) | (2008-08-04 15:43:36, 2009-02-22 04:47:58)
Energy range (MeV) | (100, 300000)
Search radius (degrees) | 30

data/gll_iem_v07.fits
data/gll_psc_v35.fit
data/iso_P8R3_SOURCE_V3_v1.txt
data/L2603111134438994BE7968_PH00.fits
data/L2603111134438994BE7968_PH01.fits
data/L2603111134438994BE7968_SC00.fits

this is my config.yaml

data:
  evfile: data/ft1.txt
  scfile: data/L2603111134438994BE7968_SC00.fits

binning:
  roiwidth: 10.0
  binsz: 0.1
  binsperdec: 8

selection:
  emin: 100
  emax: 300000
  zmax: 90
  evclass: 128
  evtype: 3
  target: "4FGL J1555.7+1111"
  tmin: 239557414
  tmax: 428903014

gtlike:
  edisp: True
  irfs: "P8R3_SOURCE_V3"
  edisp_disable: ["isodiff", "galdiff"]

model:
  src_roiwidth: 10.0
  galdiff: "/Users/michele/Software/tests/fermipy/pg1553/data/gll_iem_v07.fits"
  isodiff: "/Users/michele/Software/tests/fermipy/pg1553/data/iso_P8R3_SOURCE_V3_v1.txt"
  catalogs:
    - "/Users/michele/Software/tests/fermipy/pg1553/data/gll_psc_v35.fit"

fileio:
  outdir: results

my analysis is a simplified version of the tutorial notebook

from fermipy.gtanalysis import GTAnalysis
gta = GTAnalysis('config.yaml',logging={'verbosity': 3})

gta.setup()

gta.free_sources(distance=3.0,pars='norm')

gta.free_source('galdiff')
gta.free_source('isodiff')

target_source = '4FGL J1555.7+1111'

gta.free_source(target_source)

fit_results = gta.fit()

gta.write_roi('fit0',make_plots=True)

sed = gta.sed(target_source)

print(sed["ts"])

import numpy as np
import matplotlib.pyplot as plt

c = np.load('results/fit0.npy', allow_pickle=True).flat[0]
E = np.array(c['sources'][target_source]['model_flux']['energies'])
dnde = np.array(c['sources'][target_source]['model_flux']['dnde'])
dnde_hi = np.array(c['sources'][target_source]['model_flux']['dnde_hi'])
dnde_lo = np.array(c['sources'][target_source]['model_flux']['dnde_lo'])

plt.clf()
plt.loglog(E, (E**2)*dnde, 'k--')
plt.loglog(E, (E**2)*dnde_hi, 'k')
plt.loglog(E, (E**2)*dnde_lo, 'k')
plt.errorbar(sed['e_ctr'][:-2],
             sed['e2dnde'][:-2],
             yerr=sed['e2dnde_err'][:-2], fmt ='o')
plt.errorbar(np.array(sed['e_ctr'][-2:]),
         sed['e2dnde_ul95'][-2:], yerr=0.2*sed['e2dnde_ul95'][-2:],
             fmt='o', uplims=True)
plt.xlabel('E [MeV]')
plt.ylabel(r'E$^{2}$ dN/dE [MeV cm$^{-2}$ s$^{-1}$]')
plt.show()

Expected behavior

I should always see the first SED (if I do not change data and analysis code in any way, should I get exactly the same plot? not sure if the fit starts internally from a random seed, but it should be more or less the same no? I hope!)

Image

but instead at the 2 or 3rd time I see

Image

Log files

This log file should contain multiple runs of the notebook, the last one shows the plot with only points
and no error bars

fermipy.log

This is a sped-up gif from a screen recording that shows what I describe

Image

Environment (please complete the following information):

  • OS: (e.g. MacOS Monterey version 12.3.1)

macos 15.7.4

  • Output of uname -a

Darwin host-10-1-3-145.byod.mppmu.mpg.de 24.6.0 Darwin Kernel Version 24.6.0: Mon Jan 19 21:59:23 PST 2026; root:xnu-11417.140.69.708.3~1/RELEASE_ARM64_T6030 arm64

  • fermipy, fermitools, and fermitools-data versions (for exameple, fermipy 1.0.1, fermitools 2.0.8)

fermipy version 1.4.1
ScienceTools version 2.5.1
Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions