PEP2 upgrade - KeyError getting bigwig (not a sample_attr)

[fwzhao]

toolkit/ngs_toolkit/track_manager.py

Line 238 in 71da443

bigwig=os.path.basename(sample_attrs["bigwig"]),

Hey, happy new year! Thought you should know about this, so you don't end up wasting a couple hours like I did. Also found the bug with prj['trackhubs'] vs. prj.config['trackhubs'] in the same effort (see pull request).

So with PEP2, sample_attr does not have "bigwig", and this leads to a KeyError. I don't know where samples were getting their original bigwig file assignments before PEP2, so I came up with a workaround using the project_config.yaml for now (need to deliver browser plots)

Error message as follows

Traceback (most recent call last):
  File "/home/fzhao/.local/bin/trackmanager", line 8, in <module>
    sys.exit(main())
  File "/home/fzhao/.local/lib/python3.7/site-packages/ngs_toolkit/track_manager.py", line 432, in main
    args, prj, track_hub, bigwig_dir, genomes_hub, proj_name, proj_desc, user_email
  File "/home/fzhao/.local/lib/python3.7/site-packages/ngs_toolkit/track_manager.py", line 273, in make_ucsc_trackhub
    bigwig=os.path.basename(sample_attrs["bigwig"]),
  File "/home/fzhao/.local/lib/python3.7/site-packages/pandas/core/series.py", line 1068, in __getitem__
    result = self.index.get_value(self, key)
  File "/home/fzhao/.local/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 4744, in get_value
    raise e1
  File "/home/fzhao/.local/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 4730, in get_value
    return self._engine.get_value(s, k, tz=getattr(series.dtype, "tz", None))
  File "pandas/_libs/index.pyx", line 80, in pandas._libs.index.IndexEngine.get_value
  File "pandas/_libs/index.pyx", line 88, in pandas._libs.index.IndexEngine.get_value
  File "pandas/_libs/index.pyx", line 131, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1607, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1614, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'bigwig'

Work around using append/derive as follows - in a PEP2 compatible project_config.yaml

sample_modifiers:
  append:
    pipeline_interfaces: /home/fzhao/workspace/open_pipelines/atacseq.interface.yaml
    bigwig: bigwig
  imply:
    - if:
        organism: 'human'
      then:
        genome: 'hg38'
  derive:
    attributes: [data_source, bigwig]
    sources:
      bsf: /scratch/lab_bsf/samples/{flowcell}/{flowcell}_{lane}_samples/{flowcell}_{lane}#{BSF_name}.bam
      bigwig: /scratch/lab_bock/shared/projects/i-pad/data/{sample_name}/coverage/{sample_name}.bigWig