Skip to content

Initial data generation tasks for SPA #458

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 50 commits into
base: dev
Choose a base branch
from
Draft

Initial data generation tasks for SPA #458

wants to merge 50 commits into from

Conversation

@francisco-ebi
Copy link
Contributor

@francisco-ebi francisco-ebi commented Oct 23, 2025 •
edited
Loading

Created 3 tasks:

  • SPA Chromosome mapper (this only generates the chromosome-map.json file)
  • SPA Gene summaries
  • SPA Gene lacZ expression
  • SPA Gene histopathology
  • SPA Gene images
  • SPA Gene phenotype hits
  • SPA Gene publications
  • SPA Gene stats results
  • SPA Gene external links

Given the result of the last 2 task is splitted in several directories, I used AssetAlias to track all generated files.

@francisco-ebi francisco-ebi requested a review from ficolo October 23, 2025 13:06
@francisco-ebi
chore: add init file to impc_spa dir
0d6236e
@francisco-ebi
fix: impc_gene_external_links_mapper, repartition by mgi ID to ensure…
d25f769 
… one file per ID
@francisco-ebi
feat: generalize functions to partition data and create files
fb7d9be
@francisco-ebi
feat: update gene external links task to use new functions
0027298
@francisco-ebi
Merge branch 'dev' into feature/impc-spa
9e1bb2a
@francisco-ebi
fix: add missing param, update gene external links mapper task
ea27b64
@francisco-ebi
feat: initial Airflow task to generate gene publications.json files
36f8204
@francisco-ebi
feat: update gene publications mapper to use utils function
a852a0e
@francisco-ebi
WIP
695fa2d
@francisco-ebi
feat: initial task to create gene-histopathology json files
66eaf3c
@francisco-ebi
fix: ensure one file per gene
3a775f3
@francisco-ebi
fix: remove diseases mapper from this branch
1a8577e
@francisco-ebi
feat: update task to use utils function
9c38c39
@francisco-ebi
feat: initial task to create gene images.json file
89626d8
@francisco-ebi
feat: update task with utils functions
1b7eeff
@francisco-ebi
fix: add filename
857ec79
@francisco-ebi
feat: initial Airflow task to generate gene expression json files
76da083
@francisco-ebi
feat: update task to use utils functions
26581f4
@francisco-ebi
fix: add file name
a46bf96
@francisco-ebi
feat: initial Airflow task to produce gene phenotypehits json files
bc3ff63
@francisco-ebi
feat: update task to use utils functions
242725f
@francisco-ebi
fix: add file name
bfb9d66
@francisco-ebi
Merge branch 'feature/impc-spa-gene-histopathology-mapper' into featu…
4e03918 
…re/impc-spa
@francisco-ebi
Merge branch 'feature/impc-gene-images-mapper' into feature/impc-spa
05c59fb
@francisco-ebi francisco-ebi marked this pull request as draft November 4, 2025 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ficolo ficolo Awaiting requested review from ficolo

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@francisco-ebi