Skip to content

Add JOSS paper for submission#264

Draft
vahid-ahmadi wants to merge 37 commits intomainfrom
joss-paper
Draft

Add JOSS paper for submission#264
vahid-ahmadi wants to merge 37 commits intomainfrom
joss-paper

Conversation

@vahid-ahmadi
Copy link
Copy Markdown

@vahid-ahmadi vahid-ahmadi commented Mar 23, 2026

Summary

  • Adds paper.md and paper.bib for submission to the Journal of Open Source Software (JOSS)
  • Paper covers all 6 required JOSS sections: Summary, Statement of Need, State of the Field, Software Design, Research Impact Statement, and AI Usage Disclosure
  • Cites adoption by 10 Downing Street, HM Treasury (Algorithmic Transparency Record), NIESR UK Living Standards Review 2025, INET Oxford (Seventh Carbon Budget), IEA reports, and House of Lords parliamentary debate
  • Funded by the Nuffield Foundation

What is JOSS?

JOSS is a peer-reviewed open-access journal for research software. It publishes short papers (750–1,750 words) describing open-source software with research impact. There are no submission fees. The review process happens on GitHub via openjournals/joss-reviews. See the submission guide and paper format for full details.

JOSS requirements checklist

  • OSI-approved open-source license (AGPL-3.0)
  • Public repository with browsable source code
  • Issue tracker readable without registration
  • Public development history > 6 months
  • Demonstrated research impact (government, academic, policy citations)
  • Good open-source practices (CI, tests, docs, PRs, tagged releases)
  • Iterative development over time
  • paper.md with all 6 required sections
  • paper.bib with BibTeX references
  • AI Usage Disclosure
  • Funding acknowledgement (Nuffield Foundation)
  • ORCID IDs for all authors

Review suggestions status

From the self-review comment:

  • Add architecture diagram — architecture.png added with proper Figure 1 reference in text
  • Fix placeholder ORCIDs — All 4 authors have real ORCIDs
  • Add a minimal code example — No code snippet yet (QuantEcon-style)
  • Add comparison table in State of the Field — Feature table vs TAXSIM/UKMOD/OpenFisca
  • Add reproducibility framing to Statement of Need — Added: "Because existing proprietary models cannot be independently verified..."
  • Add scope/limitations sentence — Added: "PolicyEngine models static fiscal impacts; it does not model macroeconomic feedback effects or general equilibrium responses"
  • Condense policy research paragraph — Research Impact still ~60% of body
  • Add foundational microsimulation references — No Orcutt 1957 or Bourguignon & Spadaro 2006 yet

Additional fixes applied

  • Replaced fake pe_uk_data / pe_us_data repo citations with real Woodruff & Ghenis (2024) Enhanced CPS working paper
  • Fixed Claude citation year 2025 → 2026
  • Fixed broken Atlanta Fed Policy Rules Database URL → GitHub repo
  • Moved architecture figure after intro paragraph (text-first, then diagram)
  • Added explicit Figure 1 references in Software Design text
  • Verified all 27 bib URLs resolve correctly
  • Synced both HTML previews with paper.md

Steps before submission

  • Complete the full name of all authors in paper.md
  • ORCID IDs — All authors have real ORCIDs:
    • Max Ghenis — 0000-0002-1335-8277
    • Nikhil Woodruff — 0009-0009-5004-4910
    • Pavel Makarchuk — 0009-0003-4869-7409
    • Vahid Ahmadi — 0009-0004-1093-6272
  • All co-authors confirm agreement to be listed
  • Review and approve paper text
  • Merge this PR to main
  • Submit at https://joss.theoj.org/papers/new with the repo URL
  • Disclose conflicts of interest on the submission form (e.g., all authors employed by PolicyEngine)
  • Note related publications on the submission form (NIESR report, No 10 Innovation Fellows work, IEA reports)

After submission

  1. A JOSS editor will open a pre-review issue at joss-reviews
  2. Two reviewers will be assigned; review happens via GitHub issues
  3. Respond to reviewer feedback within 2 weeks, complete changes within 4-6 weeks
  4. Upon acceptance: create a tagged release, deposit to Zenodo for a DOI, update the review thread

Test plan

  • Verify paper.md renders correctly with JOSS preview tools
  • Confirm all paper.bib references are valid and resolve correctly

Adds paper.md and paper.bib for submission to the Journal of Open Source
Software (JOSS). Paper covers all 6 required JOSS sections and cites
adoption by 10 Downing Street, HM Treasury, NIESR, INET Oxford, IEA,
and House of Lords parliamentary debate.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
vahid-ahmadi and others added 3 commits March 23, 2026 12:07
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rewrites paper.md to describe the policyengine package as a unified
multi-country (UK + US) microsimulation framework rather than the
UK-only policyengine-uk package. Updates title, summary, statement of
need, state of the field, software design, and acknowledgements to
reflect the country-agnostic architecture. Adds TAXSIM and US data
references to paper.bib.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds congressional citation (Young Adult Tax Credit Act, H.R.7547),
NBER partnership (TAXSIM emulator MOU), Niskanen Center CTC report,
NTA conference presentation, and Georgetown Beeck Center collaboration
to the Research Impact section. Adds corresponding BibTeX entries.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
vahid-ahmadi and others added 15 commits March 23, 2026 12:27
- Convert all British spellings to American English (organised →
  organized, programme → program, labour → labor, behavioural →
  behavioral, modelling → modeling, licence → license, etc.)
- Fix Beeck Center citation: was pointing to a 2022 event but labeled
  as 2024. Replace with correct Better Government Lab collaboration
  reference (pe_bgl)
- Soften Youngman et al. claim: confirmed paper cites PolicyEngine UK
  data but removed unverifiable direct quote
- Remove unused beeck2024rac bib entry

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Change country order to "the US and the UK"
- Remove "via the Simulation class" from Summary
- Remove inline dataset names (Enhanced FRS, CPS) from Summary
- Make Statement of Need more specific about access barriers for
  existing models (UKMOD requires institutional application, TAXSIM
  covers only taxes not benefits, HMT/IFS/CBO/TPC are fully
  proprietary)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Update install instructions to match repo docs: pip install
  policyengine, policyengine[uk], policyengine[us], and uv pip
  install -e .[dev] for development
- Remove backtick code styling from "Simulation class" in Statement
  of Need
- Reorder State of the Field: US models first, then UK models

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Merge "Multi-country" and "Fully open-source" bullets into one
- Add specific US program count (over 11) with full list: federal
  income tax, payroll taxes, state income taxes, SNAP, SSI, Social
  Security, Medicare, Medicaid, EITC, CTC, and TANF
- Remove "Integration with the PolicyEngine web application" bullet

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Merge "Distributional analysis outputs" and "Labor supply dynamics"
  into single "Economic analysis" bullet
- Change "policyengine package" to "policyengine.py package"
- Delete "All code examples in the documentation are automatically
  re-executed" sentence
- Remove "built 10ds-microsim on top of" reference, simplify to
  "used PolicyEngine"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Academic research:
- USC/HHS collaboration on effective marginal tax rates (Unrath)
- DeBacker behavioral response modeling (Arnold Ventures)
- Beeck Center rules-as-code reports (2023, 2025)

Policy research:
- DC District Child Tax Credit (first local CTC in US history)
- Senator Booker Keep Your Pay Act calculator on Senate website
- Reorder US citations before UK in both sections

Acknowledgements:
- Add US funders: Arnold Ventures, NEO Philanthropy ($200K grant),
  Gerald Huff Fund for Humanity, NSF POSE program
- Keep Nuffield Foundation for UK

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add Atlanta Fed Policy Rules Database MOU for multi-model validation
  in Institutional partnership section
- Add NSF POSE Phase I award (2518372, $299,974) with nsf.gov URL
- Update NBER TAXSIM MOU reference to use taxsim.nber.org instead of
  policyengine.org link
- Update Arnold Ventures note to mention congressional district focus
- Use external/official URLs for all references where available

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Consistently place US content before UK content in every section:
Summary, Statement of Need, Software Design, Research Impact
(Government adoption, Congressional/Parliamentary, Academic,
Policy research), and Acknowledgements.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…-installable to Python

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
vahid-ahmadi and others added 3 commits March 23, 2026 13:53
- Remove "None of these tools provide a unified framework" sentence
- Merge "Programmatic reform API" and "Economic analysis" into one
  bullet: "Programmatic reform and economic analysis"
- Fix duplicate "Python Python" typo
- Now 3 bullets: framework, program coverage, reform and analysis

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Delete country-specific models paragraph from Software Design
- ghenis2026no10: gov.uk Innovation Fellowship page
- ghenis2024nta: ntanet.org conference page
- pe_usc: irp.wisc.edu extramural grants page
- pe_dctc: dccouncil.gov bill B25-0190
- pe_keepyourpay: booker.senate.gov press release
- pe_bgl and neo_philanthropy kept as PE links (no external alt exists)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@vahid-ahmadi
Copy link
Copy Markdown
Author

JOSS Paper Review: Comparison with Top Economics/Microsimulation JOSS Papers

I reviewed the most notable JOSS papers in economics and microsimulation — including AgentPy (39 citations, most-cited in the space), QuantEcon.py (9 pages, Nobel laureate co-author), neworder (dynamic microsimulation), BoARIO (indirect economic cost modeling), MEDUSA (distributional policy impacts), openmpp (R microsimulation), Economiccomplexity (9 citations), and BCEA (cost-effectiveness analysis).

Key finding: PolicyEngine would be the first JOSS paper for an open-source tax-benefit microsimulation model. Neither Tax-Calculator, OpenFisca, OG-Core, nor any PSL tool has a JOSS paper. The Research Impact section is unmatched by anything in comparable papers.


Suggested Improvements

1. Add an Architecture Diagram (inspired by AgentPy — 39 citations)

AgentPy's clean figure showing Agents → Environments → Models → Experiments → Output is likely a factor in its high citation count. The Software Design section describes 5 component groups in prose — a figure showing Dataset + Policy → Simulation → Output classes with the country-package delegation would make this immediately graspable. JOSS explicitly encourages figures.

2. Add a Code Example (inspired by QuantEcon.py)

QuantEcon's paper is mostly working code snippets. This paper has zero code. Even a short example would make it more concrete:

from policyengine import Simulation

sim = Simulation(country="us", reform={...})
sim.calculate("household_net_income")

3. Sharpen Competitive Differentiation (inspired by neworder & BCEA)

The neworder microsimulation paper does a systematic dimension-by-dimension comparison against MODGEN and LIAM2. Consider adding a comparison table:

Feature PolicyEngine TAXSIM UKMOD OpenFisca
Open source Partial
Multi-country US+UK US only UK only FR (+ forks)
Tax + benefits Both Tax only Both Both
Web interface Partial

4. Frame Around Reproducibility (inspired by Economiccomplexity — 9 citations)

Economiccomplexity's strongest move is framing open-source software as a scientific imperative for reproducibility. The Statement of Need focuses on access barriers, which is good — but adding a sentence about reproducibility of policy analysis (CBO/TPC results can't be independently verified) would resonate with JOSS reviewers who care deeply about open science.

5. Consider Trimming the Research Impact Section

This is the paper's strongest section — no other JOSS paper comes close. But it's ~60% of the body. JOSS papers typically run 3–5 pages. The government adoption and congressional citation paragraphs are the strongest; the policy research paragraph could potentially be condensed.

6. Fix Placeholder ORCIDs

Three authors have 0000-0000-0000-0000 as their ORCID. These need real ORCIDs or should be removed — JOSS reviewers will flag this immediately.

7. Engage More with Microsimulation Literature (inspired by BoARIO — 21 references)

BoARIO demonstrates deep engagement with its field's academic literature. Consider citing foundational microsimulation methodology papers (Orcutt 1957, Bourguignon & Spadaro 2006 survey) to position PolicyEngine within the academic tradition, not just against current tools.

8. State What PolicyEngine Does NOT Do (inspired by BCEA)

BCEA's "do one thing well" philosophy is effective. A brief scope statement (e.g., "PolicyEngine does not model macroeconomic feedback effects or general equilibrium") would help reviewers understand the design boundary.


Priority Summary

Priority Action Effort
🔴 High Add architecture diagram Medium
🔴 High Fix placeholder ORCIDs Low
🔴 High Add a minimal code example Low
🟡 Medium Add comparison table in State of the Field Low
🟡 Medium Add reproducibility framing to Statement of Need Low
🟡 Medium Add scope/limitations sentence Low
🟢 Lower Condense policy research paragraph Low
🟢 Lower Add foundational microsimulation references Low

vahid-ahmadi and others added 6 commits March 30, 2026 18:28
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… with docs

- Trim State of the Field, Academic/Policy research sections (~100 words saved)
- Add architecture diagram (Policies + Households + Dynamics → Simulation → Outputs)
- Add code example showing Policy/Simulation/economic_impact_analysis workflow
- Add reproducibility sentence and scope/limitations statement
- Align terminology with docs: run()/ensure(), entity hierarchy, LocalAuthorityImpact
- Fix metadata: affiliation country, AI model version
- Include paper-preview.html for review

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove pe_uk_data (no paper exists) and replace pe_us_data with
Woodruff & Ghenis 2024 working paper from policyengine-us-data/paper.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@vahid-ahmadi
Copy link
Copy Markdown
Author

@MaxGhenis Your ORCID is still the placeholder 0000-0000-0000-0000 in paper.md (line 12). Could you update it with your real ORCID? You can register for free at https://orcid.org/register if you don't have one yet.

vahid-ahmadi and others added 6 commits April 2, 2026 12:51
Update caption to match actual diagram content and add explicit
Figure 1 references in the Software Design text, following JOSS
conventions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace pe_us_data/pe_uk_data citations with woodruff2024enhanced_cps
- Update architecture figure caption and add Figure 1 text references
- Remove pe_uk_data reference entry

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Update claude2025 -> claude2026 (Anthropic 2026)
- Move architecture figure after intro paragraph in Software Design
- Rewrite "At runtime" paragraph to better reference Figure 1 inputs
- Replace broken Atlanta Fed URL with stable GitHub repo link

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move figure after intro paragraph in Software Design
- Update Claude citation year to 2026
- Replace broken Atlanta Fed URL with GitHub repo
- Update "At runtime" paragraph to match paper.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add US reform code example in Software Design (QuantEcon-style)
- Add Orcutt 1957 and Bourguignon & Spadaro 2006 to State of the Field
- Word count: 1,665 / 1,750 JOSS limit

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
vahid-ahmadi and others added 3 commits April 2, 2026 13:40
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants