Off-policy Policy Evaluation Under Unobserved Confounding

This repository contains all the code necessary to replicate the results of the paper "Off-policy Policy Evaluation Under Unobserved Confounding".

This code runs with Python 3.7. Install requirements with pip install -r requirements.txt, or with conda.

autism

Directory autism contains the code for Autism SMART trial experiment. Autism.ipynb is a notebook that generates the data for Case I, Case II and design sensitivity of this experiment. The simulator is adopted from Comparing Dynamic Treatment Regimes Using Repeated-Measures Outcomes: Modeling Considerations in SMART Studies Appendix B.

sepsis

The directory sepsis containts the code for the patinet sepsis experiments. The simulator is borrowed from Oberst, Sontag. The directory contains

• learn_policies.ipynb: This notebook is used to generate some of the data necesary for the experiments. You can skip running this notebook by unzipping the nessecary data unzip data/processed.zip. The data directory should contain the following files:

  • optimal_policy_st.pkl, mixed_policy.pkl, tx_tr.pkl, t0_policy.pkl, value_function.pkl

• sepsis_experiments.ipynb: This notebook runs the implementation of

  1. Data genration process : That uses our confounded MDP to generate data
  2. Weighted Importance Sampling esitmates
  3. Our method and Naive lowerbound