Quickstart

WhyNot makes it easy to define and run causal inference and reinforcement learning experiments in challenging simulated environments, and it provides a suite of ready-made benchmark experiments for new methods.

The first step to using WhyNot is installing the package (see Installation).

Basic Concepts - Causal Inference

Every simulator in WhyNot comes equipped with a set of experiments probing different aspects of causal inference. In this section, we show how to run experiments probing average treatment effect estimation on the World 3 Simulator. World3 is a dynamical systems model that studies the interplay between natural resource constraints, population growth, and industrial development. For a full list of supported simulators, see Simulators.

First, we examine all of the experiments available for World3.

>>> import whynot as wn
>>> experiments = wn.world3.get_experiments()
>>> print([experiment.name for experiment in experiments])
['world3_rct', 'world3_pollution_confounding', 'world3_pollution_unobserved_confounding', 'world3_pollution_mediation']

These experiments generate datasets both in the setting of a pure randomized control trial (world3_rct), as well as with (unobserved) confounding and mediation. We will run a randomized control experiment. The description property offers specific details about the experiment.

>>> rct = wn.world3.PollutionRCT
>>> rct.description
'Study effect of intervening in 1975 to decrease pollution generation on total population in 2050.'

We can run the experiment using the experiment run() function and specifying a desired sample size num_samples. The experiment then returns a causal Dataset consisting of the covariates for each unit, the treatment assignment, the outcome, and the ground truth causal effect for each unit.

dset = rct.run(num_samples=500)
(X, W, Y) = dset.covariates, dset.treatments, dset.outcomes

Using the dataset generated by WhyNot, we can then compare the true sample average treatment effect with causal estimates. Since we ran a randomized control trial, we compare the difference in means with the true effect.

>>> import numpy as np
>>> true_sate = dset.sate
>>> (X, W, Y) = dset.covariates, dset.treatments, dset.outcomes
>>> estimated_ate = np.mean(Y[W == 1.]) -  np.mean(Y[W  == 0.])
>>> relative_error = np.abs((estimated_ate - true_sate) / true_sate)
>>> print("Relative Error in causal estimate: {}".format(relative_error))
'Relative Error in causal estimate: 1.33'

Basic Concepts - Causal Estimators

After generating the dataset, WhyNot enables you to run a large collection of causal estimators for benchmarking and comparison. The main function to do this is the causal_suite() which, given the causal dataset, runs all of the available estimators on the dataset and returns an InferenceResult for each estimator containing its estimated treatment effects and uncertainty estimates like confidence intervals. For a full list of supported estimators for average treatment effects see Average Treatment Effect Estimators.

>>> import whynot as wn

# Generate the dataset
>>> rct = wn.world3.PollutionRCT
>>> dataset = rct.run(num_samples=500, show_progress=True)

# Run the suite of estimates
>>> estimated_effects = wn.causal_suite(
...     dataset.covariates, dataset.treatments, dataset.outcomes)

# Evaluate the relative error of the estimates
>>> true_sate = dataset.sate
>>> for estimator, estimate in estimated_effects.items():
...     relative_error = np.abs((estimate.ate - true_sate) / true_sate)
...     print("{}: {:.2f}".format(estimator, relative_error))
ols: 0.50
propensity_weighted_ols: 0.51
propensity_score_matching: 0.28
matching: 0.75
causal_forest: 0.06
tmle: 0.06

Basic Concepts - Reinforcement Learning

Beyond causal inference, the simulators in WhyNot provide a ready collection of environments for reinforcement learning. Each of these environments is accessible through an OpenAI gym interface, which makes it easy to test existing methods on the simulators in WhyNot. In this section, we showcase how to get started running reinforcement learning experiments on the HIV Treatment Simulator.

First, we examine all of the environments available in WhyNot.

>>> import whynot.gym as gym
>>> for env in gym.envs.registry.all():
>>>     print(env.id)

Then, we initialize the environment and set the random seed.

>>> import whynot.gym as gym
>>> env = gym.make('HIV-v0')
>>> env.seed(1)

Finally, we run a single rollout of the simulator in exactly the same way as experiments on the OpenAI gym.

>>> import whynot.gym as gym

>>> observation = env.reset()
>>> for _ in range(100):
>>>     # Replace with your treatment policy!
>>>     action = env.action_space.sample()
>>>     observation, reward, done, info = env.step(action)
>>>     if done:
>>>         observation = env.reset()

For a complete worked example on the HIV simulator, as well as more details about the action space, the observation space, and the reward function see this example notebook and the documentation.