.. _quickstart:

Quickstart
==========
WhyNot makes it easy to define and run causal inference and reinforcement
learning experiments in challenging simulated environments, and it provides a
suite of ready-made benchmark experiments for new methods.

The first step to using WhyNot is installing the package (see :ref:`installation`).

Basic Concepts - Causal Inference
---------------------------------
Every simulator in WhyNot comes equipped with a set of experiments probing
different aspects of causal inference. In this section, we show how to run
experiments probing average treatment effect estimation on the :ref:`World 3
Simulator <world3-simulator>`.  `World3` is a dynamical systems model that
studies the interplay between natural resource constraints, population growth,
and industrial development. For a full list of supported simulators, see
:ref:`simulators`.

First, we examine all of the experiments available for `World3`.

.. code:: python

    >>> import whynot as wn
    >>> experiments = wn.world3.get_experiments()
    >>> print([experiment.name for experiment in experiments])
    ['world3_rct', 'world3_pollution_confounding', 'world3_pollution_unobserved_confounding', 'world3_pollution_mediation']

These experiments generate datasets both in the setting of a pure randomized
control trial (`world3_rct`), as well as with (unobserved) confounding and
mediation. We will run a randomized control experiment. The `description`
property offers specific details about the experiment.

.. code:: python
    
    >>> rct = wn.world3.PollutionRCT
    >>> rct.description
    'Study effect of intervening in 1975 to decrease pollution generation on total population in 2050.'

We can run the experiment using the experiment :meth:`~whynot.dynamics.DynamicsExperiment.run` 
function and specifying a desired sample size ``num_samples``. 
The experiment then returns a causal :class:`~whynot.framework.Dataset`
consisting of the covariates for each unit, the treatment assignment, the
outcome, and the ground truth causal effect for each unit. 

.. code:: python

    dset = rct.run(num_samples=500)
    (X, W, Y) = dset.covariates, dset.treatments, dset.outcomes

Using the dataset generated by WhyNot, we can then compare the true sample
average treatment effect with causal estimates. Since we ran a randomized
control trial, we compare the difference in means with the true effect.

.. code:: python

    >>> import numpy as np
    >>> true_sate = dset.sate
    >>> (X, W, Y) = dset.covariates, dset.treatments, dset.outcomes
    >>> estimated_ate = np.mean(Y[W == 1.]) -  np.mean(Y[W  == 0.])
    >>> relative_error = np.abs((estimated_ate - true_sate) / true_sate)
    >>> print("Relative Error in causal estimate: {}".format(relative_error))
    'Relative Error in causal estimate: 1.33'

Basic Concepts - Causal Estimators
----------------------------------
After generating the dataset, WhyNot enables you to run a large collection of
causal estimators for benchmarking and comparison. The main function to do this
is the :func:`~whynot.causal_suite` which, given the causal dataset, runs all of
the available estimators on the dataset and returns an
:class:`~whynot.framework.InferenceResult` for each estimator containing its
estimated treatment effects and uncertainty estimates like confidence intervals.
For a full list of supported estimators for average treatment effects see
:ref:`ate-estimators`.

.. code:: python
    
    >>> import whynot as wn

    # Generate the dataset
    >>> rct = wn.world3.PollutionRCT
    >>> dataset = rct.run(num_samples=500, show_progress=True)

    # Run the suite of estimates
    >>> estimated_effects = wn.causal_suite(
    ...     dataset.covariates, dataset.treatments, dataset.outcomes)

    # Evaluate the relative error of the estimates
    >>> true_sate = dataset.sate
    >>> for estimator, estimate in estimated_effects.items():
    ...     relative_error = np.abs((estimate.ate - true_sate) / true_sate)
    ...     print("{}: {:.2f}".format(estimator, relative_error))
    ols: 0.50
    propensity_weighted_ols: 0.51
    propensity_score_matching: 0.28
    matching: 0.75
    causal_forest: 0.06
    tmle: 0.06

Basic Concepts - Reinforcement Learning
---------------------------------------
Beyond causal inference, the simulators in WhyNot provide a ready collection of
environments for reinforcement learning. Each of these environments is
accessible through an `OpenAI gym interface <https://gym.openai.com>`_, which
makes it easy to test existing methods on the simulators in WhyNot. In this
section, we showcase how to get started running reinforcement learning
experiments on the :ref:`adams-hiv-simulator`.

First, we examine all of the environments available in WhyNot.

.. code:: python

    >>> import whynot.gym as gym
    >>> for env in gym.envs.registry.all():
    >>>     print(env.id)

Then, we initialize the environment and set the random seed.

.. code:: python

    >>> import whynot.gym as gym
    >>> env = gym.make('HIV-v0')
    >>> env.seed(1)

Finally, we run a single rollout of the simulator in exactly the same way
as experiments on the OpenAI gym.

.. code:: python

    >>> import whynot.gym as gym

    >>> observation = env.reset()
    >>> for _ in range(100):
    >>>     # Replace with your treatment policy!
    >>>     action = env.action_space.sample()
    >>>     observation, reward, done, info = env.step(action)
    >>>     if done:
    >>>         observation = env.reset()

For a complete worked example on the HIV simulator, as well as more details
about the action space, the observation space, and the reward function see 
`this example notebook <https://github.com/zykls/whynot/blob/master/examples/reinforcement_learning/hiv_simulator.ipynb>`_
and the :ref:`documentation <adams-hiv-simulator>`.