Quickstart

Preliminaries

Imports

import mercs
import numpy as np
from mercs.tests import load_iris, default_dataset
from mercs.core import Mercs

import pandas as pd

Fit

Here a small MERCS testdrive for what I suppose you'll need. First, let us generate a basic dataset. Some utility-functions are integrated in MERCS so that goes like this

train, test = default_dataset(n_features=3)

df = pd.DataFrame(train)
df.head()
0 1 2 3
0 -0.878999 0.372105 -0.177663 0.0
1 -1.007950 -0.196467 -1.271123 1.0
2 0.343341 0.209659 -0.446280 0.0
3 -2.361662 -0.600424 -1.301522 0.0
4 -2.123507 0.246505 -1.323388 0.0
df.describe()
0 1 2 3
count 800.000000 800.000000 800.000000 800.000000
mean -0.026556 0.023105 -0.032320 0.495000
std 1.414683 0.982609 1.351052 0.500288
min -4.543441 -3.019512 -3.836929 0.000000
25% -1.074982 -0.629842 -1.040769 0.000000
50% -0.237825 0.000368 -0.180885 0.000000
75% 0.972748 0.668419 1.005200 1.000000
max 4.020262 3.926238 3.994644 1.000000

Now let's train a MERCS model. To know what options you have, come talk to me or dig in the code. For induction, nb_targets and nb_iterations matter most. Number of targets speaks for itself, number of iterations manages the amount of trees for each target. With n_jobs you can do multi-core learning (with joblib, really basic, but works fine on single machine), that makes stuff faster. fraction_missing sets the amount of attributes that is missing for a tree. However, this parameter only has an effect if you use the random selection algorithm. The alternative is the base algorithm, which selects targets, and uses all the rest as input.

clf = Mercs(
    max_depth=4,
    selection_algorithm="random",
    fraction_missing=0.6,
    nb_targets=2,
    nb_iterations=2,
    n_jobs=1,
    verbose=1,
    inference_algorithm="own",
    max_steps=8,
    prediction_algorithm="it",
)

You have to specify the nominal attributes yourself. This determines whether a regressor or a classifier is learned for that target. MERCS takes care of grouping targets such that no mixed sets are created.

nominal_ids = {train.shape[1]-1}
nominal_ids
{3}
clf.fit(train, nominal_attributes=nominal_ids)

So, now we have learned trees with two targets, but only a single target was nominal. If MERCS worked well, it should have learned single-target classifiers (for attribute 4) and multi-target regressors for all other target sets.

for idx, m in enumerate(clf.m_list):
    msg = """
    Model with index: {}
    {}
    """.format(idx, m.model)
    print(msg)
    Model with index: 0
    DecisionTreeRegressor(criterion='mse', max_depth=4, max_features=None,
                      max_leaf_nodes=None, min_impurity_decrease=0.0,
                      min_impurity_split=None, min_samples_leaf=1,
                      min_samples_split=2, min_weight_fraction_leaf=0.0,
                      presort=False, random_state=121958, splitter='best')


    Model with index: 1
    DecisionTreeRegressor(criterion='mse', max_depth=4, max_features=None,
                      max_leaf_nodes=None, min_impurity_decrease=0.0,
                      min_impurity_split=None, min_samples_leaf=1,
                      min_samples_split=2, min_weight_fraction_leaf=0.0,
                      presort=False, random_state=671155, splitter='best')


    Model with index: 2
    DecisionTreeRegressor(criterion='mse', max_depth=4, max_features=None,
                      max_leaf_nodes=None, min_impurity_decrease=0.0,
                      min_impurity_split=None, min_samples_leaf=1,
                      min_samples_split=2, min_weight_fraction_leaf=0.0,
                      presort=False, random_state=131932, splitter='best')


    Model with index: 3
    DecisionTreeRegressor(criterion='mse', max_depth=4, max_features=None,
                      max_leaf_nodes=None, min_impurity_decrease=0.0,
                      min_impurity_split=None, min_samples_leaf=1,
                      min_samples_split=2, min_weight_fraction_leaf=0.0,
                      presort=False, random_state=365838, splitter='best')


    Model with index: 4
    DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=4,
                       max_features=None, max_leaf_nodes=None,
                       min_impurity_decrease=0.0, min_impurity_split=None,
                       min_samples_leaf=1, min_samples_split=2,
                       min_weight_fraction_leaf=0.0, presort=False,
                       random_state=259178, splitter='best')


    Model with index: 5
    DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=4,
                       max_features=None, max_leaf_nodes=None,
                       min_impurity_decrease=0.0, min_impurity_split=None,
                       min_samples_leaf=1, min_samples_split=2,
                       min_weight_fraction_leaf=0.0, presort=False,
                       random_state=644167, splitter='best')

So, that looks good already. Let's examine up close.

clf.m_codes
array([[ 0,  1,  1, -1],
       [ 1,  1,  0,  0],
       [ 1,  1, -1,  0],
       [ 1,  0,  1, -1],
       [ 0, -1, -1,  1],
       [-1,  0, -1,  1]])

That's the matrix that summarizes everything. This can be dense to parse, and there's alternatives to gain insights, for instance;

for m_idx, m in enumerate(clf.m_list):
    msg = """
    Tree with id:          {}
    has source attributes: {}
    has target attributes: {},
    and predicts {} attributes
    """.format(m_idx, m.desc_ids, m.targ_ids, m.out_kind)
    print(msg)
    Tree with id:          0
    has source attributes: [0]
    has target attributes: [1, 2],
    and predicts numeric attributes


    Tree with id:          1
    has source attributes: [2, 3]
    has target attributes: [0, 1],
    and predicts numeric attributes


    Tree with id:          2
    has source attributes: [3]
    has target attributes: [0, 1],
    and predicts numeric attributes


    Tree with id:          3
    has source attributes: [1]
    has target attributes: [0, 2],
    and predicts numeric attributes


    Tree with id:          4
    has source attributes: [0]
    has target attributes: [3],
    and predicts nominal attributes


    Tree with id:          5
    has source attributes: [1]
    has target attributes: [3],
    and predicts nominal attributes

And that concludes my quick tour of how to fit with MERCS.

Prediction

First, we generate a query.

# Single target
q_code=np.zeros(clf.m_codes[0].shape[0], dtype=int)
q_code[-1:] = 1
print("Query code is: {}".format(q_code))

y_pred = clf.predict(test, q_code=q_code)
y_pred[:10]
Query code is: [0 0 0 1]





array([0., 0., 0., 1., 0., 0., 1., 1., 1., 1.])
clf.show_q_diagram()

svg

# Multi-target
q_code=np.zeros(clf.m_codes[0].shape[0], dtype=int)
q_code[-2:] = 1
print("Query code is: {}".format(q_code))

y_pred = clf.predict(test, q_code=q_code)
y_pred[:10]
Query code is: [0 0 1 1]





array([[ 0.15161875,  0.        ],
       [-0.07064853,  0.        ],
       [ 0.15161875,  0.        ],
       [ 0.21392281,  1.        ],
       [ 0.03979332,  0.        ],
       [-0.20459606,  0.        ],
       [ 0.21392281,  1.        ],
       [-0.20459606,  1.        ],
       [-0.31503791,  1.        ],
       [-0.17568144,  1.        ]])
clf.show_q_diagram()

svg

# Missing attributes
q_code=np.zeros(clf.m_codes[0].shape[0], dtype=int)
q_code[-1:] = 1
q_code[:2] = -1
print("Query code is: {}".format(q_code))

y_pred = clf.predict(test, q_code=q_code)
y_pred[:10]
Query code is: [-1 -1  0  1]





array([0., 0., 0., 0., 0., 0., 0., 1., 1., 0.])
clf.show_q_diagram()

svg