isanet.model_selection¶

class isanet.model_selection.model_selection.GridSearchCV(estimator, param_grid, cv=<isanet.model_selection.model_selection.Kfold object>, verbose=0)¶

Bases: object

Exhaustive search over specified parameter values for an estimator.

The parameters of the estimator used to apply these methods are optimized by cross-validated grid-search over a parameter grid.

Parameters

estimator (estimator object.) – This represents the type of task to be solved using a neural network, this can be a classifier or a regressor, implemented using the MLPClassifier and MLPRegressor classes provided by the isanet.neural_network module.
param_grid (dict or list of dictionaries) –
Dictionary with parameters names (string) as keys and lists of parameter settings to try as values, or a list of such dictionaries, in which case the grids spanned by each dictionary in the list are explored. This enables searching over any sequence of parameter settings.

NOTE

The dictionary keys must have the same name as the parameters of the estimator passed to the class.
cv (KFold object or dict, default=Kfold(n_splits=5, shuffle=True)) –
Determines the cross-validation splitting strategy. Possible inputs for cv are:
- None, to use the default 5-fold cross validation,
- A dict of two lists, one with the indices of the training and
  the other with the indices of the validation set.
verbose (integer, default=0) –
Controls the verbosity: the higher, the more messages.
- 0: no messages
- 1: grid iterations
- 2: grid iterations + folds results

grid_dim¶

The number of hyperparameters.

Type: integer

folds_dim¶

The number of folds of the KFold object.

Type: integer

tot_train¶

The total number of training needed.

Type: integer

Notes

The parameters selected are those that maximize the score of the left out data, unless an explicit score is passed in which case it is used instead. If n_jobs was set to a value higher than one, the data is copied for each point in the grid (and not n_jobs times). This is done for efficiency reasons if individual jobs take very little time, but may raise errors if the dataset is large and not enough memory is available. A workaround in this case is to set pre_dispatch. Then, the memory is copied only pre_dispatch many times. A reasonable value for pre_dispatch is 2 * n_jobs.

fit(X, Y)¶

Run fit with all sets of parameters.

Parameters

X (array-like, shape = [n_samples, n_features]) – Training vector, where n_samples is the number of samples and n_features is the number of features.
Y (array-like, shape = [n_samples] or [n_samples, n_output]) – Target relative to X for classification or regression; None for unsupervised learning.

Returns

A dict with keys as column headers and values as columns, that can be imported into a pandas DataFrame.

For instance the below given table:

+-------------------------+---------------+----------------+---------------+---+------------+
| hyper_param             | fold_results  | mean_train_mse | std_train_mse |...| time_train |
+=========================+===============+================+===============+===+============+
| {'n_layer_units': [38], | {'train_mse': |                |               |   |            |
| 'learning_rate': 0.014, | [2.019141..., |   1.966004     |   0.061030    |...|  1.786321  |
| 'max..                  | 1.95675058... |                |               |   |            |
+-------------------------+---------------+----------------+---------------+---+------------+
| {'n_layer_units': [38], | {'train_mse': |                |               |   |            |
| 'learning_rate': 0.017, | [2.019141..., |   1.853260     |   0.047401    |...|  1.532336  |
| 'max..                  | 1.95675058... |                |               |   |            |
+-------------------------+---------------+----------------+---------------+---+------------+

will be represented as follow:

{
 "hyper_param":          [{'n_layer_units': [38], 'learning_rate': 0.014...}, ...],
 "fold_results":         [{'train_mse': [1.9814435389846516, 1.865889091...}, ...],
 "mean_train_mse":       [ ... ],
 "std_train_mse":        [ ... ],
 "mean_train_mee":       [ ... ],
 "std_train_mee":        [ ... ],
 "mean_train_acc":       [ ... ],
 "std_train_acc":        [ ... ],
 "mean_val_mse":         [ ... ],
 "std_val_mse":          [ ... ],
 "mean_val_mee":         [ ... ],
 "std_val_mee":          [ ... ],
 "mean_val_acc":         [ ... ],
 "std_val_acc":          [ ... ],
 "time_train":           [ ... ]
}

NOTE

The key ‘hyper_param’ is used to store a list of parameter settings dicts for all the parameter combination candidates. Each dicts will have the same keys of ‘param_grid’ passed as parameter to the GridSearchCV object.

The ‘fold_results’ key is used to store a list of dictionaries that represent the result of cross validation (E.g. Kfolds) and will have a length equal to the number of folds specified in the ‘cv’ parameter.

E.g the folds result with K = 4:

{
 "train_mse":    [1.9814435389846516, ..., 2.013302806990128],
 "train_mee":    [ ... ],
 "train_acc":    [ ... ],
 "val_mse":      [ ... ],
 "val_mee":      [ ... ],
 "val_acc":      [ ... ],
 "init_weigth":  [ ... ],
 "final_weigth": [ ... ],
 "fold_time":    [ ... ],
}

where each list has lenght 4

Return type

dict

get_grid_dim()¶: Returns the number of hyperparameters of the GridSearchCV object.

class isanet.model_selection.model_selection.Kfold(n_splits=5, shuffle=False, random_state=None)¶

Bases: object

K-Folds cross-validator

Provides train/validation indices to split data into train/validation sets. Split dataset into k consecutive folds (without shuffling by default). Each fold is then used once as validation while the k - 1 remaining folds form the training set.

Parameters

n_splits (int, default=5) – Number of folds.
shuffle (boolean, optional) – Whether to shuffle the data before splitting into folds.
random_state (int, RandomState instance or None, optional, default=None) – If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by np.random. Only used when shuffle is True. This should be left to None if shuffle is False.

Notes

The first n_splits - 1 folds have size n_samples // n_splits, the other fold has size n_samples // n_splits + n_samples % n_splits, where n_samples is the number of samples. Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.

get_n_splits()¶: Returns the number of folds of the KFold object.

split(X, y=None)¶

Generate indices to split data into training and validation set.

Parameters

X (array-like, shape (n_samples, n_features)) – Training data, where n_samples is the number of samples and n_features is the number of features.
y (array-like, shape (n_samples,), optional) – The target variable for supervised learning problems.

Raises

ValueError – If the number of folds is greater than the number of samples.

Returns

a dict of two lists, one with the indices of the training and the other with the indices of the validation set.

Return type

dict

isanet.model_selection.model_selection.product_dict(**kwargs)¶

Does the cartesian product of a dict of lists.

Parameters: kwargs (dict of lists) –
Returns: a list of dict. Each dict is a pair od the cartesian product.
Return type: list

Examples

>>> param_grid = {"n_units": [[38], [40,50]], "learning_rate": [0.014, 0.017]}
>>> param_list = list(product_dict(**param_grid))
>>> param_list
[{"n_units": [38], "learning_rate": 0.014}, {"n_units": [38], "learning_rate": 0.017},
{"n_units": [40,50], "learning_rate": 0.014}, {"n_units": [40,50], "learning_rate": 0.017}]

isanet.model_selection.model_selection.product_list(hyper_par_list)¶

Does the cartesian product of a a list of lists.

Parameters: kwargs (list of lists) –
Returns: a list of lists. Each list is a pair od the cartesian product.
Return type: list