old version to new version migration

[dadi0005]

I have an issue on moving my code from old version to new version
rmse_scorer = make_scorer(calculate_rmse, greater_is_better=False)

config = {
'NoPreprocessing': {
},

'SavitzkyGolay': {
'filter_win': np.arange(7, 27, 2),
'deriv_order': [0, 1, 2]
},

'LocalStandardNormalVariate': {
'num_windows': [2, 3, 4, 5, 6, 7]
},

'MultipleScatterCorrection': {
},
'Normalize': {
},
'StandardNormalVariate': {
},
'Baseline': {
},

'Detrend': {
},

'sklearn.preprocessing.RobustScaler': {
},

'sklearn.preprocessing.StandardScaler': {
},
'sklearn.preprocessing.PolynomialFeatures': {
'degree': [2],
'include_bias': [True, False],
'interaction_only': [True, False]
},

'sklearn.decomposition.FastICA': {
'n_components': np.arange(1, 187, 1),
'tol': np.arange(0.0, 1.01, 0.05),
'whiten': [True, False],
'algorithm': ['parallel', 'deflation'],
'fun': ['logcosh', 'exp', 'cube']
},

'sklearn.decomposition.PCA': {
'n_components': np.arange(1, 10, 1),
'svd_solver': ['randomized'],
'iterated_power': range(1, 11)
},

'sklearn.linear_model.ElasticNetCV': {
'l1_ratio': np.arange(0.0, 1.01, 0.05),
'tol': [1e-5, 1e-4, 1e-3, 1e-2, 1e-1]
},

'sklearn.linear_model.LassoLarsCV': {
'normalize': [True, False],
'fit_intercept': [True, False]
},

'sklearn.svm.SVR': {
'loss': ["epsilon_insensitive", "squared_epsilon_insensitive"],
'kernel':['linear', 'poly', 'rbf'],
'degree':[1, 2, 3],
#'dual': [True, False],
'tol': [1e-5, 1e-4, 1e-3, 1e-2, 1e-1],
'C': [1e-4, 1e-3, 1e-2, 1e-1, 0.5, 1., 5., 10., 15., 20., 25.],
'epsilon': [1e-4, 1e-3, 1e-2, 1e-1, 1.]
},

'sklearn.linear_model.RidgeCV': {
'alpha': np.logspace(-10, 10, 50),
'fit_intercept': [True, False]
},

'xgboost.XGBRegressor': {
'n_estimators': [100],
'max_depth': range(1, 11),
'learning_rate': [1e-3, 1e-2, 1e-1, 0.5, 1.],
'subsample': np.arange(0.05, 1.01, 0.05),
'min_child_weight': range(1, 21),
'n_jobs': [1],
'verbosity': [0],
'objective': ['reg:squarederror']
},
'sklearn.cross_decomposition.PLSRegression': {
'n_components': range(5, 12)
},
'sklearn.linear_model.TweedieRegressor': {
'power':[0,1,2,3],
'alpha':np.arange(0,3,0.02),
'link':['auto', 'identity', 'log'],
'solver': ['lbfgs', 'newton-cholesky']
},

'sklearn.feature_selection.VarianceThreshold': {
'threshold': [0, .01, .1, .5]
},

'sklearn.feature_selection.SelectPercentile': {
'percentile': [5, 7.5, 10, 12, 15]
},

'sklearn.feature_selection.RFE': {
'estimator': {
'sklearn.cross_decomposition.PLSRegression': {
'n_components': range(5, 12)
}
}
},
'sklearn.feature_selection.SelectFromModel': {
'threshold': np.arange(0, 1.01, 0.05),
'estimator': {
'sklearn.cross_decomposition.PLSRegression': {
'n_components': range(5, 12)
}
}
}
}

pipeline_optimizer = TPOTRegressor(generations=100, population_size=50, cv=5,
config_dict=config, verbosity=2,random_state=42,scoring = rmse_scorer,
n_jobs=-1)

Assuming X_cal.values and y_cal.values are defined

try:
pipeline_optimizer.fit(x.values, y.values[:,0])
except Exception as e:
print(f'Error during model fitting: {e}')

Export the optimized pipeline

try:
pipeline_optimizer.export('C:/Users/abiyh/Downloads/abhiy_newest/abhiy_newest/abhiy.py')
print('Model export complete. Pipeline saved as tpot_db_bsu_pipeline.py.')
except Exception as e:
print(f'Error during model export: {e}')

[perib]

The entire configuration pipeline has significantly changed and is not backwards compatible. TPOT2 no longer uses a configuration dictionary as you provided, instead it uses the new Search Space class. I recommend looking over tutorials 1 and 2 to learn how the new API works.

https://github.com/EpistasisLab/tpot/tree/main/Tutorial

There is some code in this file that converts the old configuration dictionaries into the new format:

https://github.com/EpistasisLab/tpot/blob/main/tpot/old_config_utils/old_config_utils.py

Specifically, convert_config_dict_to_linearpipeline will convert your list into a linear search space with the classifiers at the end, and convert_config_dict_to_graphpipeline will convert your list into a graph-search space.

Note that these will not be strictly identical to the TPOT1 behavior. Specifically, TPOT1 classifiers/regressors in the inner nodes would all passthrough their inputs to the next node. This was removed in TPOT2 since it caused data duplication. I recommend sampling a few pipelines from the search space so that you have an idea of how the new pipelines from the new search space will compare.

[dadi0005]

I didn’t see where custom functions were used to generate the search space and combine with models. Can you give a basic example on how to use custom feature selection and transformer (which are scikit-learn compatible), combined with some models, and optimize the hyperparameters?

[mikeherv]

At our meeting we looked at Tutorial #3 and #4 and we agreed that will help you "dadi0005 with your question. Please reach out after reviewing them.

[perib]

This excerpt from the tutorials demonstrates how to create a custom search space

knn_configspace = ConfigurationSpace(
    space = {

        'n_neighbors': Integer("n_neighbors", bounds=(1, 10)),
        'weights': Categorical("weights", ['uniform', 'distance']),
        'p': Integer("p", bounds=(1, 3)),
        'metric': Categorical("metric", ['euclidean', 'minkowski']),
        'n_jobs': 1,
    }
)


knn_node = tpot.search_spaces.nodes.EstimatorNode(
    method = KNeighborsClassifier,
    space = knn_configspace,
)


classifier_node = tpot.search_spaces.pipelines.ChoicePipeline(
    search_spaces=[
        knn_node,
        lr_node,
        dt_node,
    ]
)
)

You can just replace the KNeighborsClassifier with your own custom classifier.

You can also look at this file for examples of how TPOT builds its built in search spaces.

If you want to include your model with all the other default model ranges, you can use this function to get the search space for any one model, or a list of models.

For example

default_list_of_classifier_nodes = get_search_space(['BernoulliNB', 'DecisionTreeClassifier', 'GaussianNB', 'KNeighborsClassifier', 'LogisticRegression', 'MultinomialNB'], return_choice_pipeline=False)
#make sure ChoicePipeline gets a flat list of EstimatorNodes if you want everything to be sampled equally.
search_space_of_my_node_and_default_nodes = ChoicePipeline(search_spaces=np.hstack(([classifier_node_1...], default_list_of_classifier_nodes )))

[perib]

If you want to search a pipeline (tree, graph, linear, etc) then you would have to build that manually that with your custom estimator.

I would recommend starting with the defaults here in template_search_spaces and modifying it to include your custom estimators.

[perib]

also to clarify, https://github.com/EpistasisLab/tpot/blob/main/tpot/old_config_utils/old_config_utils.py can be used to directly convert the old TPOT1 configuration dictionary into a TPOT2 search space