I have the same problem when using add_metric() to add metric "f1-macro" in PyCaret. My code is as following:

def my_f1(y_true, y_pred):
    return f1_score(y_true, y_pred, average='macro', )

my_f1 = make_scorer(my_f1, greater_is_better=True)
add_metric('f1_macro', 'F1-Macro', my_f1)

best = s.compare_models(sort='f1-macro', n_select=3)

It returns "AttributeError: '_Scorer' object has no attribute 'name'"