自动调参利器BayesianOptimization贝叶斯优化测试
Published on Aug. 22, 2023, 12:10 p.m.
BayesianOptimization贝叶斯优化方案测试
https://github.com/fmfn/BayesianOptimization
``python
!pip install bayesian-optimization
Requirement already satisfied: bayesian-optimization in /usr/local/lib/python3.7/dist-packages (1.2.0)
Requirement already satisfied: scikit-learn>=0.18.0 in /usr/local/lib/python3.7/dist-packages (from bayesian-optimization) (1.0.1)
Requirement already satisfied: scipy>=0.14.0 in /usr/local/lib/python3.7/dist-packages (from bayesian-optimization) (1.4.1)
Requirement already satisfied: numpy>=1.9.0 in /usr/local/lib/python3.7/dist-packages (from bayesian-optimization) (1.19.5)
Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.18.0->bayesian-optimization) (1.1.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.18.0->bayesian-optimization) (3.0.0)
```python
from bayes_opt import BayesianOptimization
# 黑盒参数
def black_box_function(x, y):
"""Function with unknown internals we wish to maximize.
This is just serving as an example, for all intents and
purposes think of the internals of this function, i.e.: the process
which generates its output values, as unknown.
"""
return (-x ** 2 - (y - 1) ** 2 + 1)*1
# 参数候选
# Bounded region of parameter space
pbounds = {'x': (200000, 400), 'y': (-30000, 100000003)}
optimizer = BayesianOptimization(
f=black_box_function,
pbounds=pbounds,
random_state=1,
)
The BayesianOptimization object will work out of the box without much tuning needed. The main method you should be aware of is maximize, which does exactly what you think it does.
There are many parameters you can pass to maximize, nonetheless, the most important ones are:
n_iter: How many steps of bayesian optimization you want to perform. The more steps the more likely to find a good maximum you are.
init_points: How many steps of random exploration you want to perform. Random exploration can help by diversifying the exploration space.
optimizer.maximize(
init_points=20,
n_iter=5,
)
| iter | target | x | y |
-------------------------------------------------
| [0m 1 [0m | [0m-5.187e+1[0m | [0m 1.168e+0[0m | [0m 7.202e+0[0m |
| [95m 2 [0m | [95m-9.128e+1[0m | [95m 2e+05 [0m | [95m 3.021e+0[0m |
| [95m 3 [0m | [95m-8.479e+1[0m | [95m 1.707e+0[0m | [95m 9.207e+0[0m |
| [0m 4 [0m | [0m-1.193e+1[0m | [0m 1.628e+0[0m | [0m 3.454e+0[0m |
| [0m 5 [0m | [0m-2.902e+1[0m | [0m 1.208e+0[0m | [0m 5.387e+0[0m |
| [0m 6 [0m | [0m-4.694e+1[0m | [0m 1.163e+0[0m | [0m 6.851e+0[0m |
| [0m 7 [0m | [0m-7.71e+15[0m | [0m 1.592e+0[0m | [0m 8.781e+0[0m |
| [0m 8 [0m | [0m-4.494e+1[0m | [0m 1.945e+0[0m | [0m 6.704e+0[0m |
| [0m 9 [0m | [0m-3.12e+15[0m | [0m 1.167e+0[0m | [0m 5.586e+0[0m |
| [0m 10 [0m | [0m-3.915e+1[0m | [0m 1.72e+05[0m | [0m 1.979e+0[0m |
| [0m 11 [0m | [0m-9.375e+1[0m | [0m 4.017e+0[0m | [0m 9.683e+0[0m |
| [0m 12 [0m | [0m-4.792e+1[0m | [0m 1.374e+0[0m | [0m 6.922e+0[0m |
| [0m 13 [0m | [0m-8.003e+1[0m | [0m 2.507e+0[0m | [0m 8.946e+0[0m |
| [95m 14 [0m | [95m-1.506e+1[0m | [95m 1.83e+05[0m | [95m 3.877e+0[0m |
| [0m 15 [0m | [0m-7.711e+1[0m | [0m 1.661e+0[0m | [0m 8.781e+0[0m |
| [0m 16 [0m | [0m-1.772e+1[0m | [0m 1.804e+0[0m | [0m 4.209e+0[0m |
| [0m 17 [0m | [0m-2.841e+1[0m | [0m 8.805e+0[0m | [0m 5.33e+07[0m |
| [0m 18 [0m | [0m-9.942e+1[0m | [0m 6.19e+04[0m | [0m 3.153e+0[0m |
| [0m 19 [0m | [0m-6.965e+1[0m | [0m 6.297e+0[0m | [0m 8.346e+0[0m |
| [0m 20 [0m | [0m-5.626e+1[0m | [0m 1.963e+0[0m | [0m 7.501e+0[0m |
| [0m 21 [0m | [0m-3.46e+14[0m | [0m 400.0 [0m | [0m 1.86e+07[0m |
| [95m 22 [0m | [95m-1.443e+1[0m | [95m 400.0 [0m | [95m 3.799e+0[0m |
| [95m 23 [0m | [95m-1.397e+1[0m | [95m 400.0 [0m | [95m 3.738e+0[0m |
| [0m 24 [0m | [0m-6.304e+1[0m | [0m 400.0 [0m | [0m 7.94e+07[0m |
| [0m 25 [0m | [0m-1.42e+13[0m | [0m 400.0 [0m | [0m 3.769e+0[0m |
=================================================
The best combination of parameters and target value found can be accessed via the property optimizer.max.
print(optimizer.max)
{'target': 12.635001220531157, 'params': {'x': 3.6921342392926375, 'y': 0.9439109674344994}}
for i in range(10):
optimizer.maximize(
init_points=2,
n_iter=3,
)
| iter | target | x | y |
-------------------------------------------------
| [95m 6 [0m | [95m 16.13 [0m | [95m 2.294 [0m | [95m-2.446 [0m |
| [0m 7 [0m | [0m 8.341 [0m | [0m 2.373 [0m | [0m-0.9266 [0m |
| [95m 8 [0m | [95m 22.76 [0m | [95m 2.989 [0m | [95m-2.85 [0m |
| [95m 9 [0m | [95m 31.0 [0m | [95m 4.0 [0m | [95m-3.0 [0m |
| [0m 10 [0m | [0m 29.56 [0m | [0m 3.998 [0m | [0m-2.817 [0m |
=================================================
| iter | target | x | y |
-------------------------------------------------
| [0m 11 [0m | [0m 7.392 [0m | [0m 2.794 [0m | [0m 0.2329 [0m |
| [0m 12 [0m | [0m 7.069 [0m | [0m 2.838 [0m | [0m 1.111 [0m |
| [0m 13 [0m | [0m 7.069 [0m | [0m 2.838 [0m | [0m 1.111 [0m |
| [0m 14 [0m | [0m 7.069 [0m | [0m 2.838 [0m | [0m 1.111 [0m |
| [0m 15 [0m | [0m 7.069 [0m | [0m 2.838 [0m | [0m 1.111 [0m |
=================================================
| iter | target | x | y |
-------------------------------------------------
| [0m 16 [0m | [0m 6.412 [0m | [0m 2.409 [0m | [0m 2.269 [0m |
| [0m 17 [0m | [0m 3.223 [0m | [0m 2.055 [0m | [0m 1.023 [0m |
| [0m 18 [0m | [0m 3.223 [0m | [0m 2.055 [0m | [0m 1.023 [0m |
| [0m 19 [0m | [0m 31.0 [0m | [0m 4.0 [0m | [0m-3.0 [0m |
| [0m 20 [0m | [0m 31.0 [0m | [0m 4.0 [0m | [0m-3.0 [0m |
=================================================
| iter | target | x | y |
-------------------------------------------------
| [0m 21 [0m | [0m 7.455 [0m | [0m 2.835 [0m | [0m 0.3521 [0m |
| [0m 22 [0m | [0m 12.11 [0m | [0m 2.281 [0m | [0m-1.811 [0m |
| [0m 23 [0m | [0m 12.11 [0m | [0m 2.281 [0m | [0m-1.811 [0m |
| [0m 24 [0m | [0m 12.11 [0m | [0m 2.281 [0m | [0m-1.811 [0m |
| [0m 25 [0m | [0m 12.11 [0m | [0m 2.281 [0m | [0m-1.811 [0m |
=================================================
| iter | target | x | y |
-------------------------------------------------
| [0m 26 [0m | [0m 15.25 [0m | [0m 3.601 [0m | [0m 2.81 [0m |
| [0m 27 [0m | [0m 5.924 [0m | [0m 2.627 [0m | [0m 1.154 [0m |
| [0m 28 [0m | [0m 5.924 [0m | [0m 2.627 [0m | [0m 1.154 [0m |
| [0m 29 [0m | [0m 5.924 [0m | [0m 2.627 [0m | [0m 1.154 [0m |
| [0m 30 [0m | [0m 5.924 [0m | [0m 2.627 [0m | [0m 1.154 [0m |
=================================================
| iter | target | x | y |
-------------------------------------------------
| [0m 31 [0m | [0m 14.95 [0m | [0m 3.753 [0m | [0m 2.368 [0m |
| [0m 32 [0m | [0m 17.89 [0m | [0m 2.17 [0m | [0m-2.766 [0m |
| [0m 33 [0m | [0m 17.89 [0m | [0m 2.17 [0m | [0m-2.766 [0m |
| [0m 34 [0m | [0m 17.89 [0m | [0m 2.17 [0m | [0m-2.766 [0m |
| [0m 35 [0m | [0m 17.89 [0m | [0m 2.17 [0m | [0m-2.766 [0m |
=================================================
| iter | target | x | y |
-------------------------------------------------
| [0m 36 [0m | [0m 6.084 [0m | [0m 2.34 [0m | [0m 2.269 [0m |
| [0m 37 [0m | [0m 5.996 [0m | [0m 2.197 [0m | [0m-0.4734 [0m |
| [0m 38 [0m | [0m 5.996 [0m | [0m 2.197 [0m | [0m-0.4734 [0m |
| [0m 39 [0m | [0m 5.996 [0m | [0m 2.197 [0m | [0m-0.4734 [0m |
| [0m 40 [0m | [0m 5.996 [0m | [0m 2.197 [0m | [0m-0.4734 [0m |
=================================================
| iter | target | x | y |
-------------------------------------------------
| [0m 41 [0m | [0m 14.97 [0m | [0m 3.916 [0m | [0m 0.199 [0m |
| [0m 42 [0m | [0m 14.89 [0m | [0m 3.384 [0m | [0m-1.107 [0m |
| [0m 43 [0m | [0m 14.89 [0m | [0m 3.384 [0m | [0m-1.107 [0m |
| [0m 44 [0m | [0m 14.89 [0m | [0m 3.384 [0m | [0m-1.107 [0m |
| [0m 45 [0m | [0m 14.89 [0m | [0m 3.384 [0m | [0m-1.107 [0m |
=================================================
| iter | target | x | y |
-------------------------------------------------
| [0m 46 [0m | [0m 11.39 [0m | [0m 3.373 [0m | [0m 2.008 [0m |
| [0m 47 [0m | [0m 3.399 [0m | [0m 2.037 [0m | [0m 1.501 [0m |
| [0m 48 [0m | [0m 3.399 [0m | [0m 2.037 [0m | [0m 1.501 [0m |
| [0m 49 [0m | [0m 3.399 [0m | [0m 2.037 [0m | [0m 1.501 [0m |
| [0m 50 [0m | [0m 3.399 [0m | [0m 2.037 [0m | [0m 1.501 [0m |
=================================================
| iter | target | x | y |
-------------------------------------------------
| [0m 51 [0m | [0m 15.06 [0m | [0m 3.978 [0m | [0m 1.489 [0m |
| [0m 52 [0m | [0m 6.099 [0m | [0m 2.561 [0m | [0m 1.736 [0m |
| [0m 53 [0m | [0m 6.099 [0m | [0m 2.561 [0m | [0m 1.736 [0m |
| [0m 54 [0m | [0m 6.099 [0m | [0m 2.561 [0m | [0m 1.736 [0m |
| [0m 55 [0m | [0m 6.099 [0m | [0m 2.561 [0m | [0m 1.736 [0m |
=================================================
print(optimizer.max)
{'target': 31.0, 'params': {'x': 4.0, 'y': -3.0}}
dir(optimizer)
['__class__',
'__delattr__',
'__dict__',
'__dir__',
'__doc__',
'__eq__',
'__format__',
'__ge__',
'__getattribute__',
'__gt__',
'__hash__',
'__init__',
'__init_subclass__',
'__le__',
'__lt__',
'__module__',
'__ne__',
'__new__',
'__reduce__',
'__reduce_ex__',
'__repr__',
'__setattr__',
'__sizeof__',
'__str__',
'__subclasshook__',
'__weakref__',
'_bounds_transformer',
'_events',
'_gp',
'_prime_queue',
'_prime_subscriptions',
'_queue',
'_random_state',
'_space',
'_verbose',
'dispatch',
'get_subscribers',
'max',
'maximize',
'probe',
'register',
'res',
'set_bounds',
'set_gp_params',
'space',
'subscribe',
'suggest',
'unsubscribe']