Fix some bugs #1174

kasyanovse · 2023-09-25T09:11:55Z

Test does not pass until aimclub/GOLEM#202 is not in release.

Fix Unserializable operation #1173 + new test
Fix Fedot cannot interpret n_jobs < -1 #1170 + new test (+ Add checks to determine_n_jobs and add test for it GOLEM#202)
Fix Lagged returns 1-dimensional target for forecast length = 1 #1159
Add test for Investigate performance for new data operations #262
Add verbose=0 for boosting models
Raise error if window_size is incorrect in LaggedImplementation.
Set non-default tag for CGRU and ransac_non_lin_reg due to high evaluation time.
Set non-default tag for CutImplementation due to specific type of operation.
Fix second point from Todo for pred_ints #1165
Set non-default tag for ExtraTreesRegressor and gbr due to more effective alternatives.

aim-pep8-bot · 2023-09-25T09:12:04Z

Hello @kasyanovse! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file fedot/core/pipelines/prediction_intervals/solvers/mutation_of_best_pipeline.py:

Line 78:9: E722 do not use bare 'except'

Comment last updated at 2023-09-28 14:14:44 UTC

pep8speaks · 2023-09-25T09:12:05Z

Hello @kasyanovse! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file fedot/core/pipelines/prediction_intervals/solvers/mutation_of_best_pipeline.py:

Line 78:9: E722 do not use bare 'except'

Comment last updated at 2023-09-28 14:14:46 UTC

codecov · 2023-09-28T13:29:36Z

Codecov Report

Merging #1174 (f92cbfa) into master (17b2ecd) will decrease coverage by 0.36%.
Report is 3 commits behind head on master.
The diff coverage is 95.23%.

@@            Coverage Diff             @@
##           master    #1174      +/-   ##
==========================================
- Coverage   79.70%   79.35%   -0.36%     
==========================================
  Files         141      141              
  Lines        9851     9784      -67     
==========================================
- Hits         7852     7764      -88     
- Misses       1999     2020      +21

Files	Coverage Δ
...edot/core/pipelines/prediction_intervals/params.py	`100.00% <100.00%> (ø)`
...ion_intervals/solvers/mutation_of_best_pipeline.py	`81.96% <100.00%> (+0.93%)`	⬆️
...core/pipelines/prediction_intervals/ts_mutation.py	`97.22% <100.00%> (-0.15%)`	⬇️
fedot/core/pipelines/prediction_intervals/utils.py	`68.42% <ø> (-1.10%)`	⬇️
...lementations/data_operations/ts_transformations.py	`89.10% <85.71%> (-0.54%)`	⬇️

... and 18 files with indirect coverage changes

valer1435 · 2023-09-28T11:45:21Z

...t/core/operations/evaluation/operation_implementations/data_operations/ts_transformations.py

-            window_size = len(time_series) - forecast_length - 10
-            self.params.update(window_size=window_size)
-            self.log.info(f"{prefix} from {previous_size} to {self.window_size}.")
+        if self.window_size + forecast_length > len(time_series):


Что предполагается делать в случае, когда длина окна больше допустимой для конкретного ряда?

Кидать ошибку. Это некорректный случай.
Идея в том, чтобы убрать скрытое поведение. Если тюнер или композер хотят поставить некорректное значение окна, то пусть они узнают об этом.

...t/core/operations/evaluation/operation_implementations/data_operations/ts_transformations.py

fedot/core/pipelines/prediction_intervals/solvers/mutation_of_best_pipeline.py

valer1435 · 2023-09-28T12:07:52Z

fedot/utilities/ts_gapfilling.py

+        forecast_length = input_data.task.task_params.forecast_length
+        data_length = input_data.features.shape[0]
+        for node in pipeline_for_forecast.nodes:
+            if node.name == 'lagged':


Мне кажется, что это должно учитываться где-то внутри. Случайный пользователь не додумается до такого

Так это и есть внутри. Разве нет?

Сейчас получается так, что, если у lagged преобразования длина окна слишком большая для конкретного ряда, то вываливается ошибка, которая нигде не обрабатывается. Это код только для замены пропусков, длина окна будет подбираться только для этой задачи, а в общем случае будет ошибка. То есть придется в любой код, в котором мы не хотим, чтобы вываливалась ошибка, писать подобные условия

Все правильно, разве нет?
Если длина окна задана некорректно, то мы получаем ошибку. Это как с пайплайном. Сейчас некорректные начальные пайплайны вываливают ошибку, а не сами как-то корректируются.

fedot/utilities/ts_gapfilling.py

test/integration/models/test_model.py

valer1435 · 2023-09-28T12:23:06Z

test/unit/validation/test_time_series_cv.py

+    simple_pipeline = get_simple_ts_pipeline(window_size=2)
+
+    max_window = int(time_series.features.shape[0] / (folds + 1)) - (forecast_len * validation_blocks) - 1
+    ppl_ss = PipelineSearchSpace({'lagged': {'window_size': {'hyperopt-dist': hp.uniformint,


Есть ли смысл явно ограничивать пространство?

Да. Там очень небольшое количество тестовых данных, тюнер легко вылетает в область недопустимых окон и все падает, так как lagged выкидывает ошибку о недопустимости размера окна.

А как сейчас эта ошибка вообще обрабатывается?

Тюнером?

Например. Или композером

С композером проблем нет потому, что оценка пайплайна обернута в try-except, пайплайн с некорректным окном получается некорректным. Тюнер вылетает. Теперь вот не знаю, откатить это изменение или скорректировать тюнер.

А можно ли тюнер обернуть в try except? (в качестве временного фикса)

Починю тюнер тогда.

Тюнер в починке не нуждается. Он корректно отрабатывает ошибки.

fedot/utilities/ts_gapfilling.py

test/integration/models/test_model.py

valer1435 · 2023-09-29T08:01:53Z

test/unit/validation/test_time_series_cv.py

+    simple_pipeline = get_simple_ts_pipeline(window_size=2)
+
+    max_window = int(time_series.features.shape[0] / (folds + 1)) - (forecast_len * validation_blocks) - 1
+    ppl_ss = PipelineSearchSpace({'lagged': {'window_size': {'hyperopt-dist': hp.uniformint,


А как сейчас эта ошибка вообще обрабатывается?

kasyanovse added 8 commits September 25, 2023 10:46

Add test for n_jobs

338a752

Add verbose=0 to boosting models

cf29ea8

Switch off cut preprocessing

e5449f8

Add error for uncorrect window size in lagged node

4291999

Fix 1159

6582449

Fix 1173#

7130416

Add speed test for operations

3c88b75

Set CGRU and ransac_non_lin_reg to non-default due to low speed

f71b690

kasyanovse linked an issue Sep 25, 2023 that may be closed by this pull request

Investigate performance for new data operations #262

Closed

kasyanovse linked an issue Sep 25, 2023 that may be closed by this pull request

Unserializable operation #1173

Closed

kasyanovse linked an issue Sep 25, 2023 that may be closed by this pull request

Lagged returns 1-dimensional target for forecast length = 1 #1159

Closed

kasyanovse added 6 commits September 25, 2023 12:24

pep8

98ecba7

Fix errors with new behavior of window_size in lagged

d5134cf

pep8

98c91fb

Fix tests and #1165 p.2

f089654

Fix test with pipeline tuning

980d038

pep8

b4beec9

kasyanovse self-assigned this Sep 25, 2023

kasyanovse added 6 commits September 26, 2023 12:45

Disable GBR

3eb740a

Disable DecisionTreeRegressor and ExtraTreesRegressor

97b4977

Disable AdaReg

ba4c6d7

Disable SVR

94f8e12

Enable adareg, dtreg, svr

b8a15fb

Fix

3ee5425

kasyanovse added the bug Something isn't working label Sep 28, 2023

kasyanovse mentioned this pull request Sep 28, 2023

Redundant logging at launch in jupyter notebook #748

Closed

Fix test

f6a0d41

kasyanovse requested a review from valer1435 September 28, 2023 09:17

Fix tests

2665e76

kasyanovse requested a review from GrigoriJasnovidov September 28, 2023 12:05

kasyanovse added 3 commits September 28, 2023 16:01

Fix tests

7bfd1eb

Fix tests

a7db620

Fix tests

212620e

valer1435 requested changes Sep 28, 2023

View reviewed changes

kasyanovse mentioned this pull request Sep 28, 2023

Add desc to not implemented errors #1177

Merged

kasyanovse added 2 commits September 28, 2023 17:13

Add requested changes

131f05a

pep8

f92cbfa

kasyanovse requested a review from valer1435 September 28, 2023 14:15

valer1435 requested changes Sep 29, 2023

View reviewed changes

kasyanovse requested a review from valer1435 September 29, 2023 08:20

GrigoriJasnovidov approved these changes Oct 3, 2023

View reviewed changes

valer1435 approved these changes Oct 17, 2023

View reviewed changes

kasyanovse merged commit a5a9b17 into master Oct 17, 2023
5 checks passed

kasyanovse deleted the 1170-fix branch October 17, 2023 12:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix some bugs #1174

Fix some bugs #1174

kasyanovse commented Sep 25, 2023 •

edited

Loading

aim-pep8-bot commented Sep 25, 2023 •

edited

Loading

pep8speaks commented Sep 25, 2023 •

edited

Loading

codecov bot commented Sep 28, 2023 •

edited

Loading

valer1435 Sep 28, 2023

kasyanovse Sep 28, 2023

valer1435 Sep 28, 2023

kasyanovse Sep 28, 2023

valer1435 Sep 29, 2023

kasyanovse Sep 29, 2023

valer1435 Sep 28, 2023

kasyanovse Sep 28, 2023

valer1435 Sep 29, 2023

kasyanovse Sep 29, 2023

valer1435 Sep 29, 2023

kasyanovse Oct 16, 2023 •

edited

Loading

valer1435 Oct 17, 2023 •

edited

Loading

kasyanovse Oct 17, 2023

kasyanovse Oct 17, 2023

valer1435 Sep 29, 2023

Fix some bugs #1174

Fix some bugs #1174

Conversation

kasyanovse commented Sep 25, 2023 • edited Loading

aim-pep8-bot commented Sep 25, 2023 • edited Loading

Comment last updated at 2023-09-28 14:14:44 UTC

pep8speaks commented Sep 25, 2023 • edited Loading

Comment last updated at 2023-09-28 14:14:46 UTC

codecov bot commented Sep 28, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kasyanovse Oct 16, 2023 • edited Loading

Choose a reason for hiding this comment

valer1435 Oct 17, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kasyanovse commented Sep 25, 2023 •

edited

Loading

aim-pep8-bot commented Sep 25, 2023 •

edited

Loading

pep8speaks commented Sep 25, 2023 •

edited

Loading

codecov bot commented Sep 28, 2023 •

edited

Loading

kasyanovse Oct 16, 2023 •

edited

Loading

valer1435 Oct 17, 2023 •

edited

Loading