Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DO-NOT-MERGE] Use go standard error group for conbench #2156

Closed
wants to merge 16 commits into from

Conversation

ykadowak
Copy link
Contributor

Description:

Related Issue:

Versions:

  • Go Version: 1.20.6
  • Docker Version: 20.10.8
  • Kubernetes Version: v1.27.3
  • NGT Version: 2.0.16

Checklist:

Special notes for your reviewer:

vankichi and others added 11 commits July 24, 2023 11:30
* Create Continuous Bench Search Job tool (#1733)

* ✨ create bench job search tools

Signed-off-by: vankichi <[email protected]>

* ✨ add load hdf5 functions

Signed-off-by: vankichi <[email protected]>

* ♻️ fix format

Signed-off-by: vankichi <[email protected]>

* ♻️ fix docker and use hdf5 data

Signed-off-by: vankichi <[email protected]>

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor benchmark job

Signed-off-by: vankichi <[email protected]>

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ fix proto

Signed-off-by: vankichi <[email protected]>

* 💚 add benchmark job image build ci

Signed-off-by: vankichi <[email protected]>

* 💚 invest

Signed-off-by: vankichi <[email protected]>

* Revert "💚 invest"

This reverts commit f0f585c.

* Apply suggestions from code review

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ apply code review

Signed-off-by: vankichi <[email protected]>

* Apply suggestions from code review

Co-authored-by: Hiroto Funakoshi <[email protected]>

* ✨ apply from feedback

Signed-off-by: vankichi <[email protected]>

* Update internal/config/benchmark.go

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ change directory path

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>

* Add crds for continuous benchmark tools (#1789)

* ✨ add crds for continuous benchmark operator

Signed-off-by: vankichi <[email protected]>

* ✨ add benchmark operator/job scheme

Signed-off-by: vankichi <[email protected]>

* ✨ rename package names and add doc.go

Signed-off-by: vankichi <[email protected]>

* ✨ create runtime object

Signed-off-by: vankichi <[email protected]>

* Apply suggestions from code review

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ apply feedback

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>

* Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825)

* ✨ ♻️ add Job reconciler & use scenario instead of operator

Signed-off-by: vankichi <[email protected]>

* ♻️ fix format & rename file

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>

* Add benchmark operator framework (#1916)

* ✨ impl benchmark reconciler

Signed-off-by: vankichi <[email protected]>

* ✨ create benchmark operator framework

Signed-off-by: vankichi <[email protected]>

* ♻️ remove unness changes

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* impl reconcile logic for create benchmark job (#1923)

* ✨ impl reconcile logic for create benchmark job

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor continuous benchmark's crds

Signed-off-by: vankichi <[email protected]>

* ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor continuous benchmark job logic

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ update charts

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ rafactor con bench config and bug fix reconcile logic

Signed-off-by: vankichi <[email protected]>

* 🐛 Bugfix: fix typo and recall function logic

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor pkg benchmark job

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
* Implement base of continuous benchmark tool (#1776)

* Create Continuous Bench Search Job tool (#1733)

* ✨ create bench job search tools

Signed-off-by: vankichi <[email protected]>

* ✨ add load hdf5 functions

Signed-off-by: vankichi <[email protected]>

* ♻️ fix format

Signed-off-by: vankichi <[email protected]>

* ♻️ fix docker and use hdf5 data

Signed-off-by: vankichi <[email protected]>

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor benchmark job

Signed-off-by: vankichi <[email protected]>

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ fix proto

Signed-off-by: vankichi <[email protected]>

* 💚 add benchmark job image build ci

Signed-off-by: vankichi <[email protected]>

* 💚 invest

Signed-off-by: vankichi <[email protected]>

* Revert "💚 invest"

This reverts commit f0f585c.

* Apply suggestions from code review

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ apply code review

Signed-off-by: vankichi <[email protected]>

* Apply suggestions from code review

Co-authored-by: Hiroto Funakoshi <[email protected]>

* ✨ apply from feedback

Signed-off-by: vankichi <[email protected]>

* Update internal/config/benchmark.go

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ change directory path

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>

* Add crds for continuous benchmark tools (#1789)

* ✨ add crds for continuous benchmark operator

Signed-off-by: vankichi <[email protected]>

* ✨ add benchmark operator/job scheme

Signed-off-by: vankichi <[email protected]>

* ✨ rename package names and add doc.go

Signed-off-by: vankichi <[email protected]>

* ✨ create runtime object

Signed-off-by: vankichi <[email protected]>

* Apply suggestions from code review

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ apply feedback

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>

* Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825)

* ✨ ♻️ add Job reconciler & use scenario instead of operator

Signed-off-by: vankichi <[email protected]>

* ♻️ fix format & rename file

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>

* Add benchmark operator framework (#1916)

* ✨ impl benchmark reconciler

Signed-off-by: vankichi <[email protected]>

* ✨ create benchmark operator framework

Signed-off-by: vankichi <[email protected]>

* ♻️ remove unness changes

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* impl reconcile logic for create benchmark job (#1923)

* ✨ impl reconcile logic for create benchmark job

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor continuous benchmark's crds

Signed-off-by: vankichi <[email protected]>

* ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor continuous benchmark job logic

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ update charts

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ rafactor con bench config and bug fix reconcile logic

Signed-off-by: vankichi <[email protected]>

* 🐛 Bugfix: fix typo and recall function logic

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor pkg benchmark job

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

* ✨ impl benchmark jobs

Signed-off-by: vankichi <[email protected]>

* ♻️ apply feedback

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
* ✨ create helm template for benchmark operator

Signed-off-by: vankichi <[email protected]>

* ✨ refactor helm template

Signed-off-by: vankichi <[email protected]>

* style: Format code with gofumpt and prettier

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
* Implement base of continuous benchmark tool (#1776)

* Create Continuous Bench Search Job tool (#1733)

* ✨ create bench job search tools

Signed-off-by: vankichi <[email protected]>

* ✨ add load hdf5 functions

Signed-off-by: vankichi <[email protected]>

* ♻️ fix format

Signed-off-by: vankichi <[email protected]>

* ♻️ fix docker and use hdf5 data

Signed-off-by: vankichi <[email protected]>

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor benchmark job

Signed-off-by: vankichi <[email protected]>

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ fix proto

Signed-off-by: vankichi <[email protected]>

* 💚 add benchmark job image build ci

Signed-off-by: vankichi <[email protected]>

* 💚 invest

Signed-off-by: vankichi <[email protected]>

* Revert "💚 invest"

This reverts commit f0f585c.

* Apply suggestions from code review

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ apply code review

Signed-off-by: vankichi <[email protected]>

* Apply suggestions from code review

Co-authored-by: Hiroto Funakoshi <[email protected]>

* ✨ apply from feedback

Signed-off-by: vankichi <[email protected]>

* Update internal/config/benchmark.go

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ change directory path

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>

* Add crds for continuous benchmark tools (#1789)

* ✨ add crds for continuous benchmark operator

Signed-off-by: vankichi <[email protected]>

* ✨ add benchmark operator/job scheme

Signed-off-by: vankichi <[email protected]>

* ✨ rename package names and add doc.go

Signed-off-by: vankichi <[email protected]>

* ✨ create runtime object

Signed-off-by: vankichi <[email protected]>

* Apply suggestions from code review

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ apply feedback

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>

* Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825)

* ✨ ♻️ add Job reconciler & use scenario instead of operator

Signed-off-by: vankichi <[email protected]>

* ♻️ fix format & rename file

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>

* Add benchmark operator framework (#1916)

* ✨ impl benchmark reconciler

Signed-off-by: vankichi <[email protected]>

* ✨ create benchmark operator framework

Signed-off-by: vankichi <[email protected]>

* ♻️ remove unness changes

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* impl reconcile logic for create benchmark job (#1923)

* ✨ impl reconcile logic for create benchmark job

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor continuous benchmark's crds

Signed-off-by: vankichi <[email protected]>

* ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor continuous benchmark job logic

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ update charts

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ rafactor con bench config and bug fix reconcile logic

Signed-off-by: vankichi <[email protected]>

* 🐛 Bugfix: fix typo and recall function logic

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor pkg benchmark job

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

* ✨ impl status handle of continuous benchmark crds (#1955)

Signed-off-by: vankichi <[email protected]>

* Impl benchmark jobs (#1977)

* Implement base of continuous benchmark tool (#1776)

* Create Continuous Bench Search Job tool (#1733)

* ✨ create bench job search tools

Signed-off-by: vankichi <[email protected]>

* ✨ add load hdf5 functions

Signed-off-by: vankichi <[email protected]>

* ♻️ fix format

Signed-off-by: vankichi <[email protected]>

* ♻️ fix docker and use hdf5 data

Signed-off-by: vankichi <[email protected]>

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor benchmark job

Signed-off-by: vankichi <[email protected]>

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ fix proto

Signed-off-by: vankichi <[email protected]>

* 💚 add benchmark job image build ci

Signed-off-by: vankichi <[email protected]>

* 💚 invest

Signed-off-by: vankichi <[email protected]>

* Revert "💚 invest"

This reverts commit f0f585c.

* Apply suggestions from code review

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ apply code review

Signed-off-by: vankichi <[email protected]>

* Apply suggestions from code review

Co-authored-by: Hiroto Funakoshi <[email protected]>

* ✨ apply from feedback

Signed-off-by: vankichi <[email protected]>

* Update internal/config/benchmark.go

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ change directory path

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>

* Add crds for continuous benchmark tools (#1789)

* ✨ add crds for continuous benchmark operator

Signed-off-by: vankichi <[email protected]>

* ✨ add benchmark operator/job scheme

Signed-off-by: vankichi <[email protected]>

* ✨ rename package names and add doc.go

Signed-off-by: vankichi <[email protected]>

* ✨ create runtime object

Signed-off-by: vankichi <[email protected]>

* Apply suggestions from code review

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ apply feedback

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>

* Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825)

* ✨ ♻️ add Job reconciler & use scenario instead of operator

Signed-off-by: vankichi <[email protected]>

* ♻️ fix format & rename file

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>

* Add benchmark operator framework (#1916)

* ✨ impl benchmark reconciler

Signed-off-by: vankichi <[email protected]>

* ✨ create benchmark operator framework

Signed-off-by: vankichi <[email protected]>

* ♻️ remove unness changes

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* impl reconcile logic for create benchmark job (#1923)

* ✨ impl reconcile logic for create benchmark job

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor continuous benchmark's crds

Signed-off-by: vankichi <[email protected]>

* ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor continuous benchmark job logic

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ update charts

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ rafactor con bench config and bug fix reconcile logic

Signed-off-by: vankichi <[email protected]>

* 🐛 Bugfix: fix typo and recall function logic

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor pkg benchmark job

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

* ✨ impl benchmark jobs

Signed-off-by: vankichi <[email protected]>

* ♻️ apply feedback

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

* ♻️ Refactor helm template and operator logic

Signed-off-by: vankichi <[email protected]>

* ♻️ Add download original dataset URL option

Signed-off-by: vankichi <[email protected]>

* ♻️ Set docker image location at the benchmark operator configmap and use it when information is set

Signed-off-by: vankichi <[email protected]>

* add search algorithm benchmark and update search aggregation algo

Signed-off-by: kpango <[email protected]>

* ✨ Add search result aggregation option

Signed-off-by: vankichi <[email protected]>

* style: Format code with prettier and gofumpt

* Improve job performance (#2061)

* 🐛 Fix job function to apply rate limiter

* ♻️ Add pyroscope setting

Signed-off-by: vankichi <[email protected]>

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* 🐛 Fix build error

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>

* 🐛 Fix docker file and add concurrencyLimit for job goroutine

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix job_template.go by feedback

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix job logic by feedback

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>
Signed-off-by: kpango <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
* Implement base of continuous benchmark tool (#1776)

* Create Continuous Bench Search Job tool (#1733)

* ✨ create bench job search tools

Signed-off-by: vankichi <[email protected]>

* ✨ add load hdf5 functions

Signed-off-by: vankichi <[email protected]>

* ♻️ fix format

Signed-off-by: vankichi <[email protected]>

* ♻️ fix docker and use hdf5 data

Signed-off-by: vankichi <[email protected]>

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor benchmark job

Signed-off-by: vankichi <[email protected]>

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ fix proto

Signed-off-by: vankichi <[email protected]>

* 💚 add benchmark job image build ci

Signed-off-by: vankichi <[email protected]>

* 💚 invest

Signed-off-by: vankichi <[email protected]>

* Revert "💚 invest"

This reverts commit f0f585c.

* Apply suggestions from code review

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ apply code review

Signed-off-by: vankichi <[email protected]>

* Apply suggestions from code review

Co-authored-by: Hiroto Funakoshi <[email protected]>

* ✨ apply from feedback

Signed-off-by: vankichi <[email protected]>

* Update internal/config/benchmark.go

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ change directory path

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>

* Add crds for continuous benchmark tools (#1789)

* ✨ add crds for continuous benchmark operator

Signed-off-by: vankichi <[email protected]>

* ✨ add benchmark operator/job scheme

Signed-off-by: vankichi <[email protected]>

* ✨ rename package names and add doc.go

Signed-off-by: vankichi <[email protected]>

* ✨ create runtime object

Signed-off-by: vankichi <[email protected]>

* Apply suggestions from code review

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ apply feedback

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>

* Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825)

* ✨ ♻️ add Job reconciler & use scenario instead of operator

Signed-off-by: vankichi <[email protected]>

* ♻️ fix format & rename file

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>

* Add benchmark operator framework (#1916)

* ✨ impl benchmark reconciler

Signed-off-by: vankichi <[email protected]>

* ✨ create benchmark operator framework

Signed-off-by: vankichi <[email protected]>

* ♻️ remove unness changes

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* impl reconcile logic for create benchmark job (#1923)

* ✨ impl reconcile logic for create benchmark job

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor continuous benchmark's crds

Signed-off-by: vankichi <[email protected]>

* ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor continuous benchmark job logic

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ update charts

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ rafactor con bench config and bug fix reconcile logic

Signed-off-by: vankichi <[email protected]>

* 🐛 Bugfix: fix typo and recall function logic

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor pkg benchmark job

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

* ✨ impl benchmark jobs

Signed-off-by: vankichi <[email protected]>

* ♻️ apply feedback

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
* Implement base of continuous benchmark tool (#1776)

* Create Continuous Bench Search Job tool (#1733)

* ✨ create bench job search tools

Signed-off-by: vankichi <[email protected]>

* ✨ add load hdf5 functions

Signed-off-by: vankichi <[email protected]>

* ♻️ fix format

Signed-off-by: vankichi <[email protected]>

* ♻️ fix docker and use hdf5 data

Signed-off-by: vankichi <[email protected]>

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor benchmark job

Signed-off-by: vankichi <[email protected]>

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ fix proto

Signed-off-by: vankichi <[email protected]>

* 💚 add benchmark job image build ci

Signed-off-by: vankichi <[email protected]>

* 💚 invest

Signed-off-by: vankichi <[email protected]>

* Revert "💚 invest"

This reverts commit f0f585c.

* Apply suggestions from code review

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ apply code review

Signed-off-by: vankichi <[email protected]>

* Apply suggestions from code review

Co-authored-by: Hiroto Funakoshi <[email protected]>

* ✨ apply from feedback

Signed-off-by: vankichi <[email protected]>

* Update internal/config/benchmark.go

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ change directory path

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>

* Add crds for continuous benchmark tools (#1789)

* ✨ add crds for continuous benchmark operator

Signed-off-by: vankichi <[email protected]>

* ✨ add benchmark operator/job scheme

Signed-off-by: vankichi <[email protected]>

* ✨ rename package names and add doc.go

Signed-off-by: vankichi <[email protected]>

* ✨ create runtime object

Signed-off-by: vankichi <[email protected]>

* Apply suggestions from code review

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ apply feedback

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>

* Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825)

* ✨ ♻️ add Job reconciler & use scenario instead of operator

Signed-off-by: vankichi <[email protected]>

* ♻️ fix format & rename file

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>

* Add benchmark operator framework (#1916)

* ✨ impl benchmark reconciler

Signed-off-by: vankichi <[email protected]>

* ✨ create benchmark operator framework

Signed-off-by: vankichi <[email protected]>

* ♻️ remove unness changes

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* impl reconcile logic for create benchmark job (#1923)

* ✨ impl reconcile logic for create benchmark job

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor continuous benchmark's crds

Signed-off-by: vankichi <[email protected]>

* ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor continuous benchmark job logic

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ update charts

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ rafactor con bench config and bug fix reconcile logic

Signed-off-by: vankichi <[email protected]>

* 🐛 Bugfix: fix typo and recall function logic

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor pkg benchmark job

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

* ✨ impl status handle of continuous benchmark crds (#1955)

Signed-off-by: vankichi <[email protected]>

* Impl benchmark jobs (#1977)

* Implement base of continuous benchmark tool (#1776)

* Create Continuous Bench Search Job tool (#1733)

* ✨ create bench job search tools

Signed-off-by: vankichi <[email protected]>

* ✨ add load hdf5 functions

Signed-off-by: vankichi <[email protected]>

* ♻️ fix format

Signed-off-by: vankichi <[email protected]>

* ♻️ fix docker and use hdf5 data

Signed-off-by: vankichi <[email protected]>

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor benchmark job

Signed-off-by: vankichi <[email protected]>

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ fix proto

Signed-off-by: vankichi <[email protected]>

* 💚 add benchmark job image build ci

Signed-off-by: vankichi <[email protected]>

* 💚 invest

Signed-off-by: vankichi <[email protected]>

* Revert "💚 invest"

This reverts commit f0f585c.

* Apply suggestions from code review

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ apply code review

Signed-off-by: vankichi <[email protected]>

* Apply suggestions from code review

Co-authored-by: Hiroto Funakoshi <[email protected]>

* ✨ apply from feedback

Signed-off-by: vankichi <[email protected]>

* Update internal/config/benchmark.go

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ change directory path

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>

* Add crds for continuous benchmark tools (#1789)

* ✨ add crds for continuous benchmark operator

Signed-off-by: vankichi <[email protected]>

* ✨ add benchmark operator/job scheme

Signed-off-by: vankichi <[email protected]>

* ✨ rename package names and add doc.go

Signed-off-by: vankichi <[email protected]>

* ✨ create runtime object

Signed-off-by: vankichi <[email protected]>

* Apply suggestions from code review

Co-authored-by: Yusuke Kato <[email protected]>

* ♻️ apply feedback

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>

* Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825)

* ✨ ♻️ add Job reconciler & use scenario instead of operator

Signed-off-by: vankichi <[email protected]>

* ♻️ fix format & rename file

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>

* Add benchmark operator framework (#1916)

* ✨ impl benchmark reconciler

Signed-off-by: vankichi <[email protected]>

* ✨ create benchmark operator framework

Signed-off-by: vankichi <[email protected]>

* ♻️ remove unness changes

Signed-off-by: vankichi <[email protected]>

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* impl reconcile logic for create benchmark job (#1923)

* ✨ impl reconcile logic for create benchmark job

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor continuous benchmark's crds

Signed-off-by: vankichi <[email protected]>

* ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor continuous benchmark job logic

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ update charts

Signed-off-by: vankichi <[email protected]>

* Format code with prettier and gofumpt

* ♻️ rafactor con bench config and bug fix reconcile logic

Signed-off-by: vankichi <[email protected]>

* 🐛 Bugfix: fix typo and recall function logic

Signed-off-by: vankichi <[email protected]>

* ♻️ refactor pkg benchmark job

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

* ✨ impl benchmark jobs

Signed-off-by: vankichi <[email protected]>

* ♻️ apply feedback

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

* ♻️ Refactor helm template and operator logic

Signed-off-by: vankichi <[email protected]>

* ♻️ Add download original dataset URL option

Signed-off-by: vankichi <[email protected]>

* ♻️ Set docker image location at the benchmark operator configmap and use it when information is set

Signed-off-by: vankichi <[email protected]>

* add search algorithm benchmark and update search aggregation algo

Signed-off-by: kpango <[email protected]>

* ✨ Add search result aggregation option

Signed-off-by: vankichi <[email protected]>

* style: Format code with prettier and gofumpt

* Improve job performance (#2061)

* 🐛 Fix job function to apply rate limiter

* ♻️ Add pyroscope setting

Signed-off-by: vankichi <[email protected]>

* ♻️ fix

Signed-off-by: vankichi <[email protected]>

* 🐛 Fix build error

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>

* 🐛 Fix docker file and add concurrencyLimit for job goroutine

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix job_template.go by feedback

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix job logic by feedback

Signed-off-by: vankichi <[email protected]>

* ♻️ Fix

Signed-off-by: vankichi <[email protected]>

---------

Signed-off-by: vankichi <[email protected]>
Signed-off-by: kpango <[email protected]>
Co-authored-by: Yusuke Kato <[email protected]>
Co-authored-by: Hiroto Funakoshi <[email protected]>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Aug 22, 2023

Deploying with  Cloudflare Pages  Cloudflare Pages

Latest commit: 4b2c685
Status: ✅  Deploy successful!
Preview URL: https://eaa796b3.vald.pages.dev
Branch Preview URL: https://tmp-bench-errgroup-test.vald.pages.dev

View logs

@vdaas-ci
Copy link
Collaborator

[WARNING:INTCFG] Changes in interal/config may require you to change Helm charts. Please check.

ENV PKG tools/benchmark/${APP_NAME}

# skipcq: DOK-DL3008
RUN apt-get update && apt-get install -y --no-install-recommends \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [hadolint] <DL3008> reported by reviewdog 🐶
Pin versions in apt get install. Instead of apt-get install <package> use apt-get install <package>=<version>


RUN mkdir -p ${GOPATH}/src

COPY --from=golang /usr/local/go $GOROOT
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [hadolint] <DL3022> reported by reviewdog 🐶
COPY --from should reference a previously defined FROM alias

ENV PKG tools/benchmark/${APP_NAME}

# skipcq: DOK-DL3008
RUN apt-get update && apt-get install -y --no-install-recommends \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [hadolint] <DL3008> reported by reviewdog 🐶
Pin versions in apt get install. Instead of apt-get install <package> use apt-get install <package>=<version>


RUN mkdir -p ${GOPATH}/src

COPY --from=golang /usr/local/go $GOROOT
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [hadolint] <DL3022> reported by reviewdog 🐶
COPY --from should reference a previously defined FROM alias


COPY .git .

RUN make REPO=${ORG} NAME=${REPO} cmd/${PKG}/${APP_NAME} \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [hadolint] <SC2086> reported by reviewdog 🐶
Double quote to prevent globbing and word splitting.

ENV PKG tools/benchmark/${APP_NAME}

# skipcq: DOK-DL3008
RUN apt-get update && apt-get install -y --no-install-recommends \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [hadolint] <DL3008> reported by reviewdog 🐶
Pin versions in apt get install. Instead of apt-get install <package> use apt-get install <package>=<version>


RUN mkdir -p ${GOPATH}/src

COPY --from=golang /usr/local/go $GOROOT
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [hadolint] <DL3022> reported by reviewdog 🐶
COPY --from should reference a previously defined FROM alias


COPY .git .

RUN make REPO=${ORG} NAME=${REPO} cmd/${PKG}/${APP_NAME} \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [hadolint] <SC2086> reported by reviewdog 🐶
Double quote to prevent globbing and word splitting.

@ykadowak ykadowak closed this Aug 22, 2023
@ykadowak ykadowak deleted the tmp/bench/errgroup-test branch August 22, 2023 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants