Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: some optimize #622

Merged
merged 38 commits into from
Sep 11, 2024
Merged
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
fa775ee
feat: add benchmark
devhaozi Sep 4, 2024
ba1b77f
chore: update mocks
devhaozi Sep 4, 2024
80b43ec
feat: bump go1.22
devhaozi Sep 4, 2024
c671bd1
workflow: fix benchmark
devhaozi Sep 4, 2024
cd7a67c
Merge remote-tracking branch 'origin/haozi/optimize' into haozi/optimize
devhaozi Sep 4, 2024
a04d408
feat: optimize session
devhaozi Sep 4, 2024
eb4ca10
chore: update mocks
devhaozi Sep 4, 2024
c476440
feat: add more benchmark cases
devhaozi Sep 4, 2024
94c7b45
fix: tests
devhaozi Sep 4, 2024
116db2d
revert: session manager Extend
devhaozi Sep 4, 2024
7dfe6db
chore: update mocks
devhaozi Sep 4, 2024
a82d4ac
fix: lint
devhaozi Sep 4, 2024
071e83a
fix: tests
devhaozi Sep 4, 2024
422a8b4
fix: tests
devhaozi Sep 4, 2024
55325f0
feat: optimize code
devhaozi Sep 4, 2024
14a8bab
refactor: change session manager Extend
devhaozi Sep 4, 2024
8835015
chore: update mocks
devhaozi Sep 4, 2024
839d37f
workflow: add govulncheck
devhaozi Sep 4, 2024
7f83a71
Merge remote-tracking branch 'origin/haozi/optimize' into haozi/optimize
devhaozi Sep 4, 2024
eda3634
workflow: add govulncheck
devhaozi Sep 4, 2024
830e51a
workflow: add govulncheck
devhaozi Sep 4, 2024
d4f2274
workflow: fix shellcheck
devhaozi Sep 4, 2024
43d37f3
refactor: rename postgresql to postgres
devhaozi Sep 4, 2024
9143806
workflow: add nilaway
devhaozi Sep 4, 2024
14c2f31
Merge branch 'master' into haozi/optimize
devhaozi Sep 4, 2024
bf97aec
workflow: optimize nilaway
devhaozi Sep 4, 2024
f5f0af1
Merge branch 'master' into haozi/optimize
devhaozi Sep 5, 2024
629010c
workflow: Update lint.yml
devhaozi Sep 5, 2024
1e66167
Merge remote-tracking branch 'origin/master' into haozi/optimize
devhaozi Sep 9, 2024
bc59486
feat: optimize code
devhaozi Sep 9, 2024
cce4247
chore: update mocks
devhaozi Sep 9, 2024
70a2f36
workflow: use .github source
devhaozi Sep 10, 2024
c412ee9
Merge branch 'master' into haozi/optimize
devhaozi Sep 10, 2024
b47dbfa
workflow: debug
devhaozi Sep 10, 2024
021515c
workflow: finish debug
devhaozi Sep 11, 2024
ca34907
Merge branch 'master' into haozi/optimize
devhaozi Sep 11, 2024
4eb0170
Merge branch 'master' into haozi/optimize
devhaozi Sep 11, 2024
c506389
chore: sort
devhaozi Sep 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 91 additions & 0 deletions .github/workflows/benchmark.yml
devhaozi marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Em, if we can‘t check the result, what's the meaning of it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Em, if we can‘t check the result, what's the meaning of it?

Move the test cases previously used for concurrency testing to the benchmark.
And although the results cannot be viewed directly, CI will automatically compare them with previous results.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI will automatically compare them with previous results.

Do you mean this?

image

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI will automatically compare them with previous results.

Do you mean this?

image

Yes, currently no data.

Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
name: Benchmark
on:
push:
branches:
- master
paths-ignore:
- "**/*.md"
pull_request:
paths-ignore:
- "**/*.md"
permissions:
contents: write
pull-requests: write
deployments: write
jobs:
benchmark:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # to be able to retrieve the last commit
- uses: actions/setup-go@v5
with:
go-version: "stable"
- name: Run Benchmark
run: set -o pipefail; go test ./... -benchmem -run=^$ -bench . | tee output.txt
# NOTE: Benchmarks could change with different CPU types
- name: Get GitHub Runner System Information
uses: kenchan0130/[email protected]
id: system-info
- name: Get master branch SHA
id: get-master-branch-sha
run: |
devhaozi marked this conversation as resolved.
Show resolved Hide resolved
SHA="$(git rev-parse origin/master)"
echo "sha=$SHA" >> $GITHUB_OUTPUT
- name: Get Benchmark Results from master branch
id: cache
uses: actions/cache/restore@v4
with:
path: ./cache
key: ${{ steps.get-master-branch-sha.outputs.sha }}-${{ runner.os }}-${{ steps.system-info.outputs.cpu-model }}-benchmark
# This will only run if we have Benchmark Results from master branch
- name: Compare PR Benchmark Results with master branch
uses: benchmark-action/[email protected]
if: steps.cache.outputs.cache-hit == 'true'
with:
tool: 'go'
output-file-path: output.txt
external-data-json-path: ./cache/benchmark-data.json
# Do not save the data (This allows comparing benchmarks)
save-data-file: false
fail-on-alert: true
# Comment on the PR if the branch is not a fork
comment-on-alert: ${{ github.event.pull_request.head.repo.fork == false }}
github-token: ${{ secrets.GITHUB_TOKEN }}
summary-always: true
alert-threshold: "150%"
- name: Store Benchmark Results for master branch
uses: benchmark-action/[email protected]
if: ${{ github.ref_name == 'master' }}
with:
tool: 'go'
output-file-path: output.txt
external-data-json-path: ./cache/benchmark-data.json
# Save the data to external file (cache)
save-data-file: true
fail-on-alert: false
github-token: ${{ secrets.GITHUB_TOKEN }}
summary-always: true
alert-threshold: "150%"
- name: Publish Benchmark Results to GitHub Pages
uses: benchmark-action/github-action-benchmark@v1
if: ${{ github.ref_name == 'master' }}
with:
tool: 'go'
output-file-path: output.txt
benchmark-data-dir-path: "benchmarks"
fail-on-alert: false
github-token: ${{ secrets.GITHUB_TOKEN }}
comment-on-alert: true
summary-always: true
# Save the data to external file (GitHub Pages)
save-data-file: true
alert-threshold: "150%"
auto-push: true
- name: Update Benchmark Results cache
uses: actions/cache/save@v4
if: ${{ github.ref_name == 'master' }}
with:
path: ./cache
key: ${{ steps.get-master-branch-sha.outputs.sha }}-${{ runner.os }}-${{ steps.system-info.outputs.cpu-model }}-benchmark
25 changes: 25 additions & 0 deletions .github/workflows/govulncheck.yml
devhaozi marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: Govulncheck
on:
schedule:
- cron: "0 */6 * * *" # Run every 6 hours
jobs:
govulncheck:
runs-on: ubuntu-latest
env:
GO111MODULE: on
steps:
- name: Fetch Repository
uses: actions/checkout@v4

- name: Install Go
uses: actions/setup-go@v5
with:
go-version: "stable"
check-latest: true
cache: false

devhaozi marked this conversation as resolved.
Show resolved Hide resolved
- name: Install Govulncheck
run: go install golang.org/x/vuln/cmd/govulncheck@latest

- name: Run Govulncheck
run: govulncheck ./...
3 changes: 3 additions & 0 deletions .github/workflows/mockery.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
name: Mockery
on:
pull_request:
permissions:
contents: write
pull-requests: write
jobs:
mockery:
runs-on: ubuntu-latest
Expand Down
10 changes: 4 additions & 6 deletions contracts/database/orm/constants.go
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
package orm

const (
DriverMysql Driver = "mysql"
// DriverPostgresql DEPRECATED, use DriverPostgres instead.
DriverPostgresql Driver = "postgresql"
DriverPostgres Driver = "postgres"
DriverSqlite Driver = "sqlite"
DriverSqlserver Driver = "sqlserver"
DriverMysql Driver = "mysql"
DriverPostgres Driver = "postgres"
DriverSqlite Driver = "sqlite"
DriverSqlserver Driver = "sqlserver"
)

type Driver string
Expand Down
4 changes: 3 additions & 1 deletion contracts/session/manager.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,7 @@ type Manager interface {
// Driver retrieves the session driver by name.
Driver(name ...string) (Driver, error)
// Extend extends the session manager with a custom driver.
Extend(driver string, handler func() Driver) Manager
Extend(driver string, handler func() Driver) error
// ReleaseSession releases the session back to the pool.
ReleaseSession(session Session)
}
32 changes: 32 additions & 0 deletions crypt/aes_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,35 @@ func (s *AesTestSuite) TestDecryptString() {
_, err = s.aes.DecryptString("eyJpdiI6IjEyMzQ1IiwidmFsdWUiOiIxMjM0NSJ9")
s.Error(err)
}

func Benchmark_EncryptString(b *testing.B) {
mockConfig := &configmock.Config{}
mockConfig.On("GetString", "app.key").Return("11111111111111111111111111111111").Once()
aes := NewAES(mockConfig, json.NewJson())
b.ResetTimer()

for i := 0; i < b.N; i++ {
_, err := aes.EncryptString("Goravel")
if err != nil {
b.Error(err)
}
}
}

func Benchmark_DecryptString(b *testing.B) {
mockConfig := &configmock.Config{}
mockConfig.On("GetString", "app.key").Return("11111111111111111111111111111111").Once()
aes := NewAES(mockConfig, json.NewJson())
payload, err := aes.EncryptString("Goravel")
if err != nil {
b.Error(err)
}
b.ResetTimer()

for i := 0; i < b.N; i++ {
_, err := aes.DecryptString(payload)
if err != nil {
b.Error(err)
}
}
}
10 changes: 5 additions & 5 deletions database/console/migrate.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,14 +52,14 @@
}

return migrate.NewWithDatabaseInstance(dir, "mysql", instance)
case orm.DriverPostgresql:
case orm.DriverPostgres:
dsn := db.NewDsnImpl(config, connection)
postgresqlDsn := dsn.Postgresql(writeConfigs[0])
if postgresqlDsn == "" {
postgresDsn := dsn.Postgres(writeConfigs[0])
if postgresDsn == "" {
return nil, nil
}

db, err := sql.Open("postgres", postgresqlDsn)
db, err := sql.Open("postgres", postgresDsn)
if err != nil {
return nil, err
}
Expand Down Expand Up @@ -114,6 +114,6 @@

return migrate.NewWithDatabaseInstance(dir, "sqlserver", instance)
default:
return nil, errors.New("database driver only support mysql, postgresql, sqlite and sqlserver")
return nil, errors.New("database driver only support mysql, postgres, sqlite and sqlserver")

Check warning on line 117 in database/console/migrate.go

View check run for this annotation

Codecov / codecov/patch

database/console/migrate.go#L117

Added line #L117 was not covered by tests
}
}
6 changes: 3 additions & 3 deletions database/console/migrate_command_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -61,14 +61,14 @@ func TestMigrateCommand(t *testing.T) {
},
},
{
name: "postgresql",
name: "postgres",
setup: func() {
var err error
docker := gorm.NewPostgresDocker(docker.Postgres())
query, err = docker.New()
assert.Nil(t, err)
mockConfig = docker.MockConfig
createPostgresqlMigrations()
createPostgresMigrations()
},
},
{
Expand Down Expand Up @@ -120,7 +120,7 @@ INSERT INTO agents (name, created_at, updated_at) VALUES ('goravel', '2023-03-11
`)
}

func createPostgresqlMigrations() {
func createPostgresMigrations() {
_ = file.Create("database/migrations/20230311160527_create_agents_table.up.sql",
`CREATE TABLE agents (
id SERIAL PRIMARY KEY NOT NULL,
Expand Down
6 changes: 3 additions & 3 deletions database/console/migrate_creator.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,12 +50,12 @@

driver := receiver.config.GetString("database.connections." + receiver.config.GetString("database.default") + ".driver")
switch orm.Driver(driver) {
case orm.DriverPostgresql:
case orm.DriverPostgres:

Check warning on line 53 in database/console/migrate_creator.go

View check run for this annotation

Codecov / codecov/patch

database/console/migrate_creator.go#L53

Added line #L53 was not covered by tests
if create {
return migration.PostgresqlStubs{}.CreateUp(), migration.PostgresqlStubs{}.CreateDown()
return migration.PostgresStubs{}.CreateUp(), migration.PostgresStubs{}.CreateDown()

Check warning on line 55 in database/console/migrate_creator.go

View check run for this annotation

Codecov / codecov/patch

database/console/migrate_creator.go#L55

Added line #L55 was not covered by tests
}

return migration.PostgresqlStubs{}.UpdateUp(), migration.PostgresqlStubs{}.UpdateDown()
return migration.PostgresStubs{}.UpdateUp(), migration.PostgresStubs{}.UpdateDown()

Check warning on line 58 in database/console/migrate_creator.go

View check run for this annotation

Codecov / codecov/patch

database/console/migrate_creator.go#L58

Added line #L58 was not covered by tests
case orm.DriverSqlite:
if create {
return migration.SqliteStubs{}.CreateUp(), migration.SqliteStubs{}.CreateDown()
Expand Down
4 changes: 2 additions & 2 deletions database/console/migrate_fresh_command_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -54,14 +54,14 @@ func TestMigrateFreshCommand(t *testing.T) {
},
},
{
name: "postgresql",
name: "postgres",
setup: func() {
var err error
docker := gorm.NewPostgresDocker(docker.Postgres())
query, err = docker.New()
assert.Nil(t, err)
mockConfig = docker.MockConfig
createPostgresqlMigrations()
createPostgresMigrations()
},
},
{
Expand Down
4 changes: 2 additions & 2 deletions database/console/migrate_refresh_command_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -54,14 +54,14 @@ func TestMigrateRefreshCommand(t *testing.T) {
},
},
{
name: "postgresql",
name: "postgres",
setup: func() {
var err error
docker := gorm.NewPostgresDocker(docker.Postgres())
query, err = docker.New()
assert.Nil(t, err)
mockConfig = docker.MockConfig
createPostgresqlMigrations()
createPostgresMigrations()
},
},
{
Expand Down
4 changes: 2 additions & 2 deletions database/console/migrate_reset_command_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,14 +44,14 @@ func TestMigrateResetCommand(t *testing.T) {
},
},
{
name: "postgresql",
name: "postgres",
setup: func() {
var err error
docker := gorm.NewPostgresDocker(docker.Postgres())
query, err = docker.New()
assert.Nil(t, err)
mockConfig = docker.MockConfig
createPostgresqlMigrations()
createPostgresMigrations()
},
},
{
Expand Down
4 changes: 2 additions & 2 deletions database/console/migrate_rollback_command_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,14 +44,14 @@ func TestMigrateRollbackCommand(t *testing.T) {
},
},
{
name: "postgresql",
name: "postgres",
setup: func() {
var err error
docker := gorm.NewPostgresDocker(docker.Postgres())
query, err = docker.New()
assert.Nil(t, err)
mockConfig = docker.MockConfig
createPostgresqlMigrations()
createPostgresMigrations()
},
},
{
Expand Down
4 changes: 2 additions & 2 deletions database/console/migrate_status_command_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,14 +44,14 @@ func TestMigrateStatusCommand(t *testing.T) {
},
},
{
name: "postgresql",
name: "postgres",
setup: func() {
var err error
docker := gorm.NewPostgresDocker(docker.Postgres())
query, err = docker.New()
assert.Nil(t, err)
mockConfig = docker.MockConfig
createPostgresqlMigrations()
createPostgresMigrations()
},
},
{
Expand Down
4 changes: 2 additions & 2 deletions database/db/dsn.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (

type Dsn interface {
Mysql(config databasecontract.Config) string
Postgresql(config databasecontract.Config) string
Postgres(config databasecontract.Config) string
Sqlite(config databasecontract.Config) string
Sqlserver(config databasecontract.Config) string
}
Expand Down Expand Up @@ -39,7 +39,7 @@ func (d *DsnImpl) Mysql(config databasecontract.Config) string {
config.Username, config.Password, host, config.Port, config.Database, charset, true, loc)
}

func (d *DsnImpl) Postgresql(config databasecontract.Config) string {
func (d *DsnImpl) Postgres(config databasecontract.Config) string {
host := config.Host
if host == "" {
return ""
Expand Down
6 changes: 3 additions & 3 deletions database/db/dsn_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,16 +52,16 @@ func (s *DsnTestSuite) TestMysql() {
testUsername, testPassword, testHost, testPort, testDatabase, charset, true, loc), dsn.Mysql(testConfig))
}

func (s *DsnTestSuite) TestPostgresql() {
connection := orm.DriverPostgresql.String()
func (s *DsnTestSuite) TestPostgres() {
connection := orm.DriverPostgres.String()
dsn := NewDsnImpl(s.mockConfig, connection)
sslmode := "disable"
timezone := "UTC"
s.mockConfig.On("GetString", fmt.Sprintf("database.connections.%s.sslmode", connection)).Return(sslmode).Once()
s.mockConfig.On("GetString", fmt.Sprintf("database.connections.%s.timezone", connection)).Return(timezone).Once()

s.Equal(fmt.Sprintf("postgres://%s:%s@%s:%d/%s?sslmode=%s&timezone=%s",
testUsername, testPassword, testHost, testPort, testDatabase, sslmode, timezone), dsn.Postgresql(testConfig))
testUsername, testPassword, testHost, testPort, testDatabase, sslmode, timezone), dsn.Postgres(testConfig))
}

func (s *DsnTestSuite) TestSqlite() {
Expand Down
Loading
Loading