Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: some optimize #622

Merged
merged 38 commits into from
Sep 11, 2024
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
fa775ee
feat: add benchmark
devhaozi Sep 4, 2024
ba1b77f
chore: update mocks
devhaozi Sep 4, 2024
80b43ec
feat: bump go1.22
devhaozi Sep 4, 2024
c671bd1
workflow: fix benchmark
devhaozi Sep 4, 2024
cd7a67c
Merge remote-tracking branch 'origin/haozi/optimize' into haozi/optimize
devhaozi Sep 4, 2024
a04d408
feat: optimize session
devhaozi Sep 4, 2024
eb4ca10
chore: update mocks
devhaozi Sep 4, 2024
c476440
feat: add more benchmark cases
devhaozi Sep 4, 2024
94c7b45
fix: tests
devhaozi Sep 4, 2024
116db2d
revert: session manager Extend
devhaozi Sep 4, 2024
7dfe6db
chore: update mocks
devhaozi Sep 4, 2024
a82d4ac
fix: lint
devhaozi Sep 4, 2024
071e83a
fix: tests
devhaozi Sep 4, 2024
422a8b4
fix: tests
devhaozi Sep 4, 2024
55325f0
feat: optimize code
devhaozi Sep 4, 2024
14a8bab
refactor: change session manager Extend
devhaozi Sep 4, 2024
8835015
chore: update mocks
devhaozi Sep 4, 2024
839d37f
workflow: add govulncheck
devhaozi Sep 4, 2024
7f83a71
Merge remote-tracking branch 'origin/haozi/optimize' into haozi/optimize
devhaozi Sep 4, 2024
eda3634
workflow: add govulncheck
devhaozi Sep 4, 2024
830e51a
workflow: add govulncheck
devhaozi Sep 4, 2024
d4f2274
workflow: fix shellcheck
devhaozi Sep 4, 2024
43d37f3
refactor: rename postgresql to postgres
devhaozi Sep 4, 2024
9143806
workflow: add nilaway
devhaozi Sep 4, 2024
14c2f31
Merge branch 'master' into haozi/optimize
devhaozi Sep 4, 2024
bf97aec
workflow: optimize nilaway
devhaozi Sep 4, 2024
f5f0af1
Merge branch 'master' into haozi/optimize
devhaozi Sep 5, 2024
629010c
workflow: Update lint.yml
devhaozi Sep 5, 2024
1e66167
Merge remote-tracking branch 'origin/master' into haozi/optimize
devhaozi Sep 9, 2024
bc59486
feat: optimize code
devhaozi Sep 9, 2024
cce4247
chore: update mocks
devhaozi Sep 9, 2024
70a2f36
workflow: use .github source
devhaozi Sep 10, 2024
c412ee9
Merge branch 'master' into haozi/optimize
devhaozi Sep 10, 2024
b47dbfa
workflow: debug
devhaozi Sep 10, 2024
021515c
workflow: finish debug
devhaozi Sep 11, 2024
ca34907
Merge branch 'master' into haozi/optimize
devhaozi Sep 11, 2024
4eb0170
Merge branch 'master' into haozi/optimize
devhaozi Sep 11, 2024
c506389
chore: sort
devhaozi Sep 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 102 additions & 0 deletions .github/workflows/benchmark.yml
devhaozi marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Em, if we can‘t check the result, what's the meaning of it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Em, if we can‘t check the result, what's the meaning of it?

Move the test cases previously used for concurrency testing to the benchmark.
And although the results cannot be viewed directly, CI will automatically compare them with previous results.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI will automatically compare them with previous results.

Do you mean this?

image

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI will automatically compare them with previous results.

Do you mean this?

image

Yes, currently no data.

Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
name: Benchmark
on:
push:
branches:
- master
paths-ignore:
- "**/*.md"
pull_request:
paths-ignore:
- "**/*.md"
permissions:
contents: write
pull-requests: write
jobs:
benchmark:
runs-on: ubuntu-latest
steps:
- name: Fetch Repository
uses: actions/checkout@v4
with:
fetch-depth: 0 # to be able to retrieve the last commit

- name: Install Go
uses: actions/setup-go@v5
with:
# NOTE: Keep this in sync with the version from go.mod
go-version: "1.22.x"

- name: Run Benchmark
run: set -o pipefail; go test ./... -benchmem -run=^$ -bench . | tee output.txt

# NOTE: Benchmarks could change with different CPU types
- name: Get GitHub Runner System Information
uses: kenchan0130/[email protected]
id: system-info

- name: Get Main branch SHA
id: get-master-branch-sha
run: |
devhaozi marked this conversation as resolved.
Show resolved Hide resolved
SHA=$(git rev-parse origin/master)
echo "sha=$SHA" >> $GITHUB_OUTPUT

- name: Get Benchmark Results from master branch
id: cache
uses: actions/cache/restore@v4
with:
path: ./cache
key: ${{ steps.get-master-branch-sha.outputs.sha }}-${{ runner.os }}-${{ steps.system-info.outputs.cpu-model }}-benchmark

# This will only run if we have Benchmark Results from master branch
- name: Compare PR Benchmark Results with master branch
uses: benchmark-action/[email protected]
if: steps.cache.outputs.cache-hit == 'true'
with:
tool: 'go'
output-file-path: output.txt
external-data-json-path: ./cache/benchmark-data.json
# Do not save the data (This allows comparing benchmarks)
save-data-file: false
fail-on-alert: true
# Comment on the PR if the branch is not a fork
comment-on-alert: ${{ github.event.pull_request.head.repo.fork == false }}
github-token: ${{ secrets.GITHUB_TOKEN }}
summary-always: true
alert-threshold: "150%"

- name: Store Benchmark Results for master branch
uses: benchmark-action/[email protected]
if: ${{ github.ref_name == 'master' }}
with:
tool: 'go'
output-file-path: output.txt
external-data-json-path: ./cache/benchmark-data.json
# Save the data to external file (cache)
save-data-file: true
fail-on-alert: false
github-token: ${{ secrets.GITHUB_TOKEN }}
summary-always: true
alert-threshold: "150%"

- name: Publish Benchmark Results to GitHub Pages
uses: benchmark-action/[email protected]
if: ${{ github.ref_name == 'master' }}
with:
tool: 'go'
output-file-path: output.txt
benchmark-data-dir-path: "benchmarks"
fail-on-alert: false
github-token: ${{ secrets.GITHUB_TOKEN }}
comment-on-alert: true
summary-always: true
# Save the data to external file (GitHub Pages)
save-data-file: true
alert-threshold: "150%"
auto-push: false

- name: Update Benchmark Results cache
uses: actions/cache/save@v4
if: ${{ github.ref_name == 'master' }}
with:
path: ./cache
key: ${{ steps.get-master-branch-sha.outputs.sha }}-${{ runner.os }}-${{ steps.system-info.outputs.cpu-model }}-benchmark
3 changes: 3 additions & 0 deletions .github/workflows/mockery.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
name: Mockery
on:
pull_request:
permissions:
contents: write
pull-requests: write
jobs:
mockery:
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion contracts/session/manager.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,5 @@ type Manager interface {
// Driver retrieves the session driver by name.
Driver(name ...string) (Driver, error)
// Extend extends the session manager with a custom driver.
Extend(driver string, handler func() Driver) Manager
Extend(driver string, handler func() Driver) (Manager, error)
Copy link
Member

@kkumar-gcc kkumar-gcc Sep 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are returning an error, we don't need to return the manager because chaining won't make sense to the user anyway. (Sorry for spamming here 😅;)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are returning an error, we don't need to return the manager because chaining won't make sense to the user anyway.

makes sense.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I think BuildSession can directly pass driver name instead of handler?

Copy link
Member

@kkumar-gcc kkumar-gcc Sep 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, we can pass the driver name directly but need to return an error as the second argument. Also, I believe we don't need the Driver method anymore if we pass driver name.

cc: @hwbrzzl

}
32 changes: 32 additions & 0 deletions crypt/aes_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,35 @@ func (s *AesTestSuite) TestDecryptString() {
_, err = s.aes.DecryptString("eyJpdiI6IjEyMzQ1IiwidmFsdWUiOiIxMjM0NSJ9")
s.Error(err)
}

func Benchmark_EncryptString(b *testing.B) {
mockConfig := &configmock.Config{}
mockConfig.On("GetString", "app.key").Return("11111111111111111111111111111111").Once()
aes := NewAES(mockConfig, json.NewJson())
b.ResetTimer()

for i := 0; i < b.N; i++ {
_, err := aes.EncryptString("Goravel")
if err != nil {
b.Error(err)
}
}
}

func Benchmark_DecryptString(b *testing.B) {
mockConfig := &configmock.Config{}
mockConfig.On("GetString", "app.key").Return("11111111111111111111111111111111").Once()
aes := NewAES(mockConfig, json.NewJson())
payload, err := aes.EncryptString("Goravel")
if err != nil {
b.Error(err)
}
b.ResetTimer()

for i := 0; i < b.N; i++ {
_, err := aes.DecryptString(payload)
if err != nil {
b.Error(err)
}
}
}
1 change: 1 addition & 0 deletions foundation/application_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -401,6 +401,7 @@ func (s *ApplicationTestSuite) TestMakeSchedule() {
func (s *ApplicationTestSuite) TestMakeSession() {
mockConfig := &configmocks.Config{}
mockConfig.On("GetInt", "session.lifetime").Return(120).Once()
mockConfig.On("GetInt", "session.gc_interval").Return(30).Once()
mockConfig.On("GetString", "session.files").Return("storage/framework/sessions").Once()

s.app.Singleton(frameworkconfig.Binding, func(app foundation.Application) (any, error) {
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/goravel/framework

go 1.21
go 1.22

require (
github.com/RichardKnop/machinery/v2 v2.0.13
Expand Down
84 changes: 72 additions & 12 deletions hash/application_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,25 +11,24 @@ import (

type ApplicationTestSuite struct {
suite.Suite
config *configmock.Config
hashers map[string]hash.Hash
}

func TestApplicationTestSuite(t *testing.T) {
mockConfig := &configmock.Config{}
argon2idHasher := getArgon2idHasher(mockConfig)
bcryptHasher := getBcryptHasher(mockConfig)

suite.Run(t, &ApplicationTestSuite{
hashers: map[string]hash.Hash{
"argon2id": argon2idHasher,
"bcrypt": bcryptHasher,
},
})
mockConfig.AssertExpectations(t)
suite.Run(t, &ApplicationTestSuite{})
}

func (s *ApplicationTestSuite) SetupTest() {
s.config = &configmock.Config{}
s.hashers = map[string]hash.Hash{
"argon2id": getArgon2idHasher(s.config),
"bcrypt": getBcryptHasher(s.config),
}
}

func (s *ApplicationTestSuite) TearDownSuite() {
s.config.AssertExpectations(s.T())
}

func (s *ApplicationTestSuite) TestMakeHash() {
Expand Down Expand Up @@ -78,6 +77,67 @@ func (s *ApplicationTestSuite) TestNeedsRehash() {
}
}

func BenchmarkMakeHash(b *testing.B) {
s := new(ApplicationTestSuite)
s.SetT(&testing.T{})
s.SetupTest()
b.StartTimer()
b.ResetTimer()
for name, hasher := range s.hashers {
b.Run(name, func(b *testing.B) {
for i := 0; i < b.N; i++ {
_, err := hasher.Make("password")
if err != nil {
b.Fatal(err)
}
}
})
}
b.StopTimer()
}

func BenchmarkCheckHash(b *testing.B) {
s := new(ApplicationTestSuite)
s.SetT(&testing.T{})
s.SetupTest()
b.StartTimer()
b.ResetTimer()
for name, hasher := range s.hashers {
b.Run(name, func(b *testing.B) {
value, err := hasher.Make("password")
if err != nil {
b.Fatal(err)
}
for i := 0; i < b.N; i++ {
if !hasher.Check("password", value) {
b.Fatal("hash check failed")
}
}
})
}
b.StopTimer()
}

func BenchmarkNeedsRehash(b *testing.B) {
s := new(ApplicationTestSuite)
s.SetT(&testing.T{})
s.SetupTest()
b.StartTimer()
b.ResetTimer()
for name, hasher := range s.hashers {
b.Run(name, func(b *testing.B) {
value, err := hasher.Make("password")
if err != nil {
b.Fatal(err)
}
for i := 0; i < b.N; i++ {
hasher.NeedsRehash(value)
}
})
}
b.StopTimer()
}

func getArgon2idHasher(mockConfig *configmock.Config) *Argon2id {
mockConfig.On("GetInt", "hashing.argon2id.memory", 65536).Return(65536).Once()
mockConfig.On("GetInt", "hashing.argon2id.time", 4).Return(4).Once()
Expand All @@ -87,7 +147,7 @@ func getArgon2idHasher(mockConfig *configmock.Config) *Argon2id {
}

func getBcryptHasher(mockConfig *configmock.Config) *Bcrypt {
mockConfig.On("GetInt", "hashing.bcrypt.rounds", 10).Return(10).Once()
mockConfig.On("GetInt", "hashing.bcrypt.rounds", 12).Return(10).Once()

return NewBcrypt(mockConfig)
}
2 changes: 1 addition & 1 deletion hash/bcrypt.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ type Bcrypt struct {
// NewBcrypt returns a new Bcrypt hasher.
func NewBcrypt(config config.Config) *Bcrypt {
return &Bcrypt{
rounds: config.GetInt("hashing.bcrypt.rounds", 10),
rounds: config.GetInt("hashing.bcrypt.rounds", 12),
}
}

Expand Down
82 changes: 82 additions & 0 deletions http/middleware/throttle_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -276,6 +276,88 @@ func TestThrottle(t *testing.T) {
}
}

func Benchmark_Throttle(b *testing.B) {
var (
ctx *TestContext
mockRateLimiterFacade *httpmocks.RateLimiter
mockCache *cachemocks.Cache
)

now := carbon.Now()
carbon.SetTestNow(now)
defer carbon.UnsetTestNow()

ctx = new(TestContext)
mockCache = &cachemocks.Cache{}
mockRateLimiterFacade = &httpmocks.RateLimiter{}
http.CacheFacade = mockCache
http.RateLimiterFacade = mockRateLimiterFacade

b.Run("WithOneLimiter", func(b *testing.B) {
mockRateLimiterFacade.On("Limiter", "test").Return(func(ctx contractshttp.Context) []contractshttp.Limit {
return []contractshttp.Limit{
limit.PerDay(10),
}
}).Times(b.N)
mockCache.On("Get", mock.MatchedBy(func(key string) bool {
return strings.HasPrefix(key, "throttle:test:0:")
})).Return(limit.NewBucket(1, time.Minute)).Times(b.N)

for i := 0; i < b.N; i++ {
Throttle("test")(ctx)
}

mockCache.AssertExpectations(b)
mockRateLimiterFacade.AssertExpectations(b)
})

b.Run("WithTwoLimiters", func(b *testing.B) {
mockRateLimiterFacade.On("Limiter", "test").Return(func(ctx contractshttp.Context) []contractshttp.Limit {
return []contractshttp.Limit{
limit.PerDay(10),
limit.PerMinute(5),
}
}).Times(b.N)
mockCache.On("Get", mock.MatchedBy(func(key string) bool {
return strings.HasPrefix(key, "throttle:test:0:")
})).Return(limit.NewBucket(10, 24*time.Hour))
mockCache.On("Get", mock.MatchedBy(func(key string) bool {
return strings.HasPrefix(key, "throttle:test:1:")
})).Return(limit.NewBucket(5, time.Minute))

for i := 0; i < b.N; i++ {
Throttle("test")(ctx)
}

mockRateLimiterFacade.AssertExpectations(b)
})

b.Run("WithThreeLimiters", func(b *testing.B) {
mockRateLimiterFacade.On("Limiter", "test").Return(func(ctx contractshttp.Context) []contractshttp.Limit {
return []contractshttp.Limit{
limit.PerMinute(5),
limit.PerHour(10),
limit.PerDay(100),
}
}).Times(b.N)
mockCache.On("Get", mock.MatchedBy(func(key string) bool {
return strings.HasPrefix(key, "throttle:test:0:")
})).Return(limit.NewBucket(10, 24*time.Hour))
mockCache.On("Get", mock.MatchedBy(func(key string) bool {
return strings.HasPrefix(key, "throttle:test:1:")
})).Return(limit.NewBucket(5, time.Minute))
mockCache.On("Get", mock.MatchedBy(func(key string) bool {
return strings.HasPrefix(key, "throttle:test:2:")
})).Return(limit.NewBucket(1, time.Second))

for i := 0; i < b.N; i++ {
Throttle("test")(ctx)
}

mockRateLimiterFacade.AssertExpectations(b)
})
}

type TestContext struct {
response contractshttp.ContextResponse
}
Expand Down
2 changes: 2 additions & 0 deletions log/logger/daily.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import (
"github.com/goravel/framework/contracts/foundation"
"github.com/goravel/framework/log/formatter"
"github.com/goravel/framework/support"
"github.com/goravel/framework/support/carbon"
)

type Daily struct {
Expand Down Expand Up @@ -43,6 +44,7 @@ func (daily *Daily) Handle(channel string) (logrus.Hook, error) {
logPath+"-%Y-%m-%d"+ext,
rotatelogs.WithRotationTime(time.Duration(24)*time.Hour),
rotatelogs.WithRotationCount(uint(daily.config.GetInt(channel+".days"))),
rotatelogs.WithClock(rotatelogs.NewClock(carbon.Now().StdTime())),
Comment on lines 46 to +47
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bug fix.

)
if err != nil {
return hook, errors.New("Config local file system for logger error: " + err.Error())
Expand Down
Loading
Loading