fatal error: concurrent map writes #563
Description
Version information
- terraform: v1.6.4
- terraform provider: v1.12.0
Describe the bug
Custom fields are cached (https://github.com/labd/terraform-provider-commercetools/blob/85d81c7642951a6a06e87c46fe6860b901819cf0/commercetools/custom_fields.go#L15C5-L15C15) when resolved, and are not using a mutex for writes, which can result in concurrent writes and panics.
I realize we're not using the latest versions of terraform and provider, but the bug is still present in the latest version.
Simply add a mutex and lock/unlock around map usage, and the problem should go away.
To Reproduce
Run multiple custom field type lookups at the same time, and hit the race condition.
I've run the provider for 2+ years without hitting this issue more than a couple of times, so it's a "rare" bug.
Expected behavior
The plugin shouldn't crash, when running multiple lookups at the same time.
Screenshots
I've provided the output below
Additional context
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
module.commercetools_production.commercetools_store.X: Creating...
module.commercetools_production.commercetools_store.Y: Creating...
module.commercetools_production.commercetools_store.Z: Creating...
╷
│ Error: Plugin did not respond
│
│ with module.commercetools_production.commercetools_store.X,
│ on ../../modules/commercetools/stores.tf line 979, in resource "commercetools_store" "X":
│ 979: resource "commercetools_store" "X" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│ with module.commercetools_production.commercetools_store.Y,
│ on ../../modules/commercetools/stores.tf line 1009, in resource "commercetools_store" "Y":
│ 1009: resource "commercetools_store" "Y" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│ with module.commercetools_production.commercetools_store.Z,
│ on ../../modules/commercetools/stores.tf line 1039, in resource "commercetools_store" "Z":
│ 1039: resource "commercetools_store" "Z" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may contain more details.
╵
Stack trace from the terraform-provider-commercetools_v1.12.0 plugin:
fatal error: concurrent map writes
goroutine 43 [running]:
github.com/labd/terraform-provider-commercetools/commercetools.getTypeResource({0x101ba9708, 0x14000807980}, 0x140001247f8, 0x140008086e0?)
github.com/labd/terraform-provider-commercetools/commercetools/custom_fields.go:256 +0x2b0
github.com/labd/terraform-provider-commercetools/commercetools.CreateCustomFieldDraft({0x101ba9708, 0x14000807980}, 0x14000404a45?, 0x5?)
github.com/labd/terraform-provider-commercetools/commercetools/custom_fields.go:43 +0x58
github.com/labd/terraform-provider-commercetools/commercetools.resourceStoreCreate({0x101ba9708?, 0x14000807980}, 0x140005d3900, {0x101b8c780?, 0x140001247f8})
github.com/labd/terraform-provider-commercetools/commercetools/resource_store.go:102 +0x4ac
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).create(0x14000323180, {0x101ba9740, 0x140006a1ad0}, 0xd?, {0x101b8c780, 0x140001247f8})
github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/resource.go:707 +0xe8
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).Apply(0x14000323180, {0x101ba9740, 0x140006a1ad0}, 0x14000136820, 0x140005d3700, {0x101b8c780, 0x140001247f8})
github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/resource.go:837 +0x888
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ApplyResourceChange(0x1400034c600, {0x101ba9740?, 0x140006a19b0?}, 0x1400070c190)
github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/grpc_provider.go:1021 +0xb78
github.com/hashicorp/terraform-plugin-mux/tf5muxserver.muxServer.ApplyResourceChange({0x140003365a0, 0x14000336600, {0x140002fde80, 0x2, 0x2}, {0x0, 0x0, 0x0}, {0x0, 0x0, ...}, ...}, ...)
github.com/hashicorp/terraform-plugin-mux@v0.10.0/tf5muxserver/mux_server_ApplyResourceChange.go:27 +0xdc
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ApplyResourceChange(0x1400033ebe0, {0x101ba9740?, 0x140006a0e70?}, 0x140002e2700)
github.com/hashicorp/terraform-plugin-go@v0.16.0/tfprotov5/tf5server/server.go:821 +0x3bc
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ApplyResourceChange_Handler({0x101b35d20?, 0x1400033ebe0}, {0x101ba9740, 0x140006a0e70}, 0x140002e2690, 0x0)
github.com/hashicorp/terraform-plugin-go@v0.16.0/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:422 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0x140001c21e0, {0x101bae838, 0x140001024e0}, 0x1400034ad80, 0x140003fecc0, 0x10235d4e0, 0x0)
google.golang.org/grpc@v1.56.0/server.go:1337 +0xc90
google.golang.org/grpc.(*Server).handleStream(0x140001c21e0, {0x101bae838, 0x140001024e0}, 0x1400034ad80, 0x0)
google.golang.org/grpc@v1.56.0/server.go:1714 +0x82c
google.golang.org/grpc.(*Server).serveStreams.func1.1()
google.golang.org/grpc@v1.56.0/server.go:959 +0x84
created by google.golang.org/grpc.(*Server).serveStreams.func1
google.golang.org/grpc@v1.56.0/server.go:957 +0x16c
goroutine 1 [select]:
github.com/hashicorp/go-plugin.Serve(0x14000374060)
github.com/hashicorp/go-plugin@v1.4.10/server.go:474 +0x1120
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.Serve({0x10167a981, 0x28}, 0x140001c1a40, {0x0, 0x0, 0x0})
github.com/hashicorp/terraform-plugin-go@v0.16.0/tfprotov5/tf5server/server.go:315 +0xa50
main.main()
github.com/labd/terraform-provider-commercetools/main.go:59 +0x380
goroutine 31 [select]:
github.com/hashicorp/go-plugin.(*gRPCBrokerServer).Recv(0x1400023e390?)
github.com/hashicorp/go-plugin@v1.4.10/grpc_broker.go:124 +0x58
github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0x14000366e10)
github.com/hashicorp/go-plugin@v1.4.10/grpc_broker.go:414 +0x40
created by github.com/hashicorp/go-plugin.(*GRPCServer).Init
github.com/hashicorp/go-plugin@v1.4.10/grpc_server.go:88 +0x420
goroutine 32 [IO wait]:
internal/poll.runtime_pollWait(0x129c80a50, 0x72)
runtime/netpoll.go:306 +0xa0
internal/poll.(*pollDesc).wait(0x140003744e0?, 0x1400048c000?, 0x1)
internal/poll/fd_poll_runtime.go:84 +0x28
internal/poll.(*pollDesc).waitRead(...)
internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x140003744e0, {0x1400048c000, 0x1000, 0x1000})
internal/poll/fd_unix.go:167 +0x200
os.(*File).read(...)
os/file_posix.go:31
os.(*File).Read(0x1400000efa0, {0x1400048c000?, 0x400?, 0x1018b1da0?})
os/file.go:118 +0x5c
bufio.(*Reader).Read(0x1400008ef30, {0x1400048e000, 0x400, 0x0?})
bufio/bufio.go:237 +0x1e0
github.com/hashicorp/go-plugin.copyChan({0x101bb2b38, 0x1400033eaa0}, 0x0?, {0x101ba4a00?, 0x1400000efa0?})
github.com/hashicorp/go-plugin@v1.4.10/grpc_stdio.go:184 +0x164
created by github.com/hashicorp/go-plugin.newGRPCStdioServer
github.com/hashicorp/go-plugin@v1.4.10/grpc_stdio.go:40 +0x10c
goroutine 33 [IO wait]:
internal/poll.runtime_pollWait(0x129c80870, 0x72)
runtime/netpoll.go:306 +0xa0
internal/poll.(*pollDesc).wait(0x140003745a0?, 0x1400010e000?, 0x1)
internal/poll/fd_poll_runtime.go:84 +0x28
internal/poll.(*pollDesc).waitRead(...)
internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x140003745a0, {0x1400010e000, 0x1000, 0x1000})
internal/poll/fd_unix.go:167 +0x200
os.(*File).read(...)
os/file_posix.go:31
os.(*File).Read(0x1400000efb0, {0x1400010e000?, 0x400?, 0x1018b1da0?})
os/file.go:118 +0x5c
bufio.(*Reader).Read(0x1400008f730, {0x14000110000, 0x400, 0x0?})
bufio/bufio.go:237 +0x1e0
github.com/hashicorp/go-plugin.copyChan({0x101bb2b38, 0x1400033eaa0}, 0x0?, {0x101ba4a00?, 0x1400000efb0?})
github.com/hashicorp/go-plugin@v1.4.10/grpc_stdio.go:184 +0x164
created by github.com/hashicorp/go-plugin.newGRPCStdioServer
github.com/hashicorp/go-plugin@v1.4.10/grpc_stdio.go:41 +0x198
goroutine 35 [syscall]:
os/signal.signal_recv()
runtime/sigqueue.go:149 +0x2c
os/signal.loop()
os/signal/signal_unix.go:23 +0x1c
created by os/signal.Notify.func1.1
os/signal/signal.go:151 +0x2c
goroutine 36 [chan receive]:
github.com/hashicorp/go-plugin.Serve.func3()
github.com/hashicorp/go-plugin@v1.4.10/server.go:439 +0x70
created by github.com/hashicorp/go-plugin.Serve
github.com/hashicorp/go-plugin@v1.4.10/server.go:436 +0xf18
goroutine 37 [IO wait]:
Error: The terraform-provider-commercetools_v1.12.0 plugin crashed!
This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.