Squashed merge of main

commit e4b378f
Author: David Reiss <[email protected]>
Date:   Mon Aug 18 15:34:34 2025 -0700

    Support subscriptions to settings with constrained defaults (#8180)

    ## What changed?
    Fill in support for subscriptions to dynamic config values with
    constrained defaults.

    ## Why?
    We'd like to use this combination of functionality.

    ## How did you test it?
    - [x] added new unit test(s)

commit 8ed0361
Author: David Reiss <[email protected]>
Date:   Mon Aug 18 15:29:17 2025 -0700

    Allow empty data in DataBlob (#8181)

    ## What changed?
    Remove check for zero-length data in NewDataBlob.

    ## Why?
    Zero-length data is a valid encoding for some encodings, e.g. proto3.
    NewDataBlob should not have an opinion on the length of data.

    ## How did you test it?
    - [ ] built
    - [ ] run locally and tested manually
    - [ ] covered by existing tests
    - [ ] added new unit test(s)
    - [ ] added new functional test(s)

    ## Potential risks
    Some code may be making assumptions about this behavior.

commit 40ac028
Author: David Reiss <[email protected]>
Date:   Mon Aug 18 12:41:41 2025 -0700

    Warn on dynamic config default values with shared structure (#8176)

    ## What changed?
    Log softassert warnings if dynamic config settings are registered with
    default values with shared structure.

    ## Why?
    This is very likely unintended and may lead to unexpected behavior of
    settings (values will be parsed on top of a copy of the default).

    ## How did you test it?
    - [x] run locally and tested manually
    - [x] added new unit test(s)

commit 9c75cd6
Author: pdoerner <[email protected]>
Date:   Fri Aug 15 12:43:47 2025 -0700

    Forward Nexus requests using same dispatch type as original request (#8199)

    ## What changed?
    When forwarding Nexus requests that were originally sent to the
    `DispatchByEndpoint` URL, the forwarding URL will also be constructed to
    send the request to the `DispatchByEndpoint` URL on the remote cluster.
    Previously, we were always sending forwarding requests using
    `DispatchByNamespaceAndTaskQueue`

    ## Why?
    bug fix

    ## How did you test it?
    existing tests

commit 21f556c
Author: Roey Berman <[email protected]>
Date:   Fri Aug 15 13:13:28 2025 -0600

    Commit generated scheduler protos (#8200)

    ## What

    - Commit generated scheduler protos.
    - Improve `make ensure-no-changes` to detect untracked files.

    ## Why?

    The protos were not generated since the tool was committed in a separate
    PR from where the protos were added.

commit 4c59cd1
Author: Roey Berman <[email protected]>
Date:   Fri Aug 15 12:09:58 2025 -0600

    Add support for protos in chasm libs (#8182)

    ## What changed?

    Added support for defining protos in chasm libs.

    ## Why?

    Keep everything local to the library.

    ## How did you test it?
    - [x] built
    - [x] run locally and tested manually

commit 08e2dfd
Author: pdoerner <[email protected]>
Date:   Thu Aug 14 16:49:44 2025 -0700

    Reconstruct failure for forwarded Nexus completion requests (#8198)

    ## What changed?
    When forwarding a `CompleteNexusOperation` HTTP request that contains a
    failure, the completion will be reconstructed instead of reusing the
    original request body.

    ## Why?
    The Nexus SDK reads and closes the HTTP request body when the operation
    state is `failed` or `canceled` so we cannot reuse it for the forwarded
    request. For `successful` operations, the SDK just passes on the result
    content in the form of a `nexus.LazyValue` which we can forward directly
    since it is not read or closed.

    ## How did you test it?
    new functional xdc tests

commit 9d82cae
Author: Lina Jodoin <[email protected]>
Date:   Thu Aug 14 16:06:01 2025 -0700

    Fix BufferedStart reference in chasm scheduler proto (#8197)

    ## What changed?
    _Describe what has changed in this PR._

    ## Why?
    _Tell your future self why have you made these changes._

    ## How did you test it?
    - [ ] built
    - [ ] run locally and tested manually
    - [ ] covered by existing tests
    - [ ] added new unit test(s)
    - [ ] added new functional test(s)

    ## Potential risks
    _Any change is risky. Identify all risks you are aware of. If none,
    remove this section._

commit 41cce70
Author: Vladyslav Simonenko <[email protected]>
Date:   Wed Aug 13 15:34:38 2025 -0700

    Produce workflow_duration metric on completion (#8185)

    ## What changed?
    This PR produces the metric workflow_duration, when the workflow
    execution completes.

    ## Why?
    Currently there is no metric that captures the duration of the workflow
    execution. It's also valuable to have the duration broken down by task
    queue, namespace, workflow type, which this PR enables

    ## How did you test it?
    - [X] built
    - [X] run locally and tested manually
    - [X] covered by existing tests
    - [X] added new unit test(s)
    - [ ] added new functional test(s)

commit a1df862
Author: Lina Jodoin <[email protected]>
Date:   Wed Aug 13 15:30:48 2025 -0700

    [CHASM Scheduler] Move scheduler protobufs to scheduler/proto package (#8189)

    ## What changed?
    - CHASM scheduler protos are moved to live alongside the scheduler
    implementation code, within the `chasm` package.

    ## Why?
    - See #8182. Sending this PR in advance, as that PR asserts protobufs
    were generated.

    ## How did you test it?
    - [ ] built
    - [ ] run locally and tested manually
    - [x] covered by existing tests
    - [ ] added new unit test(s)
    - [ ] added new functional test(s)

commit 8fe5cee
Author: Sean Kane <[email protected]>
Date:   Thu Aug 14 00:10:06 2025 +0200

    improvement: remove waits before fetching activities (#8144)

    ## What changed?
    optimize the batch operation processing in `BatchActivity` and
    `BatchActivityWithProtobuf` by removing the need to wait for entire
    pages to complete before fetching the next page.

    - Implemented proactive page fetching once a worker becomes available
    - common `processWorkflowsWithProactiveFetching` function to reduce code
    duplication

    ## Why?
    The previous implementation had workers wait for entire pages to
    complete. This optimization improves resource utilization. The
    refactoring also eliminates duplicated functions in the `BatchParams`
    struct and `BatchOperation` protobuf.

    Addresses issue #8098.

    ## How did you test it?
    - [x] built
    - [x] run locally and tested manually
    - [x] covered by existing tests
    - [ ] added new unit test(s)
    - [ ] added new functional test(s)

    The changes maintain backward compatibility.

    ## Potential risks
    While this change improves performance, it does modify the concurrency
    model of batch processing:

    1. **Timing changes**: The optimization changes when pages are fetched
    relative to task completion, which could expose edge cases in error
    handling or heartbeat timing
    2. **Memory usage**: Pages may be fetched earlier, potentially
    increasing peak memory usage if the next page is large
    3. **Rate limiting interaction**: The more aggressive task scheduling
    could interact differently with rate limiting, though the same
    per-worker limits are maintained
    4. **Heartbeat behavior**: heartbeats track the progress of an entire
    page and are applied after an entire page finishes

    The changes preserve all existing error handling, retry logic, and rate
    limiting behavior, but the different execution timing could surface
    previously hidden race conditions.

    ---------

    Co-authored-by: Roey Berman <[email protected]>

commit 469526e
Author: pdoerner <[email protected]>
Date:   Wed Aug 13 09:58:03 2025 -0700

    Change default for `component.nexusoperations.recordCancelRequestCompletionEvents` (#8191)

    ## What changed?
    Changed default for
    `component.nexusoperations.recordCancelRequestCompletionEvents` to
    `true`

    ## Why?
    Flag was added to ensure backwards compatibility. Now that 1.28 is
    released, can change the default. Flag will be removed after 1.29 is
    released.

commit 69e6b6c
Author: Rodrigo Zhou <[email protected]>
Date:   Tue Aug 12 11:31:27 2025 -0700

    Bump Temporal API to v1.52.0 (#8187)

    ## What changed?
    Bump Temporal API to v1.52.0

    ## Why?
    Bump Temporal API to v1.52.0

    ## How did you test it?
    - [ ] built
    - [ ] run locally and tested manually
    - [ ] covered by existing tests
    - [ ] added new unit test(s)
    - [ ] added new functional test(s)

    ## Potential risks

commit da90f62
Author: Stephan Behnke <[email protected]>
Date:   Fri Aug 8 14:43:15 2025 -0700

    Decode of nil data (#8179)

    ## What changed?

    Don't catch `Data: nil` in test; let it fall through to decoder. The
    decoder will return an error. An error is the better choice than a `nil`
    response since that signals to the user that the decoded data is
    usable/valid.

    ## Why?

    Follow-up to #8111; an
    internal test expects an error instead of `nil`.

    ## How did you test it?
    - [ ] built
    - [ ] run locally and tested manually
    - [ ] covered by existing tests
    - [x] added new unit test(s)
    - [ ] added new functional test(s)

    ## Potential risks

    Hard to believe that returning nil and using un-decoded data is a
    good/valid alternative.

commit b8497fa
Author: David Reiss <[email protected]>
Date:   Fri Aug 8 08:32:58 2025 -0700

    Dynamic config conversion improvements (#7052)

    ## What changed?
    - Split implementation of "constrained default" settings from "plain
    default" settings. This is more code and the diff looks complex, but the
    individual paths are both simpler than the mixed version.
    - Add conversion cache using a weak map.
    - Remove GlobalCachedTypedValue.
    - Use "raw" values for subscription dispatch deduping to avoid
    unnecessary conversions.
    - Deep copy default values when using mapstructure, to avoid problems
    with merging over shared default values.

    ## Why?
    - Fixes #6756
    - Performance improvement for "plain default" settings (almost all of
    them)
    - Performance improvement for settings with complex converters
    - Remove footgun in defaults that aren't scalar values

    ## How did you test it?
    existing+new unit tests

commit f9bd083
Author: Lina Jodoin <[email protected]>
Date:   Thu Aug 7 17:11:53 2025 -0700

    [Scheduled Actions] Update Scheduler protos for CHASM (#8163)

    ## What changed?
    - Added protos for the new Scheduler task types.
    - Added TODOs for cleanup when the HSM component is removed.

    ## Why?
    - A few fields and messages were made obsolete with the CHASM port.

commit f8b97e5
Author: Vladyslav Simonenko <[email protected]>
Date:   Thu Aug 7 16:24:54 2025 -0700

    Break out of pagination in scavenger on errors (#8133)

    ## What changed?
    Break out of the loop, when iteration through mutable states fails

    ## Why?
    Previously, we continued to iterate, leading to the panic:
    #8037

    ## How did you test it?
    - [X] run locally and tested manually
    - [X] added new unit test(s)

Author: lina-temporal

.gitignore

@@ -24,8 +24,9 @@
 /tctl*
 /tdbg
 
-# proto image
+# proto images
 /proto/image.bin
+/proto/chasm.bin
 # api+google proto dependencies
 /proto/api.binpb
 
@@ -37,3 +38,5 @@
 
 # Git SPR: https://github.com/ejoffe/spr
 .spr.yml
+
+/proto.tmp

Makefile

@@ -100,11 +100,13 @@ endef
 
 PROTO_ROOT := proto
 PROTO_FILES = $(shell find ./$(PROTO_ROOT)/internal -name "*.proto")
+CHASM_PROTO_FILES = $(shell find ./chasm/lib -name "*.proto")
 PROTO_DIRS = $(sort $(dir $(PROTO_FILES)))
 API_BINPB := $(PROTO_ROOT)/api.binpb
 # Note: If you change the value of INTERNAL_BINPB, you'll have to add logic to
 # develop/buf-breaking.sh to handle the old and new values at once.
 INTERNAL_BINPB := $(PROTO_ROOT)/image.bin
+CHASM_BINPB := $(PROTO_ROOT)/chasm.bin
 PROTO_OUT := api
 
 ALL_SRC         := $(shell find . -name "*.go")
@@ -296,11 +298,20 @@ $(INTERNAL_BINPB): $(API_BINPB) $(PROTO_FILES)
 	@printf $(COLOR) "Generate proto image..."
 	@protoc --descriptor_set_in=$(API_BINPB) -I=$(PROTO_ROOT)/internal $(PROTO_FILES) -o $@
 
+$(CHASM_BINPB): $(API_BINPB) $(INTERNAL_BINPB) $(CHASM_PROTO_FILES)
+	@printf $(COLOR) "Generate CHASM proto image..."
+	@protoc --descriptor_set_in=$(API_BINPB):$(INTERNAL_BINPB) -I=. $(CHASM_PROTO_FILES) -o $@
+
 protoc: $(PROTOGEN) $(MOCKGEN) $(GOIMPORTS) $(PROTOC_GEN_GO) $(PROTOC_GEN_GO_GRPC) $(PROTOC_GEN_GO_HELPERS) $(API_BINPB)
-	@env \
-		PROTOGEN=$(PROTOGEN) MOCKGEN=$(MOCKGEN) GOIMPORTS=$(GOIMPORTS) \
-		API_BINPB=$(API_BINPB) PROTO_ROOT=$(PROTO_ROOT) PROTO_OUT=$(PROTO_OUT) \
-		./develop/protoc.sh
+	@go run ./cmd/tools/protogen \
+		-root=$(ROOT) \
+		-proto-out=$(PROTO_OUT) \
+		-proto-root=$(PROTO_ROOT) \
+		-protogen=$(PROTOGEN) \
+		-api-binpb=$(API_BINPB) \
+		-goimports=$(GOIMPORTS) \
+		-mockgen=$(MOCKGEN) \
+		$(PROTO_DIRS)
 
 proto-codegen:
 	@printf $(COLOR) "Generate service clients..."
@@ -368,14 +379,16 @@ lint-api: $(API_LINTER) $(API_BINPB)
 	@printf $(COLOR) "Linting proto API..."
 	$(call silent_exec, $(API_LINTER) --set-exit-status -I=$(PROTO_ROOT)/internal --descriptor-set-in $(API_BINPB) --config=$(PROTO_ROOT)/api-linter.yaml $(PROTO_FILES))
 
-lint-protos: $(BUF) $(INTERNAL_BINPB)
+lint-protos: $(BUF) $(INTERNAL_BINPB) $(CHASM_BINPB)
 	@printf $(COLOR) "Linting proto definitions..."
 	@$(BUF) lint $(INTERNAL_BINPB)
+	@$(BUF) lint --config chasm/lib/buf.yaml $(CHASM_BINPB)
 
 # Edit proto/internal/buf.yaml to exclude specific files from this check.
+# TODO: buf breaking check for CHASM protos.
 buf-breaking: $(BUF) $(API_BINPB) $(INTERNAL_BINPB)
 	@printf $(COLOR) "Run buf breaking proto changes check..."
-	@env BUF=$(BUF) API_BINPB=$(API_BINPB) INTERNAL_BINPB=$(INTERNAL_BINPB) MAIN_BRANCH=$(MAIN_BRANCH) \
+	@env BUF=$(BUF) API_BINPB=$(API_BINPB) INTERNAL_BINPB=$(INTERNAL_BINPB) CHASM_BINPB=$(CHASM_BINPB) MAIN_BRANCH=$(MAIN_BRANCH) \
 		./develop/buf-breaking.sh
 
 shell-check:
@@ -637,4 +650,5 @@ go-generate: $(MOCKGEN) $(GOIMPORTS) $(STRINGER) $(GOWRAP)
 ensure-no-changes:
 	@printf $(COLOR) "Check for local changes..."
 	@printf $(COLOR) "========================================================================"
-	@git diff --name-status --exit-code || (printf $(COLOR) "========================================================================"; printf $(RED) "Above files are not regenerated properly. Regenerate them and try again."; exit 1)
+	@git status --porcelain
+	@test -z "`git status --porcelain`" || (printf $(COLOR) "========================================================================"; printf $(RED) "Above files are not regenerated properly. Regenerate them and try again."; exit 1)