-
Notifications
You must be signed in to change notification settings - Fork 215
Description
Hello brod maintainers! I noticed a slight inconsistency/oversight in how the begin offset is resolved for consumer groups.
The docs and type spec would lead me to believe that the begin_offset
in the consumer config should be treated as a :brod.offset_time()
. The atom values (:earliest
and :latest
) seem to be handled correctly. However, if you try to use a :brod.msg_ts()
type value, it is handled as a resolved offset, not a timestamp.
Obviously, not using :earliest
or :latest
is a niche use case but, in that case, handling it as a timestamp seems to be the desired behavior.
This is fairly easy to recreate, using the following child_spec. Everything behaves as expected in this case but, if you attempt to use a timestamp for the begin_offset
field of the consumer config, you'll notice it's treated as an offset and NOT a timestamp:
def child_spec(_arg) do
config = %{
client: :kafka_client,
group_id: "consumer_group",
topics: ["test_topic_1"],
cb_module: __MODULE__,
consumer_config: [begin_offset: :earliest],
init_data: [foo: "bar"],
message_type: :message_set,
group_config: [
offset_commit_policy: :consumer_managed,
offset_commit_interval_seconds: 5,
rejoin_delay_seconds: 60,
reconnect_cool_down_seconds: 60
]
}
%{
id: __MODULE__,
start: {:brod_group_subscriber_v2, :start_link, [config]},
type: :worker,
restart: :temporary,
shutdown: 5000
}
end
Not a super big deal but it would be nice to have this functionality (or at least update the spec to reduce future confusion)