emit ENDBR/BTI instructions for indirect jump targets

[kernel-patches-daemon-bpf]

Pull request for series with
subject: emit ENDBR/BTI instructions for indirect jump targets
version: 9
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=1065775

[kernel-patches-daemon-bpf]

Upstream branch: ca0f39a
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1065775
version: 9

[kernel-patches-review-bot]

commit 6890fec9f81c73e743a5fa032f250373257edce2
Author: Xu Kuohai <[email protected]>

bpf: Move constants blinding out of arch-specific JITs

This commit centralizes BPF constant blinding, moving it from each
arch-specific JIT into two common paths: bpf_prog_jit_compile() for
single-subprog programs, and jit_subprogs() for multi-subprog programs.

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 4fbacd2149cd..3ccefd13121b 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c

[ ... ]

> @@ -22843,17 +22843,23 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env)
>
>  static int jit_subprogs(struct bpf_verifier_env *env)
>  {
> -	struct bpf_prog *prog = env->prog, **func, *tmp;
> +	struct bpf_prog *orig_prog = env->prog, *prog, **func, *tmp;
>  	int i, j, subprog_start, subprog_end = 0, len, subprog;
>  	struct bpf_map *map_ptr;
>  	struct bpf_insn *insn;
>  	void *old_bpf_func;
>  	int err, num_exentries;
> -	int old_len, subprog_start_adjustment = 0;
> +	bool blinded = false;
>
>  	if (env->subprog_cnt <= 1)
>  		return 0;
>
> +	prog = bpf_jit_blind_constants(env, NULL);
> +	if (IS_ERR(prog))
> +		return -ENOMEM;
> +	if (prog != orig_prog)
> +		blinded = true;

When env is non-NULL, bpf_jit_blind_constants() uses
bpf_patch_insn_data() to insert blinding instructions.
bpf_patch_insn_data() calls adjust_insn_aux_data(), which expands
and shifts entries in env->insn_aux_data to match the now-longer
blinded program.

> +
>  	for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) {

[ ... ]

> @@ -23114,6 +23122,10 @@ static int jit_subprogs(struct bpf_verifier_env *env)
>  	}
>  	kfree(func);
>  out_undo_insn:
> +	if (blinded) {
> +		bpf_jit_prog_release_other(orig_prog, prog);
> +		env->prog = prog = orig_prog;
> +	}
>  	/* cleanup main prog to be interpreted */
>  	prog->jit_requested = 0;
>  	prog->blinding_requested = 0;
> +	for (i = 0, insn = prog->insnsi; i < prog->len; i++, insn++) {
>  		if (!bpf_pseudo_call(insn))
>  			continue;
>  		insn->off = 0;
>  		insn->imm = env->insn_aux_data[i].call_imm;
>  	}

After bpf_jit_prog_release_other() reverts to orig_prog, this loop
iterates using the original program's indices and length. However,
env->insn_aux_data was expanded and shifted by bpf_patch_insn_data()
during blinding to match the blinded program's instruction count.

The call_imm values were saved at blinded indices (earlier in
jit_subprogs, after the bpf_jit_blind_constants call), so reading
them back at original indices gets the wrong aux data entries.

On !CONFIG_BPF_JIT_ALWAYS_ON configs, could this cause incorrect
imm values to be restored into pseudo_call instructions, breaking
interpreter fallback?

---

AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

In-Reply-To-Subject: bpf: Move constants blinding out of arch-specific JITs
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/23013190773

[kernel-patches-daemon-bpf]

Forwarding comment 4048736540 via email
In-Reply-To: [email protected]
Patch: https://patchwork.kernel.org/project/netdevbpf/patch/[email protected]/

[kernel-patches-daemon-bpf]

Upstream branch: ca0f39a
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1065775
version: 9

[kernel-patches-daemon-bpf]

Upstream branch: ca0f39a
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1065775
version: 9

[kernel-patches-daemon-bpf]

Upstream branch: bb41fce
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1065775
version: 9