About the performance penalty of `KOKKOSKERNELS_DEBUG_LEVEL`

[tpadioleau]

Hi, I just discovered the macro definition KOKKOSKERNELS_DEBUG_LEVEL, which seems to be always defined at 1

kokkos-kernels/common/src/KokkosKernels_Macros.hpp

Lines 15 to 17 in 0212b3f

	#ifndef KOKKOSKERNELS_DEBUG_LEVEL
	#define KOKKOSKERNELS_DEBUG_LEVEL 1
	#endif

I see it is used as a verbosity level adding some Kokkos::printf in device functions like

kokkos-kernels/batched/dense/impl/KokkosBatched_Gbtrf_Serial_Impl.hpp

Lines 21 to 68 in 0212b3f

	#if (KOKKOSKERNELS_DEBUG_LEVEL > 0)
	const int n = AB.extent(1);
	const int npiv = ipiv.extent(0);
	if (npiv != Kokkos::min(m, n)) {
	Kokkos::printf(
	"KokkosBatched::gbtrf: the dimension of the ipiv array must "
	"satisfy ipiv.extent(0) == max(m, n): ipiv: %d, A: "
	"%d "
	"x %d \n",
	npiv, m, n);
	return 1;
	}
	if (m < 0) {
	Kokkos::printf(
	"KokkosBatched::gbtrf: input parameter m must not be less than 0: m "
	"= "
	"%d\n",
	m);
	return 1;
	}
	if (kl < 0) {
	Kokkos::printf(
	"KokkosBatched::gbtrf: input parameter kl must not be less than 0: kl "
	"= "
	"%d\n",
	kl);
	return 1;
	}
	if (ku < 0) {
	Kokkos::printf(
	"KokkosBatched::gbtrf: input parameter ku must not be less than 0: ku "
	"= "
	"%d\n",
	ku);
	return 1;
	}
	const int lda = AB.extent(0);
	if (lda < (2 * kl + ku + 1)) {
	Kokkos::printf(
	"KokkosBatched::gbtrs: leading dimension of A must be smaller than 2 * "
	"kl + ku + 1: "
	"lda = %d, kl = %d, ku = %d\n",
	lda, kl, ku);
	return 1;
	}

I was wondering if it can have some performance penalty ? If so should there be a cmake option to disable it ?

[lucbv]

Yeah that looks like an oversight, we should definitely have that be conditioned on the user wanting debug printouts and checks.