all: potential infinite recursions in method recursion/retry due to assumption that SessionPool are always working after ._getSession return 'No Session Found'

[odeke-em]

Throughout this package and in my journey of tracing this package, I've encountered the pattern of this form

nodejs-spanner/src/database.ts

Lines 3608 to 3628 in 2a19ef1

	writeAtLeastOnce(
	mutations: MutationSet,
	optionsOrCallback?: CallOptions | CommitCallback,
	callback?: CommitCallback
	): void | Promise<CommitResponse> {
	const cb =
	typeof optionsOrCallback === 'function'
	? (optionsOrCallback as CommitCallback)
	: callback;
	const options =
	typeof optionsOrCallback === 'object' && optionsOrCallback
	? (optionsOrCallback as CallOptions)
	: {};
	return startTrace('Database.writeAtLeastOnce', this._traceConfig, span => {
	this.pool_.getSession((err, session?, transaction?) => {
	if (err && isSessionNotFoundError(err as grpc.ServiceError)) {
	span.addEvent('No session available', {
	'session.id': session?.id,
	});
	this.writeAtLeastOnce(mutations, options, cb!);

writeAtLeastOnce( 
   mutations: MutationSet, 
   optionsOrCallback?: CallOptions | CommitCallback, 
   callback?: CommitCallback 
 ): void | Promise<CommitResponse> { 
...
     this.pool_.getSession((err, session?, transaction?) => { 
       if (err && isSessionNotFoundError(err as grpc.ServiceError)) { 
         span.addEvent('No session available', { 
           'session.id': session?.id, 
         }); 
         this.writeAtLeastOnce(mutations, options, cb!); 

but notice that if pool._getSession returns Session Not Found, then that method will call itself an indefinite number of times and that's non-determinism. We've got many such cases.

nodejs-spanner/src/database.ts

Lines 2097 to 2103 in 2a19ef1

	if (isSessionNotFoundError(err)) {
	span.addEvent('No session available', {
	'session.id': session?.id,
	});
	session!.lastError = err;
	this.pool_.release(session!);
	this.getSnapshot(options, (err, snapshot) => {

[alkatrivedi]

@odeke-em if there will be Session Not Found Error, we will retry the transaction after getting another session from the session pool. In every try we will be getting 'Session Not Found Error' that's very rare.

[alkatrivedi]

Could you please help me understand in which case it could lead to an issue?

[odeke-em]

If for example the pool is faulty, or the Cloud Spanner database is under load or even a billing issue on the account.

[surbhigarg92]

@odeke-em IIUC, in all above cases, sessions will not be created at the time of session pool creation.

In this case there will not be any Session not found error instead , the request will keep waiting for the session to be available and ultimately the request will get timed out. Refer
You could also set the timeout for acquiring sessions by setting SessionPoolOptions:acquireTimeout default is infinity. Refer https://cloud.google.com/blog/topics/developers-practitioners/deploying-cloud-spanner-based-nodejs-application

Please let us know if you are not seeing the above behavior and instead seeing infinite recursion for your use case ?