Wrong payload being signed

[gagdiez]

The current implementation of SignTransaction signs the transaction.encode(), which is the borsh serialized version of the transaction:

near-api-js/packages/client/src/transactions/sign_and_send.ts

Lines 18 to 25 in 9cb7e89

	const encodedTx = transaction.encode();
	const signedTransaction = new SignedTransaction({
	transaction,
	signature: new Signature({
	keyType: transaction.publicKey.keyType,
	data: await signer.signMessage(encodedTx),
	}),
	});

In reality, we need to sign the sha256 hash of the borsh serialized transaction:

const encodedTx = new Uint8Array(sha256.sha256.array(transaction.encode()));

[AlexKushnir1]

This means that signing is not working correctly. I don't fully understand how it worked beforehand and signed transactions were passed successfully and didn`t throw any errors?

[gagdiez]

@AlexKushnir1 @near-js/client is a relatively new package, which still has no proper testing

My guess is that nobody used it and detected it, or simply nobody reported the error

[AlexKushnir1]

@gagdiez I opine it has sense to clarify that we expect hash of encodedTx in the signMessage. I see it like rewrite encodedTx into encodedTransactionHash and describe that signMessage() https://github.com/near/near-api-js/blob/c49fd67535baa5163b29ca7a43f585bfcfbe16e6/packages/client/src/interfaces/dependencies.ts#L10C1-L14C1 expects a hash of encoded transaction. WDYT?

[gagdiez]

I see that actually signMessage already applies the sha256, so maybe we need to clarify that on the signMessage doc String, and add a test to try all this actually works

[AlexKushnir1]

I just thought, if we specify on the signMessage that it should get an encoded hash it will cause a misleading naming.

I tested this func and related and there came to a conclusion that is"nt working, after adding hashing it was signed correctly and passed on the chain. Regarding the tests I would describe an issue and work for all client package

[AlexKushnir1]

I"m not sure I understand where is singMessage applies the sha256. I would be really grateful for clarifying it to me!

[gagdiez]

The inMemorySigner - I think this is the only one being implemented - hashes the message before signing it:

const hash = new Uint8Array(sha256(message)); 

near-api-js/packages/signers/src/in_memory_signer.ts

Lines 65 to 80 in c49fd67

	async signMessage(message: Uint8Array, accountId?: string, networkId?: string): Promise<Signature> {
	const hash = new Uint8Array(sha256(message));
	if (!accountId) {
	throw new Error('InMemorySigner requires provided account id');
	}
	const keyPair = await this.keyStore.getKey(networkId, accountId);
	if (keyPair === null) {
	throw new Error(`Key for ${accountId} not found in ${networkId}`);
	}
	return keyPair.sign(hash);
	}
	toString(): string {
	return `InMemorySigner(${this.keyStore})`;
	}
	}

[AlexKushnir1]

Sure, I didn`t test with the signer from keystore. THANKS!

Does the KeyPair signer still have an issue?

[gagdiez]

the KeyPair.sign does not apply the sha256 hash, and that is ok since the Key should not decide on how something should be signed

[AlexKushnir1]

Just tested like this:

async function testSigners() {
  const rpcProvider = getTestnetRpcProvider();
  const signer = getSignerFromPrivateKey(PRIVATE_KEY); 

  const {result, outcome} = await SignedTransactionComposer.init({
    sender: ACCOUNT_ID,
    receiver: 'alexkushnir.testnet',
    deps: { rpcProvider, signer },
  })
  .functionCall(
    "sign",
    Buffer.from(JSON.stringify({ request : {key_version:0,path:"test",payload:[12,1,2,0,4,5,6,8,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,44]}})),
    BigInt(300000000000000),
    BigInt(1)
  )
  .signAndSend();

  console.log(`${JSON.stringify(result, null)}`);
  console.log(`${JSON.stringify(outcome, null)}`);
}

And got the next errors

ServerError: Transaction is not signed with the given public key
    at parseRpcError (/home/alex/Near/JS/test-api-js/node_modules/.pnpm/@near-js+utils@1.0.1/node_modules/@near-js/utils/lib/commonjs/errors/rpc_errors.cjs:25:19)
    at /home/alex/Near/JS/test-api-js/node_modules/.pnpm/@near-js+providers@1.0.1/node_modules/@near-js/providers/lib/commonjs/json-rpc-provider.cjs:321:57
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async exponentialBackoff (/home/alex/Near/JS/test-api-js/node_modules/.pnpm/@near-js+providers@1.0.1/node_modules/@near-js/providers/lib/commonjs/exponential-backoff.cjs:8:24)
    at async JsonRpcProvider.sendJsonRpc (/home/alex/Near/JS/test-api-js/node_modules/.pnpm/@near-js+providers@1.0.1/node_modules/@near-js/providers/lib/commonjs/json-rpc-provider.cjs:306:26)
    at async FailoverRpcProvider.withBackoff (/home/alex/Near/JS/test-api-js/node_modules/.pnpm/@near-js+providers@1.0.1/node_modules/@near-js/providers/lib/commonjs/failover-rpc-provider.cjs:54:32)
    at async SignedTransactionComposer.signAndSend (/home/alex/Near/JS/test-api-js/node_modules/.pnpm/@near-js+client@0.0.2/node_modules/@near-js/client/lib/commonjs/transactions/composers/signed_transaction_composer.cjs:96:25)
    at async testSubAccountCreation (/home/alex/Near/JS/test-api-js/src/index.js:47:33) {
  type: 'InvalidSignature',
  context: undefined
}
Switched to provider at the index 1
ServerError: Transaction is not signed with the given public key
    at parseRpcError (/home/alex/Near/JS/test-api-js/node_modules/.pnpm/@near-js+utils@1.0.1/node_modules/@near-js/utils/lib/commonjs/errors/rpc_errors.cjs:25:19)
    at /home/alex/Near/JS/test-api-js/node_modules/.pnpm/@near-js+providers@1.0.1/node_modules/@near-js/providers/lib/commonjs/json-rpc-provider.cjs:321:57
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async exponentialBackoff (/home/alex/Near/JS/test-api-js/node_modules/.pnpm/@near-js+providers@1.0.1/node_modules/@near-js/providers/lib/commonjs/exponential-backoff.cjs:8:24)
    at async JsonRpcProvider.sendJsonRpc (/home/alex/Near/JS/test-api-js/node_modules/.pnpm/@near-js+providers@1.0.1/node_modules/@near-js/providers/lib/commonjs/json-rpc-provider.cjs:306:26)
    at async FailoverRpcProvider.withBackoff (/home/alex/Near/JS/test-api-js/node_modules/.pnpm/@near-js+providers@1.0.1/node_modules/@near-js/providers/lib/commonjs/failover-rpc-provider.cjs:54:32)
    at async SignedTransactionComposer.signAndSend (/home/alex/Near/JS/test-api-js/node_modules/.pnpm/@near-js+client@0.0.2/node_modules/@near-js/client/lib/commonjs/transactions/composers/signed_transaction_composer.cjs:96:25)
    at async testSubAccountCreation (/home/alex/Near/JS/test-api-js/src/index.js:47:33) {
  type: 'InvalidSignature',
  context: undefined
}
Switched to provider at the index 0
/home/alex/Near/JS/test-api-js/node_modules/.pnpm/@near-js+providers@1.0.1/node_modules/@near-js/providers/lib/commonjs/failover-rpc-provider.cjs:63
        throw new types_1.TypedError(`Exceeded ${this.providers.length} providers to execute request`, 'RetriesExceeded');
              ^

TypedError: Exceeded 2 providers to execute request
    at FailoverRpcProvider.withBackoff (/home/alex/Near/JS/test-api-js/node_modules/.pnpm/@near-js+providers@1.0.1/node_modules/@near-js/providers/lib/commonjs/failover-rpc-provider.cjs:63:15)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async SignedTransactionComposer.signAndSend (/home/alex/Near/JS/test-api-js/node_modules/.pnpm/@near-js+client@0.0.2/node_modules/@near-js/client/lib/commonjs/transactions/composers/signed_transaction_composer.cjs:96:25)
    at async testSubAccountCreation (/home/alex/Near/JS/test-api-js/src/index.js:47:33) {
  type: 'RetriesExceeded',
  context: undefined
}

But when I pass hash function and compile js file:

async function signTransaction({ transaction, deps: { signer } }) {
    const encodedTx = new Uint8Array((0, sha256_1.sha256)(transaction.encode()));
    const signedTransaction = new transactions_1.SignedTransaction({
        transaction,
        signature: new transactions_1.Signature({
            keyType: transaction.publicKey.keyType,
            data: await signer.signMessage(encodedTx),
        }),
    });
    return {
        encodedTransactionHash: encodedTx,
        signedTransaction,
    };
}

Transaction is passing corectly

[gagdiez]

I see, then we do have a problem, because one signer implements it and another does not...

Either we make all signers hash the message, or we make all signers not hash the message, but having some doing it and some not is a huge problem

I think it is more transparent to make the signers not hash the message

Invoking @frol to see if he has an opinion here, since he built near-api-rs we could follow their take on this