Signed requests #45979

ArtificialOwl · 2024-06-19T13:49:37Z

Signed Request

Signing request allows to confirm the identity of the sender.
Signing request does not encrypt nor affect its payload.
Signing request only adds metadata to the headers of the request.

Signature

The concept is to add unique metadata and sign them using a private/public key pair.
The location of the public key used to verify the signature will confirm the origin of the request.

Signature does not affect the data of the request, it only adds headers to it:

 {
     "(request-target)": "post /path",
     "content-length": 380,
     "date": "Mon, 08 Jul 2024 14:16:20 GMT",
     "digest": "SHA-256=U7gNVUQiixe5BRbp4Tg0xCZMTcSWXXUZI2\\/xtHM40S0=",
     "host": "hostname.of.the.recipient",
     "Signature": "keyId=\"https://author.hostname/key\",algorithm=\"ras-sha256\",headers=\"content-length date digest host\",signature=\"DzN12OCS1rsA[...]o0VmxjQooRo6HHabg==\""
 }

'content-length' is the total length of the data/content
'date' is the datetime the request have been initiated
'digest' is a checksum of the data/content
'host' is the hostname of the recipient of the request (remote when signing outgoing request, local on incoming request)
'Signature' contains the signature generated using the private key, and metadata:
- 'keyId' is a unique id, formatted as an url. hostname is used to retrieve the public key via custom discovery
- 'algorithm' define the algorithm used to generate signature
- 'headers' contains a list of element used during the generation of the signature
- 'signature' is the encrypted string, using local private key, of an array containing elements
  listed in 'headers' and their value. Some elements (content-length date digest host) are mandatory
  to ensure authenticity override protection.

(Those are the minimum required headers, some can be added via options during the process)

ISignatoryManager

Because each protocol have different ways to obtain the public key of a remote instance or entity, some part of the signing/verifying process is managed by a custom provider, one for each protocol.

getProviderId should returns a unique string
getOptions should returns an array that can contains those entries:
- 'ttl' (300) is the lifetime (in secondes) of the signature
- 'ttlSignatory' (86400*3) the cache lifetime on a remote signatory
- 'extraSignatureHeaders' ([]) (list of extra headers to include to the signature)
- 'algorithm' ('sha256') is the algorithm used to generate the signature
- 'dateHeader' ("D, d M Y H:i:s T") the format of the 'date' header
getLocalSignatory should return the local signatory, including the full (public+private) key pair.
getRemoteSignatory should returns a remote signatory based on the requested data, must at least contains key id and public key

IKeyPairManager

IKeyPairManager contains a group of method to create/manage/store internal public/private key pair, stored as sensitive data using a lazy loaded IAppConfig variable. 2 strings app id and name are used to identify key pairs.

generateKeyPair generate and store public/private key pair.
hasKeyPair return if key pair is known in database.
getKeyPair return key pair from database based on app id and name.
deleteKeyPair delete key pair from database.

ISignatureManager

ISignatureManager is a service integrated to core that provide tools to set/get authenticity of/from outgoing/incoming requests.

getIncomingSignedRequest extract data from the incoming request and compare headers to confirm authenticity of remote instance
getOutgoingSignedRequest prep signature to sign an outgoing request.
signOutgoingRequestIClientPayload is the one method to call to fully process of signing and fulfilling the payload for an outgoing request using IClient
searchSignatory get a remote signatory from the database

lib/private/OCM/OCMSignatoryManager.php

lib/private/Security/Signature/Model/OutgoingSignedRequest.php

lib/private/Security/Signature/Model/SignedRequest.php

lib/private/Security/Signature/SignatureManager.php

+
+			[$k, $v] = explode('=', $entry, 2);
+			preg_match('/"([^"]+)"/', $v, $varr);
+			if ($varr[0] !== null) {


lib/private/OCM/OCMSignatoryManager.php

lib/private/Security/Signature/Model/SignedRequest.php

lib/private/Security/Signature/SignatureManager.php

+
+		$parsed = parse_url($uri);
+		$signedRequest->setHost($parsed['host'])
+					  ->setAlgorithm($options['algorithm'] ?? 'sha256')


lib/private/Security/Signature/SignatureManager.php

apps/files_sharing/lib/External/Cache.php

apps/cloud_federation_api/lib/Controller/RequestHandlerController.php

+			// remote does not support signed request.
+			// currently we still accept unsigned request until lazy appconfig
+			// core.enforce_signed_ocm_request is set to true (default: false)
+			if ($this->appConfig->getValueBool('enforce_signed_ocm_request', false, lazy: true)) {


lib/private/Security/Signature/Model/Signatory.php

lib/private/Security/Signature/SignatureManager.php

+		if ($signatory === null) {
+			throw new SignatoryNotFoundException('empty result from getRemoteSignatory');
+		}
+		if ($signatory->getKeyId() !== $signedRequest->getKeyId()) {


lib/private/Security/Signature/SignatureManager.php

apps/cloud_federation_api/lib/Controller/RequestHandlerController.php

+		if (!str_contains($entry, '@')) {
+			throw new IncomingRequestException('entry does not contains @');
+		}
+		[, $instance] = explode('@', $entry, 2);


lib/private/Security/PublicPrivateKeyPairs/Model/KeyPair.php

+	 * @inheritDoc
+	 *
+	 * @param string $publicKey
+	 * @return IKeyPair


lib/private/Security/PublicPrivateKeyPairs/Model/KeyPair.php

+	 * @inheritDoc
+	 *
+	 * @param string $privateKey
+	 * @return IKeyPair


lib/private/Security/PublicPrivateKeyPairs/Model/KeyPair.php

+	 * @inheritDoc
+	 *
+	 * @param array $options
+	 * @return IKeyPair


lib/private/Security/Signature/Model/OutgoingSignedRequest.php

+	 * @inheritDoc
+	 *
+	 * @param string $host
+	 * @return IOutgoingSignedRequest


lib/private/Security/Signature/Model/OutgoingSignedRequest.php

+	 * @param string $key
+	 * @param string|int|float|bool|array $value
+	 *
+	 * @return IOutgoingSignedRequest


lib/private/Security/Signature/Model/OutgoingSignedRequest.php

+	 *
+	 * @param string $estimated
+	 *
+	 * @return IOutgoingSignedRequest


lib/private/Security/Signature/Model/OutgoingSignedRequest.php

+	 *
+	 * @param string $algorithm
+	 *
+	 * @return IOutgoingSignedRequest


lib/private/Security/Signature/Model/SignedRequest.php

+	 * @inheritDoc
+	 *
+	 * @param array $signatureHeader
+	 * @return ISignedRequest


lib/private/Security/Signature/Model/SignedRequest.php

+	/**
+	 * @inheritDoc
+	 *
+	 * @return ISignatory


lib/private/Security/Signature/Model/SignedRequest.php

+	 * @since 30.0.0
+	 */
+	public function getSignatory(): ISignatory {
+		return $this->signatory;


to.json

nickvergessen · 2024-07-11T07:44:15Z

apps/cloud_federation_api/lib/Controller/RequestHandlerController.php

+			// if request is signed and well signed, no exception are thrown
+			// if request is not signed and host is known for not supporting signed request, no exception are thrown
+			$signedRequest = $this->getSignedRequest();
+			$this->confirmShareOrigin($signedRequest, $notification['sharedSecret'] ?? '');


sharedSecret inside the notification is already used by some OCM messages. So this would break it. Can you take another key? Or maybe you prefix with '$#$' or something alike?

this would not break it as I send the exception for the exact same reason of a missing 'sharedSecret' entry in the notifications request. The only thing is that I do this check earlier than others but I can add a prefix if you want (while I dont feel like necessary myself)

this would not break it as I send the exception for the exact same reason of a missing 'sharedSecret' entry in the notifications request.

Nextcloud Talk is sending a data field 'sharedSecret' and you either overwrite that and it breaks Talk Federation with older servers or you need to use a different key

As said before, I only use this entry because it exists in the OCM protocol and doing so to compare that the origin of the reshare request, based on the token (and the linked recipient stored in the database), confirm the identity used to sign the request.

If Talk is using this endpoint to initiate anything, signature are to be required

apps/cloud_federation_api/lib/Controller/RequestHandlerController.php

lib/private/Federation/CloudFederationProviderManager.php

lib/public/Security/Signature/SignatureAlgorithm.php

lib/public/Security/Signature/Model/SignatoryType.php

lib/private/Security/Signature/Model/SignedRequest.php

lib/private/Security/Signature/SignatureManager.php

core/Migrations/Version30000Date20240101084401.php

nickvergessen · 2024-07-11T08:11:40Z

lib/private/Security/Signature/SignatureManager.php

+		}
+	}
+
+	private function insertSignatory(ISignatory $signatory): void {


By now our Entity+QBMapper pattern is widely adapted. Any reason why you didn't go for it and instead went back to doing all this manually again?

no real reason other that personal preference

lib/private/Security/Signature/SignatureManager.php

artonge · 2024-11-14T13:50:01Z

lib/private/Security/PublicPrivateKeyPairs/KeyPairManager.php

+ *
+ * @since 31.0.0
+ */
+class KeyPairManager implements IKeyPairManager {


Could be merged with https://github.com/nextcloud/server/blob/423f57677f2b2e9d71277f6dc89a7698a1f92ea8/lib/private/Security/IdentityProof/Manager.php ?

apps/cloud_federation_api/lib/Capabilities.php

come-nc · 2024-11-14T14:04:57Z

apps/cloud_federation_api/lib/Capabilities.php

+			$this->logger->warning('cannot generate local signatory', ['exception' => $e]);
+		}
+
+		return ['ocm' => json_decode(json_encode($this->provider->jsonSerialize()), true)];


Is the json_decode(json_encode( really needed?

it is only requested for psalm as jsonSerialize returns serializable objects

apps/cloud_federation_api/lib/Controller/RequestHandlerController.php

come-nc · 2024-11-14T14:35:28Z

build/integration/federation_features/cleanup-remote-storage.feature

+    And As an "user1"
+    And sending "GET" to "/apps/files_sharing/api/v1/shares"
+    And the list of returned shares has 1 shares


What change results in this being needed?

come-nc · 2024-11-14T14:39:20Z

lib/private/Federation/CloudFederationProviderManager.php

+			// signing the payload using OCMSignatoryManager before initializing the request
+			$uri = $ocmProvider->getEndPoint() . '/notifications';
+			$payload = array_merge($this->getDefaultRequestOptions(), ['body' => json_encode($notification->getMessage())]);
+			if (!$this->appConfig->getValueBool('core', OCMSignatoryManager::APPCONFIG_SIGN_DISABLED, lazy: true)) {
+				$signedPayload = $this->signatureManager->signOutgoingRequestIClientPayload(
+					$this->signatoryManager,
+					$payload,
+					'post', $uri
+				);
+			}
+			return $client->post($uri, $signedPayload ?? $payload);


code is duplicated 4 times, please wrap it in a method.

come-nc · 2024-11-14T14:45:25Z

lib/private/Security/PublicPrivateKeyPairs/KeyPairManager.php

+ * KeyPairManager store internal public/private key pair using AppConfig, taking advantage of the encryption
+ * and lazy loading.


It should use its own table, it’s a bad habit to put everything into appconfig.

I think it is the main idea of appconfig to avoid creating a new table for only few entries, and the lazy loading helps keeping it light

come-nc · 2024-11-14T14:48:46Z

lib/private/Security/Signature/SignatureManager.php

+	 * @since 31.0.0
+	 */
+	public function getIncomingSignedRequest(
+		ISignatoryManager $signatoryManager,


Why is that not injected? It creates a dependency loop?

come-nc · 2024-11-14T14:50:33Z

lib/private/Security/Signature/SignatureManager.php

+	 * @return array new payload to be sent, including original payload and signature elements in headers
+	 * @since 31.0.0
+	 */
+	public function signOutgoingRequestIClientPayload(


Suggested change

public function signOutgoingRequestIClientPayload(

public function signOutgoingRequestPayload(

come-nc · 2024-11-14T14:51:58Z

lib/private/Security/Signature/SignatureManager.php

+	public function searchSignatory(string $host, string $account = ''): ISignatory {
+		$qb = $this->connection->getQueryBuilder();
+		$qb->select(
+			'id', 'provider_id', 'host', 'account', 'key_id', 'key_id_sum', 'public_key', 'metadata', 'type',
+			'status', 'creation', 'last_updated'
+		);
+		$qb->from(self::TABLE_SIGNATORIES);
+		$qb->where($qb->expr()->eq('host', $qb->createNamedParameter($host)));
+		$qb->andWhere($qb->expr()->eq('account', $qb->createNamedParameter($account)));
+
+		$result = $qb->executeQuery();
+		$row = $result->fetch();
+		$result->closeCursor();
+
+		if (!$row) {
+			throw new SignatoryNotFoundException('no signatory found');
+		}
+
+		$signature = new Signatory($row['key_id'], $row['public_key']);
+
+		return $signature->importFromDatabase($row);
+	}


Is there no mapper and entities for signatories?

come-nc · 2024-11-14T14:53:09Z

lib/private/Security/Signature/SignatureManager.php

+	public function extractIdentityFromUri(string $uri): string {
+		$identity = parse_url($uri, PHP_URL_HOST);
+		$port = parse_url($uri, PHP_URL_PORT);
+		if ($identity === null || $identity === false) {
+			throw new IdentityNotFoundException('cannot extract identity from ' . $uri);
+		}
+
+		if ($port !== null && $port !== false) {
+			$identity .= ':' . $port;
+		}
+
+		return $identity;
+	}


I have Déjà-vu about this code

Signed-off-by: Maxence Lange <[email protected]>

github-advanced-security bot found potential problems Jun 19, 2024

View reviewed changes

github-advanced-security bot found potential problems Jun 25, 2024

View reviewed changes

ArtificialOwl force-pushed the enh/noid/signed-request branch from 58d3efd to f45a2cb Compare July 2, 2024 04:46

github-advanced-security bot found potential problems Jul 2, 2024

View reviewed changes

ArtificialOwl force-pushed the enh/noid/signed-request branch from f45a2cb to a3dda22 Compare July 4, 2024 16:03

github-advanced-security bot found potential problems Jul 4, 2024

View reviewed changes

ArtificialOwl force-pushed the enh/noid/signed-request branch from 61cc09b to 9ef90c0 Compare July 10, 2024 12:01

github-advanced-security bot found potential problems Jul 10, 2024

View reviewed changes

ArtificialOwl added the 3. to review Waiting for reviews label Jul 10, 2024

ArtificialOwl added this to the Nextcloud 30 milestone Jul 10, 2024

ArtificialOwl marked this pull request as ready for review July 10, 2024 16:24

ArtificialOwl requested a review from nickvergessen as a code owner July 10, 2024 16:24

nickvergessen reviewed Jul 11, 2024

View reviewed changes

to.json Outdated Show resolved Hide resolved

nickvergessen requested changes Jul 11, 2024

View reviewed changes

This was referenced Jul 30, 2024

30.0.0 beta 2 #46870

Merged

30.0.0 beta 3 #46946

Merged

This was referenced Aug 5, 2024

30.0.0 beta 4 #47028

Merged

30.0.0 beta 5 #47100

Merged

skjnldsv mentioned this pull request Aug 13, 2024

30.0.0 RC1 #47190

Merged

skjnldsv modified the milestones: Nextcloud 30, Nextcloud 31 Aug 14, 2024

ArtificialOwl mentioned this pull request Aug 21, 2024

Signing request cs3org/OCM-API#92

Merged

michielbdejong mentioned this pull request Sep 20, 2024

feat(CloudFederationApi): Start implementation of 1.1.0 of OCM spec #47199

Draft

13 tasks

skjnldsv added 2. developing Work in progress and removed 3. to review Waiting for reviews labels Oct 29, 2024

ArtificialOwl force-pushed the enh/noid/signed-request branch 4 times, most recently from 444f9e7 to f3a1684 Compare October 31, 2024 12:21

ArtificialOwl requested a review from provokateurin as a code owner October 31, 2024 12:45

ArtificialOwl force-pushed the enh/noid/signed-request branch 2 times, most recently from 81d2a14 to c41e150 Compare November 8, 2024 11:46

ArtificialOwl force-pushed the enh/noid/signed-request branch 7 times, most recently from 6c69420 to ec4ed86 Compare November 12, 2024 22:08

artonge approved these changes Nov 14, 2024

View reviewed changes

come-nc reviewed Nov 14, 2024

View reviewed changes

ArtificialOwl force-pushed the enh/noid/signed-request branch 5 times, most recently from 9983ea2 to d7446ed Compare November 16, 2024 18:22

feat(ocm): signing ocm requests

14b8a00

Signed-off-by: Maxence Lange <[email protected]>

ArtificialOwl force-pushed the enh/noid/signed-request branch 2 times, most recently from 6eb73da to 5912b44 Compare November 17, 2024 00:58

fix(ocm): switching to IdentityProof

6ee56ab

Signed-off-by: Maxence Lange <[email protected]>

ArtificialOwl force-pushed the enh/noid/signed-request branch from 5912b44 to 6ee56ab Compare November 17, 2024 01:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Signed requests #45979

Signed requests #45979

ArtificialOwl commented Jun 19, 2024 •

edited

Loading

nickvergessen Jul 11, 2024

ArtificialOwl Jul 11, 2024

nickvergessen Jul 11, 2024

ArtificialOwl Oct 30, 2024

nickvergessen Jul 11, 2024

ArtificialOwl Jul 11, 2024

artonge Nov 14, 2024

come-nc Nov 14, 2024

ArtificialOwl Nov 15, 2024

come-nc Nov 14, 2024

come-nc Nov 14, 2024

come-nc Nov 14, 2024

ArtificialOwl Nov 15, 2024

come-nc Nov 14, 2024

come-nc Nov 14, 2024

come-nc Nov 14, 2024

come-nc Nov 14, 2024

ArtificialOwl Nov 15, 2024

		* KeyPairManager store internal public/private key pair using AppConfig, taking advantage of the encryption
		* and lazy loading.

	public function signOutgoingRequestIClientPayload(
	public function signOutgoingRequestPayload(

Signed requests #45979

Are you sure you want to change the base?

Signed requests #45979

Conversation

ArtificialOwl commented Jun 19, 2024 • edited Loading

Signed Request

Signature

ISignatoryManager

IKeyPairManager

ISignatureManager

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ArtificialOwl commented Jun 19, 2024 •

edited

Loading