how to use @agoric/cosmic-proto in a confined endo caplet? #1875

dckc · 2023-12-02T06:29:43Z

dckc
Dec 2, 2023
Maintainer

Protobuf is just mangling strings/integers/etc. in memory; it's a perfect candidate for being completely confined. But I'm struggling to get it to work.

I tried to endo make proto-puzzle.js where that file contains:

import { MsgWalletSpendAction } from '@agoric/cosmic-proto/swingset/msgs.js';

export const make = () => {};

and my package.json has::

    "@agoric/cosmic-proto": "^0.3.0",

The first problem is SES_EVAL_REJECTED. After some spelunking, I can hack-n-slash around that no problem:

diff --git a/node_modules/@protobufjs/inquire/index.js b/node_modules/@protobufjs/inquire/index.js
index 33778b5..c343370 100644
--- a/node_modules/@protobufjs/inquire/index.js
+++ b/node_modules/@protobufjs/inquire/index.js
@@ -1,6 +1,7 @@
 "use strict";
 module.exports = inquire;
 
+
 /**
  * Requires a module only if available.
  * @memberof util
@@ -9,7 +10,7 @@ module.exports = inquire;
  */
 function inquire(moduleName) {
     try {
-        var mod = eval("quire".replace(/^/,"re"))(moduleName); // eslint-disable-line no-eval
+        var mod = NOeval("quire".replace(/^/,"re"))(moduleName); // eslint-disable-line no-eval

Next we find that long is missing from the dependencies in cosmic-proto. hack-slash-hack-slash. There. now it's not missing.

But I'm sorely stumped by the next one: Cannot read properties of undefined (reading 'dcodeIO').
I looked all over the endo logs with less $(grep -lr 'dcodeIO' ~/.local/state/endo/) but I can't find the line where the error is thrown. Clues, anyone? @kriskowal ?

$ endo run src/proto-puzzle.js 
(TypeError#1)
TypeError#1: Cannot read properties of undefined (reading 'dcodeIO')

  at eval (eval at <anonymous> (eval at makeEvaluate (packages/ses/src/make-evaluate.js:92:27)), <anonymous>:180:52)
  at Object.execute (packages/compartment-mapper/src/parse-pre-cjs.js:40:5)
  at execute (packages/ses/src/module-instance.js:99:30)
  at compartmentImportNow (packages/ses/src/compartment.js:78:3)
  at Compartment.importNow (packages/ses/src/compartment.js:167:12)
  at require (packages/compartment-mapper/src/parse-cjs-shared-export-wrapper.js:135:35)
  at eval (eval at <anonymous> (eval at makeEvaluate (packages/ses/src/make-evaluate.js:92:27)), <anonymous>:4:17)
  at Object.execute (packages/compartment-mapper/src/parse-pre-cjs.js:40:5)
  at execute (packages/ses/src/module-instance.js:99:30)
  at compartmentImportNow (packages/ses/src/compartment.js:78:3)
  at Compartment.importNow (packages/ses/src/compartment.js:167:12)
  at require (packages/compartment-mapper/src/parse-cjs-shared-export-wrapper.js:135:35)
  at eval (eval at <anonymous> (eval at makeEvaluate (packages/ses/src/make-evaluate.js:92:27)), <anonymous>:13:25)
  at Object.execute (packages/compartment-mapper/src/parse-pre-cjs.js:40:5)
  at execute (packages/ses/src/module-instance.js:99:30)
  at compartmentImportNow (packages/ses/src/compartment.js:78:3)
  at Compartment.importNow (packages/ses/src/compartment.js:167:12)
  at require (packages/compartment-mapper/src/parse-cjs-shared-export-wrapper.js:135:35)
  at eval (eval at <anonymous> (eval at makeEvaluate (packages/ses/src/make-evaluate.js:92:27)), <anonymous>:4:18)
  at Object.execute (packages/compartment-mapper/src/parse-pre-cjs.js:40:5)
  at Object.execute (packages/ses/src/module-instance.js:99:30)
  at imports (packages/ses/src/module-instance.js:374:16)
  at eval (.../cosmic-proto/dist/agoric/swingset/msgs.js:1:133)
  at eval (.../cosmic-proto/dist/agoric/swingset/msgs.js:631:3)
  at Object.execute (packages/ses/src/module-instance.js:464:9)
  at imports (packages/ses/src/module-instance.js:374:16)
  at eval (.../ag-trace/src/proto-puzzle.js:1:145)
  at eval (.../ag-trace/src/proto-puzzle.js:4:3)
  at execute (packages/ses/src/module-instance.js:464:9)
  at compartmentImportNow (packages/ses/src/compartment.js:78:3)
  at packages/ses/src/compartment.js:141:27
  at async importBundle (packages/import-bundle/src/index.js:69:27)
  at async packages/cli/src/run.js:91:27
  at async packages/cli/src/context.js:72:7
  at async packages/cli/src/context.js:55:7
  at async packages/cli/src/context.js:43:5
  at async withInterrupt (packages/cli/src/context.js:14:5)
  at async run (packages/cli/src/run.js:37:3)
  at async Promise.all (index 0)
  at async main (packages/cli/src/endo.js:606:5)
  at async packages/cli/bin/endo.cjs:4:3

dckc · 2023-12-02T07:14:58Z

dckc
Dec 2, 2023
Maintainer Author

Hmm... maybe I found it, but I don't know how to proceed:

$ grep -r dcodeIO node_modules/ | less
...
node_modules/@agoric/cosmic-proto/node_modules/protobufjs/dist/light/protobuf.js:util.Long = /* istanbul ignore next */ util.global.dcodeIO && /* istanbul ignore next */ util.global.dcodeIO.Long
...

0 replies

dckc · 2023-12-02T07:20:00Z

dckc
Dec 2, 2023
Maintainer Author

https://github.com/bufbuild/protobuf-es looks promising... but when I used their example, I got:

ReferenceError#1: exports is not defined

  at eval (.../protobuf/dist/cjs/index.js:29:23)
  at eval (.../protobuf/dist/cjs/index.js:78:3)
  at Object.execute (packages/ses/src/module-instance.js:464:9)
  at imports (packages/ses/src/module-instance.js:374:16)
  at eval (.../protobuf/dist/proxy/index.js:1:120)
  at eval (.../protobuf/dist/proxy/index.js:2:3)
  at Object.execute (packages/ses/src/module-instance.js:464:9)
  at imports (packages/ses/src/module-instance.js:374:16)
  at eval (.../pbes/src/gen/example_pb.js:1:131)
  at eval (.../pbes/src/gen/example_pb.js:34:3)
...

somehow endo seems to pick the cjs version of their code, even though the supply esm. Their package.json has:

  "type": "module",
  "sideEffects": false,
  "main": "./dist/cjs/index.js",
  "exports": {
    ".": {
      "module": "./dist/esm/index.js",
      "require": "./dist/cjs/index.js",
      "import": "./dist/proxy/index.js"
    }
  },

my pb-puzzle.js code was pretty much the same:

import { User } from './gen/example_pb.js';

export const make = () => {};

4 replies

michaelfig Dec 3, 2023
Maintainer

Endo is definitely picking the "import" version of the protobuf code, which according to your stack trace and their package.json, uses dist/proxy/index.js and that bounces into dist/cjs/index.js.

dckc Dec 3, 2023
Maintainer Author

score!

with this patch:

diff --git a/node_modules/@bufbuild/protobuf/package.json b/node_modules/@bufbuild/protobuf/package.json
index 81b39d9..177f75e 100644
--- a/node_modules/@bufbuild/protobuf/package.json
+++ b/node_modules/@bufbuild/protobuf/package.json
@@ -27,7 +27,7 @@
     ".": {
       "module": "./dist/esm/index.js",
       "require": "./dist/cjs/index.js",
-      "import": "./dist/proxy/index.js"
+      "import": "./dist/esm/index.js"
     }

I can import the generated code without error:

$ endo make src/pb-puzzle.js 
undefined

dckc Dec 3, 2023
Maintainer Author

I'm inclined to open a protobuf-es issue, but I'm struggling to find documentation that says that the way their import field goes to CJS code is not The Right Way.

Clues?

dckc Dec 3, 2023
Maintainer Author

I filed an issue:

package import key leads to CJS code via proxy bufbuild/protobuf-es#639

dckc · 2023-12-02T17:20:13Z

dckc
Dec 2, 2023
Maintainer Author

This bit of endo doc looked possibly relevant, but I don't understand it well enough to apply it:

The exports property describes package entry points and can be influenced by build tags. Currently, the only tag supported by the compartment mapper is import, which indicates that the module map should use ESM modules over CommonJS modules or other variants.

0 replies

kriskowal · 2023-12-03T04:26:42Z

kriskowal
Dec 3, 2023
Maintainer

I agree the patch is valid upstream. Docs here might help https://nodejs.org/api/packages.html#exports

…

On Sat, Dec 2, 2023 at 8:23 PM Dan Connolly ***@***.***> wrote: I'm inclined to open a protobuf-es issue <https://github.com/bufbuild/protobuf-es/issues>, but I'm struggling to find documentation that says that the way their import field goes to CJS code is not The Right Way. Clues? — Reply to this email directly, view it on GitHub <#1875 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAOXBUG5FTAWIA446BJW4LYHP5FVAVCNFSM6AAAAABADZKWQWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TONBRHA4DM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

0 replies

dckc · 2023-12-03T08:02:31Z

dckc
Dec 3, 2023
Maintainer Author

I made an analog of @agoric-cosmic-proto using protobuf-es:

cosmic-proto-es

Using that, I can end make proto-puzzle.js fully confined:

import { MsgWalletSpendAction } from 'cosmic-proto-es/swingset/msgs.js';

export const make = () => true;

But there's lots of other stubs in cosmjs-types built with protobufjs where I get TypeError#1: Cannot read properties of undefined (reading 'dcodeIO')

So I couldn't confine agoricZone.js let alone smartWallet.js.

dckc/finquick#65

0 replies

michaelfig · 2023-12-04T21:14:57Z

michaelfig
Dec 4, 2023
Maintainer

You may want to take a look at agoric-labs/protobuf.js where I patched protobufjs not to break when getting dcodeIO.

0 replies

dckc · 2024-05-11T08:22:30Z

dckc
May 11, 2024
Maintainer Author

looks like this is addressed in...

proto messages in vats Agoric/agoric-sdk#9112

0 replies

how to use @agoric/cosmic-proto in a confined endo caplet? #1875

Uh oh!

Uh oh!

dckc Dec 2, 2023 Maintainer

Replies: 7 comments · 4 replies

Uh oh!

dckc Dec 2, 2023 Maintainer Author

Uh oh!

Uh oh!

dckc Dec 2, 2023 Maintainer Author

Uh oh!

michaelfig Dec 3, 2023 Maintainer

Uh oh!

dckc Dec 3, 2023 Maintainer Author

Uh oh!

dckc Dec 3, 2023 Maintainer Author

Uh oh!

dckc Dec 3, 2023 Maintainer Author

Uh oh!

dckc Dec 2, 2023 Maintainer Author

Uh oh!

kriskowal Dec 3, 2023 Maintainer

Uh oh!

Uh oh!

dckc Dec 3, 2023 Maintainer Author

Uh oh!

michaelfig Dec 4, 2023 Maintainer

Uh oh!

dckc May 11, 2024 Maintainer Author

dckc
Dec 2, 2023
Maintainer

Replies: 7 comments 4 replies

dckc
Dec 2, 2023
Maintainer Author

dckc
Dec 2, 2023
Maintainer Author

michaelfig Dec 3, 2023
Maintainer

dckc Dec 3, 2023
Maintainer Author

dckc Dec 3, 2023
Maintainer Author

dckc Dec 3, 2023
Maintainer Author

dckc
Dec 2, 2023
Maintainer Author

kriskowal
Dec 3, 2023
Maintainer

dckc
Dec 3, 2023
Maintainer Author

michaelfig
Dec 4, 2023
Maintainer

dckc
May 11, 2024
Maintainer Author