Provide support for solidity in `serde_generate` by MathieuDutSik · Pull Request #61 · zefchain/serde-reflection

MathieuDutSik · 2025-01-14T14:31:04Z

Summary

Solidity is the Javascript of blockchains, therefore having a way to support solidity in one of the codes is quite important.

Solidity has a number of limitations that force some adaptations:

The only enum supported are of the elementary types, therefore, the more complex enums are supported as a "choice" entry in the code.
There is no Tuple, so again a struct is introduced.
There is no Option type, therefore, we again introduce a struct construction.
The TupleArray [T; N] are also encoded as struct with the length being part of the typename.

There are also some more mundane restrictions where we choose to simply panic with an explicit error if this occurs in the input code:

No floating point.
No empty struct
No empty enum
Enum with at most 256 entries.
No recursive structure like trees.

The main implication of the use of struct to represent nested structure is that it breaks the design of ContainerFormat/ Format. So, we need to introduce the recursive type SolFormat type that provides support for this.

Other design choices:

No runtime is used. There is a way to have libraries in solidity, but it complicates the development since the library needs to be deployed first followed by the other contract.
Everything uses the "memory" location. I tried to use the more efficient "calldata" without success. Not sure if that is possible.
The get_keywords is a standard function instead of being a static one as in the OCaml generator. I think it does not matter.
Solidity has the abi.encode / abi.decode that works with padded serialization. It has an abi.encodePacked that provides a compact serialization, but it has no abi.decodePacked. Added to that the BCS serialization of 42_u32 is [42,0,0,0] but the solidity serialization is [0,0,0,42]. Therefore hand-crafted code has been written for processing all primitive types.
No assembly is used for the operation. There is room for improvement here.

Test Plan

The code is tested by running an EVM virtual machine. This is nicely provided by the REVM crate.
The only external files used are the temporary ones created during the test which makes the development smooth.

serde-generate-bin/tests/cli.rs

serde-generate/src/solidity.rs

ma2bd · 2025-01-14T18:50:07Z

serde-generate/src/solidity.rs

+    }
+}
+
+fn need_memory(sol_format: &SolFormat, sol_registry: &SolRegistry) -> bool {


Can we make that a method of either SolRegistry or SolFormat?

serde-generate/src/solidity.rs

serde-generate/tests/solidity_runtime.rs

ma2bd · 2025-01-15T03:42:38Z

serde-generate/src/solidity.rs

+        }
+    }
+
+    pub fn output<T: std::io::Write>(&self, out: &mut IndentedWriter<T>) -> Result<()> {


Probably ok for a first version but I'm assuming re-using a library would cheaper in terms of gas?

I think the opposite:

When we have all the source code in one single file, then the LLVM intermediate representation is fully available therefore optimizations are available.

When a library is available, then the linking occurs when the library is deployed. This of course prevents some optimization. I think it has its uses if the library is an external one. However, we do not gain anything by having a library.

We can add an option later.

ma2bd · 2025-01-17T05:58:00Z

#62 should help with CI. Let's see if it comes back green.

serde-reflection/src/lib.rs

rust-toolchain.toml

.github/workflows/main.yml

rust-toolchain.toml

This reverts commit 91adf02.

ma2bd · 2025-01-30T01:46:55Z

serde-generate/tests/solidity_generation.rs

+        generator.output(&mut test_file, &registry).unwrap();
+    }
+
+    let _bytecode = get_bytecode(path, "test.sol", "test").unwrap();


let _bytecode = is not necessarily (and slightly counter-productive. E.g. it disables the warning that would be triggered if you forget the .unwrap() here).

ma2bd · 2025-01-30T01:49:15Z

serde-generate/tests/solidity_runtime.rs

+        let ExecutionResult::Success {
+            reason: _,
+            gas_used: _,
+            gas_refunded: _,
+            logs: _,
+            output,
+        } = result


let ExecutionResult::Success { output, .. } = result is probably fine, especially for testing.

ma2bd · 2025-01-30T01:49:38Z

serde-generate/tests/solidity_runtime.rs

+        output: _,
+    } = result
+    else {
+        panic!("The TxKind::Call execution failed to be done");


remove "to be done"

ma2bd · 2025-01-30T02:22:39Z

serde-generate/src/solidity.rs

+impl SolRegistry {
+    fn insert(&mut self, sol_format: SolFormat) {
+        let key_name = sol_format.key_name();
+        if matches!(sol_format, SolFormat::Primitive(Primitive::I8)) {


The cases look disjoint. Why not match sol_format { .. } ?

ma2bd · 2025-01-30T21:43:50Z

serde-generate/src/solidity.rs

+            _ => {}
        }
+        // Typename entries are by definition already covered and do not need
+        // to be inserted. Others do.
        if !matches!(sol_format, SolFormat::TypeName(_)) {
            self.names.insert(key_name, sol_format);
        }


There's really no need for matches! here.

SolFormat::Primitive(SolFormat::TypeName(_)) => { // Typename entries do not need to be inserted. } _ => { self.names.insert(key_name, sol_format); }

## Summary Fix CI ## Test Plan CI

MathieuDutSik requested a review from ma2bd as a code owner January 14, 2025 14:31

deuszx mentioned this pull request Jan 14, 2025

Solidity deserializers for Linera types linera-io/linera-protocol#3127

Closed

ma2bd reviewed Jan 16, 2025

View reviewed changes

MathieuDutSik force-pushed the serde_generate_solidity branch from f1c98ad to 3dec9d1 Compare January 18, 2025 13:08

ma2bd reviewed Jan 18, 2025

View reviewed changes

serde-reflection/src/lib.rs Show resolved Hide resolved

ma2bd reviewed Jan 20, 2025

View reviewed changes

rust-toolchain.toml Show resolved Hide resolved

ma2bd reviewed Jan 28, 2025

View reviewed changes

.github/workflows/main.yml Outdated Show resolved Hide resolved

rust-toolchain.toml Show resolved Hide resolved

MathieuDutSik added 22 commits January 29, 2025 08:55

First insertion of new code.

5839a44

First version of the code.

fb18128

Some correction for compilation.

136dac9

Some additional corrections.

1d4b9eb

Now it compiles, continuing the work.

21d3ec1

Further code insertion.

72691b0

Further code write.

349e777

Another significant insertion.

81d1d48

A bunch of corrections coming from code inspection.

30670d8

Resolve more bugs.

a7e28b6

Have the first code working correctly for SimpleEnum.

5bb64c5

Several more corrections.

e8fbbe1

Address one more bug.

2cb4d52

Address a number of bugs related to the memory location.

f2c8144

Resolve several more issues. More work is needed.

15976b0

Further problem.

73f7982

Cleanup of the code. Now it finally compiles.

f01033d

Some simplification of the data_location stuff.

a5d6ca0

Add code for testing for circular dependencies.

8acefd3

Debug the circular dependency code.

f3cf483

Insert the code for the solidity generation.

23a7d1e

Some basic corrections for compilation.

4b04ad2

MathieuDutSik added 18 commits January 29, 2025 08:55

Another cleanup.

1857c45

Make the simple enum work correctly.

d707ae5

Correct the support for the boolean and introduce another test case.

bd982da

Iron out another bug.

ef3d594

Add one more test to the code and make the code more readable.

844415a

Clippy corrections and elimination of code used during the development.

1ad1973

Now formatting corrections.

e0ea445

Reformatting error forgotten.

5995b66

Address some clippy issues.

d6def0c

Switch the keywords to a static variable.

e542e1d

Make the routines method of SolRegistry.

5a9d846

Formatting corrections.

db9d2ce

Remove the expect statement + add ViaIR stuff.

06ef836

Add the solidity compiler.

6ef58ba

Removing correction induced by clippy.

d01b860

Revert "Removing correction induced by clippy."

59f3fd2

This reverts commit 91adf02.

Some corrections at having the solc working.

5b8396a

Correct the solidity entry in the CI.

a2b5f4a

MathieuDutSik force-pushed the serde_generate_solidity branch from e25765b to a2b5f4a Compare January 29, 2025 08:00

ma2bd reviewed Jan 30, 2025

View reviewed changes

Simplifications from the reviewing process.

0859878

ma2bd reviewed Jan 30, 2025

View reviewed changes

Remove the matches! entry.

25071dc

ma2bd approved these changes Jan 31, 2025

View reviewed changes

ma2bd merged commit dce87ea into zefchain:main Jan 31, 2025

ma2bd added a commit that referenced this pull request Jan 31, 2025

improve docs after #61

01c8cc8

ma2bd added a commit to ma2bd/serde-reflection that referenced this pull request Jan 31, 2025

simplify/fix CI after zefchain#61

72fc5ff

ma2bd added a commit to ma2bd/serde-reflection that referenced this pull request Jan 31, 2025

simplify/fix CI after zefchain#61

40c571d

ma2bd added a commit that referenced this pull request Jan 31, 2025

simplify/fix CI after #61 (#64)

8dc4933

## Summary Fix CI ## Test Plan CI

Conversation

MathieuDutSik commented Jan 14, 2025

Summary

Test Plan

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ma2bd commented Jan 17, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ma2bd Jan 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ma2bd Jan 30, 2025 •

edited

Loading