Skip to content

Commit

Permalink
Merge pull request #175 from nevillegrech/storage-refactor-api
Browse files Browse the repository at this point in the history
Introduce Storage Modeling API
  • Loading branch information
sifislag authored Feb 10, 2025
2 parents f68d1f9 + 4efed98 commit b39719d
Show file tree
Hide file tree
Showing 7 changed files with 350 additions and 119 deletions.
115 changes: 115 additions & 0 deletions clientlib/storage_modeling/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
# Storage Modeling
Recovers the contract's storage layout and information about statements that deal with storage.
This documentation currently covers the core parts of the storage modeling functionality.
See the [API](storage_modeling_api.dl) and [clienthelpers](clienthelpers.dl) for additional functionality.

## Type Declarations
## `StorageConstruct`
`StorageConstruct` is the main type that defines arbitrarily nested data structures using a nested ADT type.

### Definition
```solidity
.type StorageConstruct = Constant {value: Value}
| StaticArray {parConstruct: StorageConstruct, arraySize: number}
| Array {parConstruct: StorageConstruct}
| Mapping {parConstruct: StorageConstruct}
| Offset {parConstruct: StorageConstruct, offset: number}
| Variable {construct: StorageConstruct}
| TightlyPackedVariable {construct: StorageConstruct, byteLow: number, byteHigh: number}
```

#### Example
The following example can be used to intuitively understand how nested structures are modeled.
Smart contract declarations:
```solidity
contract StorageExample {
uint256 public supply; // slot 0x0
address public owner; // slot 0x1
bool public isPaused; // slot 0x1
uint256[] public supplies; // slot 0x2
mapping (address => bool) public admins; // slot 0x3
struct vals {uint256 field0; uint256 field1;}
mapping (address => mapping(uint256 => vals)) public complex; // slot 0x4
}
```

Construct inferences:
```solidity
Variable(Constant(0x0)) // uint256 supply
Variable(Constant(0x1)) // address owner , bool isPaused
Variable(Array(Constant(0x2))) // uint256 [] supplies
Variable(Mapping(Constant(0x3))) // mapping admins
// the 2 fields of struct value of nested mapping complex :
Variable(Mapping(Mapping(Constant(0x4))))
Variable(Offset(Mapping(Mapping(Constant(0x4))), 1))
```

## `StorageStmtKind`
The `StorageStmtKind` type is used to describe the kind of load or write a storage statement performs.

### Definition
```solidity
.type StorageStmtKind = VariableAccess {}
| ArrayDataStart {}
| ArrayLength {}
```

Storage statement kind can be:
* `VariableAccess` which can be used to read or write either:
* A top-level variable
* A struct field
* An array or mapping element
* `ArrayDataStart` which loads or writes to the start location of an array
* `ArrayLength` which is used to load or write the length of an array

## `VarOrConstWrite`
The `VarOrConstWrite` type is introduced to unify writes of variables and constants.

### Definition
```solidity
.type VarOrConstWrite = VariableWrite {var: Variable}
| ConstantWrite {const: Value}
```

## Main relations

---
`.decl StorageStmtKindAndConstruct(stmt: Statement, kind: StorageStmtKind, construct: StorageConstruct)`
Maps Storage Stmts to their corresponding construct and their `StorageStmtKind`
__Note__: Storage stmts do not have to be `SSTORE` and `SLOAD` statements, see `StorageLoad` for more info

---
`.decl StorageStmt_HighLevelUses(stmt: Statement, accessVar: Variable, offset: number, i: number, nestedness: number)`
Information about storage stmts on potentially nested data structures.
Can be used to get all information on index variables and field offsets at every step.

---
`.decl StorageLoad(load: Statement, cons: StorageConstruct, loadedVar: Variable)`
Storage load information.
In cases of packed variables a single `SLOAD` can load multiple high-level variables.
In these cases we consider the `load` to not be the `SLOAD` but rather the statement that ends up
extracting the bytes that define the packed variable (cast, etc).

---
`.decl StorageStore(store: Statement, cons: StorageConstruct, write: VarOrConstWrite)`
`SSTORE` statement write information.
A write can be either a Variable or a Constant in cases where the actual written constant doesn't have a corresponding `Variable_Value` fact.
__Note__: Due to optimized packed variable write patterns, one `SSTORE` can write multiple different constructs

---
`.decl ArrayDeleteOp(sstore: Statement, loop: Block, array: StorageConstruct)`
An `array` is deleted by setting its length to zero in `sstore` and then erasing all its contents in a following `loop`.

---
`.decl DataStructureValueIsStruct(cons: StorageConstruct, structID: symbol, elemNum: number)`
Data structure construct has a value (Mappings) or element (Arrays) that is a struct identified by its `structID`

---
`.decl StructToString(structID: symbol, stringStruct: symbol)`
Map struct identified by `structID` to solidity-like struct definition `stringStruct`.

---



---
18 changes: 18 additions & 0 deletions clientlib/storage_modeling/clienthelpers.dl
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@

/**
Storage modeling client helpers
Relations that can be computed using relations in the API but are useful standalone
*/

// Doesn't contain TightlyPackedVariable, maybe revisit
.decl StorageConstruct_ParentAndOffset(cons: StorageConstruct, parentCons: StorageConstruct, offset: number)


.decl StorageConstructPredecesor(cons: StorageConstruct, predCons: StorageConstruct)

StorageConstructPredecesor(cons, parentCons):-
StorageConstruct_ParentAndOffset(cons, parentCons, _).

StorageConstructPredecesor(cons, predParentCons):-
StorageConstructPredecesor(cons, predCons),
StorageConstruct_ParentAndOffset(predCons, predParentCons, _).
121 changes: 75 additions & 46 deletions clientlib/storage_modeling/data_structures.dl
Original file line number Diff line number Diff line change
@@ -1,8 +1,4 @@

/***
New Storage Model
Aims to support arbituary nested data structures
***/

/**
`StorageIndex` contains information on the data structure used as well as the variables
Expand All @@ -15,16 +11,6 @@
| MappingAccessIndex {parIndex: StorageIndex, keyVar: Variable}
| OffsetIndex {parIndex: StorageIndex, offset: number}

/**
`StorageConstruct` contains the information of `StorageIndex`, stripped of indexing/access vars
*/
.type StorageConstruct = Constant {value: Value}
| StaticArray {parConstruct: StorageConstruct, arraySize: number}
| Array {parConstruct: StorageConstruct}
| Mapping {parConstruct: StorageConstruct}
| Offset {parConstruct: StorageConstruct, offset: number}
| Variable {construct: StorageConstruct}
| TightlyPackedVariable {construct: StorageConstruct, byteLow: number, byteHigh: number}

/**
Syntactic translation of `StorageIndex` to `StorageConstruct`
Expand All @@ -51,12 +37,20 @@
// Indexes in this relation can only be used directly to construct offset indexes
.decl LikelyVariableLoadingStorageIndex(index: StorageIndex)

/**
Maps `SSTORE`, `SLOAD` (and more, for packed vars) statements to the high-level information:
- `kind` can be "ACCESS", "LENGTH", "GETDATASTART"
- `index` can be used to get the high-level uses of the composite storage data structure accesses and writes
- `construct` is the data structure that is being accessed/written to
*/
.decl StorageStmtToIndexAndConstruct(stmt: Statement, kind: symbol, index: StorageIndex, construct: StorageConstruct)

.decl StorageIndex_HighLevelUses(index: StorageIndex, accessVar: Variable, offset: number, i: number, nestedness: number)

// Construct is not used as a variable (i.e. is used as an array or mapping)
// Used to find the ones that are end-level variables
.decl NonVariableConstruct(cons: StorageConstruct)

.decl StorageIndex_HighLevelUses(index: StorageIndex, accessVar: Variable, offset: number, i: number, nestedness: number)

.decl IsStorageConstruct(cons: StorageConstruct)

// StorageConstruct is array or mapping
Expand All @@ -65,37 +59,13 @@
// Number of elements the value of a data structure construct (Array or Mapping) has
.decl DataStructure_ElemNum(cons: StorageConstruct, elemNum: number)

// Data structure construct has a value (Mappings) or element (Arrays) that is a struct
.decl DataStructureValueIsStruct(cons: StorageConstruct, structID: symbol, elemNum: number)

// value type for mapping or element type for arrays
.decl DataStructure_ValueOrElementType(cons: StorageConstruct, type: symbol)

// key type for mapping
.decl Mapping_KeyType(cons: StorageConstruct, type: symbol)

// Map `structID` to solidity-like struct definition `stringStruct`
.decl StructToString(structID: symbol, stringStruct: symbol)
// Internal predicate, used to compute `StructToString`
.decl StructToStringUpTo(structID: symbol, stringStruct: symbol, upTo: number)

// Map tightly packed field in storage struct to solidity-like def used by `StructToString`
.decl PackedStructWordToString(structID: symbol, offset: number, stringRep: symbol)
.decl PackedStructWordToStringUpTo(structID: symbol, offset: number, stringRep: symbol, upToByte: number, upToIndex: number)

// Note: Can probably be unified with `StorageVariable_Type`
.decl DataStructure_Type(cons: StorageConstruct, type: symbol)

// Doesn't contain TightlyPackedVariable, maybe revisit
.decl StorageConstruct_ParentAndOffset(cons: StorageConstruct, parentCons: StorageConstruct, offset: number)

/**
Maps `SSTORE` and `SLOAD` statements to the high-level information:
- `kind` can be "ACCESS", "LENGTH", "GETDATASTART"
- `index` can be used to get the high-level uses of the composite storage data structure accesses and writes
- `construct` is the data structure that is being accessed/written to
*/
.decl StorageStmtToIndexAndConstruct(stmt: Statement, kind: symbol, index: StorageIndex, construct: StorageConstruct)

// Same as `StorageStmtToIndexAndConstruct`, populated before the tight packing component
.decl InitialStorageStmtToIndexAndConstruct(stmt: Statement, kind: symbol, index: StorageIndex, construct: StorageConstruct)

Expand Down Expand Up @@ -496,6 +466,7 @@ StorageStmtToIndexAndConstruct(stmt, "ACCESS", index, $Variable(cons)):-
StorageAccessOp(stmt, var),
ProcessedStorageVariable($Variable(cons), $Variable(cons)).

StorageLoad(stmt, $TightlyPackedVariable(cons, actualLow, actualHigh), var),
StorageStmtToIndexAndConstruct(stmt, "ACCESS", index, $TightlyPackedVariable(cons, actualLow, actualHigh)):-
LikelyVariableLoadingStorageIndex(index),
StorageIndex_StorageConstruct(index, cons),
Expand All @@ -516,6 +487,68 @@ StorageStmtToIndexAndConstruct(store, "ACCESS", index, $TightlyPackedVariable(co
NormalizeStorageVarByteLimits($Variable(cons), low, high, actualLow, actualHigh),
ProcessedStorageVariable($Variable(cons), $TightlyPackedVariable(cons, actualLow, actualHigh)).

// New API relation computation
DEBUG_OUTPUT(StorageLoad)
DEBUG_OUTPUT(StorageStore)

StorageLoad(stmt, $Variable(cons), loadedVar):-
LikelyVariableLoadingStorageIndex(index),
StorageIndex_StorageConstruct(index, cons),
Variable_StorageIndex(var, index),
SLOAD(stmt, var, loadedVar),
ProcessedStorageVariable($Variable(cons), $Variable(cons)).

StorageStore(store, $Variable(cons), $VariableWrite(storedVar)):-
LikelyVariableLoadingStorageIndex(index),
StorageIndex_StorageConstruct(index, cons),
Variable_StorageIndex(var, index),
SSTORE(store, var, storedVar),
ProcessedStorageVariable($Variable(cons), $Variable(cons)).

StorageStore(store, $TightlyPackedVariable(cons, actualLow, actualHigh), $VariableWrite(storedVar)):-
LikelyVariableLoadingStorageIndex(index),
StorageIndex_StorageConstruct(index, cons),
Variable_StorageIndex(indexVar, index),
StorageAccessOp(store, indexVar),
VarWrittenToBytesOfStorVarFinal(storedVar, store, $Variable(cons), low, high),
NormalizeStorageVarByteLimits($Variable(cons), low, high, actualLow, actualHigh),
ProcessedStorageVariable($Variable(cons), $TightlyPackedVariable(cons, actualLow, actualHigh)).

StorageStore(store, $TightlyPackedVariable(cons, actualLow, actualHigh), $ConstantWrite(const)):-
LikelyVariableLoadingStorageIndex(index),
StorageIndex_StorageConstruct(index, cons),
Variable_StorageIndex(indexVar, index),
StorageAccessOp(store, indexVar),
ConstWrittenToBytesOfStorVarProcessed(_, const, store, _, $Variable(cons), low, high),
NormalizeStorageVarByteLimits($Variable(cons), low, high, actualLow, actualHigh),
ProcessedStorageVariable($Variable(cons), $TightlyPackedVariable(cons, actualLow, actualHigh)),
!VarWrittenToBytesOfStorVarFinal(_, store, $Variable(cons), low, high).

// Not sure if this is needed
// StorageStore(store, $Variable(cons), $ConstantWrite("0x0")):-
// LikelyVariableLoadingStorageIndex(index),
// StorageIndex_StorageConstruct(index, cons),
// Variable_StorageIndex(indexVar, index),
// StorageAccessOp(store, indexVar),
// DeleteOfStructSlot(store, $Variable(cons)),
// NormalizeStorageVarByteLimits($Variable(cons), _, _, actualLow, actualHigh),
// ProcessedStorageVariable($Variable(cons), $TightlyPackedVariable(cons, actualLow, actualHigh)).



StorageStmtKindAndConstruct(stmt, $VariableAccess(), construct):-
StorageStmtToIndexAndConstruct(stmt, "ACCESS", _, construct).

StorageStmtKindAndConstruct(stmt, $ArrayLength(), construct):-
StorageStmtToIndexAndConstruct(stmt, "LENGTH", _, construct).

StorageStmtKindAndConstruct(stmt, $ArrayDataStart(), construct):-
StorageStmtToIndexAndConstruct(stmt, "GETDATASTART", _, construct).

StorageStmt_HighLevelUses(stmt, accessVar, offset, i, nestedness):-
StorageStmtToIndexAndConstruct(stmt, _, index, _),
StorageIndex_HighLevelUses(index, accessVar, offset, i, nestedness).

.decl StorageOffset_Type(offset: Value, type: symbol)

StorageOffset_Type(offset, type):-
Expand Down Expand Up @@ -631,12 +664,8 @@ StorageConstantToHash(const, zeroPaded, as(@hex_keccak_256(zeroPaded), Value)):-
Helper_Zeros(zeros, 64 - strlen(cut)),
zeroPaded = cat("0x", cat(zeros, cut)).

/**
An `array` is deleted by setting its length to zero in `sstore` and then erasing all its contents in a following `loop`.
*/
.decl ArrayDeleteOp(sstore: Statement, loop: Block, array: StorageConstruct)
DEBUG_OUTPUT(ArrayDeleteOp)

DEBUG_OUTPUT(ArrayDeleteOp)
ArrayDeleteOp(sstore, loop, array):-
SSTORE(sstore, _, zeroVar),
Variable_Value(zeroVar, "0x0"),
Expand Down
4 changes: 4 additions & 0 deletions clientlib/storage_modeling/storage_modeling.dl
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,16 @@

#define PACKED_FIELD(fieldNum,byteLow,byteHigh) MERGED_STORAGE_VAR(cat("field", to_string(fieldNum)),byteLow,byteHigh)

#define MISSING_VAR_PLACEHOLDER "0xNoVar"

#include "../dominators.dl"
#include "../flows.dl"
#include "../casts_shifts.dl"

#include "storage_modeling_api.dl"
#include "data_structures.dl"
#include "tight_packing.dl"
#include "type_inference.dl"
#include "clienthelpers.dl"
#include "legacy_data_structures.dl"
#include "metrics.dl"
Loading

0 comments on commit b39719d

Please sign in to comment.