Skip to content

Add EnumerableSetExtended and EnumerableMapExtended #89

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Apr 12, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/checks.yml
Original file line number Diff line number Diff line change
@@ -45,6 +45,8 @@ jobs:
run: npm run test:inheritance
- name: Check pragma consistency between files
run: npm run test:pragma
- name: Check procedurally generated contracts are up-to-date
run: npm run test:generation

coverage:
runs-on: ubuntu-latest
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
## XX-XX-XXXX

- `EnumerableSetExtended` and `EnumerableMapExtended`: Extensions of the `EnumerableSet` and `EnumerableMap` libraries with more types, including non-value types.

## 03-04-2025

- `PaymasterERC20`: Extension of `PaymasterCore` that sponsors user operations against payment in ERC-20 tokens.
7 changes: 7 additions & 0 deletions contracts/utils/README.adoc
Original file line number Diff line number Diff line change
@@ -9,6 +9,7 @@ Miscellaneous contracts and libraries containing utility functions you can use t
* {ERC7739}: An abstract contract to validate signatures following the rehashing scheme from `ERC7739Utils`.
* {ERC7739Utils}: Utilities library that implements a defensive rehashing mechanism to prevent replayability of smart contract signatures based on ERC-7739.
* {SignerECDSA}, {SignerP256}, {SignerRSA}: Implementations of an {AbstractSigner} with specific signature validation algorithms.
* {EnumerableSetExtended} and {EnumerableMapExtended}: Extensions of the `EnumerableSet` and `EnumerableMap` libraries with more types, including non-value types.
* {Masks}: Library to handle `bytes32` masks.

== Cryptography
@@ -27,6 +28,12 @@ Miscellaneous contracts and libraries containing utility functions you can use t

{{SignerRSA}}

== Structs

{{EnumerableSetExtended}}

{{EnumerableMapExtended}}

== Libraries

{{Masks}}
281 changes: 281 additions & 0 deletions contracts/utils/structs/EnumerableMapExtended.sol
Original file line number Diff line number Diff line change
@@ -0,0 +1,281 @@
// SPDX-License-Identifier: MIT
// This file was procedurally generated from scripts/generate/templates/EnumerableMapExtended.js.

pragma solidity ^0.8.20;

import {EnumerableSet} from "@openzeppelin/contracts/utils/structs/EnumerableSet.sol";
import {EnumerableSetExtended} from "./EnumerableSetExtended.sol";

/**
* @dev Library for managing an enumerable variant of Solidity's
* https://solidity.readthedocs.io/en/latest/types.html#mapping-types[`mapping`]
* type for non-value types as keys.
*
* Maps have the following properties:
*
* - Entries are added, removed, and checked for existence in constant time
* (O(1)).
* - Entries are enumerated in O(n). No guarantees are made on the ordering.
* - Map can be cleared (all entries removed) in O(n).
*
* ```solidity
* contract Example {
* // Add the library methods
* using EnumerableMapExtended for EnumerableMapExtended.BytesToUintMap;
*
* // Declare a set state variable
* EnumerableMapExtended.BytesToUintMap private myMap;
* }
* ```
*
* The following map types are supported:
*
* - `bytes -> uint256` (`BytesToUintMap`)
* - `string -> string` (`StringToStringMap`)
*
* [WARNING]
* ====
* Trying to delete such a structure from storage will likely result in data corruption, rendering the structure
* unusable.
* See https://github.com/ethereum/solidity/pull/11843[ethereum/solidity#11843] for more info.
*
* In order to clean an EnumerableMap, you can either remove all elements one by one or create a fresh instance using an
* array of EnumerableMap.
* ====
*
* NOTE: Extensions of openzeppelin/contracts/utils/struct/EnumerableMap.sol.
*/
library EnumerableMapExtended {
using EnumerableSet for *;
using EnumerableSetExtended for *;

/**
* @dev Query for a nonexistent map key.
*/
error EnumerableMapNonexistentBytesKey(bytes key);

struct BytesToUintMap {
// Storage of keys
EnumerableSetExtended.BytesSet _keys;
mapping(bytes key => uint256) _values;
}

/**
* @dev Adds a key-value pair to a map, or updates the value for an existing
* key. O(1).
*
* Returns true if the key was added to the map, that is if it was not
* already present.
*/
function set(BytesToUintMap storage map, bytes memory key, uint256 value) internal returns (bool) {
map._values[key] = value;
return map._keys.add(key);
}

/**
* @dev Removes a key-value pair from a map. O(1).
*
* Returns true if the key was removed from the map, that is if it was present.
*/
function remove(BytesToUintMap storage map, bytes memory key) internal returns (bool) {
delete map._values[key];
return map._keys.remove(key);
}

/**
* @dev Removes all the entries from a map. O(n).
*
* WARNING: Developers should keep in mind that this function has an unbounded cost and using it may render the
* function uncallable if the map grows to the point where clearing it consumes too much gas to fit in a block.
*/
function clear(BytesToUintMap storage map) internal {
uint256 len = length(map);
for (uint256 i = 0; i < len; ++i) {
delete map._values[map._keys.at(i)];
}
map._keys.clear();
}

/**
* @dev Returns true if the key is in the map. O(1).
*/
function contains(BytesToUintMap storage map, bytes memory key) internal view returns (bool) {
return map._keys.contains(key);
}

/**
* @dev Returns the number of key-value pairs in the map. O(1).
*/
function length(BytesToUintMap storage map) internal view returns (uint256) {
return map._keys.length();
}

/**
* @dev Returns the key-value pair stored at position `index` in the map. O(1).
*
* Note that there are no guarantees on the ordering of entries inside the
* array, and it may change when more entries are added or removed.
*
* Requirements:
*
* - `index` must be strictly less than {length}.
*/
function at(BytesToUintMap storage map, uint256 index) internal view returns (bytes memory key, uint256 value) {
key = map._keys.at(index);
value = map._values[key];
}

/**
* @dev Tries to returns the value associated with `key`. O(1).
* Does not revert if `key` is not in the map.
*/
function tryGet(BytesToUintMap storage map, bytes memory key) internal view returns (bool exists, uint256 value) {
value = map._values[key];
exists = value != uint256(0) || contains(map, key);
}

/**
* @dev Returns the value associated with `key`. O(1).
*
* Requirements:
*
* - `key` must be in the map.
*/
function get(BytesToUintMap storage map, bytes memory key) internal view returns (uint256 value) {
bool exists;
(exists, value) = tryGet(map, key);
if (!exists) {
revert EnumerableMapNonexistentBytesKey(key);
}
}

/**
* @dev Return the an array containing all the keys
*
* WARNING: This operation will copy the entire storage to memory, which can be quite expensive. This is designed
* to mostly be used by view accessors that are queried without any gas fees. Developers should keep in mind that
* this function has an unbounded cost, and using it as part of a state-changing function may render the function
* uncallable if the map grows to a point where copying to memory consumes too much gas to fit in a block.
*/
function keys(BytesToUintMap storage map) internal view returns (bytes[] memory) {
return map._keys.values();
}

/**
* @dev Query for a nonexistent map key.
*/
error EnumerableMapNonexistentStringKey(string key);

struct StringToStringMap {
// Storage of keys
EnumerableSetExtended.StringSet _keys;
mapping(string key => string) _values;
}

/**
* @dev Adds a key-value pair to a map, or updates the value for an existing
* key. O(1).
*
* Returns true if the key was added to the map, that is if it was not
* already present.
*/
function set(StringToStringMap storage map, string memory key, string memory value) internal returns (bool) {
map._values[key] = value;
return map._keys.add(key);
}

/**
* @dev Removes a key-value pair from a map. O(1).
*
* Returns true if the key was removed from the map, that is if it was present.
*/
function remove(StringToStringMap storage map, string memory key) internal returns (bool) {
delete map._values[key];
return map._keys.remove(key);
}

/**
* @dev Removes all the entries from a map. O(n).
*
* WARNING: Developers should keep in mind that this function has an unbounded cost and using it may render the
* function uncallable if the map grows to the point where clearing it consumes too much gas to fit in a block.
*/
function clear(StringToStringMap storage map) internal {
uint256 len = length(map);
for (uint256 i = 0; i < len; ++i) {
delete map._values[map._keys.at(i)];
}
map._keys.clear();
}

/**
* @dev Returns true if the key is in the map. O(1).
*/
function contains(StringToStringMap storage map, string memory key) internal view returns (bool) {
return map._keys.contains(key);
}

/**
* @dev Returns the number of key-value pairs in the map. O(1).
*/
function length(StringToStringMap storage map) internal view returns (uint256) {
return map._keys.length();
}

/**
* @dev Returns the key-value pair stored at position `index` in the map. O(1).
*
* Note that there are no guarantees on the ordering of entries inside the
* array, and it may change when more entries are added or removed.
*
* Requirements:
*
* - `index` must be strictly less than {length}.
*/
function at(
StringToStringMap storage map,
uint256 index
) internal view returns (string memory key, string memory value) {
key = map._keys.at(index);
value = map._values[key];
}

/**
* @dev Tries to returns the value associated with `key`. O(1).
* Does not revert if `key` is not in the map.
*/
function tryGet(
StringToStringMap storage map,
string memory key
) internal view returns (bool exists, string memory value) {
value = map._values[key];
exists = bytes(value).length != 0 || contains(map, key);
}

/**
* @dev Returns the value associated with `key`. O(1).
*
* Requirements:
*
* - `key` must be in the map.
*/
function get(StringToStringMap storage map, string memory key) internal view returns (string memory value) {
bool exists;
(exists, value) = tryGet(map, key);
if (!exists) {
revert EnumerableMapNonexistentStringKey(key);
}
}

/**
* @dev Return the an array containing all the keys
*
* WARNING: This operation will copy the entire storage to memory, which can be quite expensive. This is designed
* to mostly be used by view accessors that are queried without any gas fees. Developers should keep in mind that
* this function has an unbounded cost, and using it as part of a state-changing function may render the function
* uncallable if the map grows to a point where copying to memory consumes too much gas to fit in a block.
*/
function keys(StringToStringMap storage map) internal view returns (string[] memory) {
return map._keys.values();
}
}
415 changes: 415 additions & 0 deletions contracts/utils/structs/EnumerableSetExtended.sol

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion lib/@openzeppelin-contracts
2 changes: 1 addition & 1 deletion lib/@openzeppelin-contracts-upgradeable
Submodule @openzeppelin-contracts-upgradeable updated 75 files
+0 −5 .changeset/brave-islands-sparkle.md
+0 −5 .changeset/brown-seals-sing.md
+0 −5 .changeset/brown-turkeys-marry.md
+0 −5 .changeset/cyan-taxis-travel.md
+0 −5 .changeset/dirty-bananas-shake.md
+0 −5 .changeset/famous-timers-compare.md
+0 −5 .changeset/good-cameras-rush.md
+0 −5 .changeset/good-cameras-serve.md
+0 −5 .changeset/green-drinks-report.md
+0 −5 .changeset/long-walls-draw.md
+5 −0 .changeset/mighty-melons-cheer.md
+0 −5 .changeset/pretty-lobsters-tan.md
+0 −5 .changeset/proud-cooks-do.md
+0 −5 .changeset/seven-insects-taste.md
+0 −5 .changeset/sixty-tips-wink.md
+0 −5 .changeset/ten-fishes-fold.md
+0 −5 .changeset/ten-hats-begin.md
+0 −5 .changeset/ten-peas-mix.md
+0 −5 .changeset/thin-eels-cross.md
+50 −0 CHANGELOG.md
+ audits/2025-04-v5.3.pdf
+1 −0 audits/README.md
+1 −1 contracts/access/AccessControlUpgradeable.sol
+1 −1 contracts/finance/VestingWalletUpgradeable.sol
+1 −1 contracts/governance/GovernorUpgradeable.sol
+1 −1 contracts/governance/TimelockControllerUpgradeable.sol
+1 −1 contracts/governance/extensions/GovernorCountingFractionalUpgradeable.sol
+1 −1 contracts/governance/extensions/GovernorCountingOverridableUpgradeable.sol
+1 −0 contracts/governance/extensions/GovernorProposalGuardianUpgradeable.sol
+1 −0 contracts/governance/extensions/GovernorSequentialProposalIdUpgradeable.sol
+1 −1 contracts/governance/extensions/GovernorStorageUpgradeable.sol
+1 −0 contracts/governance/extensions/GovernorSuperQuorumUpgradeable.sol
+1 −1 contracts/governance/extensions/GovernorTimelockAccessUpgradeable.sol
+1 −1 contracts/governance/extensions/GovernorTimelockCompoundUpgradeable.sol
+1 −1 contracts/governance/extensions/GovernorTimelockControlUpgradeable.sol
+1 −1 contracts/governance/extensions/GovernorVotesQuorumFractionUpgradeable.sol
+2 −0 contracts/governance/extensions/GovernorVotesSuperQuorumFractionUpgradeable.sol
+1 −1 contracts/metatx/ERC2771ForwarderUpgradeable.sol
+5 −5 contracts/mocks/AuthorityMockUpgradeable.sol
+0 −7 contracts/mocks/WithInit.sol
+2 −2 contracts/mocks/docs/access-control/AccessControlModifiedUpgradeable.sol
+0 −23 contracts/mocks/docs/access-control/AccessControlNonRevokableAdminUpgradeable.sol
+1 −1 contracts/mocks/token/ERC20ForceApproveMockUpgradeable.sol
+2 −2 contracts/package.json
+1 −1 contracts/proxy/utils/Initializable.sol
+1 −1 contracts/proxy/utils/UUPSUpgradeable.sol
+2 −3 contracts/token/ERC20/ERC20Upgradeable.sol
+5 −5 contracts/token/ERC20/README.adoc
+1 −1 contracts/token/ERC20/extensions/ERC4626Upgradeable.sol
+1 −1 contracts/token/ERC20/extensions/draft-ERC20TemporaryApprovalUpgradeable.sol
+5 −5 contracts/token/ERC6909/draft-ERC6909Upgradeable.sol
+1 −0 contracts/token/ERC6909/extensions/draft-ERC6909ContentURIUpgradeable.sol
+1 −0 contracts/token/ERC6909/extensions/draft-ERC6909MetadataUpgradeable.sol
+2 −1 contracts/token/ERC6909/extensions/draft-ERC6909TokenSupplyUpgradeable.sol
+1 −1 contracts/token/ERC721/extensions/ERC721ConsecutiveUpgradeable.sol
+1 −1 contracts/token/ERC721/extensions/ERC721URIStorageUpgradeable.sol
+1 −1 contracts/token/common/ERC2981Upgradeable.sol
+1 −1 contracts/utils/MulticallUpgradeable.sol
+1 −1 contracts/utils/PausableUpgradeable.sol
+1 −1 contracts/utils/ReentrancyGuardTransientUpgradeable.sol
+1 −1 contracts/utils/cryptography/EIP712Upgradeable.sol
+4 −4 docs/modules/ROOT/pages/utilities.adoc
+1 −0 hardhat.config.js
+1 −1 hardhat/ignore-unreachable-warnings.js
+1 −1 lib/openzeppelin-contracts
+126 −133 package-lock.json
+2 −2 package.json
+1 −1 scripts/generate/templates/TransientSlot.js
+11 −1 test/access/manager/AuthorityUtils.test.js
+1 −1 test/governance/extensions/GovernorVotesSuperQuorumFraction.test.js
+1 −0 test/helpers/constants.js
+6 −6 test/token/ERC20/extensions/ERC4626.test.js
+0 −8 test/token/ERC721/extensions/ERC721Consecutive.test.js
+22 −0 test/utils/cryptography/P256.t.sol
+56 −30 test/utils/cryptography/P256.test.js
2 changes: 1 addition & 1 deletion lib/forge-std
Submodule forge-std updated 2 files
+68 −0 src/Vm.sol
+2 −2 test/Vm.t.sol
2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
@@ -22,7 +22,9 @@
"lint:sol": "prettier --log-level warn --ignore-path .gitignore '{contracts,test}/**/*.sol' --check && solhint '{contracts,test}/**/*.sol'",
"lint:sol:fix": "prettier --log-level warn --ignore-path .gitignore '{contracts,test}/**/*.sol' --write",
"coverage": "scripts/checks/coverage.sh",
"generate": "scripts/generate/run.js",
"test": "hardhat test",
"test:generation": "scripts/checks/generation.sh",
"test:inheritance": "scripts/checks/inheritance-ordering.js artifacts/build-info/*",
"test:pragma": "scripts/checks/pragma-consistency.js artifacts/build-info/*"
},
6 changes: 6 additions & 0 deletions scripts/checks/generation.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/usr/bin/env bash

set -euo pipefail

npm run generate
git diff -R --exit-code
38 changes: 38 additions & 0 deletions scripts/generate/run.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
#!/usr/bin/env node

const cp = require('child_process');
const fs = require('fs');
const path = require('path');
const format = require('@openzeppelin/contracts/scripts/generate/format-lines');

function getVersion(path) {
try {
return fs.readFileSync(path, 'utf8').match(/\/\/ OpenZeppelin Community Contracts \(last updated v[^)]+\)/)[0];
} catch {
return null;
}
}

function generateFromTemplate(file, template, outputPrefix = '') {
const script = path.relative(path.join(__dirname, '../..'), __filename);
const input = path.join(path.dirname(script), template);
const output = path.join(outputPrefix, file);
const version = getVersion(output);
const content = format(
'// SPDX-License-Identifier: MIT',
...(version ? [version + ` (${file})`] : []),
`// This file was procedurally generated from ${input}.`,
'',
require(template).trimEnd(),
);
fs.writeFileSync(output, content);
cp.execFileSync('prettier', ['--write', output]);
}

// Contracts
for (const [file, template] of Object.entries({
'utils/structs/EnumerableSetExtended.sol': './templates/EnumerableSetExtended.js',
'utils/structs/EnumerableMapExtended.sol': './templates/EnumerableMapExtended.js',
})) {
generateFromTemplate(file, template, './contracts/');
}
46 changes: 46 additions & 0 deletions scripts/generate/templates/Enumerable.opts.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
const { capitalize, mapValues } = require('@openzeppelin/contracts/scripts/helpers');

const typeDescr = ({ type, size = 0, memory = false }) => {
memory |= size > 0;

const name = [type == 'uint256' ? 'Uint' : capitalize(type), size].filter(Boolean).join('x');
const base = size ? type : undefined;
const typeFull = size ? `${type}[${size}]` : type;
const typeLoc = memory ? `${typeFull} memory` : typeFull;
return { name, type: typeFull, typeLoc, base, size, memory };
};

const toSetTypeDescr = value => ({
name: value.name + 'Set',
value,
});

const toMapTypeDescr = ({ key, value }) => ({
name: `${key.name}To${value.name}Map`,
keySet: toSetTypeDescr(key),
key,
value,
});

const SET_TYPES = [
{ type: 'bytes32', size: 2 },
{ type: 'string', memory: true },
{ type: 'bytes', memory: true },
]
.map(typeDescr)
.map(toSetTypeDescr);

const MAP_TYPES = [
{ key: { type: 'bytes', memory: true }, value: { type: 'uint256' } },
{ key: { type: 'string', memory: true }, value: { type: 'string', memory: true } },
]
.map(entry => mapValues(entry, typeDescr))
.map(toMapTypeDescr);

module.exports = {
SET_TYPES,
MAP_TYPES,
typeDescr,
toSetTypeDescr,
toMapTypeDescr,
};
179 changes: 179 additions & 0 deletions scripts/generate/templates/EnumerableMapExtended.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
const format = require('@openzeppelin/contracts/scripts/generate/format-lines');
const { SET_TYPES, MAP_TYPES } = require('./Enumerable.opts');

const header = `\
pragma solidity ^0.8.20;
import {EnumerableSet} from "@openzeppelin/contracts/utils/structs/EnumerableSet.sol";
import {EnumerableSetExtended} from "./EnumerableSetExtended.sol";
/**
* @dev Library for managing an enumerable variant of Solidity's
* https://solidity.readthedocs.io/en/latest/types.html#mapping-types[\`mapping\`]
* type for non-value types as keys.
*
* Maps have the following properties:
*
* - Entries are added, removed, and checked for existence in constant time
* (O(1)).
* - Entries are enumerated in O(n). No guarantees are made on the ordering.
* - Map can be cleared (all entries removed) in O(n).
*
* \`\`\`solidity
* contract Example {
* // Add the library methods
* using EnumerableMapExtended for EnumerableMapExtended.BytesToUintMap;
*
* // Declare a set state variable
* EnumerableMapExtended.BytesToUintMap private myMap;
* }
* \`\`\`
*
* The following map types are supported:
*
* - \`bytes -> uint256\` (\`BytesToUintMap\`)
* - \`string -> string\` (\`StringToStringMap\`)
*
* [WARNING]
* ====
* Trying to delete such a structure from storage will likely result in data corruption, rendering the structure
* unusable.
* See https://github.com/ethereum/solidity/pull/11843[ethereum/solidity#11843] for more info.
*
* In order to clean an EnumerableMap, you can either remove all elements one by one or create a fresh instance using an
* array of EnumerableMap.
* ====
*
* NOTE: Extensions of openzeppelin/contracts/utils/struct/EnumerableMap.sol.
*/
`;

const map = ({ name, keySet, key, value }) => `\
/**
* @dev Query for a nonexistent map key.
*/
error EnumerableMapNonexistent${key.name}Key(${key.type} key);
struct ${name} {
// Storage of keys
${SET_TYPES.some(el => el.name == keySet.name) ? 'EnumerableSetExtended' : 'EnumerableSet'}.${keySet.name} _keys;
mapping(${key.type} key => ${value.type}) _values;
}
/**
* @dev Adds a key-value pair to a map, or updates the value for an existing
* key. O(1).
*
* Returns true if the key was added to the map, that is if it was not
* already present.
*/
function set(${name} storage map, ${key.typeLoc} key, ${value.typeLoc} value) internal returns (bool) {
map._values[key] = value;
return map._keys.add(key);
}
/**
* @dev Removes a key-value pair from a map. O(1).
*
* Returns true if the key was removed from the map, that is if it was present.
*/
function remove(${name} storage map, ${key.typeLoc} key) internal returns (bool) {
delete map._values[key];
return map._keys.remove(key);
}
/**
* @dev Removes all the entries from a map. O(n).
*
* WARNING: Developers should keep in mind that this function has an unbounded cost and using it may render the
* function uncallable if the map grows to the point where clearing it consumes too much gas to fit in a block.
*/
function clear(${name} storage map) internal {
uint256 len = length(map);
for (uint256 i = 0; i < len; ++i) {
delete map._values[map._keys.at(i)];
}
map._keys.clear();
}
/**
* @dev Returns true if the key is in the map. O(1).
*/
function contains(${name} storage map, ${key.typeLoc} key) internal view returns (bool) {
return map._keys.contains(key);
}
/**
* @dev Returns the number of key-value pairs in the map. O(1).
*/
function length(${name} storage map) internal view returns (uint256) {
return map._keys.length();
}
/**
* @dev Returns the key-value pair stored at position \`index\` in the map. O(1).
*
* Note that there are no guarantees on the ordering of entries inside the
* array, and it may change when more entries are added or removed.
*
* Requirements:
*
* - \`index\` must be strictly less than {length}.
*/
function at(
${name} storage map,
uint256 index
) internal view returns (${key.typeLoc} key, ${value.typeLoc} value) {
key = map._keys.at(index);
value = map._values[key];
}
/**
* @dev Tries to returns the value associated with \`key\`. O(1).
* Does not revert if \`key\` is not in the map.
*/
function tryGet(
${name} storage map,
${key.typeLoc} key
) internal view returns (bool exists, ${value.typeLoc} value) {
value = map._values[key];
exists = ${value.memory ? 'bytes(value).length != 0' : `value != ${value.type}(0)`} || contains(map, key);
}
/**
* @dev Returns the value associated with \`key\`. O(1).
*
* Requirements:
*
* - \`key\` must be in the map.
*/
function get(${name} storage map, ${key.typeLoc} key) internal view returns (${value.typeLoc} value) {
bool exists;
(exists, value) = tryGet(map, key);
if (!exists) {
revert EnumerableMapNonexistent${key.name}Key(key);
}
}
/**
* @dev Return the an array containing all the keys
*
* WARNING: This operation will copy the entire storage to memory, which can be quite expensive. This is designed
* to mostly be used by view accessors that are queried without any gas fees. Developers should keep in mind that
* this function has an unbounded cost, and using it as part of a state-changing function may render the function
* uncallable if the map grows to a point where copying to memory consumes too much gas to fit in a block.
*/
function keys(${name} storage map) internal view returns (${key.type}[] memory) {
return map._keys.values();
}
`;

// GENERATE
module.exports = format(
header.trimEnd(),
'library EnumerableMapExtended {',
format(
[].concat('using EnumerableSet for *;', 'using EnumerableSetExtended for *;', '', MAP_TYPES.map(map)),
).trimEnd(),
'}',
);
316 changes: 316 additions & 0 deletions scripts/generate/templates/EnumerableSetExtended.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,316 @@
const format = require('@openzeppelin/contracts/scripts/generate/format-lines');
const { SET_TYPES } = require('./Enumerable.opts');

const header = `\
pragma solidity ^0.8.20;
import {Arrays} from "@openzeppelin/contracts/utils/Arrays.sol";
import {Hashes} from "@openzeppelin/contracts/utils/cryptography/Hashes.sol";
/**
* @dev Library for managing
* https://en.wikipedia.org/wiki/Set_(abstract_data_type)[sets] of non-value
* types.
*
* Sets have the following properties:
*
* - Elements are added, removed, and checked for existence in constant time
* (O(1)).
* - Elements are enumerated in O(n). No guarantees are made on the ordering.
* - Set can be cleared (all elements removed) in O(n).
*
* \`\`\`solidity
* contract Example {
* // Add the library methods
* using EnumerableSetExtended for EnumerableSetExtended.StringSet;
*
* // Declare a set state variable
* EnumerableSetExtended.StringSet private mySet;
* }
* \`\`\`
*
* Sets of type \`string\` (\`StringSet\`), \`bytes\` (\`BytesSet\`) and
* \`bytes32[2]\` (\`Bytes32x2Set\`) are supported.
*
* [WARNING]
* ====
* Trying to delete such a structure from storage will likely result in data corruption, rendering the structure
* unusable.
* See https://github.com/ethereum/solidity/pull/11843[ethereum/solidity#11843] for more info.
*
* In order to clean an EnumerableSet, you can either remove all elements one by one or create a fresh instance using an
* array of EnumerableSet.
* ====
*
* NOTE: This is an extension of openzeppelin/contracts/utils/struct/EnumerableSet.sol.
*/
`;

const set = ({ name, value }) => `\
struct ${name} {
// Storage of set values
${value.type}[] _values;
// Position is the index of the value in the \`values\` array plus 1.
// Position 0 is used to mean a value is not in the set.
mapping(${value.type} value => uint256) _positions;
}
/**
* @dev Add a value to a set. O(1).
*
* Returns true if the value was added to the set, that is if it was not
* already present.
*/
function add(${name} storage self, ${value.type} memory value) internal returns (bool) {
if (!contains(self, value)) {
self._values.push(value);
// The value is stored at length-1, but we add 1 to all indexes
// and use 0 as a sentinel value
self._positions[value] = self._values.length;
return true;
} else {
return false;
}
}
/**
* @dev Removes a value from a set. O(1).
*
* Returns true if the value was removed from the set, that is if it was
* present.
*/
function remove(${name} storage self, ${value.type} memory value) internal returns (bool) {
// We cache the value's position to prevent multiple reads from the same storage slot
uint256 position = self._positions[value];
if (position != 0) {
// Equivalent to contains(self, value)
// To delete an element from the _values array in O(1), we swap the element to delete with the last one in
// the array, and then remove the last element (sometimes called as 'swap and pop').
// This modifies the order of the array, as noted in {at}.
uint256 valueIndex = position - 1;
uint256 lastIndex = self._values.length - 1;
if (valueIndex != lastIndex) {
${value.type} memory lastValue = self._values[lastIndex];
// Move the lastValue to the index where the value to delete is
self._values[valueIndex] = lastValue;
// Update the tracked position of the lastValue (that was just moved)
self._positions[lastValue] = position;
}
// Delete the slot where the moved value was stored
self._values.pop();
// Delete the tracked position for the deleted slot
delete self._positions[value];
return true;
} else {
return false;
}
}
/**
* @dev Removes all the values from a set. O(n).
*
* WARNING: Developers should keep in mind that this function has an unbounded cost and using it may render the
* function uncallable if the set grows to the point where clearing it consumes too much gas to fit in a block.
*/
function clear(${name} storage set) internal {
uint256 len = length(set);
for (uint256 i = 0; i < len; ++i) {
delete set._positions[set._values[i]];
}
Arrays.unsafeSetLength(set._values, 0);
}
/**
* @dev Returns true if the value is in the set. O(1).
*/
function contains(${name} storage self, ${value.type} memory value) internal view returns (bool) {
return self._positions[value] != 0;
}
/**
* @dev Returns the number of values on the set. O(1).
*/
function length(${name} storage self) internal view returns (uint256) {
return self._values.length;
}
/**
* @dev Returns the value stored at position \`index\` in the set. O(1).
*
* Note that there are no guarantees on the ordering of values inside the
* array, and it may change when more values are added or removed.
*
* Requirements:
*
* - \`index\` must be strictly less than {length}.
*/
function at(${name} storage self, uint256 index) internal view returns (${value.type} memory) {
return self._values[index];
}
/**
* @dev Return the entire set in an array
*
* WARNING: This operation will copy the entire storage to memory, which can be quite expensive. This is designed
* to mostly be used by view accessors that are queried without any gas fees. Developers should keep in mind that
* this function has an unbounded cost, and using it as part of a state-changing function may render the function
* uncallable if the set grows to a point where copying to memory consumes too much gas to fit in a block.
*/
function values(${name} storage self) internal view returns (${value.type}[] memory) {
return self._values;
}
`;

const arraySet = ({ name, value }) => `\
struct ${name} {
// Storage of set values
${value.type}[] _values;
// Position is the index of the value in the \`values\` array plus 1.
// Position 0 is used to mean a value is not in the set.
mapping(bytes32 valueHash => uint256) _positions;
}
/**
* @dev Add a value to a set. O(1).
*
* Returns true if the value was added to the set, that is if it was not
* already present.
*/
function add(${name} storage self, ${value.type} memory value) internal returns (bool) {
if (!contains(self, value)) {
self._values.push(value);
// The value is stored at length-1, but we add 1 to all indexes
// and use 0 as a sentinel value
self._positions[_hash(value)] = self._values.length;
return true;
} else {
return false;
}
}
/**
* @dev Removes a value from a set. O(1).
*
* Returns true if the value was removed from the set, that is if it was
* present.
*/
function remove(${name} storage self, ${value.type} memory value) internal returns (bool) {
// We cache the value's position to prevent multiple reads from the same storage slot
bytes32 valueHash = _hash(value);
uint256 position = self._positions[valueHash];
if (position != 0) {
// Equivalent to contains(self, value)
// To delete an element from the _values array in O(1), we swap the element to delete with the last one in
// the array, and then remove the last element (sometimes called as 'swap and pop').
// This modifies the order of the array, as noted in {at}.
uint256 valueIndex = position - 1;
uint256 lastIndex = self._values.length - 1;
if (valueIndex != lastIndex) {
${value.type} memory lastValue = self._values[lastIndex];
// Move the lastValue to the index where the value to delete is
self._values[valueIndex] = lastValue;
// Update the tracked position of the lastValue (that was just moved)
self._positions[_hash(lastValue)] = position;
}
// Delete the slot where the moved value was stored
self._values.pop();
// Delete the tracked position for the deleted slot
delete self._positions[valueHash];
return true;
} else {
return false;
}
}
/**
* @dev Removes all the values from a set. O(n).
*
* WARNING: Developers should keep in mind that this function has an unbounded cost and using it may render the
* function uncallable if the set grows to the point where clearing it consumes too much gas to fit in a block.
*/
function clear(${name} storage self) internal {
${value.type}[] storage v = self._values;
uint256 len = length(self);
for (uint256 i = 0; i < len; ++i) {
delete self._positions[_hash(v[i])];
}
assembly ("memory-safe") {
sstore(v.slot, 0)
}
}
/**
* @dev Returns true if the value is in the set. O(1).
*/
function contains(${name} storage self, ${value.type} memory value) internal view returns (bool) {
return self._positions[_hash(value)] != 0;
}
/**
* @dev Returns the number of values on the set. O(1).
*/
function length(${name} storage self) internal view returns (uint256) {
return self._values.length;
}
/**
* @dev Returns the value stored at position \`index\` in the set. O(1).
*
* Note that there are no guarantees on the ordering of values inside the
* array, and it may change when more values are added or removed.
*
* Requirements:
*
* - \`index\` must be strictly less than {length}.
*/
function at(${name} storage self, uint256 index) internal view returns (${value.type} memory) {
return self._values[index];
}
/**
* @dev Return the entire set in an array
*
* WARNING: This operation will copy the entire storage to memory, which can be quite expensive. This is designed
* to mostly be used by view accessors that are queried without any gas fees. Developers should keep in mind that
* this function has an unbounded cost, and using it as part of a state-changing function may render the function
* uncallable if the set grows to a point where copying to memory consumes too much gas to fit in a block.
*/
function values(${name} storage self) internal view returns (${value.type}[] memory) {
return self._values;
}
`;

const hashes = `\
function _hash(bytes32[2] memory value) private pure returns (bytes32) {
return Hashes.efficientKeccak256(value[0], value[1]);
}
`;

// GENERATE
module.exports = format(
header.trimEnd(),
'library EnumerableSetExtended {',
format(
[].concat(
SET_TYPES.filter(({ value }) => value.size == 0).map(set),
SET_TYPES.filter(({ value }) => value.size > 0).map(arraySet),
hashes,
),
).trimEnd(),
'}',
);
66 changes: 66 additions & 0 deletions test/utils/structs/EnumerableMapExtended.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
const { ethers } = require('hardhat');
const { loadFixture } = require('@nomicfoundation/hardhat-network-helpers');

const { mapValues } = require('@openzeppelin/contracts/test/helpers/iterate');
const { generators } = require('@openzeppelin/contracts/test/helpers/random');
const { MAP_TYPES } = require('../../../scripts/generate/templates/Enumerable.opts');

const { shouldBehaveLikeMap } = require('@openzeppelin/contracts/test/utils/structs/EnumerableMap.behavior');

async function fixture() {
const mock = await ethers.deployContract('$EnumerableMapExtended');

const env = Object.fromEntries(
MAP_TYPES.map(({ name, key, value }) => [
name,
{
key,
value,
keys: Array.from({ length: 3 }, generators[key.type]),
values: Array.from({ length: 3 }, generators[value.type]),
zeroValue: generators[value.type].zero,
methods: mapValues(
{
set: `$set(uint256,${key.type},${value.type})`,
get: `$get(uint256,${key.type})`,
tryGet: `$tryGet(uint256,${key.type})`,
remove: `$remove(uint256,${key.type})`,
clear: `$clear_EnumerableMapExtended_${name}(uint256)`,
length: `$length_EnumerableMapExtended_${name}(uint256)`,
at: `$at_EnumerableMapExtended_${name}(uint256,uint256)`,
contains: `$contains(uint256,${key.type})`,
keys: `$keys_EnumerableMapExtended_${name}(uint256)`,
},
fnSig =>
(...args) =>
mock.getFunction(fnSig)(0, ...args),
),
events: {
setReturn: `return$set_EnumerableMapExtended_${name}_${key.type}_${value.type}`,
removeReturn: `return$remove_EnumerableMapExtended_${name}_${key.type}`,
},
error: key.memory || value.memory ? `EnumerableMapNonexistent${key.name}Key` : `EnumerableMapNonexistentKey`,
},
]),
);

return { mock, env };
}

describe('EnumerableMapExtended', function () {
beforeEach(async function () {
Object.assign(this, await loadFixture(fixture));
});

for (const { name, key, value } of MAP_TYPES) {
describe(`${name} (enumerable map from ${key.type} to ${value.type})`, function () {
beforeEach(async function () {
Object.assign(this, this.env[name]);
[this.keyA, this.keyB, this.keyC] = this.keys;
[this.valueA, this.valueB, this.valueC] = this.values;
});

shouldBehaveLikeMap();
});
}
});
62 changes: 62 additions & 0 deletions test/utils/structs/EnumerableSetExtended.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
const { ethers } = require('hardhat');
const { loadFixture } = require('@nomicfoundation/hardhat-network-helpers');

const { mapValues } = require('@openzeppelin/contracts/test/helpers/iterate');
const { generators } = require('@openzeppelin/contracts/test/helpers/random');
const { SET_TYPES } = require('../../../scripts/generate/templates/Enumerable.opts');

const { shouldBehaveLikeSet } = require('@openzeppelin/contracts/test/utils/structs/EnumerableSet.behavior');

async function fixture() {
const mock = await ethers.deployContract('$EnumerableSetExtended');

const env = Object.fromEntries(
SET_TYPES.map(({ name, value }) => [
name,
{
value,
values: Array.from(
{ length: 3 },
value.size ? () => Array.from({ length: value.size }, generators[value.base]) : generators[value.type],
),
methods: mapValues(
{
add: `$add(uint256,${value.type})`,
remove: `$remove(uint256,${value.type})`,
contains: `$contains(uint256,${value.type})`,
clear: `$clear_EnumerableSetExtended_${name}(uint256)`,
length: `$length_EnumerableSetExtended_${name}(uint256)`,
at: `$at_EnumerableSetExtended_${name}(uint256,uint256)`,
values: `$values_EnumerableSetExtended_${name}(uint256)`,
},
fnSig =>
(...args) =>
mock.getFunction(fnSig)(0, ...args),
),
events: {
addReturn: `return$add_EnumerableSetExtended_${name}_${value.type.replace(/[[\]]/g, '_')}`,
removeReturn: `return$remove_EnumerableSetExtended_${name}_${value.type.replace(/[[\]]/g, '_')}`,
},
},
]),
);

return { mock, env };
}

describe('EnumerableSetExtended', function () {
beforeEach(async function () {
Object.assign(this, await loadFixture(fixture));
});

for (const { name, value } of SET_TYPES) {
describe(`${name} (enumerable set of ${value.type})`, function () {
beforeEach(function () {
Object.assign(this, this.env[name]);
[this.valueA, this.valueB, this.valueC] = this.values;
});

shouldBehaveLikeSet();
});
}
});