Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate summaries of opcode automatically into the evm semantics #2710

Open
wants to merge 130 commits into
base: master
Choose a base branch
from

Conversation

Stevengre
Copy link
Contributor

@Stevengre Stevengre commented Feb 27, 2025

This PR is based on #2676, integrating all the summaries into the evm semantics. Specifically, it includes the following changes:

  1. Enhance summarize command with opcode-specific and clear options.
  2. Modify the function to generate correct summary files that can be used directly.
  3. Provide new kdist build targets using these summary rules.
  4. Modify the .gitignore to integrate the summary files by default.

Copy link
Contributor

@tothtamas28 tothtamas28 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a description for the PR, and make sure tests pass.

@Stevengre Stevengre self-assigned this Mar 4, 2025
@Stevengre Stevengre force-pushed the jh/summary-integration branch from f6646f4 to 82c51a7 Compare March 7, 2025 13:12
@Stevengre
Copy link
Contributor Author

Change files:

  • summarizer.py: Although there are 113 change files, the key file to this PR is summarizer.py, where I modify the summarize function to print kompileable K files.
  • summaries/: All the K files in summaries are generated automatically through the summarizer.py and tested by the kidst build.
  • driver.md & edsl.md: I provided two new modules for the kidst to target.
  • __main.py & cli.py: Here, I made a better option for the summarize command.

@Stevengre
Copy link
Contributor Author

Stevengre commented Mar 7, 2025

Should I provide tests to test the summary rules? Just like what we've done for the exising one, showing that the summary rules do the same as the previous ones. Or should we verify if the rules are correct or not?

Should we provide these tests inside of the CI? Should delete the previous CI test for the summarzation after we've provided the one for testing/verifying the summarization result.

@Stevengre Stevengre requested a review from ehildenb March 7, 2025 14:08
@Stevengre Stevengre marked this pull request as ready for review March 7, 2025 14:08
@Stevengre
Copy link
Contributor Author

Or maybe I can verify the correctness of these summary rules and migrate all the tests through the normal semantics to the summary ones?

Comment on lines +19 to +24
module EDSL-SUMMARY
imports EDSL-PURE
imports SUMMARY
endmodule

module EDSL-SUMMARIZE
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference between these two modules?

@@ -643,13 +698,12 @@ def batch_summarize(num_processes: int = 4) -> None:
num_processes: Number of parallel processes to use. Defaults to 4.
"""

opcodes_to_process = OPCODES.keys()
OPCODES.keys()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
OPCODES.keys()

Comment on lines 609 to 610
def _remove_inf_gas(res_line: str) -> str:
return re.sub(r'#gas (\([^)]*\))', r'\1', res_line)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to do these transformations directly on the KRule produced by summarization?

@Stevengre Stevengre force-pushed the jh/summary-integration branch from ce061ca to 12fe54e Compare March 12, 2025 17:07
- Add 'requires "../evm.md"' at the start of the module
- Insert 'imports EVM' after the module declaration
- Simplify module writing logic to handle imports and requires
- Rename 'summary' target to 'summarize' in kdist plugin
- Add new LLVM and Haskell summary targets
- Modify driver.md and edsl.md to support summary module imports
- Update summarizer to generate more robust summary files
- Add summary.k file to aggregate individual summary modules
- Implement text processing for summary file generation (remove inf gas, clean variable names)
- Include requires and imports for a wide range of EVM opcode summary specifications
- Expand the SUMMARY module to cover numerous opcodes from basic arithmetic to memory and storage operations
- Ensure complete coverage of EVM instruction summaries in the module
- Remove unnecessary account constraints and DotAccountVar references
- Modify summarizer to handle complex regex replacements for cleaner spec files
- Update opcode symbol parsing to handle parentheses in opcode names
- Remove redundant requires and andBool conditions in summary specs
- Rename an undefined opcode summary file to fix import syntax
…constraints

- Modify summarizer to comment out complex regex replacements
- Update summary files to use simpler account representation
- Remove parenthesized account and DotAccountVar references
- Restore _use_legal_remainder function with commented-out complex logic
- Update summary files for numerous EVM opcodes
- Modify basic block rules in summary specification files
- Systematically process and generate summary files for various EVM instructions
- Add function to replace LHS function with assignment in summary-balance specs
- Modify account ID representation in balance summary files
- Update regex transformation to simplify account ID constraints
- Reduce summarization test timeout from 360 to 150 seconds
- Remove duplicate method in KEVM class
- Update poetry.lock dependencies and extras
- Modify batch summarization processing logic
- Clean up regex transformations in summary specification generation
- Modify Poetry lock file to add platform-specific extras and dependency constraints
- Update Poetry version from 2.0.1 to 2.1.1
- Add conditional dependencies for different Python implementations and versions
- Refine package extras with more precise platform and version requirements
- Upgrade Hypothesis from 6.125.2 to 6.127.9
- Update setuptools from 75.8.0 to 75.8.2
- Add watchdog dependency to Hypothesis extras
- Refresh package hashes and version constraints
- Modify account ID replacement logic in summarizer
- Remove commented-out explanatory notes
- Streamline regex pattern for summary-balance specifications
- Reformat target argument definition for better readability
- Maintain existing target choices for KEVM CLI
@Stevengre Stevengre force-pushed the jh/summary-integration branch from 30a0dcc to 3423f02 Compare March 14, 2025 18:00
- Reduced timeout for the 'Summarization' test suite from 150 to 30 seconds.
- Updated dependencies: filelock to version 3.18.0, hypothesis to version 6.129.1, and setuptools to version 76.0.0 in poetry.lock.
…from summarizer and summary.k files to address potential issues with log2 calculations.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants