Skip to content

Improve gather_result, remove filter, fix inf loop in truncation #167

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 30 commits into from
Jul 24, 2025

Conversation

xingdi-eric-yuan
Copy link
Collaborator

@xingdi-eric-yuan xingdi-eric-yuan commented Jun 26, 2025

This pull request introduces several changes across multiple files, focusing on enhancing the analysis of agent behavior, simplifying the codebase, and removing obsolete functionality. The most significant updates include modifications to the analysis/gather_results.py file to improve data grouping and path extraction, the removal of the entire trajectory filtering module, and the addition of a new error handling case in the debug_gym environment.

Enhancements to analysis/gather_results.py:

  • Removed the map_uuid function, simplifying the handling of UUIDs.
  • Added logic to extract relative paths from log file paths and include them in the results dictionary.
  • Updated the data grouping logic to include the path field, enabling finer-grained analysis.
  • Enhanced success rate reporting to include the path and uuid in the output.

Removal of trajectory filtering module:

  • Deleted the README.md file for trajectory filtering, removing documentation for the module.
  • Removed the criteria.py file, which contained various filtering criteria for analyzing trajectories.
  • Deleted the filter.py file, which implemented trajectory filtering logic and command-line tools.

Updates to debug_gym environment:

  • Added a new error handling case in get_triggered_tools to return a descriptive message when no tool call is generated by the agent.
  • fixed inf loop issue in message truncation, added test cases.

@xingdi-eric-yuan xingdi-eric-yuan marked this pull request as ready for review July 9, 2025 18:01
@xingdi-eric-yuan xingdi-eric-yuan changed the title Improve filter [wip] Improve filter Jul 21, 2025
@xingdi-eric-yuan xingdi-eric-yuan changed the title [wip] Improve filter Improve filter Jul 21, 2025
@xingdi-eric-yuan xingdi-eric-yuan changed the title Improve filter Improve gather_result, remove filter Jul 21, 2025
@xingdi-eric-yuan xingdi-eric-yuan changed the title Improve gather_result, remove filter Improve gather_result, remove filter, fix inf loop in truncation Jul 21, 2025
@xingdi-eric-yuan xingdi-eric-yuan merged commit e20081f into main Jul 24, 2025
6 checks passed
@xingdi-eric-yuan xingdi-eric-yuan deleted the improve_filter branch July 24, 2025 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants