[feat]: Adding azure NLP graders and python grader

[yeshsurya]

This pull request introduces significant enhancements to the reward scoring and grading functionality within the ACPT-RFT environment. The main updates include the addition of new modular grader scripts for Azure-based and Python-based grading, an extensible reward scoring entry point, and several dependency updates and Dockerfile improvements to support these features.

Reward scoring and grading enhancements:

• Added a new default_compute_score function in __init__.py to serve as a unified entry point for reward scoring across multiple data sources, supporting both legacy and new grader modules.
• Introduced azure_grader.py, a flexible grader supporting both string matching and text similarity metrics (BLEU, ROUGE, METEOR, etc.), with fallback logic if the HuggingFace evaluate library is unavailable.
• Added azure_python_grader.py, a customizable Python code grader that validates syntax using AST parsing, intended for user extension and custom grading logic.

Dockerfile and dependency updates:

• Upgraded verl from version 0.6.0 to 0.6.1, and updated vllm from 0.11.1 to 0.13.0; added new dependencies openai and DeepGEMM, and copied new grader scripts into the appropriate locations in the Docker image. [1] [2]
• Ensured new grader modules (azure_grader.py, azure_python_grader.py, and updated __init__.py) are available in the verl.utils.reward_score package by copying them in the Docker build.

These changes collectively provide a more modular, extensible, and robust reward scoring framework, enabling easier customization and improved evaluation capabilities for different data sources and grading requirements.

Test links

• Custom Grader
• Python Grader
• Experiment with other nlp graders tests

[github-actions]

Test Results for assets-test

0 tests   0 ✅  0s ⏱️
0 suites  0 💤
0 files    0 ❌

Results for commit 3388c1e.

♻️ This comment has been updated with latest results.