Deprecate TransformerEvalWrapper, LMEvalInputRecorder

[namgyu-youn]

Summary:
Calibration-based PTQ methods like AWQ and GPTQ have been used TransformerEvalWrapper, which wraps ao model to run lm-eval. Instead, we want to run lm-eval directly, by using HFLM. The change log can be summarized to

# Old
from torchao._models._eval import TransformerEvalWrapper

TransformerEvalWrapper(
    model=model,
    tokenizer=tokenizer,
    max_seq_length=max_seq_length,
).run_eval(
    tasks=tasks,
    limit=calibration_limit,
)

# New (direct lm-eval)
from lm_eval import evaluator
from lm_eval.models.huggingface import HFLM

evaluator.simple_evaluate(
    HFLM(pretrained=model, tokenizer=tokenizer),
    tasks=tasks,
    limit=calibration_limit,
    batch_size=1,
)

Test plan:

python quantize_and_upload.py --model_id meta-llama/Llama-3.2-1B --quant AWQ-INT4

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3617

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[namgyu-youn]

@pytorchbot label "topic: deprecation"

[jerryzh168]

oh does this work with gemma3 as well? I remember I need to make some modifications before: 2a98f58

[jerryzh168]

code examples?

[namgyu-youn]

I didn't consider example code because: 1) not familiar with GPTQ workflow, 2) GPTQ api migration is work in progress at #3517. How about focusing on migration instead?

[namgyu-youn]

Updated the code example for temporary filling up, please take a look.

[namgyu-youn]

oh does this work with gemma3 as well? I remember I need to make some modifications before: 2a98f58

Not yet, but isn't it lm-eval issue? Should we update this opaque wrapper whenever architecture issue happen?