Skip to content

Support for Autoregressive generation with LLMs  #1136

Open
@apresunreve

Description

@apresunreve

Hi, I was able to run llama for a single forward, but when I tried to make it generate texts in an autoregressive way, there were errors with input shapes. Looks like a fixed sized input is set when exporting the model (correct me if I'm wrong). I would like to know if LLM text generation is currently supported. Thanks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions