Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Confidence is not returned in the decoding example? #783

Open
liuzhuang1024 opened this issue Dec 26, 2023 · 0 comments
Open

Confidence is not returned in the decoding example? #783

liuzhuang1024 opened this issue Dec 26, 2023 · 0 comments
Labels
bug Something isn't working

Comments

@liuzhuang1024
Copy link

liuzhuang1024 commented Dec 26, 2023

Branch/Tag/Commit

main

Docker Image Version

nvcr.io/nvidia/pytorch:22.08-py3

GPU name

A100

CUDA Driver

470.161.03

Reproduced Steps

In src/fastertransformer/models/decoding/Decoding.h:134

    Decoding(Decoding<T> const& Decoding);

    ~Decoding();

    void forward(std::vector<Tensor>*       output_tensors,
                 const std::vector<Tensor>* input_tensors,
                 const DecodingWeight<T>*   Decoding_weights);

No corresponding confidence information is sent from this place?

@liuzhuang1024 liuzhuang1024 added the bug Something isn't working label Dec 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant