Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accessing data in requester pays bucket #1435

Open
tomlimi opened this issue Nov 11, 2023 · 0 comments
Open

Accessing data in requester pays bucket #1435

tomlimi opened this issue Nov 11, 2023 · 0 comments

Comments

@tomlimi
Copy link

tomlimi commented Nov 11, 2023

I'm trying to pretrain mt5 model on mc4 data available at the requester pays bucket, following the example:

TFDS_DATA_DIR="gs://allennlp-tensorflow-datasets/c4/multilingual/3.0.1"

python3 ${T5X_DIR}/t5x/train.py \
  --gin_file="${T5X_DIR}/t5x/examples/pretrain_mt5_mc4.gin" \
  --gin.MODEL_DIR="'${MODEL_DIR}'" \
  --tfds_data_dir=${TFDS_DATA_DIR}

I get the following error caused because I cannot authenticate my project. How should I add my project ID?

tensorflow.python.framework.errors_impl.InvalidArgumentError: Error executing an HTTP request: HTTP response code 400 with body '{
  "error": {
    "code": 400,
    "message": "Bucket is a requester pays bucket but no user project provided.",
    "errors": [
      {
        "message": "Bucket is a requester pays bucket but no user project provided.",
        "domain": "global",
        "reason": "required"
      }
    ]
  }
}
@tomlimi tomlimi changed the title Accessing data from requester pays bucket Accessing data in requester pays bucket Nov 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant