Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

XFAIL failing tflite test #19123

Closed
wants to merge 2 commits into from
Closed

XFAIL failing tflite test #19123

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
895af6f
XFAIL failining tflite test
nirvedhmeshram Nov 12, 2024
88ea156
add new line
nirvedhmeshram Nov 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 13 additions & 13 deletions integrations/tensorflow/test/iree_tfl_tests/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,22 +6,22 @@ update_tflite_model_documentation.py to update this table.

| Model | Status |
| ------------------ | ------------------ |
vulkan_mobilenet_v1 | PASS ✓
llvmcpu_mobilenet_v3-large_uint8 | FAIL ✗
vmvx_mobilebert_tf2_quant | PASS ✓
llvmcpu_mobilebert_tf2_quant | PASS ✓
vulkan_mobilenet_v3-large_uint8 | FAIL ✗
person_detect | PASS ✓
cartoon_gan | PASS ✓
east_text_detector | PASS ✓
vulkan_resnet_50_int8 | FAIL ✗
east_text_detector | FAIL ✗
vmvx_person_detect | PASS ✓
gpt2 | PASS ✓
llvmcpu_mobilebert_tf2_quant | PASS ✓
llvmcpu_mobilenet_v1 | PASS ✓
llvmcpu_mobilenet_v3-large_uint8 | FAIL ✗
llvmcpu_posenet_i8 | FAIL ✗
mobilenet_v3 | PASS ✓
llvmcpu_resnet_50_int8 | PASS ✓
llvmcpu_posenet_i8 | FAIL ✗
mnasnet | PASS ✓
mobilenet_v3 | PASS ✓
person_detect | PASS ✓
vmvx_mobilebert_tf2_quant | PASS ✓
vulkan_posenet_i8 | FAIL ✗
vmvx_mobilenet_v3-large_uint8 | FAIL ✗
vmvx_person_detect | PASS ✓
vulkan_mobilebert_tf2_quant | FAIL ✗
vulkan_mobilenet_v1 | PASS ✓
vulkan_mobilenet_v3-large_uint8 | FAIL ✗
vulkan_posenet_i8 | FAIL ✗
vulkan_resnet_50_int8 | FAIL ✗
llvmcpu_mobilenet_v1 | FAIL ✗
1 change: 1 addition & 0 deletions integrations/tensorflow/test/iree_tfl_tests/llvmcpu_mobilenet_v1.run
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
# REQUIRES: llvmcpu
# RUN: %PYTHON -m iree_tfl_tests.mobilenet_v1_test --target_backend=llvmcpu --artifacts_dir=%t
# XFAIL: *
Comment on lines 1 to +3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While XFAIL is generally useful, I would actually exclude the test here with REQUIRES, using this pattern:

iree/integrations/tensorflow/test/iree_tfl_tests/cartoon_gan.run

Lines 1 to 3 in e0cd5b8

# REQUIRES: bugfix
# tfhub/kaggle broke their download endpoint, update URLs
# RUN: %PYTHON -m iree_tfl_tests.cartoon_gan_test --artifacts_dir=%t

The specific file this test is trying to download is

iree/integrations/tensorflow/test/python/iree_tfl_tests/mobilenet_v1_test.py

Line 11 in e0cd5b8

model_path = "https://storage.googleapis.com/iree-model-artifacts/tflite-integration-tests/mobilenet_v1.tflite"

That URL is returning an error. We have a tracking issue covering migrating iree-model-artifacts GCP files to other locations: #18518 .

I'd rather keep the test disabled until we take intentional action to move the file, instead of XFAIL it and then have the test maybe start unexpectedly passing if the URL becomes live again.

2 changes: 1 addition & 1 deletion integrations/tensorflow/test/iree_tfl_tests/update_tflite_model_documentation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ def main():
"| Model | Status |\n"
"| ------------------ | ------------------ |\n"
)
tflite_model_documentation.write(create_markdown_table(models))
tflite_model_documentation.write(create_markdown_table(models) + "\n")

new_lines = read_file(readme_file_path)
if new_lines == old_lines:
Expand Down
Loading