[Thinkit/Tests] Changes to support arbitration test

[sreeiyer-1]

This PR contains the changes required in sonic-pins for the arbitration_test to pass.

Four command line flags are added to bypass the gNMI calls that are not yet supported.

bazel test //tests/thinkit:arbitration_test --define=absl=1 --test_strategy=standalone --test_arg=--gnmi_deviceid_support=false --test_arg=--gnmi_push_support=false --test_arg=--gnmi_boottime_support=false --test_arg=--gnmi_get_config_support=false --test_arg=--pins_p4info_file="/data/sonic-pins/sai_p4/instantiations/google/middleblock.p4info.pb.txt"

gnmi_deviceid_support - device id/node-id
gnmi_push_support - gnmi push
gnmi_boottime_support - get boot time
gnmi_get_config_support - gnmi get type 'CONFIG' is supported

By default, all four flags are true. For the test to pass, all of them should be set to false. They can be gradually removed as and when each feature is supported in gNMI.

Some of the fields/attributes in acl_ingress.p4 are not supported in p4orch at the moment. These are tagged as "@unsupported" and the middleblock.p4info.pb.txt regenerated.

Keyword check

~/bin/keyword_check.sh ../pins.diff
Keyword check Passed.

Arbitration test

bazel test //tests/thinkit:arbitration_test --define=absl=1 --test_strategy=standalone --test_arg=--gnmi_deviceid_support=false --test_arg=--gnmi_push_support=false --test_arg=--gnmi_boottime_support=false --test_arg=--gnmi_get_config_support=false --test_arg=--pins_p4info_file="/data/sonic-pins/sai_p4/instantiations/google/middleblock.p4info.pb.txt"
INFO: Analyzed target //tests/thinkit:arbitration_test (297 packages loaded, 22856 targets configured).
INFO: Found 1 test target...
Target //tests/thinkit:arbitration_test up-to-date:
bazel-bin/tests/thinkit/arbitration_test
INFO: Elapsed time: 52.145s, Critical Path: 49.21s
INFO: 2 processes: 1 internal, 1 local.
INFO: Build completed successfully, 2 total actions
//tests/thinkit:arbitration_test PASSED in 47.6s

Executed 1 out of 1 test: 1 test passes.

Build and test results

bazel build $BAZEL_BUILD_OPTS ...

INFO: Analyzed 774 targets (0 packages loaded, 0 targets configured).
INFO: Found 774 targets...
INFO: Elapsed time: 0.541s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action

bazel test $BAZEL_BUILD_OPTS --cache_test_results=no ...
INFO: Analyzed 774 targets (0 packages loaded, 390 targets configured).
INFO: Found 538 targets and 236 test targets...
INFO: Elapsed time: 205.593s, Critical Path: 141.10s
INFO: 293 processes: 346 linux-sandbox, 19 local.
INFO: Build completed successfully, 293 total actions
//dvaas:dataplane_validation_diff_test PASSED in 0.3s
//dvaas:dataplane_validation_test PASSED in 0.3s
//dvaas:dvaas_detective_diff_test PASSED in 0.3s
//dvaas:dvaas_detective_test PASSED in 0.2s
//dvaas:port_id_map_test PASSED in 2.4s
//dvaas:test_run_validation_golden_test PASSED in 0.5s
//dvaas:test_run_validation_test PASSED in 2.8s
//dvaas:test_run_validation_test_runner PASSED in 0.1s
//dvaas:test_vector_stats_diff_test PASSED in 0.3s
//dvaas:test_vector_stats_test PASSED in 0.2s
//dvaas:test_vector_test PASSED in 2.2s
//dvaas:user_provided_packet_test_vector_diff_test PASSED in 0.5s
//dvaas:user_provided_packet_test_vector_test PASSED in 0.5s
//gutil:collections_test PASSED in 1.6s
//gutil:io_test PASSED in 1.4s
//gutil:proto_matchers_test PASSED in 1.8s
//gutil:proto_ordering_test PASSED in 1.9s
//gutil:proto_test PASSED in 1.6s
//gutil:status_matchers_test PASSED in 1.7s
//gutil:test_artifact_writer_test PASSED in 1.8s
//gutil:testing_test PASSED in 1.6s
//gutil:timer_test PASSED in 5.1s
//gutil:version_test PASSED in 4.1s
//lib:basic_switch_test PASSED in 2.4s
//lib:ixia_helper_test PASSED in 2.6s
//lib:ixia_protocol_helper_test PASSED in 2.4s
//lib:lacp_ixia_helper_test PASSED in 1.9s
//lib/basic_traffic:basic_p4rt_util_test PASSED in 2.6s
//lib/basic_traffic:basic_traffic_test PASSED in 17.3s
//lib/gnmi:gnmi_helper_test PASSED in 4.7s
//lib/gnoi:gnoi_helper_test PASSED in 1.2s
//lib/p4rt:p4rt_port_test PASSED in 1.5s
//lib/p4rt:p4rt_programming_context_test PASSED in 2.5s
//lib/utils:generic_testbed_utils_test PASSED in 3.4s
//lib/utils:json_test_utils_test PASSED in 1.4s
//lib/utils:json_utils_test PASSED in 1.5s
//lib/validator:validator_backend_test PASSED in 1.7s
//lib/validator:validator_lib_test PASSED in 27.4s
//p4_fuzzer:constraints_test PASSED in 9.8s
//p4_fuzzer:fuzz_util_test PASSED in 141.0s
//p4_fuzzer:fuzzer_config_test PASSED in 2.2s
//p4_fuzzer:oracle_util_test PASSED in 4.9s
//p4_fuzzer:switch_state_assert_entry_equality_test PASSED in 4.1s
//p4_fuzzer:switch_state_assert_entry_equality_test_runner PASSED in 3.7s
//p4_fuzzer:switch_state_test PASSED in 2.1s
//p4_pdpi:built_ins_test PASSED in 1.6s
//p4_pdpi:entity_keys_test PASSED in 1.5s
//p4_pdpi:ir_properties_test PASSED in 1.6s
//p4_pdpi:ir_to_ir_test PASSED in 1.5s
//p4_pdpi:ir_tools_test PASSED in 1.9s
//p4_pdpi:names_test PASSED in 2.0s
//p4_pdpi:p4_runtime_matchers_test PASSED in 1.7s
//p4_pdpi:p4info_union_test PASSED in 2.5s
//p4_pdpi:reference_annotations_test PASSED in 3.2s
//p4_pdpi:references_test PASSED in 2.5s
//p4_pdpi:sequencing_util_test PASSED in 2.5s
//p4_pdpi:ternary_test PASSED in 2.1s
//p4_pdpi/netaddr:ipv4_address_and_network_address_test PASSED in 1.5s
//p4_pdpi/netaddr:ipv6_address_test PASSED in 1.3s
//p4_pdpi/netaddr:mac_address_test PASSED in 1.7s
//p4_pdpi/packetlib:packetlib_debugging_test PASSED in 1.3s
//p4_pdpi/packetlib:packetlib_fuzzer_test PASSED in 20.5s
//p4_pdpi/packetlib:packetlib_matchers_test PASSED in 1.7s
//p4_pdpi/packetlib:packetlib_test PASSED in 0.3s
//p4_pdpi/packetlib:packetlib_test_runner PASSED in 0.2s
//p4_pdpi/packetlib:packetlib_unit_test PASSED in 6.0s
//p4_pdpi/string_encodings:bit_string_test PASSED in 1.9s
//p4_pdpi/string_encodings:byte_string_test PASSED in 1.4s
//p4_pdpi/string_encodings:decimal_string_test PASSED in 0.2s
//p4_pdpi/string_encodings:decimal_string_test_runner PASSED in 0.2s
//p4_pdpi/string_encodings:hex_string_test PASSED in 0.2s
//p4_pdpi/string_encodings:hex_string_test_runner PASSED in 0.1s
//p4_pdpi/string_encodings:readable_byte_string_test PASSED in 1.4s
//p4_pdpi/testing:helper_function_test PASSED in 1.9s
//p4_pdpi/testing:info_test PASSED in 0.3s
//p4_pdpi/testing:info_test_runner PASSED in 0.3s
//p4_pdpi/testing:main_pd_test PASSED in 0.1s
//p4_pdpi/testing:mock_p4_runtime_server_test PASSED in 1.8s
//p4_pdpi/testing:packet_io_test PASSED in 0.2s
//p4_pdpi/testing:packet_io_test_runner PASSED in 0.2s
//p4_pdpi/testing:references_test PASSED in 0.2s
//p4_pdpi/testing:references_test_runner PASSED in 0.3s
//p4_pdpi/testing:rpc_test PASSED in 0.4s
//p4_pdpi/testing:rpc_test_runner PASSED in 0.3s
//p4_pdpi/testing:sequencing_test PASSED in 0.3s
//p4_pdpi/testing:sequencing_test_runner PASSED in 0.3s
//p4_pdpi/testing:sequencing_util_test PASSED in 0.3s
//p4_pdpi/testing:sequencing_util_test_runner PASSED in 0.3s
//p4_pdpi/testing:table_entry_gunit_test PASSED in 2.0s
//p4_pdpi/testing:table_entry_test PASSED in 0.3s
//p4_pdpi/testing:table_entry_test_runner PASSED in 0.3s
//p4_pdpi/utils:annotation_parser_test PASSED in 1.7s
//p4_pdpi/utils:ir_test PASSED in 1.7s
//p4_symbolic:z3_util_test PASSED in 2.5s
//p4_symbolic/bmv2:ipv4_routing_exact_test PASSED in 0.2s
//p4_symbolic/bmv2:ipv4_routing_subset_test PASSED in 0.3s
//p4_symbolic/bmv2:port_hardcoded_exact_test PASSED in 0.1s
//p4_symbolic/bmv2:port_hardcoded_subset_test PASSED in 0.4s
//p4_symbolic/bmv2:port_table_exact_test PASSED in 0.2s
//p4_symbolic/bmv2:port_table_subset_test PASSED in 0.3s
//p4_symbolic/bmv2:reflector_exact_test PASSED in 0.1s
//p4_symbolic/bmv2:reflector_subset_test PASSED in 0.3s
//p4_symbolic/bmv2:set_invalid_exact_test PASSED in 0.0s
//p4_symbolic/bmv2:set_invalid_subset_test PASSED in 0.2s
//p4_symbolic/bmv2:table_hit_1_exact_test PASSED in 0.2s
//p4_symbolic/bmv2:table_hit_1_subset_test PASSED in 0.2s
//p4_symbolic/bmv2:table_hit_2_exact_test PASSED in 0.1s
//p4_symbolic/bmv2:table_hit_2_subset_test PASSED in 0.3s
//p4_symbolic/ir:cfg_test PASSED in 2.2s
//p4_symbolic/ir:complex_conditional_test PASSED in 0.1s
//p4_symbolic/ir:conditional_sequence_test PASSED in 0.1s
//p4_symbolic/ir:conditional_test PASSED in 0.1s
//p4_symbolic/ir:default_transition_test PASSED in 0.1s
//p4_symbolic/ir:extract_parser_operation_test PASSED in 0.2s
//p4_symbolic/ir:fall_through_transition_test PASSED in 0.2s
//p4_symbolic/ir:hex_string_transition_test PASSED in 0.2s
//p4_symbolic/ir:ipv4_routing_test PASSED in 0.1s
//p4_symbolic/ir:partial_icmp_test PASSED in 0.1s
//p4_symbolic/ir:port_hardcoded_test PASSED in 0.1s
//p4_symbolic/ir:port_table_test PASSED in 0.2s
//p4_symbolic/ir:primitive_parser_operation_test PASSED in 0.1s
//p4_symbolic/ir:reflector_test PASSED in 0.1s
//p4_symbolic/ir:sai_parser_test PASSED in 0.0s
//p4_symbolic/ir:set_invalid_test PASSED in 0.1s
//p4_symbolic/ir:set_parser_operation_test PASSED in 0.2s
//p4_symbolic/ir:string_optional_test PASSED in 0.2s
//p4_symbolic/ir:table_hit_1_test PASSED in 0.1s
//p4_symbolic/ir:table_hit_2_test PASSED in 0.1s
//p4_symbolic/packet_synthesizer:packet_synthesis_criteria_test PASSED in 2.0s
//p4_symbolic/sai:criteria_generator_test PASSED in 37.2s
//p4_symbolic/sai:packet_synthesizer_test PASSED in 42.7s
//p4_symbolic/sai:sai_test PASSED in 9.9s
//p4_symbolic/symbolic:conditional_sequence_test_packets PASSED in 0.0s
//p4_symbolic/symbolic:condtional_test_packets PASSED in 0.2s
//p4_symbolic/symbolic:deparser_test PASSED in 4.8s
//p4_symbolic/symbolic:parser_test PASSED in 2.5s
//p4_symbolic/symbolic:port_hardcoded_test_packets PASSED in 0.1s
//p4_symbolic/symbolic:port_table_test_packets PASSED in 0.2s
//p4_symbolic/symbolic:symbolic_table_entry_test PASSED in 2.8s
//p4_symbolic/symbolic:util_test PASSED in 2.9s
//p4_symbolic/symbolic:values_test PASSED in 2.2s
//p4_symbolic/tests:sai_p4_integration_test PASSED in 22.2s
//p4_symbolic/tests:symbolic_table_entries_test PASSED in 4.0s
//p4rt_app/event_monitoring:app_state_db_port_table_event_test PASSED in 3.4s
//p4rt_app/event_monitoring:app_state_db_send_to_ingress_port_table_event_test PASSED in 3.9s
//p4rt_app/event_monitoring:config_db_node_cfg_table_event_test PASSED in 3.0s
//p4rt_app/event_monitoring:config_db_port_table_event_test PASSED in 3.0s
//p4rt_app/event_monitoring:config_db_queue_table_event_test PASSED in 3.4s
//p4rt_app/event_monitoring:debug_data_dump_events_test PASSED in 3.2s
//p4rt_app/event_monitoring:state_verification_events_test PASSED in 3.5s
//p4rt_app/p4runtime:ir_translation_test PASSED in 2.9s
//p4rt_app/p4runtime:p4info_reconcile_test PASSED in 2.9s
//p4rt_app/p4runtime:p4info_verification_schema_test PASSED in 3.0s
//p4rt_app/p4runtime:p4info_verification_test PASSED in 3.3s
//p4rt_app/p4runtime:packetio_helpers_test PASSED in 3.8s
//p4rt_app/p4runtime:queue_translator_test PASSED in 2.1s
//p4rt_app/p4runtime:resource_utilization_test PASSED in 2.8s
//p4rt_app/sonic:app_db_acl_def_table_manager_test PASSED in 2.7s
//p4rt_app/sonic:app_db_manager_test PASSED in 3.4s
//p4rt_app/sonic:app_db_to_pdpi_ir_translator_test PASSED in 3.1s
//p4rt_app/sonic:hashing_test PASSED in 2.1s
//p4rt_app/sonic:packet_replication_entry_translation_test PASSED in 2.6s
//p4rt_app/sonic:packetio_impl_test PASSED in 2.4s
//p4rt_app/sonic:packetio_port_test PASSED in 2.1s
//p4rt_app/sonic:response_handler_test PASSED in 2.6s
//p4rt_app/sonic:state_verification_test PASSED in 2.4s
//p4rt_app/sonic:swss_utils_test PASSED in 2.3s
//p4rt_app/sonic:vlan_entry_translation_test PASSED in 3.0s
//p4rt_app/sonic:vrf_entry_translation_test PASSED in 2.4s
//p4rt_app/sonic/adapters:fake_sonic_db_table_test PASSED in 1.0s
//p4rt_app/sonic/adapters:fake_warm_boot_state_adapter_test PASSED in 0.3s
//p4rt_app/tests:acl_table_test PASSED in 2.2s
//p4rt_app/tests:action_set_test PASSED in 1.6s
//p4rt_app/tests:api_access_test PASSED in 1.3s
//p4rt_app/tests:arbitration_test PASSED in 1.5s
//p4rt_app/tests:cpu_queue_name_and_id_test PASSED in 2.6s
//p4rt_app/tests:debug_data_dump_test PASSED in 1.6s
//p4rt_app/tests:fixed_l3_tables_test PASSED in 4.5s
//p4rt_app/tests:forwarding_pipeline_config_test PASSED in 10.7s
//p4rt_app/tests:front_panel_queue_name_and_id_test PASSED in 1.6s
//p4rt_app/tests:grpc_behavior_test PASSED in 5.6s
//p4rt_app/tests:p4_constraints_test PASSED in 1.3s
//p4rt_app/tests:p4_constraints_test_runner PASSED in 0.5s
//p4rt_app/tests:p4_programs_test PASSED in 2.4s
//p4rt_app/tests:packet_replication_table_test PASSED in 3.4s
//p4rt_app/tests:packetio_test PASSED in 4.6s
//p4rt_app/tests:port_name_and_id_test PASSED in 4.3s
//p4rt_app/tests:resource_limits_test PASSED in 6.3s
//p4rt_app/tests:response_path_test PASSED in 5.4s
//p4rt_app/tests:role_test PASSED in 1.7s
//p4rt_app/tests:state_verification_test PASSED in 3.0s
//p4rt_app/tests:vrf_table_test PASSED in 2.5s
//p4rt_app/tests/lib:app_db_entry_builder_test PASSED in 0.2s
//p4rt_app/utils:event_data_tracker_test PASSED in 0.3s
//p4rt_app/utils:table_utility_test PASSED in 2.7s
//sai_p4/instantiations/google:clos_stage_test PASSED in 2.0s
//sai_p4/instantiations/google:fabric_border_router_p4info_up_to_date_test PASSED in 0.2s
//sai_p4/instantiations/google:middleblock_p4info_up_to_date_test PASSED in 0.0s
//sai_p4/instantiations/google:preprocessed_sai_programs_test PASSED in 0.3s
//sai_p4/instantiations/google:sai_nonstandard_platforms_build_test PASSED in 0.1s
//sai_p4/instantiations/google:sai_nonstandard_platforms_cc_test PASSED in 2.3s
//sai_p4/instantiations/google:sai_p4info_fetcher_test PASSED in 2.5s
//sai_p4/instantiations/google:sai_p4info_test PASSED in 3.7s
//sai_p4/instantiations/google:sai_pd_proto_test PASSED in 0.0s
//sai_p4/instantiations/google:sai_pd_util_test PASSED in 2.2s
//sai_p4/instantiations/google:tor_p4info_up_to_date_test PASSED in 0.3s
//sai_p4/instantiations/google:union_p4info_up_to_date_test PASSED in 0.5s
//sai_p4/instantiations/google:wbb_p4info_up_to_date_test PASSED in 0.2s
//sai_p4/instantiations/google/test_tools:table_entry_generator_helper_test PASSED in 6.7s
//sai_p4/instantiations/google/test_tools:test_entries_test PASSED in 4.4s
//sai_p4/instantiations/google/tests:p4_fuzzer_integration_test PASSED in 9.6s
//sai_p4/tools:auxiliary_entries_for_v1model_targets_test PASSED in 3.2s
//sai_p4/tools:p4info_tools_test PASSED in 2.7s
//sai_p4/tools:packetio_tools_test PASSED in 2.3s
//tests:thinkit_gnmi_interface_util_tests PASSED in 3.5s
//tests/forwarding:hash_statistics_util_test PASSED in 2.8s
//tests/integration/system/nsf:compare_p4flows_test PASSED in 2.0s
//tests/lib:common_ir_table_entries_test PASSED in 2.6s
//tests/lib:p4rt_fixed_table_programming_helper_test PASSED in 3.1s
//tests/lib:switch_test_setup_helpers_golden_test PASSED in 0.4s
//tests/lib:switch_test_setup_helpers_golden_test_runner PASSED in 0.4s
//tests/qos:gnmi_parsers_test PASSED in 0.3s
//tests/qos:gnmi_parsers_test_runner PASSED in 0.3s
//tests/sflow:sflow_util_test PASSED in 8.9s
//thinkit:bazel_test_environment_test PASSED in 1.8s
//thinkit:generic_testbed_test PASSED in 2.6s
//thinkit:mock_control_device_test PASSED in 1.9s
//thinkit:mock_generic_testbed_test PASSED in 2.4s
//thinkit:mock_mirror_testbed_test PASSED in 2.1s
//thinkit:mock_ssh_client_test PASSED in 0.2s
//thinkit:mock_switch_test PASSED in 2.6s
//thinkit:mock_test_environment_test PASSED in 0.3s
//thinkit:switch_test PASSED in 2.4s
//tests/lib:packet_generator_test PASSED in 65.9s
Stats over 4 runs: max = 65.9s, min = 64.0s, avg = 64.8s, dev = 0.7s
//sai_p4/instantiations/google/tests:p4_constraints_integration_test PASSED in 3.2s
Stats over 5 runs: max = 3.2s, min = 2.4s, avg = 2.7s, dev = 0.3s
//sai_p4/instantiations/google/test_tools:table_entry_generator_test PASSED in 47.6s
Stats over 50 runs: max = 47.6s, min = 1.7s, avg = 6.5s, dev = 11.8s

Executed 236 out of 236 tests: 236 tests pass.

[mssonicbld]

/azp run

[azure-pipelines]

No pipelines are associated with this pull request.

[mssonicbld]

/azp run

[azure-pipelines]

No pipelines are associated with this pull request.

[mssonicbld]

/azp run

[azure-pipelines]

No pipelines are associated with this pull request.

[mssonicbld]

/azp run

[azure-pipelines]

No pipelines are associated with this pull request.

[sreeiyer-1]

@kishanps - Rebased, added comments. table_entry_generator_test failed because the acl_ingress_qos_table referred to the unsupported set_qos_queue_and_cancel_copy_above_rate_limit (which was not happening before rebase). Moved acl_ingress_qos table under unsupported table, commented and retested.

Original error logs :

//sai_p4/instantiations/google/test_tools:table_entry_generator_test     FAILED in 1 out of 50 in 46.1s
  Stats over 50 runs: max = 46.1s, min = 1.8s, avg = 6.2s, dev = 11.7s


Actual: INVALID_ARGUMENT: Table entry for 'acl_ingress_qos_table' is invalid: Action 'set_qos_queue_and_cancel_copy_above_rate_limit' is invalid: Action has @unsupported annotation. (of type absl::lts_20240116::Status)
Google Test trace:
sai_p4/instantiations/google/test_tools/table_entry_generator_test.cc:116: Failed to generate entry at index 0
sai_p4/instantiations/google/test_tools/table_entry_generator_test.cc:122: Failure
Failed
pdpi::IrTableEntryToPi(ir_p4info, ir_table_entry) failed: INVALID_ARGUMENT: Table entry for 'acl_ingress_qos_table' is invalid: Action 'set_qos_queue_and_cancel_copy_above_rate_limit' is invalid: Action has @unsupported annotation.
Google Test trace:

Pass logs after the changes :

INFO: Elapsed time: 248.640s, Critical Path: 159.58s
INFO: 2714 processes: 5 internal, 2709 linux-sandbox.
INFO: Build completed successfully, 2714 total actions
sbiyer@57d205691b4a:/sonic/src/sonic-p4rt/sonic-pins$ pins_test  
INFO: Analyzed 778 targets (0 packages loaded, 391 targets configured).
INFO: Found 541 targets and 237 test targets...
INFO: Elapsed time: 216.935s, Critical Path: 153.23s
INFO: 294 processes: 347 linux-sandbox, 19 local.
INFO: Build completed successfully, 294 total actions
//dvaas:dataplane_validation_diff_test                                   PASSED in 0.3s

<snip>
//sai_p4/instantiations/google/test_tools:table_entry_generator_helper_test PASSED in 7.0s
<snip>
//tests/lib:packet_generator_test                                        PASSED in 66.0s
  Stats over 4 runs: max = 66.0s, min = 64.0s, avg = 65.0s, dev = 0.8s
//sai_p4/instantiations/google/tests:p4_constraints_integration_test     PASSED in 3.1s
  Stats over 5 runs: max = 3.1s, min = 1.9s, avg = 2.6s, dev = 0.4s
//sai_p4/instantiations/google/test_tools:table_entry_generator_test     PASSED in 45.4s
  Stats over 50 runs: max = 45.4s, min = 1.6s, avg = 5.4s, dev = 10.2s

Executed 237 out of 237 tests: 237 tests pass.

[mssonicbld]

/azp run

[azure-pipelines]

No pipelines are associated with this pull request.

[mssonicbld]

/azp run

[azure-pipelines]

No pipelines are associated with this pull request.

[sreeiyer-1]

Moving these changes to #1670