Skip to content

Commit

Permalink
feature: events deadlock detection in validation layer
Browse files Browse the repository at this point in the history
Related-To: NEO-12810

Signed-off-by: Chandio, Bibrak Qamar <[email protected]>
  • Loading branch information
bibrak committed Nov 12, 2024
1 parent 3091196 commit abde7db
Show file tree
Hide file tree
Showing 8 changed files with 13 additions and 744 deletions.
8 changes: 5 additions & 3 deletions source/layers/validation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ By default, no validation modes will be enabled. The individual validation modes

- `ZE_ENABLE_PARAMETER_VALIDATION`
- `ZE_ENABLE_HANDLE_LIFETIME`
- `ZEL_ENABLE_EVENTSDEADLOCK_CHECKER`
- `ZEL_ENABLE_EVENTS_CHECKER`
- `ZE_ENABLE_MEMORY_TRACKER` (Not yet Implemeneted)
- `ZE_ENABLE_THREADING_VALIDATION` (Not yet Implemeneted)

Expand Down Expand Up @@ -49,9 +49,11 @@ This mode maintains an internal mapping of each handle type to a state structure
- Additional per handle state checks added as needed
- Example - Check ze_cmdlist_handle_t open or closed

### `ZEL_ENABLE_EVENTSDEADLOCK_CHECKER`
### `ZEL_ENABLE_EVENTS_CHECKER`

The Events Deadlock Checker is designed to detect potential deadlocks that might occur due to improper event usage in the Level Zero API. It prints out wairning messages for user when it detects a potential deadlock.
The Events Checker validates usage of events.
- It is designed to detect potential deadlocks that might occur due to improper event usage in the Level Zero API. It prints out warning messages for user when it detects a potential deadlock.
- In some cases it may also detect whether an event is being used more than once without being reset. Consider a case in which a single event is signaled from twice.

### `ZE_ENABLE_THREADING_VALIDATION` (Not yet Implemeneted)

Expand Down
2 changes: 1 addition & 1 deletion source/layers/validation/checkers/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
add_subdirectory(events_deadlock)
add_subdirectory(events_checker)
add_subdirectory(parameter_validation)
add_subdirectory(template)
12 changes: 0 additions & 12 deletions source/layers/validation/checkers/events_deadlock/CMakeLists.txt

This file was deleted.

41 changes: 0 additions & 41 deletions source/layers/validation/checkers/events_deadlock/DESIGN.md

This file was deleted.

This file was deleted.

This file was deleted.

6 changes: 3 additions & 3 deletions test/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ set_property(TEST tests_both_gpu PROPERTY ENVIRONMENT "ZE_ENABLE_NULL_DRIVER=1")
add_test(NAME tests_both_npu COMMAND tests --gtest_filter=*GivenLevelZeroLoaderPresentWhenCallingzeInitThenZeInitDriversThenBothCallsSucceedWithNPUTypes*)
set_property(TEST tests_both_npu PROPERTY ENVIRONMENT "ZE_ENABLE_NULL_DRIVER=1")
add_test(NAME tests_event_deadlock COMMAND tests --gtest_filter=*GivenLevelZeroLoaderPresentWhenCallingzeCommandListAppendMemoryCopyWithCircularDependencyOnEventsThenValidationLayerPrintsWarningOfDeadlock*)
set_property(TEST tests_event_deadlock PROPERTY ENVIRONMENT "ZE_ENABLE_NULL_DRIVER=1;ZE_ENABLE_VALIDATION_LAYER=1;ZEL_ENABLE_EVENTSDEADLOCK_CHECKER=1")
set_property(TEST tests_event_deadlock PROPERTY ENVIRONMENT "ZE_ENABLE_NULL_DRIVER=1;ZE_ENABLE_VALIDATION_LAYER=1;ZEL_ENABLE_EVENTS_CHECKER=1")
add_test(NAME tests_event_deadlock_reset COMMAND tests --gtest_filter=*GivenLevelZeroLoaderPresentWhenCallingzeCommandListAppendMemoryCopyWithCircularDependencyOnEventsAndExplicitCallzeEventHostSignalThenValidationLayerPrintsWarningOfIllegalUsage*)
set_property(TEST tests_event_deadlock_reset PROPERTY ENVIRONMENT "ZE_ENABLE_NULL_DRIVER=1;ZE_ENABLE_VALIDATION_LAYER=1;ZEL_ENABLE_EVENTSDEADLOCK_CHECKER=1")
set_property(TEST tests_event_deadlock_reset PROPERTY ENVIRONMENT "ZE_ENABLE_NULL_DRIVER=1;ZE_ENABLE_VALIDATION_LAYER=1;ZEL_ENABLE_EVENTS_CHECKER=1")
add_test(NAME tests_event_reset_reuse COMMAND tests --gtest_filter=*GivenLevelZeroLoaderPresentWhenCallingzeEventHostResetWithAlreadySignaledEventThenUsingEventAgainThenValidationLayerDoesNotPrintsWarningOfIllegalUsage*)
set_property(TEST tests_event_reset_reuse PROPERTY ENVIRONMENT "ZE_ENABLE_NULL_DRIVER=1;ZE_ENABLE_VALIDATION_LAYER=1;ZEL_ENABLE_EVENTSDEADLOCK_CHECKER=1")
set_property(TEST tests_event_reset_reuse PROPERTY ENVIRONMENT "ZE_ENABLE_NULL_DRIVER=1;ZE_ENABLE_VALIDATION_LAYER=1;ZEL_ENABLE_EVENTS_CHECKER=1")
7 changes: 4 additions & 3 deletions third_party/xla/graphcycles.cc
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ int32_t GraphCycles::NewNode() {
if (rep_->free_nodes_.empty()) {
Node n;
n.visited = false;
n.rank = rep_->nodes_.size();
n.rank = static_cast<int32_t>(rep_->nodes_.size());
rep_->nodes_.emplace_back(n);
rep_->node_io_.emplace_back();
rep_->node_data_.push_back(nullptr);
Expand Down Expand Up @@ -522,8 +522,9 @@ std::vector<int32_t> GraphCycles::AllNodesInPostOrder() const {
all_nodes.reserve(rep_->nodes_.size() - free_nodes_set.size());

for (int64_t i = 0, e = rep_->nodes_.size(); i < e; i++) {
if (!contains(free_nodes_set, i)) {
all_nodes.push_back(i);
int32_t index = static_cast<int32_t>(i);
if (!contains(free_nodes_set, index)) {
all_nodes.push_back(index);
}
}

Expand Down

0 comments on commit abde7db

Please sign in to comment.