Feat/event data search merged #1106
Conversation
1. 新增事件字段筛选及策略相关接口与常量 2. 增加事件合流入库 doris - 增加事件 doris 合流表初始化 - 审计策略-规则审计，模型审计支持 doris 合流入库（已兼容历史策略） 3. 系统初始化 - 支持策略标签资源纳入权限模型并同步 IAM 初始化 - 增加事件相关资产反向拉取初始化 - 定时同步资产快照结果表并完善插件入库任务 - 移除风险模型标签冗余字段并优化 IAM Provider 4. 优化风险列表筛选条件,增加日志打印
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1106 +/- ##
==========================================
+ Coverage 77.23% 78.30% +1.07%
==========================================
Files 622 633 +11
Lines 27689 29301 +1612
==========================================
+ Hits 21385 22944 +1559
- Misses 6304 6357 +53 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting
| def _build_event_filter_condition(self, filter_item: Dict[str, Any], index: int) -> str: | ||
| alias = f"risk_event_{index}" | ||
| join_conditions = [f"{alias}.strategy_id = base_query.strategy_id"] | ||
|
|
||
| field_expression = self._build_event_field_expression(alias, filter_item) | ||
| if not field_expression: | ||
| return "" | ||
|
|
||
| comparison = self._build_event_filter_expression(field_expression, filter_item) | ||
| if not comparison: | ||
| return "" | ||
|
|
||
| join_clause = " AND ".join(join_conditions) | ||
| table_reference = self._get_risk_event_table_reference() |
There was a problem hiding this comment.
The BKBase event filter predicate only joins on strategy_id when determining whether a risk matches the supplied event field filters. If two risks originate from the same strategy but have different event payloads, the EXISTS clause will return true for all of them as soon as any event for that strategy satisfies the filter, because the subquery never correlates on the risk’s raw_event_id (or another risk-specific key). This means filtering by event fields can return unrelated risks. The join should include a risk-specific identifier (e.g. raw_event_id or risk_id) and the column must be selected in the base subquery so that each risk is filtered against its own event record.
Useful? React with 👍 / 👎.
1. 增加支持过滤的field的id。添加过滤operator支持。
1. 添加时间分区过滤条件。
1. 优化时间分区过滤条件。 feat: 新增关联事件字段搜索功能 --story=126826559 1. 使用sqllot重构sql生成逻辑，确保可用性。 2. 优化测试用例。
1. 简化sql语句，修正遗漏。 2. 优化测试用例。
1. 新增资源反向拉取 2. 修复 github 单测异常 3. 资产表反向拉取支持 JSON 字段 (cherry picked from commit eaffc55) Signed-off-by: raja <[email protected]>
1. 修正兼容性差的*写法。 (cherry picked from commit 1121f8f)
1. 添加doris后缀。
1. 新增工单权限表资源反向拉取的初始化 Signed-off-by: raja <[email protected]>
1. 兼容query某些情况下提前为空的情况。
1. 优化资产反向拉取支持 JSON 字段逻辑 Signed-off-by: raja <[email protected]> (cherry picked from commit afc3b21)
1. 修复事件表 json 字段入库配置 Signed-off-by: raja <[email protected]>
1. 优化资产反向拉取支持 JSON 字段逻 Signed-off-by: raja <[email protected]> (cherry picked from commit a633c8b)
1. 移除表标识符格式化中的反引号包裹。
1. 移除表标识符格式化中的反引号包裹。
1. 去掉多余的event risk_id对应。
1. 去掉多余的event risk_id对应。
1. 去掉多余的event risk_id对应。
1. 优化风险资产反向拉取时间判断逻辑 Signed-off-by: raja <[email protected]> (cherry picked from commit 3791db3)
1. 新增事件时间关联约束
1. 新增事件和风险raw_event_id约束
…into feat/event_data_search_merged
1. 修改事件时间戳条件并移除LIKE表达式转义
1. 优化模型审计策略更新逻辑，兼容 doris 存储节点变更 Signed-off-by: raja <[email protected]>
1. 新增数值过滤器测试及优化数值操作符处理逻辑
1. 支持工单权限表初始化 Signed-off-by: raja <[email protected]>
1. 新增重复字段处理逻辑及测试，优化类型与格式
1. 新增 BKBase SQL 记录与票权限表替换
3 participants
No description provided.