forked from tikv/tikv
-
Notifications
You must be signed in to change notification settings - Fork 9
Commit fbd13ee
Cherry pick remaining commits from raftstore-proxy-6.1 to raftstore-proxy (#72)
* ctl: validate column family name when scanning data using ctl (#11269)
* ctl: should check column name validation when scan data using ctl
close #11268
Signed-off-by: yuqi1129 <[email protected]>
* fix format problem
Signed-off-by: yuqi1129 <[email protected]>
Co-authored-by: Xinye Tao <[email protected]>
* txn: Do assertions on data for mutations to reduce potential data inconsistency (#10977)
* Add mutation checking logic
Signed-off-by: MyonKeminta <[email protected]>
* Fix build
Signed-off-by: MyonKeminta <[email protected]>
* Fix usages of mutation
Signed-off-by: MyonKeminta <[email protected]>
* Update kvproto and renaming
Signed-off-by: MyonKeminta <[email protected]>
* Add a simple test
Signed-off-by: MyonKeminta <[email protected]>
* Fix incorrect assertion
Signed-off-by: MyonKeminta <[email protected]>
* Support check existance during pessimistic lock
Signed-off-by: MyonKeminta <[email protected]>
* Add assertion level support
Signed-off-by: MyonKeminta <[email protected]>
* Add tests
Signed-off-by: MyonKeminta <[email protected]>
* Add more test cases
Signed-off-by: MyonKeminta <[email protected]>
* Add test for check_existence of pessimistic_lock request
Signed-off-by: MyonKeminta <[email protected]>
* Fix build
Signed-off-by: MyonKeminta <[email protected]>
* ref #11542
Signed-off-by: MyonKeminta <[email protected]>
* Add a metric for counting different way of checking assertion, for reference about the performance impact
Signed-off-by: MyonKeminta <[email protected]>
Co-authored-by: MyonKeminta <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* metrics: fix memory metrics (#11656)
Signed-off-by: Shuaipeng Yu <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* coprocesser: refactor encoding and add gbk_chinese_ci map (#11661)
* close #11660
Signed-off-by: xiongjiwei <[email protected]>
* close #11660
Signed-off-by: xiongjiwei <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* storage: Add API V2 check for RawKV and TxnKV requests (txn part) (#11216)
* wip
Signed-off-by: pingyu <[email protected]>
* wip
Signed-off-by: pingyu <[email protected]>
* ref #10974: Add API V2 check for RawKV and TxnKV requests (txn)
Signed-off-by: pingyu <[email protected]>
* ref #10974: Add API V2 check for RawKV and TxnKV requests (txn)
Signed-off-by: pingyu <[email protected]>
* ref #10974: Add API V2 check for RawKV and TxnKV requests (txn)
Signed-off-by: pingyu <[email protected]>
* ref #10974: Add API V2 check for RawKV and TxnKV requests (txn)
Signed-off-by: pingyu <[email protected]>
* ref #10974: Add API V2 check for RawKV and TxnKV requests (txn)
Signed-off-by: pingyu <[email protected]>
* ref #10974: Add API V2 check for RawKV and TxnKV requests (txn)
Signed-off-by: pingyu <[email protected]>
* fix enable_ttl
Signed-off-by: andylokandy <[email protected]>
* simplify test
Signed-off-by: andylokandy <[email protected]>
* ref #10974: Add API V2 check for RawKV and TxnKV requests (txn)
Signed-off-by: pingyu <[email protected]>
Co-authored-by: andylokandy <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* cdc: load uninlined value more effectively (#11615)
Signed-off-by: qupeng <[email protected]>
* cdc: load old value with prefix seek if possible (#11643)
* close #11642
Signed-off-by: qupeng <[email protected]>
* a more comment
Signed-off-by: qupeng <[email protected]>
* ctl: split into multiple modules (#11658)
* ctl: split into multiple modules
ref #10938
Signed-off-by: andylokandy <[email protected]>
* refactor if let to match
Signed-off-by: andylokandy <[email protected]>
* address comment
Signed-off-by: andylokandy <[email protected]>
* address comment
Signed-off-by: andylokandy <[email protected]>
* address comment
Signed-off-by: andylokandy <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* Fix build issues on non-linux platforms (#11671)
* close #11676
Signed-off-by: devillove084 <[email protected]>
* all: Migrate from std:: types to primitives. (#11677)
* Deprecation is planned for MIN,MAX,EPSLION,INFINITY,NEG_INFINITY of
the (i|u|f)(8|16|32|64) types in standard.
ref #11678
```release-note
NONE
```
Signed-off-by: Harold Dost <[email protected]>
* metrics alert: fix apply cpu alert (#11672)
Signed-off-by: zhangjinpeng1987 <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* resrc_mtr: introduce scanned keys statistics (#11085)
close #11690
Signed-off-by: mornyx <[email protected]>
* file_system: support collect the io flow by proc (#11667)
* file_system: support collect the read flow by proc
Signed-off-by: MuZhou233 <[email protected]>
* fix lint
Signed-off-by: MuZhou233 <[email protected]>
* fix clippy
Signed-off-by: MuZhou233 <[email protected]>
* Delete unused content & Adjust copyright text
Signed-off-by: MuZhou233 <[email protected]>
* Add bench_fetch_io_bytes
Signed-off-by: MuZhou233 <[email protected]>
* fix lint
Signed-off-by: MuZhou233 <[email protected]>
* Integrate thread_io
Signed-off-by: MuZhou233 <[email protected]>
* record bytes in request
Signed-off-by: MuZhou233 <[email protected]>
* fix logical bug
Signed-off-by: MuZhou233 <[email protected]>
* Update test write_bytes
Signed-off-by: MuZhou233 <[email protected]>
* Delete unused & Rename func
Signed-off-by: MuZhou233 <[email protected]>
* Add test for read & Update test for write
Signed-off-by: MuZhou233 <[email protected]>
* Add buffered thread io
Signed-off-by: MuZhou233 <[email protected]>
* Replace DashMap with ThreadLocal
Signed-off-by: MuZhou233 <[email protected]>
* Rename variables and function & Remove unused call
Signed-off-by: MuZhou233 <[email protected]>
* Change fetch logic. close #10867
Signed-off-by: MuZhou233 <[email protected]>
* Add AtomicIOBytes
Signed-off-by: MuZhou233 <[email protected]>
* Split ThreadIOSentinel
Signed-off-by: MuZhou233 <[email protected]>
* add THREAD_IO_TOTAL for metrics bytes fetcher
Signed-off-by: MuZhou233 <[email protected]>
* Adjust code style & fix
Signed-off-by: MuZhou233 <[email protected]>
* improve write_bytes test
Signed-off-by: MuZhou233 <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* *: check epoch and validness of lock table when writing pessimistic locks (#11591)
ref #11452
Signed-off-by: Yilin Chen <[email protected]>
* rsmeter: introduce datasink (#11688)
* add datasink trait
Signed-off-by: Zhenchi <[email protected]>
* RAII removes on_reporter_closing
Signed-off-by: Zhenchi <[email protected]>
* fmt
Signed-off-by: Zhenchi <[email protected]>
* fix build
ref #11691
Signed-off-by: Zhenchi <[email protected]>
* datasink -> data_sink
Signed-off-by: Zhenchi <[email protected]>
* fmt
Signed-off-by: Zhenchi <[email protected]>
* fix build
Signed-off-by: Zhenchi <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* clippy: Fixing up clippy errors. (#11193)
* clippy: Fixing up clippy errors.
* Fix crate references
* Fix anonymous lifetimes references.
ref #4301
Signed-off-by: Harold Dost <[email protected]>
* Test Revert of Iterations
ref #4301
Signed-off-by: Harold Dost <[email protected]>
* clippy: Enable check for rust-2018-idioms
ref #4301
Signed-off-by: Harold Dost <[email protected]>
* rsmeter: add centralized place to construct protobuf data (#11696)
ref #11691
Signed-off-by: Zhenchi <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* tikv_util: update procfs to 0.12.0 (#11703)
Signed-off-by: linning <[email protected]>
* Use generic for api version to reduce runtime branching (#11687)
* Use generic for api version to reduce runtime branching
ref #10938
Signed-off-by: andylokandy <[email protected]>
* fix test
Signed-off-by: andylokandy <[email protected]>
* add test
Signed-off-by: andylokandy <[email protected]>
* fix clippy
Signed-off-by: andylokandy <[email protected]>
* address comment
Signed-off-by: andylokandy <[email protected]>
* fmt
Signed-off-by: andylokandy <[email protected]>
* address comment
Signed-off-by: andylokandy <[email protected]>
* address comment
Signed-off-by: andylokandy <[email protected]>
* rename vars
Signed-off-by: andylokandy <[email protected]>
* improve vars
Signed-off-by: andylokandy <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* sst_importer: check api version when importing (#11664)
* sst_importer: check api version when importing
ref #10938
Signed-off-by: Peng Guanwen <[email protected]>
* Fix bugs
Signed-off-by: Peng Guanwen <[email protected]>
* fix bug of Iterator::{scan,scan_cf}
Signed-off-by: Peng Guanwen <[email protected]>
* remove redundant log
Signed-off-by: Peng Guanwen <[email protected]>
* Show the entire key
Signed-off-by: Peng Guanwen <[email protected]>
* Fix lint issue
Signed-off-by: Peng Guanwen <[email protected]>
* resolve conflict
Signed-off-by: andylokandy <[email protected]>
Co-authored-by: Andy Lok <[email protected]>
* backup: pipeline scan and save step (#11528)
* br: sperate io and scan threads
Signed-off-by: Yu Juncen <[email protected]>
* br/stream: make clippy happy
Signed-off-by: yujuncen <[email protected]>
* br/stream: fix tests
Signed-off-by: Yu Juncen <[email protected]>
* backup: fix a dummy bug
Signed-off-by: Yu Juncen <[email protected]>
* backup: add config of io-threads
Signed-off-by: yujuncen <[email protected]>
* backup: ref #11350: some minior change
Signed-off-by: yujuncen <[email protected]>
* backup: added some metrics
Signed-off-by: yujuncen <[email protected]>
* backup: better tuning concurrency, and added a metric
Signed-off-by: yujuncen <[email protected]>
* backup: added some slow log and metrics
Signed-off-by: yujuncen <[email protected]>
* backup: address comments
Signed-off-by: yujuncen <[email protected]>
* backup: fix build
Signed-off-by: yujuncen <[email protected]>
* Revert "backup: fix build"
This reverts commit 74537be56c410d0f91af2f48b7e65356ab53720e.
Signed-off-by: yujuncen <[email protected]>
* Revert "backup: address comments"
This reverts commit 77d75756028bea3496f878f0ad8bbc562f5d00aa.
Signed-off-by: yujuncen <[email protected]>
* backup: address comments
Signed-off-by: yujuncen <[email protected]>
* backup: always set IO type to Export
Signed-off-by: yujuncen <[email protected]>
* backup:make clippy happy
Signed-off-by: yujuncen <[email protected]>
* backup: address comments
Signed-off-by: yujuncen <[email protected]>
* metrics: fix the metrics cannot be displayed (#11710)
ref #11662
Signed-off-by: Ryan Leung <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* backup: return api-version to BR when backup (#11704)
* Update kvproto
Signed-off-by: Peng Guanwen <[email protected]>
* Return api-version for br
ref #10938
Signed-off-by: Peng Guanwen <[email protected]>
* Reformat code
Signed-off-by: Peng Guanwen <[email protected]>
* Update components/external_storage/export/src/export.rs
Signed-off-by: Peng Guanwen <[email protected]>
Co-authored-by: Andy Lok <[email protected]>
* format code
Signed-off-by: Peng Guanwen <[email protected]>
Co-authored-by: Andy Lok <[email protected]>
* file_system: limit thread_io target os (#11715)
* limit thread_io target os
Signed-off-by: MuZhou233 <[email protected]>
* limit thread_io target os. typo. close #11698
Signed-off-by: MuZhou233 <[email protected]>
Co-authored-by: Yilin Chen <[email protected]>
* file_system: bypass file allocate implementation in fs2 (#11700)
Close #10688
Patch fs2 to https://github.com/tabokie/fs2-rs/tree/tikv. In which, `posix_fallocate` is replaced with `fallocate` (https://github.com/danburkert/fs2-rs/pull/42).
* bypass file allocate implementation in fs2
Signed-off-by: tabokie <[email protected]>
* metrics: remove min legend of some Grafana panels (#11487)
Minimum legend isn't very useful, and it occupied valuable visual real estate.
Remove some min legends from Grafana metrics, except for the following:
- Level 0 chance
- Number files at each level
- File Count
- Entry Count
- Allocator Stats
- Encrypted files
- Encryption meta files size
Ref #11119
Signed-off-by: tabokie <[email protected]>
* coprocessor: not return rows when there is no input for simple aggregation (#11708)
* not return rows when there is no input for simple aggregation
Signed-off-by: xufei <[email protected]>
* close #11735, and address comments
Signed-off-by: xufei <[email protected]>
* backup: reduce the default thread pool size of backup, enable auto tune by default (#11699)
* backup: clamp auto tune values
Signed-off-by: yujuncen <[email protected]>
* ref #11000: change the default backup threads.
Signed-off-by: yujuncen <[email protected]>
* backup: run tasks in background
Signed-off-by: yujuncen <[email protected]>
* backup: run rustfmt
Signed-off-by: yujuncen <[email protected]>
* backup: set remain threads to 20% of vcpu
Signed-off-by: yujuncen <[email protected]>
* br-stream: fix build
Signed-off-by: yujuncen <[email protected]>
* Backup: add S3 metrics && add s3_multi_part_size config (#11666)
* s3: add request metrics for s3 storage
Signed-off-by: 3pointer <[email protected]>
* s3: add grafana json
Signed-off-by: 3pointer <[email protected]>
* br: add config for s3 multi part upload
Signed-off-by: 3pointer <[email protected]>
* update comment
Signed-off-by: 3pointer <[email protected]>
* update comment
Signed-off-by: 3pointer <[email protected]>
* address comment
Signed-off-by: 3pointer <[email protected]>
* hidden the new config
Signed-off-by: 3pointer <[email protected]>
* only hidden config in config-template.toml
Signed-off-by: 3pointer <[email protected]>
* address comment
Signed-off-by: 3pointer <[email protected]>
* close #11727 && format
Signed-off-by: 3pointer <[email protected]>
* fix
Signed-off-by: 3pointer <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* rsmeter: support multiple datasinks (#11707)
* rsmeter: support multiple datasinks
Signed-off-by: Zhenchi <[email protected]>
* wrap config notifier
Signed-off-by: Zhenchi <[email protected]>
* add unit tests
Signed-off-by: Zhenchi <[email protected]>
* polish
ref #11691
Signed-off-by: Zhenchi <[email protected]>
* address comments
Signed-off-by: Zhenchi <[email protected]>
* fix test
Signed-off-by: Zhenchi <[email protected]>
* fix outdated comments
Signed-off-by: Zhenchi <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* engine: properly estimate store size (#11728)
* properly estimate store size, ref #11119
Signed-off-by: tabokie <[email protected]>
* address comments
Signed-off-by: tabokie <[email protected]>
* implement global config client for tikv (#11685)
* implement global config client
close #11686
Signed-off-by: lemonhx <[email protected]>
* implement unit test for global config in pd client
Signed-off-by: lemonhx <[email protected]>
* resolved formatting issues
Signed-off-by: lemonhx <[email protected]>
* impl pd client asyncly
Signed-off-by: lemonhx <[email protected]>
* according to reviewer's opinion
removing store method implementation
Signed-off-by: lemonhx <[email protected]>
* Merge branch 'master' of https://github.com/tikv/tikv into global_conf
Signed-off-by: lemonhx <[email protected]>
* write test case for watch global config when
grpc server is closed.
Signed-off-by: lemonhx <[email protected]>
* resove issues
Signed-off-by: lemonhx <[email protected]>
* clippy
Signed-off-by: lemonhx <[email protected]>
* watch global config rentry future in test deadlock just removed
Signed-off-by: lemonhx <[email protected]>
Co-authored-by: Zhenchi <[email protected]>
* storage: skip scanning lock when using rc (#11701)
* skip scanning lock when using rc, ref #11485
Signed-off-by: you06 <[email protected]>
* format code
Signed-off-by: you06 <[email protected]>
* add test
Signed-off-by: you06 <[email protected]>
* address comment
Signed-off-by: you06 <[email protected]>
* add an integration test
Signed-off-by: you06 <[email protected]>
* format code
Signed-off-by: you06 <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* copr: pushdown substring to tikv (#11494)
* corp: pushdown substring to tikv
Signed-off-by: guo-shaoge <[email protected]>
* fix case. close #11495
Signed-off-by: guo-shaoge <[email protected]>
* fix comment
Signed-off-by: guo-shaoge <[email protected]>
* rsmeter: distinguish between collectors and observers within recorder (#11712)
* rsmeter: support multiple datasinks
Signed-off-by: Zhenchi <[email protected]>
* wrap config notifier
Signed-off-by: Zhenchi <[email protected]>
* add unit tests
Signed-off-by: Zhenchi <[email protected]>
* polish
ref #11691
Signed-off-by: Zhenchi <[email protected]>
* rsmeter: distinguish between collectors and observers within recorder
Signed-off-by: Zhenchi <[email protected]>
* fmt & try to resolve #11689
Signed-off-by: Zhenchi <[email protected]>
* add unit tests
ref #11691
Signed-off-by: Zhenchi <[email protected]>
* retrigger test
Signed-off-by: Zhenchi <[email protected]>
* address comments
Signed-off-by: Zhenchi <[email protected]>
* fix test
Signed-off-by: Zhenchi <[email protected]>
* fix outdated comments
Signed-off-by: Zhenchi <[email protected]>
* Update components/resource_metering/src/recorder/sub_recorder/mod.rs
Signed-off-by: Zhenchi <[email protected]>
Co-authored-by: Yexiang Zhang <[email protected]>
* address comment
Signed-off-by: Zhenchi <[email protected]>
* remove enabled
Signed-off-by: Zhenchi <[email protected]>
* fix
Signed-off-by: Zhenchi <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
Co-authored-by: Wenxuan <[email protected]>
Co-authored-by: Yexiang Zhang <[email protected]>
* config: validate online configurable thread pools (tikv#11159) (#11714)
ref #11159
Signed-off-by: Wenbo Zhang <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* res_meter: upgrade kvproto (#11749)
* Upgrade kvproto
close #11748
Signed-off-by: mornyx <[email protected]>
* Fix test compilation
Signed-off-by: mornyx <[email protected]>
* Fix test compilation
Signed-off-by: mornyx <[email protected]>
* Fix test compilation
Signed-off-by: mornyx <[email protected]>
* Use master kvproto
Signed-off-by: mornyx <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* rsmeter: add pubsub datasink (#11719)
* rsmeter: add pubsub datasink
close #11691
Signed-off-by: Zhenchi <[email protected]>
* fix build
Signed-off-by: Zhenchi <[email protected]>
* Update components/resource_metering/src/reporter/pubsub.rs
Signed-off-by: Zhenchi <[email protected]>
Co-authored-by: Wenxuan <[email protected]>
* fix build
Signed-off-by: Zhenchi <[email protected]>
Co-authored-by: Wenxuan <[email protected]>
* status_server: Support online config update for configuration hosting platform (#11693)
close #11692
Signed-off-by: Wenbo Zhang <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* res_meter: optimize top k (#11753)
* Keep top k on recorder
Signed-off-by: mornyx <[email protected]>
* Modify comments
Signed-off-by: mornyx <[email protected]>
* Fix pd.rs
Signed-off-by: mornyx <[email protected]>
* Refactor
Signed-off-by: mornyx <[email protected]>
* Remove import
Signed-off-by: mornyx <[email protected]>
* max_resource_groups 200 -> 100
Signed-off-by: mornyx <[email protected]>
* Fix ut
Signed-off-by: mornyx <[email protected]>
* Use iterator to avoid clone
Signed-off-by: mornyx <[email protected]>
* Optimize
close #11754
Signed-off-by: mornyx <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftclient: delay flush (#11705)
* raftclient: delay flush
Signed-off-by: Jay Lee <[email protected]>
* remove raftstore wait
Signed-off-by: Jay Lee <[email protected]>
* Ref #11309.
tikv#11310 introduces delay in raftstore thread to reduce the flush rate.
It can only reduce flush rate from one thread. If there are many
raftstore threads or IO threads, messages can still be flush frequently.
This PR reduces the flush rate by introducing delay in raft client. So
delay will work at connection level and achieve the maximun batch
affect.
Signed-off-by: Jay Lee <[email protected]>
* remove useless field
Signed-off-by: Jay Lee <[email protected]>
* address comment
Signed-off-by: Jay Lee <[email protected]>
* fix unstable raw get case
Signed-off-by: Jay Lee <[email protected]>
Co-authored-by: Liqi Geng <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* cloud: support azure blob storage as external storage backend (#11732)
* support azure storage blob for external storage
Signed-off-by: Leavrth <[email protected]>
* modify the priority of the azure login methods
Signed-off-by: Leavrth <[email protected]>
* fix feature match branch
Signed-off-by: Leavrth <[email protected]>
* implement token update retriable
Signed-off-by: Leavrth <[email protected]>
* fix some problems
Signed-off-by: Leavrth <[email protected]>
* close #11731
Signed-off-by: Leavrth <[email protected]>
* rustfmt
Signed-off-by: Leavrth <[email protected]>
* make clippy success
Signed-off-by: Leavrth <[email protected]>
* simplify uploader
Signed-off-by: Leavrth <[email protected]>
* commit some suggestions
Signed-off-by: Leavrth <[email protected]>
* coprocesser: update encoding lib (#11506)
Signed-off-by: xiongjiwei <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* coprocesser: implement upper and lower function on GBK charset (#11756)
Signed-off-by: xiongjiwei <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* logger: support archiving and rotation (#11651) (#11657)
* logger: support archiving and rotation (#11651)
close #11651
Signed-off-by: Wenbo Zhang <[email protected]>
* logger: add testcases to test dt_from_file_name (#11651)
ref #11651
Signed-off-by: Wenbo Zhang <[email protected]>
* logger: simplify DateTime format (#11651)
ref #11651
Signed-off-by: Wenbo Zhang <[email protected]>
* logger: do a little refactoring (#11651)
ref #11651
Signed-off-by: Wenbo Zhang <[email protected]>
* logger: do a little refactoring (#11651)
ref #11651
Signed-off-by: Wenbo Zhang <[email protected]>
* logger: do a little refactoring (#11651)
ref #11651
Signed-off-by: Wenbo Zhang <[email protected]>
* logger: do a little refactoring (#11651)
ref #11651
Signed-off-by: Wenbo Zhang <[email protected]>
* logger: do a little refactoring (#11651)
ref #11651
Signed-off-by: Wenbo Zhang <[email protected]>
* diagnostics: refactor (#11651)
ref #11651
Signed-off-by: Wenbo Zhang <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* *: run clippy for integration tests (#11768)
* run clippy --fix with nightly-2021-12-17
Signed-off-by: tabokie <[email protected]>
* manual clippy
Signed-off-by: tabokie <[email protected]>
* format and clippy
Signed-off-by: tabokie <[email protected]>
* enable default features when running clippy, ref #11119
Signed-off-by: tabokie <[email protected]>
* fix build
Signed-off-by: tabokie <[email protected]>
* revert box_collection lint
Signed-off-by: tabokie <[email protected]>
* Update tests/integrations/raftstore/test_bootstrap.rs
Signed-off-by: Xinye Tao <[email protected]>
Co-authored-by: Lei Zhao <[email protected]>
Co-authored-by: Lei Zhao <[email protected]>
* logger: fix config-template's log unit (#11651) (#11777)
ref #11651
Signed-off-by: Wenbo Zhang <[email protected]>
* logger: improve testcase (#11651) (#11778)
ref #11651
Signed-off-by: Wenbo Zhang <[email protected]>
* raftstore: propose in-memory pessimistic locks before leader transfer (#11577)
* *: check epoch and validness of lock table when writing pessimistic locks
ref #11452
Signed-off-by: Yilin Chen <[email protected]>
* raftstore: propose in-memory pessimistic locks before leader transfer
ref #11452
Signed-off-by: Yilin Chen <[email protected]>
* mark delete in scheduler and delete after apply
Signed-off-by: Yilin Chen <[email protected]>
* Add tests for two different orders between proposing locks and write command
Signed-off-by: Yilin Chen <[email protected]>
* do not clear pessimistic locks before transfer leader
Signed-off-by: Yilin Chen <[email protected]>
* delete memory locks in the apply thread and add more term and version check
Signed-off-by: Yilin Chen <[email protected]>
* remove potentially incorrect assert
Signed-off-by: Yilin Chen <[email protected]>
* check term and version when reading the lock table
Signed-off-by: Yilin Chen <[email protected]>
* revert a log change
Signed-off-by: Yilin Chen <[email protected]>
* change term when becoming follower
Signed-off-by: Yilin Chen <[email protected]>
* fix lint
Signed-off-by: Yilin Chen <[email protected]>
* do not check version when reading locks (temporarily)
Signed-off-by: Yilin Chen <[email protected]>
* fix another test
Signed-off-by: Yilin Chen <[email protected]>
* only treat MsgTransferLeader with context with valid msg
Signed-off-by: Yilin Chen <[email protected]>
* fix a tiny comment error
Signed-off-by: Yilin Chen <[email protected]>
* make clippy happy
Signed-off-by: Yilin Chen <[email protected]>
* res_meter: ignore the read keys test temporarily (#11779)
ref #11765
Signed-off-by: mornyx <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* logger: change critical to fatal (#11651) (#11780)
* logger: change critical to fatal (#11651)
ref #11651
Signed-off-by: Wenbo Zhang <[email protected]>
* logger: fix test_parse_log_level (#11651)
ref #11651
Signed-off-by: Wenbo Zhang <[email protected]>
* update CHANGELOG.md (#11790)
* update from tikv-server v5.0.1(20210423)
from https://github.com/tikv/tikv/releases
Signed-off-by: xiejiandong <[email protected]>
* update CHANGELOG.md close #11167
Signed-off-by: xiejiandong <[email protected]>
* config: relax the bounds of online configurable worker number (#11651) (#11798)
close #11776
Signed-off-by: Wenbo Zhang <[email protected]>
* logger: invoke logger_compatible_adjust before run_tikv (#11651) (#11792)
close #11789
Signed-off-by: Wenbo Zhang <[email protected]>
Co-authored-by: Xinye Tao <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* *: unify thread related native APIs (#11785)
* *: unify thread related native APIs
Now there are duplicated codes calling syscalls to fetch thread
informations. This PR provides unified APIs and remove duplicated codes.
Close #11784.
Signed-off-by: Jay Lee <[email protected]>
* address comment
Signed-off-by: Jay Lee <[email protected]>
* declare platform compatability
Signed-off-by: Jay Lee <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* Fix method on_tick will not update metric when enable_auto_tune is false (#11791)
* Fix method on_tick will not update metric when enable_auto_tune is false
ref #11787
Signed-off-by: yuqi1129 <[email protected]>
* resolve format problem in discussion
Signed-off-by: yuqi1129 <[email protected]>
* resolve format problem in discussion again
Signed-off-by: yuqi1129 <[email protected]>
Co-authored-by: Xinye Tao <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftstore: add a new tick to renew lease in advance (#6427)
* renew lease in raft tick
Signed-off-by: 5kbpers <[email protected]>
* check lease before sending read index
Signed-off-by: 5kbpers <[email protected]>
* renew lease on raft tick to cover lease till next tick
Signed-off-by: linning <[email protected]>
* refactor
Signed-off-by: linning <[email protected]>
* don't use inspect_lease
Signed-off-by: linning <[email protected]>
* fix ci
Signed-off-by: linning <[email protected]>
* add test
Signed-off-by: linning <[email protected]>
* remove unneed comment
Signed-off-by: linning <[email protected]>
* close #5388
Signed-off-by: 5kbpers <[email protected]>
* check pending_read_count before hibernating
Signed-off-by: 5kbpers <[email protected]>
* increase the range to check lease expired
Signed-off-by: 5kbpers <[email protected]>
* renew leader lease after not hibernate
Signed-off-by: 5kbpers <[email protected]>
* fix hibernate states metrics
Signed-off-by: 5kbpers <[email protected]>
* check writes and pre_read_index before proposing read index
Signed-off-by: 5kbpers <[email protected]>
* fix test
Signed-off-by: 5kbpers <[email protected]>
* use a individual tick to renewing lease
Signed-off-by: 5kbpers <[email protected]>
* fix condition
Signed-off-by: 5kbpers <[email protected]>
* stablize test_node_renew_lease
Signed-off-by: 5kbpers <[email protected]>
* cleanup unused changes
Signed-off-by: 5kbpers <[email protected]>
* add config item for interval of tick
Signed-off-by: 5kbpers <[email protected]>
* rename tick
Signed-off-by: 5kbpers <[email protected]>
* disable the tick for test_inconsistent_configuration
Signed-off-by: 5kbpers <[email protected]>
* change deafault value of check_leader_lease_interval
Signed-off-by: 5kbpers <[email protected]>
* address comments
Signed-off-by: 5kbpers <[email protected]>
* disable the tick for test_read_hibernated_region
Signed-off-by: 5kbpers <[email protected]>
* address comments
Signed-off-by: 5kbpers <[email protected]>
* count renew lease read index into raft metrics
Signed-off-by: 5kbpers <[email protected]>
* address comments
Signed-off-by: 5kbpers <[email protected]>
* disable the tick for test_renew_lease
Signed-off-by: 5kbpers <[email protected]>
* address comments
Signed-off-by: 5kbpers <[email protected]>
* address comments
Signed-off-by: 5kbpers <[email protected]>
* fix lint error
Signed-off-by: 5kbpers <[email protected]>
* add a 100ms buffer for renew_bound
Signed-off-by: 5kbpers <[email protected]>
* address comments
Signed-off-by: 5kbpers <[email protected]>
* address comments
Signed-off-by: 5kbpers <[email protected]>
* make clippy happy
Signed-off-by: 5kbpers <[email protected]>
Co-authored-by: linning <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* cloud: Add retry for azure blob server busy error (#11813)
* add retry for azure blob server busy error
Signed-off-by: Leavrth <[email protected]>
* close#11812
Signed-off-by: Leavrth <[email protected]>
* use to_string
Signed-off-by: Leavrth <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* tikv_util: make cgroup parsing more robust (#11786)
* make cgroup parsing more robust
Signed-off-by: tabokie <[email protected]>
* fix error message
Signed-off-by: tabokie <[email protected]>
* parse cpu quota in floats
Signed-off-by: tabokie <[email protected]>
* *: make it compile on macos (#11825)
* *: make it compile on macos
close #11823
Signed-off-by: Jay Lee <[email protected]>
* remove extra comment
Signed-off-by: Jay Lee <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* resolved_ts: add more metrics (#11809)
* add metrics
Signed-off-by: 5kbpers <[email protected]>
* add slow log
Signed-off-by: 5kbpers <[email protected]>
* remove duplicate metric
Signed-off-by: 5kbpers <[email protected]>
* add resolved ts metrics for leader peers
Signed-off-by: 5kbpers <[email protected]>
* close #11767
Signed-off-by: 5kbpers <[email protected]>
* init endpoint with store meta
Signed-off-by: 5kbpers <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* rsmeter: fix perf regression caused by arc swap (#11833)
* rsmeter: fix perf regression caused by arc swap
Signed-off-by: Zhenchi <[email protected]>
* use atomic cell instead
Signed-off-by: Zhenchi <[email protected]>
* debug assert
Signed-off-by: Zhenchi <[email protected]>
Co-authored-by: Wenxuan <[email protected]>
* load base split: more accurate sample for large key ranges batch (#11039)
* more accurate sample for large key ranges
Signed-off-by: lhy1024 <[email protected]>
* update
Signed-off-by: lhy1024 <[email protected]>
* ref #11521
Signed-off-by: lhy1024 <[email protected]>
* address comment
Signed-off-by: lhy1024 <[email protected]>
* fix test
Signed-off-by: lhy1024 <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftclient: enable delay only in high load (#11772)
and improve CPU efficiency under high load. But it turns out
this can lead to significant regression in normal load. So
this PR adds CPU stats and only enabling delay in high load.
This is also the same strategy used in v4.x.
Close #11769.
Signed-off-by: Jay Lee <[email protected]>
* tikv-ctl: fix tikv-ctl's output is incomplete on calling process::exit (#11231)
Signed-off-by: Yao Zongyou <[email protected]>
Co-authored-by: Xinye Tao <[email protected]>
Co-authored-by: Connor <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftstore: skip deleting snapshot files in peer pending_remove is true (#11782)
* raftstore: skip deleting snapshot files in peer when the peer is pending removal and the snapshot is being applied and canceled -- close #11746
This is to avoid the potential panic when the snapshot files are deleted, but the peer's status (Tombstone) is not persisted in disk due to tikv crash.
Signed-off-by: tonyxuqqi <[email protected]>
* address code review feedback -- close #11746
Signed-off-by: qi.xu <[email protected]>
* address code review feedback 2 -- close #11746
Signed-off-by: qi.xu <[email protected]>
* address code review feedback 2 -- close #11746
Signed-off-by: qi.xu <[email protected]>
* address code review feedback 2 -- close #11746
Signed-off-by: qi.xu <[email protected]>
* address code review feedback 2 -- close #11746
Signed-off-by: qi.xu <[email protected]>
Co-authored-by: qi.xu <[email protected]>
* load_statis: fix compilation on macos (#11851)
close #11772
Signed-off-by: Jay Lee <[email protected]>
* config: override rocksdb config when flow control enabled (#11840)
Signed-off-by: Connor1996 <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* rocksdb: consider env sharing when configuring background threads (#11760)
#9649 increased the default setting of kvdb background flushes (3 for 8c and 4 for 16c).
This PR takes another approach: use smaller concurrency for individual dbs, but configure the global thread pool size with the sum of kvdb and raftdb concurrency. This way we won't over-configure flush workers when raft-engine is enabled.
* consider env sharing when configuring background threads
ref #11119
Signed-off-by: tabokie <[email protected]>
* server/config: keep compatible using option (#11862)
* server/config: keep compatible using option
Using `Option` for `heavy_load_wait_duration` to keep compatible with
versions prior to v5.3.
Close #11861.
Signed-off-by: Jay Lee <[email protected]>
* fix format
Signed-off-by: Jay Lee <[email protected]>
* fix clippy
Signed-off-by: Jay Lee <[email protected]>
Co-authored-by: zhouqiang <[email protected]>
* raftstore: fix missing workers' stop on shutdown (#11864)
ref #11159
Signed-off-by: Wenbo Zhang <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftstore: move in-memory pessimistic locks to split regions (#11655)
ref #11452
Signed-off-by: Yilin Chen <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* *: Remove the entrance of enabling io snooper. (#11394)
* *: Remove the entrance of enabling io snooper. close #10867
Signed-off-by: MuZhou233 <[email protected]>
* fix
Signed-off-by: MuZhou233 <[email protected]>
* fix test
Signed-off-by: MuZhou233 <[email protected]>
* fix test
Signed-off-by: MuZhou233 <[email protected]>
Co-authored-by: Xinye Tao <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftstore: change PeerTicks to enum (#11849)
* raftstore: change PeerTicks to enum
close #11848
Signed-off-by: Yilin Chen <[email protected]>
* use bool array for registry
Signed-off-by: Yilin Chen <[email protected]>
* rename PeerTicks to PeerTick
Signed-off-by: Yilin Chen <[email protected]>
* add associated const for number of PeerTick types
Signed-off-by: Yilin Chen <[email protected]>
* change naming
Signed-off-by: Yilin Chen <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* *: check memory limit when inserting in-memory pessimistic locks (#11841)
* *: check memory limit when inserting in-memory pessimistic locks
ref #11452
Signed-off-by: Yilin Chen <[email protected]>
* fix merged upstream
Signed-off-by: Yilin Chen <[email protected]>
* fix test
Signed-off-by: Yilin Chen <[email protected]>
* fix typo
Signed-off-by: Yilin Chen <[email protected]>
* *: fix invalid failpoint caused by typo (#11709)
* fix invalid failpoint caused by typo
close #11734
Signed-off-by: Ryan Leung <[email protected]>
* address the comment
Signed-off-by: Ryan Leung <[email protected]>
* address the comment
Signed-off-by: Ryan Leung <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftstore: Extract significant router (#11750)
* ref#11409 add significant router
Signed-off-by: Connor1996 <[email protected]>
* make format
Signed-off-by: Connor1996 <[email protected]>
* address comment
Signed-off-by: Connor1996 <[email protected]>
* raftstore: move scan delete to raft log gc worker (#11853)
* raftstore: move scan delete to raft log gc worker
When clearing raft metas, raftstore will scan raft logs and delete them
one by one. Seeking can be slow if there are a lot of tombstone keys.
This PR moves the operation to raft log gc worker to reduce the impact.
The final solution should be also moving remaining IO operations to
async write IO threads.
Close #10210.
Signed-off-by: Jay Lee <[email protected]>
* address comment
Signed-off-by: Jay Lee <[email protected]>
* speed up destroy
Signed-off-by: Jay Lee <[email protected]>
* fix compile
Signed-off-by: Jay Lee <[email protected]>
* further speed up
Signed-off-by: Jay Lee <[email protected]>
* revert test case configuration
Signed-off-by: Jay Lee <[email protected]>
* address comment
Signed-off-by: Jay Lee <[email protected]>
* address comment
Signed-off-by: Jay Lee <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* doc: update new rules for linking issue and commit message (#11832)
close #11097, close #11831
Signed-off-by: zhangyangyu <[email protected]>
Co-authored-by: Mini256 <[email protected]>
Co-authored-by: Mini256 <[email protected]>
Co-authored-by: Xiaoguang Sun <[email protected]>
* *: update rust toolchain to 2022-01-07 (#11875)
* update rust toolchain to 2022-01-17
Signed-off-by: tabokie <[email protected]>
* address comment: clean up unnecessary `as_ref()`
Signed-off-by: tabokie <[email protected]>
* try fixing the underflow
Signed-off-by: tabokie <[email protected]>
* mute underflow warning for raft metrics
Signed-off-by: tabokie <[email protected]>
* clean up unused data members
Signed-off-by: tabokie <[email protected]>
* format
Signed-off-by: tabokie <[email protected]>
* step back to 2022-01-07
Signed-off-by: tabokie <[email protected]>
* readme: change images based on github theme (#11795)
Signed-off-by: sloorush <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftstore: don't remove other peer's read delegate (#11882)
* don't remove read delegate besides peer_destroy
Signed-off-by: linning <[email protected]>
* add test case
Signed-off-by: linning <[email protected]>
* make clippy happy
Signed-off-by: linning <[email protected]>
* address comment
Signed-off-by: linning <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* logger: use eprintln! if the logger is not initialized (#11869)
ref #11651
Signed-off-by: Wenbo Zhang <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* copr: fix the calculation of total column size in analyze (#11884)
Signed-off-by: xuyifan <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* engine: update raft-engine for data consistency fix (#11885)
Fix https://github.com/tikv/raft-engine/issues/142.
When encountering this bug, TiKV will panic with message "applied index > max(commit index, recorded commit index)" after restart.
* update raft-engine
Signed-off-by: tabokie <[email protected]>
* update one more commit
Signed-off-by: tabokie <[email protected]>
* check docker build during clippy, ref #11312 (#11819)
Signed-off-by: tabokie <[email protected]>
* gc_worker: fix incorrect scheduled_tasks counting (#11904)
* gc_worker: fix incorrect scheduled_tasks counting
close #11903
Signed-off-by: Yilin Chen <[email protected]>
* remove check_is_busy totally
Signed-off-by: Yilin Chen <[email protected]>
* do not use wildcard match in error handling
Signed-off-by: Yilin Chen <[email protected]>
* raftstore: Introduce raft log fetcher (#11900)
* ref#11409 introduce raft log fetcher
Signed-off-by: Connor1996 <[email protected]>
* update kvproto
Signed-off-by: Connor1996 <[email protected]>
* address comment
Signed-off-by: Connor1996 <[email protected]>
* fix test build
Signed-off-by: Connor1996 <[email protected]>
* fix clippy
Signed-off-by: Connor1996 <[email protected]>
* limit capacity
Signed-off-by: Connor1996 <[email protected]>
* update kvproto
Signed-off-by: Connor1996 <[email protected]>
* call stop on worker
Signed-off-by: Connor1996 <[email protected]>
* rename worker
Signed-off-by: Connor1996 <[email protected]>
* status_server: add pprof flamegraph response header (#10951)
* tweak(status_server): add pprof flamegraph response header
Signed-off-by: Suhaha <[email protected]>
* test(status_server): add Content-Type asset to test_pprof_profile_service
Signed-off-by: Suhaha <[email protected]>
* tweak(status_server): add pprof flamegraph response header
Signed-off-by: Suhaha <[email protected]>
* tweak(status_server): add pprof flamegraph response header close #11917
Signed-off-by: Suhaha <[email protected]>
* close #11917
Signed-off-by: Suhaha <[email protected]>
Co-authored-by: goroutine <[email protected]>
Co-authored-by: Connor <[email protected]>
* raftstore: renew leader lease in advance when handle read request (#9307)
* renew lease advance
Signed-off-by: linning <[email protected]>
* add log
Signed-off-by: linning <[email protected]>
* make clippy happy
Signed-off-by: linning <[email protected]>
* ref #11579
Signed-off-by: 5kbpers <[email protected]>
* reset raft tick
Signed-off-by: 5kbpers <[email protected]>
* set has ready
Signed-off-by: 5kbpers <[email protected]>
* address comments
Signed-off-by: 5kbpers <[email protected]>
* address comment
Signed-off-by: 5kbpers <[email protected]>
* address comment
Signed-off-by: 5kbpers <[email protected]>
* address comment
Signed-off-by: 5kbpers <[email protected]>
* add renew_leader_lease_advance_duration config
Signed-off-by: 5kbpers <[email protected]>
* address comment
Signed-off-by: 5kbpers <[email protected]>
* fix panic
Signed-off-by: 5kbpers <[email protected]>
* address comment
Signed-off-by: 5kbpers <[email protected]>
* address comment
Signed-off-by: 5kbpers <[email protected]>
* disable renewing for test_lease_unsafe_during_leader_transfers
Signed-off-by: 5kbpers <[email protected]>
Co-authored-by: 5kbpers <[email protected]>
* pprof: ignore cpu profiling on non-x86 arch (#11925)
* Disable cpu profiling on non-x86_64 arch
Signed-off-by: mornyx <[email protected]>
* Fix warns
Signed-off-by: mornyx <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* github: add new pr requirement for linking issue and commit message (#11887)
Signed-off-by: zhangyangyu <[email protected]>
Co-authored-by: Xiaoguang Sun <[email protected]>
* engine: enable raft engine by default (#11928)
* enable raft engine by default and synchronize docs
Signed-off-by: tabokie <[email protected]>
* update raft engine
Signed-off-by: tabokie <[email protected]>
* update raft-engine and fix tests
Signed-off-by: tabokie <[email protected]>
* gc_worker: Limit the key range to scan for GcKeys tasks (#11922)
close tikv/tikv#11752, close tikv/tikv#11902
Signed-off-by: MyonKeminta <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftstore: TransferLeader support multiple target peers (#11063)
ref tikv/tikv#822, ref tikv/tikv#4214, close tikv/tikv#10602
Signed-off-by: MrCroxx <[email protected]>
* raftstore: remove the leaders field of `StoreMeta` (#11934)
close tikv/tikv#11933
Remove `StoreMeta.leaders`
```
### Related changes
### Check List <!--REMOVE the items that are not applicable-->
Tests <!-- At least one of them must be included. -->
- Manual test (add detailed scripts or steps below)
`cargo check`
### Release note <!-- bugfixes or new feature need a release note -->
```release-note
None
Signed-off-by: linning <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftstore: Fetch raft log in async manner (#11409)
close tikv/tikv#10408, close tikv/tikv#11320
Signed-off-by: Connor1996 <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* copr: add quarter function (#11935)
ref tikv/tikv#5751
Signed-off-by: zhongyong jin <[email protected]>
* Enable full debug info for dev and test (#11949)
ref tikv/tikv#5049, ref tikv/tikv#5572, close tikv/tikv#5572
```
### Related changes
- PR to update `pingcap/docs`/`pingcap/docs-cn`:
- PR to update `pingcap/tidb-ansible`:
- Need to cherry-pick to the release branch
-->
### Check List <!--REMOVE the items that are not applicable-->
Tests <!-- At least one of them must be included. -->
- Manual test (add detailed scripts or steps below)
- No code
Manual test:
Linux box with 8 cores (3:50 -> 4:00):
```
# Before this PR:
$ make clean
cargo clean
rm -rf bin dist
$ time make build
cargo build --no-default-features --features "jemalloc mem-profiling portable sse test-engines-rocksdb cloud-aws cloud-gcp cloud-azure"
Compiling libc v0.2.106
...
Compiling server v0.0.1 (tikv/components/server)
Finished dev [unoptimized + debuginfo] target(s) in 3m 50s
real 3m50,487s
user 38m11,859s
sys 3m9,540s
# After this PR:
$ git diff
diff --git a/Cargo.toml b/Cargo.toml
index 71f5329d3..67cb9d183 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -267,7 +267,7 @@ default-members = ["cmd/tikv-server", "cmd/tikv-ctl"]
[profile.dev]
opt-level = 0
-debug = 1 # required for line numbers in tests, see tikv #5049
+debug = true
codegen-units = 4
lto = false
incremental = true
@@ -293,7 +293,7 @@ codegen-units = 4
[profile.test]
opt-level = 0
-debug = 1 # enable line numbers by default for easy test debugging
+debug = true
codegen-units = 16
lto = false
incremental = true
mattias@msig:~/repos/tikv$ time make clean
cargo clean
rm -rf bin dist
real 0m0,975s
user 0m0,148s
sys 0m0,828s
mattias@msig:~/repos/tikv$ time make build
cargo build --no-default-features --features "jemalloc mem-profiling portable sse test-engines-rocksdb cloud-aws cloud-gcp cloud-azure"
Compiling libc v0.2.106
...
Compiling server v0.0.1 (tikv/components/server)
Finished dev [unoptimized + debuginfo] target(s) in 4m 00s
real 4m0,201s
user 39m45,037s
sys 3m16,397s
```
Macbook Air M1:
```
# Before:
% time make build
cargo build --no-default-features --features " jemalloc test-engines-rocksdb cloud-aws cloud-gcp cloud-azure"
Compiling libc v0.2.106
....
Compiling server v0.0.1 (tikv/components/server)
Finished dev [unoptimized + debuginfo] target(s) in 4m 01s
make build 1107.42s user 116.20s system 506% cpu 4:01.46 total
# After:
% time make build
cargo build --no-default-features --features " jemalloc test-engines-rocksdb cloud-aws cloud-gcp cloud-azure"
Compiling libc v0.2.106
....
Compiling server v0.0.1 (tikv/components/server)
Finished dev [unoptimized + debuginfo] target(s) in 4m 10s
make build 1179.39s user 120.74s system 518% cpu 4:10.98 total
```
Side effects
- Performance regression, Only when building, to the benefit of full debug info by default.
### Release note <!-- bugfixes or new feature need a release note -->
```release-note
None
Signed-off-by: Mattias Jonsson <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* correct a metric about compaction filter (#11938)
correct a metric about compaction filter
Signed-off-by: qupeng <[email protected]>
* raft: Fix possible panic on entries fetched callback (#11961)
close tikv/tikv#11951
Signed-off-by: Connor1996 <[email protected]>
* split_controller: refine the sample function and add some comments (#11952)
Signed-off-by: JmPotato <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* metrics: fix grid position in TiKV Details (#11936)
ref tikv/tikv#11119
Fix grid positions in TiKV Details dashboard.
Signed-off-by: tabokie <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* cdc: capture old value from txn layer dynamically (#11896)
close tikv/tikv#10091
Signed-off-by: hi-rustin <[email protected]>
Signed-off-by: qupeng <[email protected]>
Co-authored-by: hi-rustin <[email protected]>
* split_controller: reorganize the structs and add more comments (#11967)
Reorganize the structs and add more comments.
Signed-off-by: JmPotato <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* copr: support Bit column push downBit column (#11968)
close tikv/tikv#11893, ref tikv/tikv#11893, ref pingcap/tidb#31884
Signed-off-by: xiejiandong <[email protected]>
* copr: support push down mod, sysdate to TiKV (#11970)
close tikv/tikv#11916, ref tikv/tikv#11916
Signed-off-by: xiejiandong <[email protected]>
* raftstore: propose in-memory pessimistic locks before prepare merge (#11758)
ref tikv/tikv#11452
Signed-off-by: Yilin Chen <[email protected]>
* stats_monitor: reformat stats_monitor tick (#11972)
Signed-off-by: lhy1024 <[email protected]>
* copr: support extract scalar value from bit (#11980)
close tikv/tikv#11893
Signed-off-by: yisaer <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* diagnosis: support get cpu time of threads for macOS (#11978)
close tikv/tikv#11977
Add the variant for macOS into `components/tikv_util/src/sys/thread.rs`
Signed-off-by: Zhenchi <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
Co-authored-by: Jay <[email protected]>
* rust: update toolchain to fix missing rls (#11954)
close tikv/tikv#11953
Signed-off-by: Connor1996 <[email protected]>
Signed-off-by: Yilin Chen <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
Co-authored-by: Yilin Chen <[email protected]>
* cdc: separate resolved region outliers (#11991)
Separate broadcasing outlier regions and normal regions,
so 1) downstreams know where they should send resolve lock requests,
and 2) resolved ts of normal regions does not fallback.
close pingcap/tiflow#4516
close pingcap/tiflow#4311
ref pingcap/tiflow#4146
Signed-off-by: Neil Shen <[email protected]>
* *: collect key ranges for the read request in ResourceMeteringTag (#11995)
close tikv/tikv#11988
*: collect key ranges for the read request in ResourceMeteringTag
Signed-off-by: JmPotato <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* split_controller: move sample_threshold check out of the split_key iteration (#11986)
split_controller: move sample_threshold check out of the split_key iteration
Signed-off-by: JmPotato <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* copr: support insertutf8 and lowerutf8 (#11987)
ref tikv/tikv#5751
Signed-off-by: yisaer <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* copr: support rest greatest/least functions (#12003)
ref tikv/tikv#5751
Signed-off-by: yisaer <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftstore: force compact with gentleness (#12006)
* force compact with gentleness
Signed-off-by: tabokie <[email protected]>
* address comment
Signed-off-by: tabokie <[email protected]>
* split_controller: refine the LOAD_BASE_SPLIT_EVENT metrics label definitions (#12010)
split_controller: refine the LOAD_BASE_SPLIT_EVENT metrics label definitions
Signed-off-by: JmPotato <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* split_controller: fix the incorrect sampled key ranges number check (#12013)
close tikv/tikv#12012
split_controller: fix the incorrect sampled key ranges number check
Signed-off-by: JmPotato <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftstore: Support adjusting max_inflight_msgs dynamically (#11866)
close tikv/tikv#11865
Signed-off-by: Wenbo Zhang <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* copr: fix missing flag solvement for bit column in decoding (#12016)
ref tikv/tikv#7, close pingcap/tidb#32506
Signed-off-by: yisaer <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* file_system: overhaul proc-based io tracing utilities (#11873)
ref tikv/tikv#10867, fix tikv/tikv#11775
Signed-off-by: tabokie <[email protected]>
* raftstore: Wait for the apply index equals to the commit index (#11716)
ref tikv/tikv#10483
Signed-off-by: v01dstar <[email protected]>
* copr: revert 11968/11980/12016 (#12030)
ref tikv/tikv#11968, ref tikv/tikv#11980, ref tikv/tikv#12016, ref pingcap/tidb#32506
Signed-off-by: yisaer <[email protected]>
* build: fix a Mac build issue (#12027)
ref tikv/tikv#10867, ref tikv/tikv#11873
Signed-off-by: tabokie <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* copr: Extra physical table id column (#11931)
close tikv/tikv#11888
Added EXTRA_PHYSICAL_TABLE_ID_COL_ID support, to support TiDB's table partition dynamic prune mode, where a single request includes multiple partitions, but when Pessimistic lock (like `SELECT FOR UPDATE`) or ongoing transaction (having something in the tidb session local transaction buffer) each row needs its partition id / physical table ID to either lock that row or to check against the transaction buffer.
Signed-off-by: Mattias Jonsson <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* *: hotfix panic from tokio-timer (#12004)
close tikv/tikv#11940, ref tikv/tikv#11940
*: hotfix panic from tokio-timer by enlarging the length of the level vector
Signed-off-by: you06 <[email protected]>
* build: add arm64 check on SSE in makefile (#12035)
close tikv/tikv#12034
Signed-off-by: Jin Dong <[email protected]>
Co-authored-by: Yilin Chen <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* replication mode: sync state to pd (#11751)
Signed-off-by: disksing <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftstore: revert pessimistic locks status when failing to propose PrepareMerge (#11985)
ref tikv/tikv#11452
Signed-off-by: Yilin Chen <[email protected]>
* copr: support push bit column down (#12037)
ref pingcap/tidb#30738
Signed-off-by: yisaer <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* split_controller: add more sample function test cases (#12058)
split_controller: add more sample function test cases
Signed-off-by: JmPotato <[email protected]>
* copr: fix greatest/least time/date args type (#12056)
Signed-off-by: yisaer <[email protected]>
* cdc: advancing resolved ts correctly for clusters with tiflash (#12050)
close pingcap/tiflow#4461
cdc: advancing resolved ts correctly for clusters with tiflash
Signed-off-by: qupeng <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* *: bump master branch version to v6.0.0-alpha (#12077)
close tikv/tikv#12075
Signed-off-by: Yilin Chen <[email protected]>
* raftstore: check uninitialized destroy for merge (#12055)
close tikv/tikv#12048
When a peer is destroyed without being initialized, it will store
itself to peer list and the region epoch is missing. In merge if
such state is detected, it should abort merging instead of panicking.
Signed-off-by: Jay Lee <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* *: Add TruncateManager and TruncateWorker (#11553)
ref tikv/rfcs#81
Signed-off-by: longfangsong <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* TiKV supports buckets (#11763)
ref tikv/tikv#11759
Signed-off-by: qi.xu <[email protected]>
Co-authored-by: qi.xu <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* api_version: Codec for RawKV key (#12036)
ref tikv/tikv#11965
Key of RawKV is encoded as `user-key + memcomparable-padding + timestamp` in API v2.
Signed-off-by: pingyu <[email protected]>
Co-authored-by: Andy Lok <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* *: fix some typos (#12066)
Signed-off-by: cuishuang <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftstore: fix stale message cause panic (#12054)
close tikv/tikv#12023
raftstore: fix stale message cause panic
Signed-off-by: linning <[email protected]>
* raftstore: ignore async fetch result when the peer is pending removed (#12038)
close tikv/tikv#11973, close tikv/tikv#12026
Signed-off-by: Connor1996 <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* *: remove part about wechat in doc (#12101)
close tikv/tikv#12100
Signed-off-by: jackwener <[email protected]>
* engine: upgrade raft engine (#12095)
ref tikv/tikv#165, ref tikv/raft-engine#165, ref tikv/tikv#11119
Signed-off-by: Randy <[email protected]>
Co-authored-by: Xinye Tao <[email protected]>
* *: Fix possible undefined behavior for transmuting vec (#12096)
close tikv/tikv#12070
Signed-off-by: Connor1996 <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* github: Remove the pingcap/tidb-ansible from the PR template (#12102)
close tikv/tikv#12103
Signed-off-by: hi-rustin <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* copr: extract reschedule to inject a fail point (#12108)
Signed-off-by: Zhenchi <[email protected]>
* raftstore: reactivate in-memory pessimistic locking when leader transfer fails (#11883)
ref tikv/tikv#11452
Signed-off-by: Yilin Chen <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* *: check memory locks for replica read only on the leader (#12115)
close tikv/tikv#12109
Consider a reader sends read index to the leader it supposed to be, but when
the leader receives the read index message, it has stepped down to a
follower. Without this commit, a peer will check the locks and re-fill the
read index context with the result no matter what role it is. So, when the
read index request is redirected to the new leader again, it no longer carries
the context of lock checking.
This commmit changes to only do the check when the peer is a leader. Then, the
read index request will remain unchanged before being redirected to the leader.
If the lease of the leader expires, it is still safe to check on the
uncertained leader. If the heartbeat check passes and no new leader is elected,
then the check works fine. If there is a new leader, when the old leader
becomes a follower, it will clear its pending read index. Then, the reader has
to resend the read index again to the correct leader. So, generally it is safe
as long as we guarantee the check only happens on the leader.
Signed-off-by: Yilin Chen <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* apiv2 rawkv: logical deletion flag (#12090)
ref tikv/tikv#11965
Signed-off-by: haojinming <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* copr: add projection (#10689)
close tikv/tikv#12114
Signed-off-by: ichn-hu <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
Co-authored-by: Alex Chi <[email protected]>
* raftstore: adjust raft max msg size dynamically (#12018)
close tikv/tikv#12017
Signed-off-by: glorv <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* *: update deps (#12098)
- Upgrade grpcio to 0.10
- Upgrade openssl to 1.1.1m
- Remove dependency on prost
Signed-off-by: Jay Lee <[email protected]>
* tests: make raftstore tests about transactions more stable (#12122)
close tikv/tikv#12120
A new fail point "after_propose" is added to insert callback from the test.
Then we can ensure we release other key fail points after they take effect.
The other change is to retry request automatically to avoid stale command
errors.
Signed-off-by: Yilin Chen <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* metrics: support multi k8s in grafana dashboards (#12008)
close tikv/tikv#12104
Signed-off-by: just1900 <[email protected]>
* storage: set in-memory pessimistic lock feature gate to 6.0.0 (#12078)
ref tikv/tikv#11452
The in-memory pessimistic lock feature should be only enabled after
all TiKV instances have been upgraded to the new version. Otherwise,
transferring leader may be blocked because of the new protocol and
procedure when in-memory pessimistic locks exist.
So, we need to use a feature gate to make sure this feature is only
enabled after all TiKVs have been upgraded to the new version.
Signed-off-by: Yilin Chen <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftstore,cdc: pass leader transferee to cdc observer (#12124)
Signed-off-by: qupeng <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* config: simplify handling of nullable configurations (#12133)
Support deriving `OnlineConfig` on any `Option<T>`. One caveat is such `T` must implement `Default`.
* simplify handling of nullable configurations
Signed-off-by: tabokie <[email protected]>
* add credit for copied code
Signed-off-by: tabokie <[email protected]>
* patch unit test
Signed-off-by: tabokie <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftstore: support dynamic adjust max_batch_size (#11974)
close tikv/tikv#11982
Signed-off-by: glorv <[email protected]>
* server: disallow coexist of raft engine and raftdb (#12047)
* disallow coexist of raft engine and raftdb
Signed-off-by: tabokie <[email protected]>
* add defer block
Signed-off-by: tabokie <[email protected]>
* minor cleanup
Signed-off-by: tabokie <[email protected]>
* check not clear
Signed-off-by: tabokie <[email protected]>
* print out rm failure
Signed-off-by: tabokie <[email protected]>
* print log for non-empty target engine
Signed-off-by: tabokie <[email protected]>
* even more logs
Signed-off-by: tabokie <[email protected]>
* use a state machine to monitor the switch
Signed-off-by: tabokie <[email protected]>
* fix and add test
Signed-off-by: tabokie <[email protected]>
* clippy
Signed-off-by: tabokie <[email protected]>
* address comments
Signed-off-by: tabokie <[email protected]>
* address comments
Signed-off-by: tabokie <[email protected]>
* remove tempfile from engine_traits
Signed-off-by: tabokie <[email protected]>
* address comments
Signed-off-by: tabokie <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* *: add more metrics and logs for in-memory pessimistic locking (#12089)
ref tikv/tikv#11452
It adds logs when proposing locks during transferring leader and moving locks to new split regions.
The metrics about the memory used by pessimistic locks and how many locks fail to be inserted due to
reaching the memory limit are displayed in the grafana.
Signed-off-by: Yilin Chen <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* *: notify snapshot apply (#12126)
close tikv/tikv#12057
In the past, it relies on base tick to check if a snapshot is applied.
If the base tick is set to a very large value, it will take a long time
to detect snapshot is finished. This PR adds a casual message to notify
peer actively.
Signed-off-by: Jay Lee <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* txn: Fix assertion fail on non-pessimistic key with conflict (#12123)
ref tikv/tikv#12113
txn: Fix assertion fail on non-pessimistic key with conflict (ref #12113)
Signed-off-by: MyonKeminta <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* cmd: link c++ explicitly (#12145)
close tikv/tikv#12144
In the past, we rely on librocksdb-sys to include the required stdc++
library. This is not reliable after upgrading grpcio, which also needs
the same library. I suspect it doesn't include all the required symbols.
Anyway, this PR chooses to link to stdc++ explicitly in binary.
Signed-off-by: Jay Lee <[email protected]>
* Fix logic of error string match in `bad-ssts` (#12049)
Signed-off-by: Xintao <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* APIV2: Implement RawMvccSnapshot & RawMvccIterator (#12087)
ref tikv/tikv#11965
Signed-off-by: haojinming <[email protected]>
* *: add tip in contributing.md (#12129)
close tikv/tikv#12142
polish contributing.md
Signed-off-by: jackwener <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* dr-ausosync: report min resolved ts (#12052)
ref tikv/pd#4686
Signed-off-by: lhy1024 <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* txn: support read-consistency read with tso checking (#12097)
Signed-off-by: cfzjywxk <[email protected]>
* raftstore: collect statistics of buckets (#12080)
ref tikv/tikv#11718
Collect read/write flow of buckets.
Signed-off-by: 5kbpers <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* config: enable in-memory pessimistic locks by default (#12153)
close tikv/tikv#11452
Signed-off-by: Yilin Chen <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* rsmeter: add integration tests for cpu (#12140)
Signed-off-by: Zhenchi <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* cdc: don't emit resolved timestamps before scan (#12156)
Signed-off-by: qupeng <[email protected]>
* resolved_ts: fix failpoints (#12170)
Signed-off-by: lhy1024 <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* cdc: add a metrics for count of pending tasks (#12169)
Signed-off-by: qupeng <[email protected]>
* raft_log_engine: fix managedfs create encrypted file (#12163)
close tikv/tikv#12162
Signed-off-by: glorv <[email protected]>
Co-authored-by: Xinye Tao <[email protected]>
* config: limit backup workers cap (#12141)
ref #11804
Signed-off-by: Wenbo Zhang <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* split_controller: refine the sample function (#12177)
split_controller: refine the sample function
Signed-off-by: JmPotato <[email protected]>
* cdc: remove useless deps and fix typos (#12165)
close tikv/tikv#12183
Remove usless deps and fix typos
Signed-off-by: hi-rustin <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* storage: support scale store scheduler pool size (#12068)
close tikv/tikv#12067
Signed-off-by: glorv <[email protected]>
* *: update jemalloc version to fix profiling problem (#12179)
close tikv/tikv#12180
*: upgrade jemalloc version to fix sampling problem during profiling
Signed-off-by: cosven <[email protected]>
Co-authored-by: Jay <[email protected]>
* split_controller: refine and add some test cases for sample function (#12189)
ref tikv/tikv#12185
split_controller: refine and add some test cases for sample function
Signed-off-by: JmPotato <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* Foreground Quota Limiter: limit the cpu, bandwidth to provide stable QPS (#12151)
close tikv/tikv#11855, ref tikv/tikv#11856
Add a global forefront quota limiter based speed limiter to record the CPU resource overhead occupied by various requests. When the limiter reaches the limit quota, the request will be forced to sleep for a period of time to compensate. Statistics are only done on the part of process and are therefore approximate.
Signed-off-by: tonyxuqqi <[email protected]>
Signed-off-by: qi.xu <[email protected]>
Signed-off-by: zhangjinpeng1987 <[email protected]>
Signed-off-by: Xintao <[email protected]>
Co-authored-by: tonyxuqqi <[email protected]>
Co-authored-by: qi.xu <[email protected]>
Co-authored-by: zhangjinpeng1987 <[email protected]>
* UnifiedReadPool: support adjusting the pool size dynamically (#11783)
close tikv/tikv#11781
Signed-off-by: Wenbo Zhang <[email protected]>
* raftstore: drop raft message by replication state (#12076)
close tikv/tikv#12074
Signed-off-by: disksing <[email protected]>
* copr: return buckets version for unary call (#12184)
Signed-off-by: youjiali1995 <[email protected]>
* tikv-ctl: support raft engine ctl (#12019)
* tikv-ctl: support raft engine ctl
Signed-off-by: tabokie <[email protected]>
* kebab case
Signed-off-by: tabokie <[email protected]>
* fix build
Signed-off-by: tabokie <[email protected]>
* switch to 0.1.x branch
Signed-off-by: tabokie <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* dr-ausosync: make report interval configurable (#12157)
ref tikv/pd#4686
Signed-off-by: lhy1024 <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* resolved_ts: fix oom and add more metrics (#12190)
* add metrics
Signed-off-by: 5kbpers <[email protected]>
* fix
Signed-off-by: 5kbpers <[email protected]>
* remove unused code
Signed-off-by: 5kbpers <[email protected]>
* add timeout
Signed-off-by: 5kbpers <[email protected]>
Co-authored-by: qupeng <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* avoid to banish region leaders when rolling upgrade (#12167)
Signed-off-by: qupeng <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* rocksdb: use new pipelined-write to replace multibatch-write (#12060)
close tikv/tikv#12059
Signed-off-by: Little-Wallace <[email protected]>
* add vdso to the pprof blocklist (#12201) (#12209)
close tikv/tikv#9765, ref tikv/tikv#12201
Signed-off-by: ti-srebot <[email protected]>
Co-authored-by: YangKeao <[email protected]>
* raftstore:Fix memory metrics overflow on raft msgs (#12186) (#12224)
close tikv/tikv#12160, ref tikv/tikv#12186
Fix raft msg memory metrics overflow.
Signed-off-by: ti-srebot <[email protected]>
Co-authored-by: tier-cap <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* do not force compact unreplicated entries (#12181) (#12228)
close tikv/tikv#12161, ref tikv/tikv#12181
Signed-off-by: ti-srebot <[email protected]>
Co-authored-by: Xinye Tao <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* resolved_ts: fix shutdown panic (#12244) (#12245)
close tikv/tikv#12231, ref tikv/tikv#12244
resolved_ts: fix shutdown panic
Signed-off-by: ti-srebot <[email protected]>
Co-authored-by: NingLin-P <[email protected]>
* Quota limiter: make method to delay cpu more precise (#12219) (#12242)
close tikv/tikv#12218, ref tikv/tikv#12219
Use the results returned by the cpu limiter directly without extra processing
Signed-off-by: ti-srebot <[email protected]>
Co-authored-by: Xintao <[email protected]>
Co-authored-by: you06 <[email protected]>
* raftstore: validate bucket config only when it's enabled (#12237) (#12239)
close tikv/tikv#12235, ref tikv/tikv#12236, ref tikv/tikv#12237
Shouldn't report config related errors when bucket is disabled.
Signed-off-by: ti-srebot <[email protected]>
Co-authored-by: Lei Zhao <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* res_meter: fix assertion failure (#12252) (#12261)
close tikv/tikv#12234, ref tikv/tikv#12252
Signed-off-by: ti-srebot <[email protected]>
Co-authored-by: Yexiang Zhang <[email protected]>
* cdc: remove downstreams correctly when uninitialized (#12262) (#12268)
ref tikv/tikv#12262
cdc: remove downstreams correctly when uninitialized
Signed-off-by: ti-srebot <[email protected]>
Co-authored-by: qupeng <[email protected]>
* Update procinfo-rs to support pid status parse in kernel 5.x (#12263) (#12265)
ref tikv/tikv#12263
Update procinfo-rs to support pid status parse in kernel 5.x.
Signed-off-by: ti-srebot <[email protected]>
Co-authored-by: Xintao <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* Quota limiter: increase refill duration to avoid much context switch cost (#12264) (#12266)
ref tikv/tikv#12264
Quota limiter: increase refill duration to avoid much context switch cost.
Signed-off-by: ti-srebot <[email protected]>
Co-authored-by: Xintao <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftstore: only persist merge target if the merge is known to be succeeded (#12251) (#12271)
close tikv/tikv#12232, ref tikv/tikv#12251
raftstore: only persist merge target if the merge is known to be succeeded
Signed-off-by: ti-srebot <[email protected]>
Co-authored-by: NingLin-P <[email protected]>
Co-authored-by: you06 <[email protected]>
* storage: send old value to CDC after in-memory pessimistic locking (#12280) (#12282)
close tikv/tikv#12279, ref tikv/tikv#12280
Send old values through the TxnExtraScheduler after in-memory pessimisitc
locking successfully. This will help improve the old value hit rate of CDC when
enabling in-memory pessimistic locks.
Signed-off-by: ti-srebot <[email protected]>
Co-authored-by: Yilin Chen <[email protected]>
* server: fix issues with critical config check (#12250) (#12284)
close tikv/tikv#12238, ref tikv/tikv#12238, ref tikv/tikv#12250
Signed-off-by: ti-srebot <[email protected]>
Co-authored-by: Xinye Tao <[email protected]>
* metrics: remove the incorrect rate() for worker pending tasks (#12152) (#12249)
close tikv/tikv#11915, ref tikv/tikv#12152
tikv_worker_pending_task_total is a gauge. We should directly use its value
to represent the pending tasks instead of using rate(). This PR removes the
rate() on these metrics.
Signed-off-by: ti-srebot <[email protected]>
Co-authored-by: Yilin Chen <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* server: do not rename directories when migrating raft data (#12273) (#12298)
close tikv/tikv#12269, ref tikv/tikv#12269, ref tikv/tikv#12273
Signed-off-by: ti-srebot <[email protected]>
Co-authored-by: Xinye Tao <[email protected]>
* *: update grpcio (#12203) (#12299)
close tikv/tikv#12198, close tikv/tikv#12202, ref tikv/tikv#12203
update grpcio to fix UAF
Signed-off-by: ti-srebot <[email protected]>
Co-authored-by: Jay <[email protected]>
* refine some versions
Signed-off-by: CalvinNeo <[email protected]>
* add except exec_ctx
Signed-off-by: CalvinNeo <[email protected]>
* use exec_log_index/term
Signed-off-by: CalvinNeo <[email protected]>
* remove exec_ctc
Signed-off-by: CalvinNeo <[email protected]>
* before handle cdc
Signed-off-by: CalvinNeo <[email protected]>
* it compiles
Signed-off-by: CalvinNeo <[email protected]>
* fmt
Signed-off-by: CalvinNeo <[email protected]>
* Default disable raft engine (#12317)
* disable raft engine
Signed-off-by: tabokie <[email protected]>
* fix lint and test
Signed-off-by: tabokie <[email protected]>
* bump version v6.0.0
Signed-off-by: purelind <[email protected]>
* fix linux
Signed-off-by: CalvinNeo <[email protected]>
* address test deps
Signed-off-by: CalvinNeo <[email protected]>
* Fix test
Signed-off-by: CalvinNeo <[email protected]>
* f
Signed-off-by: CalvinNeo <[email protected]>
* fix linux
Signed-off-by: CalvinNeo <[email protected]>
* fix
Signed-off-by: CalvinNeo <[email protected]>
* fix some files
Signed-off-by: CalvinNeo <[email protected]>
* fix
Signed-off-by: CalvinNeo <[email protected]>
* fix
Signed-off-by: CalvinNeo <[email protected]>
* ignore some unsupported tests
Signed-off-by: CalvinNeo <[email protected]>
* Fix test_delegate_subscribe_unsubscribe flaking test (#12309) (#12315)
close tikv/tikv#12278, ref tikv/tikv#12309
Fix test_delegate_subscribe_unsubscribe flaking test.
Use the corresponding id, otherwise there will be failures due to concurrent runs of the test. (We will assign downstream ids concurrently).
Signed-off-by: ti-srebot <[email protected]>
Co-authored-by: 二手掉包工程师 <[email protected]>
Co-authored-by: qupeng <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* fix fmt
Signed-off-by: CalvinNeo <[email protected]>
* disable more disk_full tests
Signed-off-by: CalvinNeo <[email protected]>
* f
Signed-off-by: CalvinNeo <[email protected]>
* disable
Signed-off-by: CalvinNeo <[email protected]>
* remove stale peer
Signed-off-by: CalvinNeo <[email protected]>
* pick pr 12301
Signed-off-by: CalvinNeo <[email protected]>
* Fix build when openssl v1.1 and v3 are both installed (#59)
Signed-off-by: Wish <[email protected]>
* Enable frame pointer (#60)
Signed-off-by: mornyx <[email protected]>
* Upgrade pprof-rs to v0.9 (use frame-pointer), and enable CPU Profiling on ARM (#61)
Signed-off-by: mornyx <[email protected]>
* reduce log
Signed-off-by: CalvinNeo <[email protected]>
* raftstore:Fix append log entry oom (#12247)
close tikv/tikv#11379, close tikv/tikv#11598, close tikv/tikv#12107
when memory usage is high water, add flow control on the append log entries.
disable log entry cache evict functions.
Signed-off-by: tier-cap <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
Signed-off-by: CalvinNeo <[email protected]>
* fix pd.rs of reporting to PD
Signed-off-by: CalvinNeo <[email protected]>
* fix
Signed-off-by: CalvinNeo <[email protected]>
* Raftstore proxy 6.0 try (#63)
* raftstore: Introduce force leader state (#11932)
close tikv/tikv#6107, ref tikv/tikv#10483
Signed-off-by: Connor1996 <[email protected]>
* raftstore: Wait ticks for hibernated peer when doing force leader (#12364)
ref tikv/tikv#10483
Force leader is rejected on a peer who is already a leader. For the hibernated leader,
it doesn't step down to follower when quorum is missing due to not ticking election.
So wait ticks in force leader process for hibernated peers to make sure election ticking
is performed.
Signed-off-by: Connor1996 <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
* raftstore: Make unsafe recovery wait apply cover snapshot apply cases ref #10483 (#12308)
ref tikv/tikv#10483
Signed-off-by: v01dstar <[email protected]>
* raftstore: Execute recovery plan via raft (#12022)
Signed-off-by: Connor1996 <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
* Hotfix pprof (#65)
Signed-off-by: mornyx <[email protected]>
* remove log
Signed-off-by: CalvinNeo <[email protected]>
* remove log
Signed-off-by: CalvinNeo <[email protected]>
Co-authored-by: Qi Yu <[email protected]>
Co-authored-by: Xinye Tao <[email protected]>
Co-authored-by: MyonKeminta <[email protected]>
Co-authored-by: MyonKeminta <[email protected]>
Co-authored-by: Ti Chi Robot <[email protected]>
Co-authored-by: Jack Yu <[email protected]>
Co-authored-by: xiongjiwei <[email protected]>
Co-authored-by: Ping Yu <[email protected]>
Co-authored-by: andylokandy <[email protected]>
Co-authored-by: qupeng <[email protected]>
Co-authored-by: Devillove084 <[email protected]>
Co-authored-by: Harold Dost <[email protected]>
Co-authored-by: zhangjinpeng1987 <[email protected]>
Co-authored-by: Yexiang Zhang <[email protected]>
Co-authored-by: 沐 <[email protected]>
Co-authored-by: Yilin Chen <[email protected]>
Co-authored-by: Zhenchi <[email protected]>
Co-authored-by: Harold Dost <[email protected]>
Co-authored-by: NingLin-P <[email protected]>
Co-authored-by: Peng Guanwen <[email protected]>
Co-authored-by: 山岚 <[email protected]>
Co-authored-by: Ryan Leung <[email protected]>
Co-authored-by: xufei <[email protected]>
Co-authored-by: 3pointer <[email protected]>
Co-authored-by: LemonHX <[email protected]>
Co-authored-by: you06 <[email protected]>
Co-authored-by: guo-shaoge <[email protected]>
Co-authored-by: Wenxuan <[email protected]>
Co-authored-by: Zwb <[email protected]>
Co-authored-by: Wenxuan <[email protected]>
Co-authored-by: Jay <[email protected]>
Co-authored-by: Liqi Geng <[email protected]>
Co-authored-by: Jianjun Liao <[email protected]>
Co-authored-by: Lei Zhao <[email protected]>
Co-authored-by: xiejiandong <[email protected]>
Co-authored-by: 5kbpers <[email protected]>
Co-authored-by: lhy1024 <[email protected]>
Co-authored-by: Yao Zongyou <[email protected]>
Co-authored-by: Connor <[email protected]>
Co-authored-by: tonyxuqqi <[email protected]>
Co-authored-by: qi.xu <[email protected]>
Co-authored-by: zhouqiang <[email protected]>
Co-authored-by: Xiang Zhang <[email protected]>
Co-authored-by: Mini256 <[email protected]>
Co-authored-by: Xiaoguang Sun <[email protected]>
Co-authored-by: Aarush Bhat <[email protected]>
Co-authored-by: Yifan Xu <[email protected]>
Co-authored-by: Suhaha <[email protected]>
Co-authored-by: goroutine <[email protected]>
Co-authored-by: Croxx <[email protected]>
Co-authored-by: zhongyong jin <[email protected]>
Co-authored-by: Mattias Jonsson <[email protected]>
Co-authored-by: JmPotato <[email protected]>
Co-authored-by: hi-rustin <[email protected]>
Co-authored-by: Song Gao <[email protected]>
Co-authored-by: Neil Shen <[email protected]>
Co-authored-by: Yang Zhang <[email protected]>
Co-authored-by: Jin Dong <[email protected]>
Co-authored-by: disksing <[email protected]>
Co-authored-by: 龙方淞 <[email protected]>
Co-authored-by: cui fliter <[email protected]>
Co-authored-by: jakevin <[email protected]>
Co-authored-by: randy <[email protected]>
Co-authored-by: haojinming <[email protected]>
Co-authored-by: 虎 <[email protected]>
Co-authored-by: Alex Chi <[email protected]>
Co-authored-by: glorv <[email protected]>
Co-authored-by: Justin Huang <[email protected]>
Co-authored-by: Xintao <[email protected]>
Co-authored-by: haojinming <[email protected]>
Co-authored-by: cfzjywxk <[email protected]>
Co-authored-by: cosven <[email protected]>
Co-authored-by: Wallace <[email protected]>
Co-authored-by: ti-srebot <[email protected]>
Co-authored-by: YangKeao <[email protected]>
Co-authored-by: tier-cap <[email protected]>
Co-authored-by: purelind <[email protected]>
Co-authored-by: Wenxuan <[email protected]>1 parent 8415ad1 commit fbd13eeCopy full SHA for fbd13ee
1 file changed
+1
-1
lines changedcomponents/server/src/lib.rs
Copy file name to clipboardexpand all lines: components/server/src/lib.rs+1-1
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
| 37 | + | |
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
| |||
0 commit comments