feat: check resource exists before CreateOrUpdate

[PetterZhukov]

What problem does this PR solve?

There will be a large number of 409 errors when confirming the existence of resources through creation operations in CreateOrUpdate
Modify the logic to confirm the existence of resources through Exist.

Closes #6234 #6322

What is changed and how does it work?

Code changes

• Has Go code change
• Has CI related scripts change

Tests

• Unit test
• E2E test
• Manual test
• No code

Side effects

• Breaking backward compatibility
• Other side effects:

Related changes

• Need to cherry-pick to the release branch
• Need to update the documentation

Release Notes

Please refer to Release Notes Language Style Guide before writing the release note.

Fix the issue that instead of creating the resource in every sync cycle, first check whether it already exists because just use create-api will cause many 409 responses.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign weekface for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

Hi @PetterZhukov. Thanks for your PR.

I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[ti-chi-bot]

Welcome @PetterZhukov! It looks like this is your first PR to pingcap/tidb-operator 🎉

[pingcap-cla-assistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ liubog2008
❌ PetterZhukov
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[liubog2008]

please also change the comment

[liubog2008]

/run-all-tests

[liubog2008]

/ok-to-test

[codecov-commenter]

Codecov Report

❌ Patch coverage is 60.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 47.02%. Comparing base (a16b2e8) to head (440edf2).
⚠️ Report is 65 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (a16b2e8) and HEAD (440edf2). Click for more details.

HEAD has 1 upload less than BASE

Flag	BASE (a16b2e8)	HEAD (440edf2)
unittest	1	0

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #6472       +/-   ##
===========================================
- Coverage   57.19%   47.02%   -10.18%     
===========================================
  Files         259      218       -41     
  Lines       33233    32124     -1109     
===========================================
- Hits        19008    15105     -3903     
- Misses      12291    15106     +2815     
+ Partials     1934     1913       -21     

Flag	Coverage Δ
e2e	47.02% <60.00%> (?)
unittest	?

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[liubog2008]

/run-all-tests

[liubog2008]

/run-pull-e2e-kind-across-kubernetes

[liubog2008]

cc @csuzhangxc