Adds better abilities to check, what exactly was rate limited #2901

TheCataliasTNT2k · 2024-06-12T22:47:11Z

Add "OwnedRoute" to "RatelimitInfo" which allows to check the cause of the rate limit and which identifiers were affected (e.g. which channel).

…erenity-rs#2646) This avoids having to allocate to store fixed length (replaced with normal array) or fixed capacity (replaced with `ArrayVec`) collections as vectors for the purposes of putting them through the `Request` plumbing. Slight behavioral change - before, setting `params` to `Some(vec![])` would still append a question mark to the end of the url. Now, we check if the params array `is_empty` instead of `is_some`, so the question mark won't be appended if the params list is empty. Co-authored-by: Michael Krasnitski <[email protected]>

These are unnecessary. Accepting `impl Into<Arc<T>>` allows passing either `T` or `Arc<T>`.

This trades a heap allocation for messages sent along with thread creation for `Message`'s inline size dropping from 1176 bytes to 760 bytes,

…l models (serenity-rs#2656) This shrinks type sizes by a lot; however, it makes the user experience slightly different: - `FixedString` must be converted to String with `.into()` or `.into_string()` before it can be pushed to, but dereferences to `&str` as is. - `FixedArray` must be converted to `Vec` with `.into()` or `.into_vec()` before it can be pushed to, but dereferences to `&[T]` as is. The crate of these types is currently a Git dependency, but this is fine for the `next` branch. It needs some basic testing, which Serenity is perfect for, before a release will be made to crates.io.

…s push over the limit (serenity-rs#2660)

Fixes serenity-rs#2664

…enity-rs#2668) This commit: - switches from `u64` to `i64` in `CreateCommandOption::min_int_value` and `CreateCommandOption::max_int_value` to accommodate negative integers in Discord's integer range (between -2^53 and 2^53). Values outside this range will cause Discord's API to return an error. - switches from `i32` to `i64` in `CreateCommandOption::add_int_choice` and `CreateCommandOption::add_int_choice_localized` to accommodate Discord's complete integer range (between -2^53 and 2^53). Values outside this range will cause Discord's API to return an error.

This cache was just duplicating information already present in `Guild::members` and therefore should be removed. This saves around 700 MBs for my bot (pre-`FixedString`). This has to refactor `utils::content_safe` to always take a `Guild` instead of`Cache`, but in practice it was mostly pulling from the guild cache anyway and this means it is more likely to respect nicknames and other information, while losing the ability to clean mentions from DMs, which do not matter.

`Embed::fields` previously had to stay as a `Vec` due to `CreateEmbed` wrapping around it, but by implementing `Serialize` manually we can overwrite the `Embed::fields` with a normal `Vec`, for a small performance hit on serialization while saving some space for all stored `Embed`s.

Simply missed these when finding and replacing.

This uses the `bool_to_bitflags` macro to remove boolean (and optional boolean) fields from structs and pack them into a bitflags invocation, so a struct with many bools will only use one or two bytes, instead of a byte per bool as is. This requires using getters and setters for the boolean fields, which changes user experience and is hard to document, which is a significant downside, but is such a nice change and will just become more and more efficient as time goes on.

…rs#2681) This swaps fields that store `Option<Int>` for `Option<NonMaxInt>` where the maximum value would be ludicrous. Since `nonmax` uses `NonZero` internally, this gives us niche optimisations, so model sizes can drop some more. I have had to include a workaround for [serenity-rs#17] in `optional_string` by making my own `TryFrom<u64>`, so that should be removable once that issue is fixed. [serenity-rs#17]: LPGhatguy/nonmax#17

A couple of clippy bugs have been fixed and I have shrunk model sizes enough to make `clippy::large_enum_variant` go away.

A discord bot library should not be using the tools reserved for low level OS interaction/data structure libraries.

)

Discord seems to internally default Ids to 0, which is a bug whenever exposed, but this makes ID parsing more resilient. I also took the liberty to remove the `From<NonZero*>` implementations, to prevent future headaches, as it was impossible to not break public API as we exposed `NonZero` in `*Id::parse`.

…nity-rs#2694) This, 1. shrinks the size of Request, when copied around, as it doesn't have to store the max capacity at all times 2. shrinks llvm-lines (compile time metric) for my bot in debug from `1,153,519` to `1,131,480` as no monomorphisation has to be performed for `MAX_PARAMS`.

Follow-up to serenity-rs#2694. When `Request::params` was made into an ArrayVec, the `Option` around it was removed in order to avoid having to add a turbofish on `None` to specify the value of `MAX_PARAMS`. Also, `Request::new` also needed to be changed so that the value of `MAX_PARAMS` could be inferred. Now that the field is a slice again, we can wrap it in `Option` again (at no cost to size, thanks to niche opts). We ensure we never store the redundant `Some(&[])` by checking for an empty slice and storing `None` instead. This way, we ensure we never read an empty slice out of the `Some` variant.

The instrument macros generate 2% of Serenity's release mode llvm-lines, and are proc-macros so hurt compile time in that way, so this limits them to opt-in. This commit also fixes the issues that the instrument macro was hiding, such as results that didn't ever error and missing documentation.

This signature is hard to use as `None` cannot infer the type of the generic. I also replaced `Option<u8>` with `Option<NonMaxU8>` as it's more efficient and will make the user think of the maximum value.

…nity-rs#2698) This removes inefficient `IntoIterator` generics and instead takes what is actually required. I also reworked `reorder_channels` to allow for keeping the generic, as it actually does only just need iterator semantics.

Previously, someone assumed that `Ratelimiter` was going to be cloned, so put a ton of `Arc`s everywhere. This was unneeded, and before dashmap, so the buckets were also stored massively inefficiently. This fixes all that. I had to shuffle around the `Ratelimit` methods a little bit to return their sleep time instead of sleeping themselves, so I didn't have to hold a dashmap lock over an `.await`.

This removes multiple error variants and overall cleans up the codebase by moving overflow checks into two `ModelError` variants.

Shrinks `size_of::<Error>` from 136 bytes to 64 bytes, while removing unused variants. This will improve performance for any method returning `Result<T>` where `T` is less than the size of `Error` as both `Result`'s `Ok` and `Err` have to be allocated stack space.

The compiler knows best as inlining is quite complicated. This should help with compile times, significantly.

This field is documented as nullable but that isn't reflected in the model. I also took the opportunity to replace the Option<Vec> with a default Vec.

This was mistakenly removed in serenity-rs#2278 and wasn't caught because of the wildcard pattern in the match.

…-rs#2852)

Users should realistically be checking the permissions themselves, or handling the HTTP error from Discord. This removes any cases where permission checking inside the library is broken because of Discord's changes or due to oversights. This also changes the documentation on the prune functionality, this was recently changed to also require `MANAGE_GUILDS` as well as `KICK_MEMBERS`.

…enity-rs#2856)

This isn't needed anymore because simd-json has been removed on `next`.

TheCataliasTNT2k · 2024-06-12T22:51:21Z

The failing check seems broken, at least I do not get any error when looking at the details...

jamesbt365 · 2024-06-12T22:54:02Z

The failing check seems broken, at least I do not get any error when looking at the details...

cargo +nightly fmt --all

src/http/routing.rs

GnomedDev · 2024-06-16T15:28:34Z

src/http/routing.rs

+        #[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
+        enum RouteKind {
+            $($name,)+
+        }


Why was this moved?

To have two similar blocks next to another (Route and OwnedRoute).
I could move it back, but in my opinion it is more readable like this.

GnomedDev · 2024-06-16T15:29:21Z

src/http/routing.rs

+            /// please match the variants you are interested in.
+            #[must_use]
+            #[allow(unused_variables)] // prevent compiler from complaining about unused variables
+            pub fn get_common_identifiers(&self) -> RatelimitCause {


Suggested change

pub fn get_common_identifiers(&self) -> RatelimitCause {

pub fn cause(&self) -> RatelimitCause {

This does not really return the "cause" of the rate limit but the parts of the path, which was rate limited.
The "cause" would be the deletion of a message or something like that.

So the type should not be caused RatelimitCause, it should be called RatelimitPathParts or something else.

That is better, yes.
Can the method name stay then?

GnomedDev · 2024-06-16T15:30:55Z

src/http/routing.rs

@@ -27,23 +27,24 @@ enum RatelimitingKind {
 macro_rules! routes {
    ($lt:lifetime, {
        $(
-            $name:ident $({ $($field_name:ident: $field_type:ty),* })?,
+            $name:ident $({ $($field_name_route:ident: $field_type:ty | $field_type_owned:ty),* })?,


Instead of making the macro syntax messier, can you make this use traits instead? I'm pretty sure ToOwned should handle this, but if it doesn't you can just write a simple private trait.

How? I somehow need to know the type, which the OwnedRoute should use. I do not know, how a trait would be able to do that.
The problem is the "&str" type. Can you please provide an example for how to do it?

The owned type is simply <$field_type as ToOwned>::Owned, and if that doesn't return the correct types you can duplicate the definition of ToOwned and implement it for the types needed.

Will do that.

GnomedDev · 2024-06-16T15:33:45Z

src/http/routing.rs

+            }
+
+            #[must_use]
+            pub fn get_owned_route(&self) -> OwnedRoute {


Rust convention is to name getters without the prefix, but since this is a &self -> Owned I think the convention would be to_owned_route... however that's just ToOwned, so implement ToOwned instead.

I did not use toOwned, because this is not really a clone and should not be treated as one, which is expected by the trait however.

With that logic ToOwned should not be a thing, but this is exactly what it's for. This is internally just converting str -> String which is exactly the point of ToOwned.

GnomedDev and others added 30 commits June 9, 2024 22:33

Remove deprecated items (serenity-rs#2645)

007c360

Remove more deprecated items (serenity-rs#2651)

8df3f63

Remove *_arc methods (serenity-rs#2654)

651380e

These are unnecessary. Accepting `impl Into<Arc<T>>` allows passing either `T` or `Arc<T>`.

Clean up ShardManager/ShardQueuer/ShardRunner (serenity-rs#2653)

c564016

Put Message::thread behind a Box (serenity-rs#2658)

be79bdb

This trades a heap allocation for messages sent along with thread creation for `Message`'s inline size dropping from 1176 bytes to 760 bytes,

Fix truncation when using FixedString<u8> where multibyte character…

2db8487

…s push over the limit (serenity-rs#2660)

Encode timestamps in RFC3339 (serenity-rs#2665)

f20e8fc

Fixes serenity-rs#2664

Use FixedArray and FixedString in model enums (serenity-rs#2675)

eb0a598

Simply missed these when finding and replacing.

Remove unused warning #[allow(...)]s (serenity-rs#2682)

e92d095

A couple of clippy bugs have been fixed and I have shrunk model sizes enough to make `clippy::large_enum_variant` go away.

Get rid of unsafe (serenity-rs#2686)

22a8ec8

A discord bot library should not be using the tools reserved for low level OS interaction/data structure libraries.

Swap Git dependencies for crates.io published versions (serenity-rs#2685

613c04c

)

Rewrite builders to take Cows (serenity-rs#2688)

205ef79

Remove unnecessary Vec in WsClient (serenity-rs#2705)

918ab9b

Remove impl Into<Option> (serenity-rs#2701)

a99e06b

This signature is hard to use as `None` cannot infer the type of the generic. I also replaced `Option<u8>` with `Option<NonMaxU8>` as it's more efficient and will make the user think of the maximum value.

Handle overflow checks in central locations (serenity-rs#2697)

79d87e7

This removes multiple error variants and overall cleans up the codebase by moving overflow checks into two `ModelError` variants.

Remove manual #[inline] attributes (serenity-rs#2702)

49a4580

The compiler knows best as inlining is quite complicated. This should help with compile times, significantly.

GnomedDev and others added 14 commits June 9, 2024 23:28

Fix AuditLogEntry::user_id (serenity-rs#2846)

87345a0

This field is documented as nullable but that isn't reflected in the model. I also took the opportunity to replace the Option<Vec> with a default Vec.

Restore match arm for Error::Gateway variant (serenity-rs#2847)

b213438

This was mistakenly removed in serenity-rs#2278 and wasn't caught because of the wildcard pattern in the match.

Remove multiple event handler support (serenity-rs#2849)

3d50a65

Use RwLock for active collectors collection (serenity-rs#2851)

6406655

Provide old presence data from the cache on presence update (serenity…

8edab28

…-rs#2852)

Allow filtering events before they are dispatched (serenity-rs#2853)

fe0f087

Implement ToArrayString for Mention (serenity-rs#2857)

74659fc

Add PartialGuild::greater_member_hierarchy + member_highest_role (ser…

776185b

…enity-rs#2856)

Remove recently deprecated items

e855379

Update bool_to_bitflags to fix compilation without unstable

7f18f11

Remove RUSTFLAGS from .cargo/config.toml (serenity-rs#2875)

3e03891

This isn't needed anymore because simd-json has been removed on `next`.

Remove deprecated link_ensured method (serenity-rs#2898)

eb8495f

Adds "OwnedRoute"

e715b5e

TheCataliasTNT2k changed the base branch from current to next June 12, 2024 22:47

github-actions bot added the http Related to the `http` module. label Jun 12, 2024

Reformats Code to pass checks

964c006

GnomedDev requested changes Jun 16, 2024

View reviewed changes

Resolves change requests

54e207c

GnomedDev force-pushed the next branch 2 times, most recently from c01e57d to 842373b Compare June 22, 2024 20:38

arqunis force-pushed the next branch from adf4673 to 61d6ddb Compare July 30, 2024 17:27

GnomedDev force-pushed the next branch from 9d41131 to b167b4d Compare August 16, 2024 13:22

GnomedDev force-pushed the next branch from 2645fcc to 4d54465 Compare October 7, 2024 22:47

GnomedDev force-pushed the next branch from 4d54465 to 391b234 Compare October 20, 2024 17:32

GnomedDev force-pushed the next branch 3 times, most recently from 9296017 to efb5820 Compare November 15, 2024 19:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds better abilities to check, what exactly was rate limited #2901

Adds better abilities to check, what exactly was rate limited #2901

TheCataliasTNT2k commented Jun 12, 2024

TheCataliasTNT2k commented Jun 12, 2024

jamesbt365 commented Jun 12, 2024

GnomedDev Jun 16, 2024

TheCataliasTNT2k Jun 20, 2024

GnomedDev Jun 16, 2024

TheCataliasTNT2k Jun 20, 2024

GnomedDev Jun 22, 2024

TheCataliasTNT2k Jun 24, 2024

GnomedDev Jun 16, 2024

TheCataliasTNT2k Jun 20, 2024

GnomedDev Jun 22, 2024

TheCataliasTNT2k Jun 24, 2024

GnomedDev Jun 16, 2024

TheCataliasTNT2k Jun 20, 2024

GnomedDev Jun 22, 2024

	pub fn get_common_identifiers(&self) -> RatelimitCause {
	pub fn cause(&self) -> RatelimitCause {

Adds better abilities to check, what exactly was rate limited #2901

Are you sure you want to change the base?

Adds better abilities to check, what exactly was rate limited #2901

Conversation

TheCataliasTNT2k commented Jun 12, 2024

TheCataliasTNT2k commented Jun 12, 2024

jamesbt365 commented Jun 12, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment