Skip to content

Users autoblock feature #305

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 81 commits into
base: main
Choose a base branch
from
Open

Users autoblock feature #305

wants to merge 81 commits into from

Conversation

entantoencuanto
Copy link
Contributor

@entantoencuanto entantoencuanto commented May 8, 2024

This PR implements a feature which allows to detect spam users and block them based on a set of custom rules which generate a score for each user. By defining a threshold, the users who are candidates for blocking are classified.

The scores calculation generates a file that is used to generate a report which is sent to admins once generated and can be downloaded.

The page of the feature allows to manage the rules and configure different actions:

  • If a threshold is set the admins can calculate the scores and receive a report with the detected users
  • If the admin checks the "Perform users blocking" option the scores calculation is performed and the detected users are blocked. In this case the admin can configure blocked users to receive a notification. If the notifications are set to true a mandatory block justification message has to be configured: This message will be included in the notifications of each blocked user
  • There is also a "Allow to perform the configured users block from a task" check. If it's marked there is a task (to execute it run bundle exec rails decidim_decidim_awesome:autoblock_users:run_scheduled_block) that can be scheduled with cron jobs. The task inspects the configuration of the feature on each organization and runs the automatic block if this option was marked using the threshold, the users notification settings and the admin who saved the configuration (an admin is required to block users).

A rule can be defined with several criteria:

  • About user section is blank.
  • User has not created contents (the application inspects the presence of comments, comment_votes, proposals, proposals_votes, collaborative_drafts, amendments, debates, initiatives, initiatives_votes, meetings, meetings_answers, budgets_orders, forms_answers, blog_posts, follows, endorsements or reports authored by the user).
  • About user section or comments created by the user contains links.
  • The user has not confirmed their email address.
  • The user email belongs to a list.
  • The about section or comments created by the user contain URLs with certain domains.

Each rule, in addition to the criterion, has other attributes:

  • Weight: If the user If the user complies with the rule, he/she scores with it.
  • Application type: The user is scored when the condition of the criterion is met or when the condition is not met.
  • Allow list: A list of domains separated by spaces. The users with the email domain in the list will not be considered by the rule. In the case of the last criterion the list is applied to the URLs found in the about section or comments
  • Block list: A list of domains separated by spaces. Only the users with the email domain in the list will be considered by the rule. In the case of the last criterion the list is applied to the URLs found in the about section or comments

image

Base automatically changed from develop to main August 1, 2024 09:33
@aramollis
Copy link

@microstudi we finished the job here some months ago. Anything pending to be able to add to next release of Awesome?

@microstudi
Copy link
Contributor

@microstudi we finished the job here some months ago. Anything pending to be able to add to next release of Awesome?

This PR is marked as a Draft, also it does say "WIP" in the description and, with a quick look, it does not have any specs. Please fix things and I'll review it!

@aramollis
Copy link

@microstudi we finished the job here some months ago. Anything pending to be able to add to next release of Awesome?

This PR is marked as a Draft, also it does say "WIP" in the description and, with a quick look, it does not have any specs. Please fix things and I'll review it!

Can you complete @entantoencuanto ? Tx!

@entantoencuanto entantoencuanto force-pushed the users_autoblock branch 2 times, most recently from 1b7982f to bc0e0ee Compare December 2, 2024 17:57
aramollis added a commit to PopulateTools/decidim-module-decidim_awesome that referenced this pull request Dec 3, 2024
Updated the README to reflect changes introduced with the PR for the Autoblock feature: decidim-ice#305, addressing issue decidim-ice#299.
@aramollis
Copy link

Can you review @microstudi ?

@microstudi
Copy link
Contributor

Sorry, I didn't notice this @aramollis. I see that you have a README updated in another place, can you port it here? meanwhile I'll test and review this between this and the next week

@aramollis
Copy link

Sorry, I didn't notice this @aramollis. I see that you have a README updated in another place, can you port it here? meanwhile I'll test and review this between this and the next week

I'm not sure where should be defined and I tried to add in the main readme of the module.
I think I haven't access everywhere, so I copy here, and you or @entantoencuanto can put in the right place? 😅
I can add more detail if it is needed...

#### 22. Users Automatic Blocks
Automatically block users based on activities or suspicious data using customizable rules. 
![treshold_actions](https://github.com/user-attachments/assets/6138226b-e33e-4e15-838b-452e80853b40)
##### 1) Add Rules
Define how and when each rule applies:
- About user section is blank
- User has not created content
- Comments or about section contains links
- User email is not confirmed
- Email domain included in the list (can also act as an allow/block list)
![new_rule](https://github.com/user-attachments/assets/cfd5c2fe-6ae8-4680-8683-35c7d5546f52)
##### 2) Define the Threshold
Each rule contributes weight to the "spam risk index," and users who exceed the defined threshold can be blocked.
For example, if the threshold is set to "6" and each of the two defined rules has a weight of 5, both conditions must be met for a user to be flagged, as shown in the example screenshot.
##### 3) Possible Actions
- If "Perform users blocking" is unchecked: Calculates scores and generates a CSV with flagged users for review.
- If checked: Prompts for confirmation, blocks flagged users, and generates a CSV with blocked users.
##### Recommendation:
Review the CSV of flagged users before confirming to block them. However, blocked users can still be reviewed and unblocked later through Global Moderation > Blocked Users (/admin/moderated_users?blocked=true).

@aramollis
Copy link

Sorry, I didn't notice this @aramollis. I see that you have a README updated in another place, can you port it here? meanwhile I'll test and review this between this and the next week

I'm not sure where should be defined and I tried to add in the main readme of the module. I think I haven't access everywhere, so I copy here, and you or @entantoencuanto can put in the right place? 😅 I can add more detail if it is needed...

ping @entantoencuanto @microstudi Tx!

@microstudi
Copy link
Contributor

@entantoencuanto nice to see that you are finishing this, please try to merge main if you can. Let me know when ready !

@entantoencuanto
Copy link
Contributor Author

Yes, I hope to get everything ready in the next few days.

entantoencuanto pushed a commit that referenced this pull request May 27, 2025
Updated the README to reflect changes introduced with the PR for the Autoblock feature: #305, addressing issue #299.
Updated the README to reflect changes introduced with the PR for the Autoblock feature: #305, addressing issue #299.
Copy link

codecov bot commented May 27, 2025

Codecov Report

Attention: Patch coverage is 50.74906% with 263 lines in your changes missing coverage. Please review.

Project coverage is 88.05%. Comparing base (41fdbca) to head (78a32c8).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
...cidim_awesome/admin/users_autoblocks_controller.rb 26.80% 71 Missing ⚠️
...s/decidim/decidim_awesome/admin/autoblock_users.rb 25.60% 61 Missing ⚠️
...b/tasks/decidim_awesome_autoblock_users_tasks.rake 21.15% 41 Missing ⚠️
...ecidim_awesome/users_autoblocks_scores_exporter.rb 32.25% 21 Missing ⚠️
.../decidim_awesome/users_autoblocks_report_mailer.rb 26.92% 19 Missing ⚠️
...cidim_awesome/admin/update_users_autoblock_rule.rb 42.85% 12 Missing ⚠️
...cidim_awesome/admin/create_users_autoblock_rule.rb 45.00% 11 Missing ⚠️
...idim_awesome/admin/destroy_users_autoblock_rule.rb 50.00% 9 Missing ⚠️
...idim/decidim_awesome/block_user_mailer_override.rb 41.66% 7 Missing ⚠️
...idim_awesome/admin/users_autoblocks_config_form.rb 76.19% 5 Missing ⚠️
... and 3 more
Additional details and impacted files
@@             Coverage Diff             @@
##             main     #305       +/-   ##
===========================================
+ Coverage   60.06%   88.05%   +27.98%     
===========================================
  Files         148      162       +14     
  Lines        3794     4328      +534     
===========================================
+ Hits         2279     3811     +1532     
+ Misses       1515      517      -998     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@entantoencuanto entantoencuanto marked this pull request as ready for review May 29, 2025 13:39
@entantoencuanto
Copy link
Contributor Author

entantoencuanto commented May 29, 2025

Hi, @microstudi , I have rebased the branch on main and added some tests and I think that the PR is ready for review. Ups, the description of the PR is missing, sorry, I'm going to add it

@entantoencuanto entantoencuanto marked this pull request as draft May 29, 2025 13:44
@microstudi
Copy link
Contributor

Is ready now @entantoencuanto can you put it ready to review?

@entantoencuanto
Copy link
Contributor Author

Yes, I'm going to add a few more tests but I don't expect to change the main code.

@entantoencuanto entantoencuanto marked this pull request as ready for review May 29, 2025 16:01
@microstudi
Copy link
Contributor

@entantoencuanto I've deployed this to https://edge-awesome.dev.pokecode.net so we can test it!

Copy link
Contributor

@microstudi microstudi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, looks amazing.
I have some initial comments while we test it. Let me know what you think.
I would specially change the use of constants, instead let's use initializers or registry manifests to make this more expandable.

BTW, pleas add a CHANGELOG entry!

end

def default_block_justification_message
DEFAULT_BLOCK_JUSTIFICATION_MESSAGE
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't we just use a default i18n key here?

Comment on lines +11 to +18
USERS_AUTOBLOCKS_TYPES = {
"about_blank" => { has_variable: false, default_blocklist: true },
"activities_blank" => { has_variable: false, default_blocklist: true },
"links_in_comments_or_about" => { has_variable: true, default_blocklist: true },
"email_unconfirmed" => { has_variable: true, default_blocklist: true },
"email_domain" => { has_variable: true, default_blocklist: true },
"links_in_comments_or_about_with_domains" => { has_variable: true, default_blocklist: true }
}.freeze
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to put this in an initializer, this facilitates future expansions don't you think? It also allows implementator to remove some of the types if needed

Comment on lines +161 to +179
DEFAULT_ACTIVITIES = {
comments: ->(user) { Decidim::Comments::Comment.where(author: user).count },
comment_votes: ->(user) { Decidim::Comments::CommentVote.where(author: user).count },
proposals: ->(user) { Decidim::Proposals::Proposal.coauthored_by(user).count },
proposals_votes: ->(user) { Decidim::Proposals::ProposalVote.where(author: user).where(author: user).count },
collaborative_drafts: ->(user) { Decidim::Proposals::CollaborativeDraft.coauthored_by(user).count },
amendments: ->(user) { Decidim::Amendment.where(amender: user).count },
debates: ->(user) { Decidim::Debates::Debate.where(author: user).count },
initiatives: ->(user) { Decidim::Initiative.where(author: user).count },
initiatives_votes: ->(user) { Decidim::InitiativesVote.where(author: user).count },
meetings: ->(user) { Decidim::Meetings::Meeting.where(author: user).count },
meetings_answers: ->(user) { Decidim::Meetings::Answer.where(user:).count },
budgets_orders: ->(user) { Decidim::Budgets::Order.where(user:).count },
forms_answers: ->(user) { Decidim::Forms::Answer.where(user:).count },
blog_posts: ->(user) { Decidim::Blogs::Post.where(author: user).count },
follows: ->(user) { Decidim::Follow.where(user:).count },
endorsements: ->(user) { Decidim::Endorsement.where(author: user).count },
reports: ->(user) { Decidim::Report.where(user:).count }
}.freeze
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I see this is coupled with the available_keys.
I think a better approach for this would be to add a manifest where to register the user possible activities. This is for two reasons, first it will encapsulate the logic of detecting them in a separate, isolated place.
The other reason is that there are more external modules that can be applied here and it would be nice for these modules to incorporate their own resources here.

I imagine having an initializer in the engine.rb such as:

initializer "decidim_decidim_awesome.autoblock_users" do |_app|
  if DecidimAwesome.enabled?(:users_autoblock)
   if Decidim.module_installed?(:comments)
     Decidim::DecidimAwesome.user_activities_registry.register(:comments, ->(user) { Decidim::Comments::Comment.where(author: user).count })
     Decidim::DecidimAwesome.user_activities_registry.register(:comment_votess, ->(user) { Decidim::Comments::CommentVote.where(author: user).count })
    end
  end
end

Then other plugins (or applications) could register other resources in a similar faishon:

Decidim::DecidimAwesome.configure do 
  Decidim::DecidimAwesome.user_activities_registry.register(:suggestion, ->(user) { Decidim::ParticipatoryDocuments::Suggestion.where(author: user).count })
end

@@ -66,6 +66,10 @@ module DecidimAwesome
true
end

config_accessor :users_autoblocks do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a description in here, also specify that this can be disabled as well :disabled


initializer "decidim_decidim_awesome.override_blocked_users_notifications" do
config.to_prepare do
Decidim::BlockUserMailer.prepend(Decidim::DecidimAwesome::BlockUserMailerOverride)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Decidim::BlockUserMailer.prepend(Decidim::DecidimAwesome::BlockUserMailerOverride)
if DecidimAwesome.enabled?(:users_autoblock)
Decidim::BlockUserMailer.prepend(Decidim::DecidimAwesome::BlockUserMailerOverride)
end

decidim_admin_decidim_awesome.users_autoblocks_path,
position: 11,
icon_name: "user-unfollow-line",
if: menus[:users_autoblocks]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are missing to specify the menus[:users_autoblock]

def menus
          @menus ||= {
            users_autoblock: config_enabled?(:users_autoblock),

@entantoencuanto
Copy link
Contributor Author

Thanks for this, looks amazing. I have some initial comments while we test it. Let me know what you think. I would specially change the use of constants, instead let's use initializers or registry manifests to make this more expandable.

BTW, pleas add a CHANGELOG entry!

Yes, you are right, It's much more flexible the use of initializers. I'm going to add all the changes you suggested. I'll let you know when they're all ready.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants