Default package sources (#4364)

* Set default package sources
* Add release notes
* Add documentation
* Index URL -> Source URL

internal/manager/config/config.go

@@ -138,8 +138,14 @@ const (
 	PluginsSettingPrefix = PluginsSetting + "."
 	DisabledPlugins      = "plugins.disabled"
 
-	PluginPackageSources  = "plugins.package_sources"
-	ScraperPackageSources = "scrapers.package_sources"
+	sourceDefaultPath = "stable"
+	sourceDefaultName = "Community (stable)"
+
+	PluginPackageSources        = "plugins.package_sources"
+	pluginPackageSourcesDefault = "https://stashapp.github.io/CommunityScripts/stable/index.yml"
+
+	ScraperPackageSources        = "scrapers.package_sources"
+	scraperPackageSourcesDefault = "https://stashapp.github.io/CommunityScrapers/stable/index.yml"
 
 	// i18n
 	Language = "language"
@@ -1658,6 +1664,18 @@ func (i *Config) setDefaultValues() {
 
 	// Set NoProxy default
 	i.main.SetDefault(NoProxy, noProxyDefault)
+
+	// set default package sources
+	i.main.SetDefault(PluginPackageSources, map[string]string{
+		"name":       sourceDefaultName,
+		"url":        pluginPackageSourcesDefault,
+		"local_path": sourceDefaultPath,
+	})
+	i.main.SetDefault(ScraperPackageSources, map[string]string{
+		"name":       sourceDefaultName,
+		"url":        scraperPackageSourcesDefault,
+		"local_path": sourceDefaultPath,
+	})
 }
 
 // setExistingSystemDefaults sets config options that are new and unset in an existing install,

ui/v2.5/src/docs/en/Manual/Plugins.md

@@ -10,7 +10,43 @@ Plugin tasks can be implemented using embedded Javascript, or by calling an exte
 
 > **⚠️ Note:** Plugin support is still experimental and is likely to change.
 
-# Adding plugins
+## Managing Plugins
+
+Plugins can be installed and managed from the `Settings > Plugins` page. 
+
+Scrapers are installed using the `Available Plugins` section. This section allows configuring sources from which to install plugins. The `Community (stable)` source is configured by default. This source contains plugins for the current _stable_ version of stash.
+
+These are the plugin sources maintained by the stashapp organisation:
+
+| Name | Source URL | Recommended Local Path | Notes |
+|------|-----------|------------------------|-------|
+| Community (stable) | `https://stashapp.github.io/CommunityScripts/stable/index.yml` | `stable` | For the current stable version of stash. |
+| Community (develop) | `https://stashapp.github.io/CommunityScripts/develop/index.yml` | `develop` | For the develop version of stash. |
+
+Installed plugins can be updated or uninstalled from the `Installed Plugins` section.
+
+### Source URLs
+
+The source URL must return a yaml file containing all the available packages for the source. An example source yaml file looks like the following:
+
+```
+- id: <package id>
+  name: <package name>
+  version: <version>
+  date: <date>
+  requires:
+  - <ids of packages required by this package (optional)>
+  - ...
+  path: <path to package zip file>
+  sha256: <sha256 of zip>
+  metadata:
+    <optional key/value pairs for extra information>
+- ...
+```
+
+Path can be a relative path to the zip file or an external URL.
+
+# Adding plugins manually
 
 By default, Stash looks for plugin configurations in the `plugins` sub-directory of the directory where the stash `config.yml` is read. This will either be the `$HOME/.stash` directory or the current working directory.
 

ui/v2.5/src/docs/en/Manual/Scraping.md

@@ -30,18 +30,51 @@ Stash provides the following built-in scrapers:
 | Freeones | `search` Performer scraper for freeones.xxx. |
 | Auto Tag | Scene `fragment` scraper that matches existing performers, studio and tags using the filename. |
 
-## Adding Scrapers
+## Managing Scrapers
 
+Scrapers can be installed and managed from the `Settings > Metadata Providers` page. 
+
+Scrapers are installed using the `Available Scrapers` section. This section allows configuring sources from which to install scrapers. The `Community (stable)` source is configured by default. This source contains scrapers for the current _stable_ version of stash.
+
+These are the scraper sources maintained by the stashapp organisation:
+
+| Name | Source URL | Recommended Local Path | Notes |
+|------|-----------|------------------------|-------|
+| Community (stable) | `https://stashapp.github.io/CommunityScrapers/stable/index.yml` | `stable` | For the current stable version of stash. |
+| Community (develop) | `https://stashapp.github.io/CommunityScrapers/develop/index.yml` | `develop` | For the develop version of stash. |
+
+Installed scrapers can be updated or uninstalled from the `Installed Scrapers` section.
+
+### Source URLs
+
+The source URL must return a yaml file containing all the available packages for the source. An example source yaml file looks like the following:
+
+```
+- id: <package id>
+  name: <package name>
+  version: <version>
+  date: <date>
+  requires:
+  - <ids of packages required by this package (optional)>
+  - ...
+  path: <path to package zip file>
+  sha256: <sha256 of zip>
+  metadata:
+    <optional key/value pairs for extra information>
+- ...
+```
+
+Path can be a relative path to the zip file or an external URL.
+
+## Adding Scrapers manually
 
 By default, Stash looks for scraper configurations in the `scrapers` sub-directory of the directory where the stash `config.yml` is read. This will either be the `$HOME/.stash` directory or the current working directory.
 
-Scrapers are added by placing yaml configuration files (format: `scrapername.yml`) in the `scrapers` directory.
+Scrapers are added manually by placing yaml configuration files (format: `scrapername.yml`) in the `scrapers` directory.
 
 > **⚠️ Note:** Some scrapers may require more than just the yaml file, consult the individual scraper documentation
 
 After the yaml files are added, removed or edited while stash is running, they can be reloaded going to `Settings > Metadata Providers > Scrapers` and clicking `Reload Scrapers`.
-
-The stash community maintains a number of custom scraper configuration files that can be found [here](https://github.com/stashapp/CommunityScrapers).
 
 ## Using Scrapers
 

ui/v2.5/src/docs/en/ReleaseNotes/index.ts

@@ -1,5 +1,6 @@
 import v0170 from "./v0170.md";
 import v0200 from "./v0200.md";
+import v0240 from "./v0240.md";
 
 export interface IReleaseNotes {
   // handle should be in the form of YYYYMMDD
@@ -9,6 +10,11 @@ export interface IReleaseNotes {
 }
 
 export const releaseNotes: IReleaseNotes[] = [
+  {
+    date: 20231212,
+    version: "v0.24.0",
+    content: v0240,
+  },
   {
     date: 20230301,
     version: "v0.20.0",

ui/v2.5/src/docs/en/ReleaseNotes/v0240.md

@@ -0,0 +1,5 @@
+This release introduces scraper and plugin management interfaces. This allows installing, updating and removing scrapers and plugins from the WebUI. 
+
+Default package sources have been automatically configured to point at the _stable_ branches of the `CommunityScrapers` and `CommunityScripts` respositories. These branches will correspond to the current stable version of stash. 
+
+**Note:** existing scrapers and plugins will _not_ be able to be managed using the management interface. It is recommended that any existing scrapers and plugins that are available from the community repositories are backed up and removed from the applicable `scrapers` or `plugins` directory, and reinstalled using the management UI.

ui/v2.5/src/locales/en-GB.json

@@ -1088,7 +1088,7 @@
     "required_by": "Required by {packages}",
     "source": {
       "name": "Name",
-      "url": "Index URL",
+      "url": "Source URL",
       "local_path": {
         "heading": "Local Path",
         "description": "Relative path to store packages for this source. Note that changing this requires the packages to be moved manually."