From 84a14328020865b51ec7058af40c415b2cfe0dba Mon Sep 17 00:00:00 2001 From: Eric Hwang Date: Fri, 14 Jun 2024 15:51:19 -0700 Subject: [PATCH] [docs] Improvements to Racer model docs - Overall: - Document `*Promised` methods added in racer@1.1.0, with examples - Change the very outdated "store" term to use "backend" instead - Main Models page: - Add textual overview of important sub-pages - Rewrite docs on creating models for clarity - Add docs for `model.close()` - Contexts: - Small rewrites for clarity - Events: - Recommend that new code use `{useEventObjects: true}` - Mention that TS types only support that event-objects callback format - Hide full docs on legacy event callbacks by default - Getters: - Add docs for `getOrThrow` and `getOrDefault` added in racer@2.1.0 - Make a louder warning about in-place modififications of values returned by non-copying `get*` methods - Mutators: - Add section about error handling - Rewrite info about confirming mutations, now in its own section with examples - Add docs about `*Promised` mutator methods in that section - Document `whenNothingPending()` with appropriate warnings - Paths: - Include full example of model data structure - Update docs on local vs remote collections for clarity - Queries: - Add explanations on how to subscribe/fetch a query, via both callback and promise APIs - Add examples - Reactive functions: - Move `model.evaluate` to its own paragraph, to make it more clear what options applies to it vs `model.start` - Minor wording updates and clarifications - References: - Mention similarity to filesystem symlinks --- docs/README.md | 2 +- docs/_config.yml | 9 ++++ docs/apps.md | 4 +- docs/models.md | 79 ++++++++++++++++++--------- docs/models/backends.md | 63 ++++++++++++++++------ docs/models/contexts.md | 23 +++++--- docs/models/events.md | 18 +++++-- docs/models/getters.md | 41 ++++++++++---- docs/models/mutators.md | 87 ++++++++++++++++++++++++++---- docs/models/paths.md | 73 +++++++++++++++++-------- docs/models/queries.md | 89 ++++++++++++++++++++++++++++--- docs/models/reactive-functions.md | 42 +++++++++++---- docs/models/refs.md | 4 +- 13 files changed, 415 insertions(+), 119 deletions(-) diff --git a/docs/README.md b/docs/README.md index 4364ffd9..2df60387 100644 --- a/docs/README.md +++ b/docs/README.md @@ -24,7 +24,7 @@ The site is viewable at `http://localhost:4000/derby/`. ## With Ruby in Docker container -One-time container creation: +One-time container creation, with current directory pointing at this repo's root: ``` docker run --name derby-docs-ruby -v "$(pwd)/docs:/derby-docs" -p 127.0.0.1:4000:4000 ruby:2.7 bash -c 'cd derby-docs && bundle install && bundle exec jekyll serve -H 0.0.0.0 -P 4000 --trace' diff --git a/docs/_config.yml b/docs/_config.yml index fa63bf90..c1783eea 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -41,6 +41,15 @@ theme: just-the-docs color_scheme: derby-light # just-the-docs theme customization permalink: /:path/:name +# just-the-docs customization +callouts: + warning-red: + title: Warning + color: red + warning-yellow: + title: Warning + color: yellow + # Front matter defaults defaults: - diff --git a/docs/apps.md b/docs/apps.md index 923d4347..9166ca51 100644 --- a/docs/apps.md +++ b/docs/apps.md @@ -58,9 +58,9 @@ The server includes an app with a standard Node.js require statement. It can then use the `app.router()` method to create a router middleware for Express that handles all of the app's routes. -The server also needs to create a `store` object, which is what creates models, +The server also needs to create a `backend` object, which is what creates models, coordinates data syncing, and interfaces with databases. Stores are created via -the `derby.createStore()` method. See [Backends](models/backends). +the `derby.createBackend()` method. See [Backends](models/backends). > A typical setup can be seen in the [derby-starter](https://github.com/derbyjs/derby-starter/blob/master/lib/server.js) project, which is a node module for getting started with Derby. > diff --git a/docs/models.md b/docs/models.md index 6d56dbc9..cd312c03 100644 --- a/docs/models.md +++ b/docs/models.md @@ -6,7 +6,18 @@ has_children: true # Models -DerbyJS models are provided by [Racer](https://github.com/derbyjs/racer), a realtime model synchronization engine. By building on ShareDB, Racer enables multiple users and services to interact with the same data objects with realtime conflict resolution. Racer models have a simple getter/setter and event interface for writing application logic. +DerbyJS's models are provided by [Racer](https://github.com/derbyjs/racer), a realtime model synchronization engine built for Derby. By building on ShareDB, Racer enables multiple users and services to interact with the same data objects with realtime conflict resolution. + +Racer models can also be used without Derby, such as in backend code to interact with ShareDB-based data, or even in frontend code with another UI framework. + +A model's data can be thought of as a JSON object tree - see the [Paths documentation](models/paths) for details. + +Racer models provide functionality useful for writing real-time application logic: +- Methods to [load data into the model](backends#loading-data-into-a-model), including via [database queries](queries) +- Null-safe [getter methods](getters) and [mutator (setter) methods](mutators) +- [Reactive functions](reactive-functions) for automatically producing output data whenever any input data changes + - Built-in reactive [filter and sort](filters-sorts) functions +- [Data change events](events) for more complex situations not covered by pure reactive functions ## Racer and ShareDB @@ -14,47 +25,65 @@ Racer provides a single interface for working with local data stored in memory a Remotely synced data is stored via [ShareDB](https://github.com/share/sharedb), which means that different clients can modify the same data at the same time. ShareDB uses [Operational Transformation (OT)](https://en.wikipedia.org/wiki/Operational_transformation) to automatically resolve conflicts in realtime or offline. -On the server, Racer provides a `store`, which configures a connection to a database and pub/sub adapter. Every store connected to the same database and pub/sub system is synchronized in realtime. +On the server, Racer provides a `backend` that extends the [ShareDB Backend](https://share.github.io/sharedb/api/backend). It configures a connection to a database and pub/sub adapter. Every backend connected to the same database and pub/sub system is synchronized in realtime. -Stores create `model` objects. Models have a synchronous interface similar to interacting directly with objects. They maintain their own copy of a subset of the global state. This subset is defined via [subscriptions](backends#loading-data-into-a-model) to certain queries or documents. Models perform operations independently, and they automatically synchronize their state. +Backends create `model` objects. Models have a synchronous interface similar to interacting directly with objects. They maintain their own copy of a subset of the global state. This subset is defined via [subscriptions](backends#loading-data-into-a-model) to certain queries or documents. Models perform operations independently, and they automatically synchronize their state. Models emit events when their contents are updated, which DerbyJS uses to update the view in realtime. ## Creating models -Derby provides a model when calling application routes. On the server, it creates an empty model from the `store` associated with an app. When the server renders the page, the model is serialized. It is then reinitialized into the same state on the client. This model object is passed to app routes rendered on the client. +On the server, `backend.modelMiddleware()` provides an Express-compatible middleware function that, when run during request handling, creates a new empty model for the request and attaches it to `req.model`. Custom middleware can be added between the `modelMiddleware()` and the application routes to customize the model's data. + +When the server runs an application route, it uses `req.model` to render the page. The model state is serialized into the server-side rendered page, and then in the browser, the model is reinitialized into the same state. This model object is passed to app routes rendered on the client. + +```js +// Middleware to add req.model on each request +expressApp.use(backend.modelMiddleware()); -Derby uses the model supplied by the store.modelMiddleware by calling `req.getModel()`. To pass data from server-side express middleware or routes, the model can be retrieved via this same method and data can be set on it before passing control to the app router. +// Subsequent middleware can use the model +expressApp.use((req, res, next) => { + req.model.set('_session.userId', 'test-user'); + next(); +}); -If you would like to get or set data outside of the app on the server, you can create models directly via `store.createModel()`. +// Derby application routes use req.model for rendering +expressApp.use(derbyApp.router()); +``` -> `model = store.createModel(options)` +If you would like to get or set data on the server outside of the context of a request, you can create models directly via `backend.createModel()`. + +> `model = backend.createModel(options)` > * `options:` > * `fetchOnly` Set to true to make model.subscribe calls perform a fetch instead -> * `model` Returns a model instance associated with the given store +> * `model` Returns a model instance associated with the given backend -## Store +## Closing models -Typically, a project will have only one store, even if it has multiple apps. It is possible to have multiple stores, but a model can be associated with only a single store, and a page can have only a single model. +Models created by `modelMiddleware()` are automatically closed when the Express request ends. -> `store = derby.createStore(options)` -> * `options` See the [Backends](backends) section for information on configuration -> * `store` Returns a Racer store instance +To close a manually-created model, you can use `model.close()`. The `close()` method will wait for all pending operations to finish before closing the model. -### Methods +> `backend.close([callback])` +> * `callback` - `() => void` - Optional callback, called once the model has finished closing. -> `middleware = store.modelMiddleware()` -> * `middleware` Returns a connect middleware function +> `closePromise = backend.closePromised()` +> * Returns a `Promise` that resolves when the model has finished closing. The promise will never be rejected. -The model middleware adds a `req.getModel()` function which can be called to create or get a model (if one was already created) for a given request. It also closes this model automatically at the end of the request. +## Backend -Model's created from `req.getModel()` specify the option `{fetchOnly: true}`. This means that calls to `model.subscribe()` actually only fetch data and don't subscribe. This is more efficient during server-side rendering, since the model is only created for long enough to handle the route and render the page. The model then gets subscribed when it initializes in the browser. +Typically, a project will have only one backend, even if it has multiple apps. It is possible to have multiple backends, but a model can be associated with only a single backend, and a page can have only a single model. -```js -var expressApp = express(); -expressApp.use(store.modelMiddleware()); +> `backend = derby.createBackend(options)` +> `backend = racer.createBackend(options)` +> * `options` See the [Backends](backends) section for information on configuration +> * `backend` Returns a Racer backend instance -expressApp.get('/', function(req, res, next) { - var model = req.getModel(); -}); -``` +### Methods + +> `middleware = backend.modelMiddleware()` +> * `middleware` Returns an Express-compatible middleware function + +The model middleware creates a new model for each request and adds a `req.model` reference to that model. It also closes this model automatically at the end of the request. + +Models created by `modelMiddleware()` use the `{fetchOnly: true}` option. That means during server-side rendering, `model.subscribe()` doesn't actually register with the pub/sub system, which is more efficient since the model is only open for the short lifetime of the request. It's still tracked as a subscription so that when the model is re-initialized in the browser, the browser can register the actual subscriptions. diff --git a/docs/models/backends.md b/docs/models/backends.md index 85b35bfd..1901d4a7 100644 --- a/docs/models/backends.md +++ b/docs/models/backends.md @@ -92,31 +92,64 @@ It is not possible to set or delete an entire collection, or get the list of col ## Loading data into a model -The `subscribe`, `fetch`, `unsubscribe`, and `unfetch` methods are used to load and unload data from ShareJS. These methods don't return data directly. Rather, they load the data into a model. Once loaded, the data are then accessed via model getter methods. +The `subscribe`, `fetch`, `unsubscribe`, and `unfetch` methods are used to load and unload data from the database. These methods don't return data directly. Rather, they load the data into a model. Once loaded, the data are then accessed via model getter methods. -`subscribe` and `fetch` both return data initially, but subscribe also registers with pub/sub on the server to receive ongoing updates as the data change. +`subscribe` and `fetch` both load the requested data into the model. Subscribe also registers with pub/sub, automatically applying remote updates to the locally loaded data. -> `model.subscribe(items..., callback(err))` -> `model.fetch(items..., callback(err))` -> `model.unsubscribe(items..., callback(err))` -> `model.unfetch(items..., callback(err))` -> * `items` Accepts one or more subscribe-able items, including a document path, scoped model, or query -> * `callback` Calls back once all of the data for each query and document has been loaded or when an error is encountered +> `model.subscribe(items, callback)` +> `model.fetch(items, callback)` +> `model.unsubscribe(items, callback)` +> `model.unfetch(items, callback)` +> * `items` - `string | ChildModel | Query | Array` - Specifier(s) for one or more loadable items, such as a document path, scoped model, or query +> * `callback` - `(error?: Error) => void` - Calls back once all of the data for each item has been loaded or when an error is encountered + +There are also promise-based versions of the methods, available since racer@1.1.0. + +> `model.subscribePromised(items)` +> `model.fetchPromised(items)` +> `model.unsubscribePromised(items)` +> `model.unfetchPromised(items)` +> * These each return a `Promise` that is resolved when the requested item(s) are loaded or rejected on errors. Avoid subscribing or fetching queries by document id like `model.query('users', {_id: xxx})`. You can achieve the same result passing `'users.xxx'` or `model.at('users.xxx')` to subscribe or fetch, and it is much more efficient. If you only have one argument in your call to subscribe or fetch, you can also call `subscribe`, `fetch`, `unsubscribe`, and `unfetch` on the query or scoped model directly. ```js -var user = model.at('users.' + userId); -var todosQuery = model.query('todos', {creatorId: userId}); -model.subscribe(user, todosQuery, function(err) { +// Subscribing to a single document with a path string. +const userPath = `users.${userId}`; +model.subscribe(userPath, (error) => { if (err) return next(err); - console.log(user.get(), todosQuery.get()); - page.render(); + console.log(model.get(userPath)); }); +// Subscribing to two things at once: a document via child model and a query. +const userModel = model.at(userPath); +const todosQuery = model.query('todos', {creatorId: userId}); +model.subscribe([userModel, todosQuery], function(err) { + if (err) return next(err); + console.log(userModel.get(), todosQuery.get()); +}); + +// Promise-based API +model.subscribePromised(userPath).then( + () => { + console.log(model.get(userPath)); + }, + (error) => { next(error); } +); +// Promise-based API with async/await +try { + await model.subscribePromised([userModel, todosQuery]); + console.log(userModel.get(), todosQuery.get()); +} catch (error) { + // Handle subscribe error +} ``` -Racer internally keeps track of the context in which you call subscribe or fetch, and it counts the number of times that each item is subscribed or fetched. To actually unload a document from the model, you must call the unsubscribe method the same number of times that subscribe is called and the unfetch method the same number of times that fetch is called. However, you generally don't need to worry about calling unsubscribe and unfetch manually. +Racer internally keeps track of the context in which you call subscribe or fetch, and it counts the number of times that each item is subscribed or fetched. To actually unload a document from the model, you must call the unsubscribe method the same number of times that subscribe is called and the unfetch method the same number of times that fetch is called. + +However, you generally don't need to worry about calling unsubscribe and unfetch manually. Instead, the `model.unload()` method can be called to unsubscribe and unfetch from all of the subscribes and fetches performed since the last call to unload. + +Derby unloads all contexts when doing a full page render, right before invoking route handlers. By default, the actual unsubscribe and unfetch happens after a short delay, so if something gets resubscribed during routing, the item will never end up getting unsubscribed and it will callback immediately. -Instead, the `model.unload()` method can be called to unsubscribe and unfetch from all of the subscribes and fetches performed since the last call to unload. Derby calls this method on every full page render right before entering a route. By default, the actual unsubscribe and unfetch happens after a short delay, so if something gets resubscribed during routing, the item will never end up getting unsubscribed and it will callback immediately. +See the [Contexts documentation](contexts) for more details. diff --git a/docs/models/contexts.md b/docs/models/contexts.md index 8715041e..e620b3f8 100644 --- a/docs/models/contexts.md +++ b/docs/models/contexts.md @@ -7,21 +7,32 @@ grand_parent: Models # Data loading contexts -As data is loaded into a model with calls to fetch and subscribe, Racer tracks the number of fetches and subscribes per document path and query. Data is not removed from a model until it is released by calling unfetch and unsubscribe the matching number of times for each document or query. For example, after calling `subscribe()` on a query twice, then `unsubscribe()` once, the query would remain subscribed. It would be unsubscribed and its data would be removed from the model only after `unsubscribe()` was called once more. +Data loading contexts are an advanced feature, useful for features like pop-over dialogs and modals that need to load their own data independently from the parent page. -This behavior is helpful, since multiple parts of an application may need the same resource, but they may want perform data loading and unloading independently. For example, an edit dialog may be opened and closed while some of the same data may be displayed in a list; or a migration script may fetch data in batches in order to process a large amount of data without loading all of it into memory simultaneously. +Racer uses something like [reference counting](https://en.wikipedia.org/wiki/Reference_counting) for fetches and subscribes. As data is loaded into a model with calls to fetch and subscribe, Racer tracks the number of fetches and subscribes for each document and query. Data is not removed from a model until it is released by calling unfetch and unsubscribe the matching number of times for each document or query. -Contexts provide a way to track a group of related fetches and subscribes. In addition, they provide an `unload()` method that unfetches and unsubscribes the corresponding number of times. By default, all fetches and subscribes happen within the `'root'` context. Additional context names may be used to isolate the loading and unloading of data within the same model for independent purposes. +For example, after calling `subscribe()` on a query twice, then `unsubscribe()` once, the query would remain subscribed. It would be unsubscribed and its data would be removed from the model only after `unsubscribe()` was called once more. + +This behavior is helpful, since multiple parts of a page may need the same resource, but they may want perform data loading and unloading independently. For example, an edit dialog may be opened and closed while some of the same data may be displayed in a list; or a migration script may fetch data in batches in order to process a large amount of data without loading all of it into memory simultaneously. + +A model's context tracks all fetches and subscribes made under its context name. Calling `model.unload()` on the model will "undo" the unfetch and unsubscribe counts made under its context, while not affecting fetches and subscribes made under other contexts. + +By default, all fetches and subscribes happen within the context named `'root'`. Additional context names may be used to isolate the loading and unloading of data within the same model for independent purposes. + +Child models created with `model.at()`, `model.scope()`, etc. will inherit the context name from the parent model. > `childModel = model.context(name)` > * `name` A string uniquely identifying a context. Calling `model.context()` again with the same string will refer to the same context. By default, models have the context name `'root'` -> * `childModel` Returns a model with a context of `name`, overriding the parent model's context name. All fetch, subscribe, and unload actions performed on this childModel will have this context +> * `childModel` Returns a model with a context of `name`. All fetch, subscribe, and unload actions performed on this child model will be tracked under the new named context. The child model's path is inherited from the parent. > `model.unload([name])` -> * `name` *(optional)* Unfetch and unsubscribe from all documents and queries for the corresponding number of times they were fetched and subscribed. This will end subscriptions and remove the data from the model if no remaining fetches or subscribes hold the data in the model under a different context. Defaults to the current model context name. Specifying a `name` argument overrides the default +> * `name` *(optional)* - A specific context name to unload. Defaults to the current model's context name. +> * Undoes the fetches and subscribes for all documents and queries loaded under the context. For each piece of data, if no other contexts hold fetches or subscribes on it, then this will end the subscription and remove the data from the model. > `model.unloadAll()` -> * Unload each context within a model. Results in all remotely loaded data being removed from a model. (Data within [local collections](paths#local-and-remote-collections) will remain.) +> * Unload all contexts within a model. Results in all remotely loaded data being removed from a model. +> * Data within [local collections](paths#local-and-remote-collections) will remain. +> * This is automatically called by Derby prior to doing a client-side render of a new page. ## Usage example diff --git a/docs/models/events.md b/docs/models/events.md index 1d78cd3c..efbcb1c0 100644 --- a/docs/models/events.md +++ b/docs/models/events.md @@ -15,14 +15,16 @@ Racer emits events whenever it mutates data via `model.set()`, `model.push()`, e `model.on()` and `model.once()` accept a second argument for these mutation events. The second argument is a path pattern that will filter emitted events, calling the handler function only when a mutator matches the pattern. Path patterns support a single segment wildcard (`*`) anywhere in a path, and a multi-segment wildcard (`**`) at the end of the path. The multi-segment wildcard alone (`'**'`) matches all paths. > `listener = model.on(method, path, [options], eventCallback)` -> * `method` Name of the mutator method: `'change'`, `'insert'`, `'remove'`, `'move'`, `'load'`, `'unload'`, or `'all'` -> * `path` Pattern matching the path being mutated. For example: `'_page.user'`, `'users.*.name'`, `'users.*'`, `'users.**'` / `'users**'`, or `'**'`. `**` is valid only by itself or at the end of the path. +> * `method` Name of the event to listen to: `'change'`, `'insert'`, `'remove'`, `'move'`, `'load'`, `'unload'`, or `'all'` +> * `path` Pattern matching the path to listen to. For example: `'_page.user'`, `'users.*.name'`, `'users.*'`, `'users.**'` / `'users**'`, or `'**'`. `**` is valid only by itself or at the end of the path. > * `options` (optional) > * `useEventObjects` - If true, the callback is called with a structured event object instead of with a variable number of arguments. _Introduced in [racer@0.9.6](https://github.com/derbyjs/racer/releases/tag/v0.9.6)._ > * `eventCallback` Function to call when a matching method and path are mutated > * Returns `listener` - the listener function subscribed to the event emitter. This is the function that should be passed to `model.removeListener` -### `eventCallback` with `{useEventObjects: true}` +New code should use the `{useEventObjects: true}` option, since the structured event objects are easier to work with. The TypeScript definitions _require_ the option, since the legacy callback's dynamic arguments are impossible to type correctly. + +## Event callbacks with `{useEventObjects: true}` _Introduced in [racer@0.9.6](https://github.com/derbyjs/racer/releases/tag/v0.9.6)._ @@ -81,7 +83,11 @@ model.on('all', '**', {useEventObjects: true}, function(event, captures) { }); ``` -### `eventCallback` when `useEventObjects` is false or undefined +## Legacy event callbacks, when `useEventObjects` is false or undefined + +
+ +Click here to show documentation for legacy event callbacks The event callback receives a number of arguments based on the path pattern and method. The arguments are: @@ -133,7 +139,9 @@ model.on('all', '**', function(path, event, args...) { }); ``` -### Passing data to event listeners +
+ +## Passing data to event listeners > `model.pass(object)` > * `object` An object whose properties will each be set on the `passed` argument diff --git a/docs/models/getters.md b/docs/models/getters.md index eead8f13..5e6f16ed 100644 --- a/docs/models/getters.md +++ b/docs/models/getters.md @@ -16,9 +16,34 @@ As model document snapshots change from local or remote mutations, the `model.ro model.get('_session.account') === model.root.data._session.account; ``` -> `value = model.get([path])` -> * `path` *(optional)* Path of object to get. Not supplying a path will return all data in the model starting from the current scope -> * `value` Current value of the object at the given path. Note that objects are returned by reference and should not be modified directly +## Basic get methods + +The basic `get` methods are fastest for most use-cases, where you don't need to do directly manipulate returned objects/arrays. + +> `value = model.get([subpath])` +> * `path` *(optional)* Subpath of object to get. Not supplying a subpath will return the entire value at the current model's path. +> * `value` Returns the current value at the given subpath. + +> `value = model.getOrThrow(subpath)` _(since racer@2.1.0)_ +> * `path` Subpath of object to get +> * `value` Returns the current value at the given subpath, if not null-ish. If the current value is `null` or `undefined`, an exception will be thrown. + +> `value = model.getOrDefault(subpath, defaultValue)` _(since racer@2.1.0)_ +> * `path` *(optional)* Subpath of object to get +> * `value` Returns the current value at the given subpath, if not null-ish. If the current value is `null` or `undefined`, the provided `defaultValue` will be returned instead. +> +> This method will _not_ put the default into the model. It's equivalent to using the relatively newer [JS nullish coalescing operator](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing) as `model.get(subpath) ?? defaultValue`. + +{: .warning-red } +> When using the non-copying `get` methods above to get an object or array, do NOT modify or sort the returned value in-place. +> +> The returned values are references to Racer's internal data tree, and direct manipulations can cause hard-to-debug issues. To make changes to model data, use [the mutator methods](mutators). +> +> If you do need to modify the value in-place, such as for sorting or for a later `setDiffDeep`, use the copying getters below. +> +> The TypeScript types indicate this restriction by returning a `ReadonlyDeep` version of the type parameter. + +## Copying get methods > `shallowCopy = model.getCopy([path])` > * `path` *(optional)* Path of object to get @@ -28,16 +53,10 @@ model.get('_session.account') === model.root.data._session.account; > * `path` *(optional)* Path of object to get > * `deepCopy` Deep copy of current value -## Values returned by reference - -`model.get()` returns values by reference. Racer will fail to function correctly if data in the model is manipulated directly instead of via its mutator methods, such as `model.set()`. You should *never* mutate objects returned from `model.get()` directly. - -As a convenience, Racer also provides a `model.getCopy()` method that returns a shallow copy and `model.getDeepCopy()` method that returns a deep copy. It is safe to mutate copied objects. Changes in these objects can then be updated back into the model using `model.setDiffDeep()`. - ```js -// WARNING: Do NOT directly manipulate objects in the model +// Do NOT directly manipulate objects in the model var user = model.get('users.' + userId); -user.name = 'John'; +/* BAD */ user.name = 'John'; /* BAD */ // Instead, use the model setter methods var user = model.get('users.' + userId); diff --git a/docs/models/mutators.md b/docs/models/mutators.md index 222eef90..ca79728b 100644 --- a/docs/models/mutators.md +++ b/docs/models/mutators.md @@ -4,7 +4,9 @@ title: Mutators parent: Models --- -# Setters +# Mutators + +Mutator methods synchronously update data in the local model, then for remote (DB-backed) collections, they will also send the operation to the server. If there's an error committing the op, the local model will roll back the mutation so that the local model reflects the database state. Models allow getting and setting to nested undefined paths. Setting such a path first sets each undefined or null parent to an empty object or empty array. Whether or not a path segment is a number determines whether the implied parent is created as an object or array. @@ -14,9 +16,27 @@ model.set('cars.DeLorean.DMC12.color', 'silver'); console.log(model.get()); ``` -All model mutators modify data and emit events synchronously. This is only safe to do, because all remote data is synchronized with Operational Transformation, and every client will eventually see a consistent view of the same data. With a more naive approach to syncing data to the server and other clients, updates to the data would need to wait until they were confirmed successful from the server. +## Error handling + +In frontend Derby apps, it's usually fine to use the mutator methods synchronously, since ShareDB's operational transform logic allows the local model to optimisically reflect the update before it's committed to the database. + +Unhandled mutation errors are emitted as `'error'` events on the root model. For frontend code, these can be handled at the top level like this: + +```js +// The 'ready' event is only emitted in the browser. +app.on('ready', () => { + app.model.on('error', (error) => { + // Handle the error appropriately, such as displaying an error message + // to the user and asking them to refresh. + displayErrorMessage(); + // Report the error to your error-handling tool manually, or + // just re-throw for reporting. + throw error; + }); +}); +``` -As well as a synchronous interface, model mutators have an optional callback with an error argument `callback(err)`. This callback is called when the operation is confirmed from the server, which may be desired to confirm that data was saved before updating the UI in some rare cases. This callback should be used very rarely in practice, and data updates should be treated as sychronous, so that the UI responds immediately even if a user has a high latency connection or is currently disconnected. +To handle errors in individual mutation calls via callbacks or promises, see the [Confirming mutations](#confirming-mutations) section below. ## General methods @@ -24,20 +44,19 @@ As well as a synchronous interface, model mutators have an optional callback wit > * `path` Model path to set > * `value` Value to assign > * `previous` Returns the value that was set at the path previously -> * `callback` *(optional)* Invoked upon completion of a successful or failed operation > `obj = model.del(path, [callback])` -> * `path` Model path of object to delete -> * `obj` Returns the deleted object +> * `path` Model path of value to delete +> * `obj` Returns the deleted value > `obj = model.setNull(path, value, [callback])` > * `path` Model path to set -> * `value` Value to assign only if the path is null or undefined +> * `value` Value to assign only if the current value is null or undefined > * `obj` Returns the object at the path if it is not null or undefined. Otherwise, returns the new value > `previous = model.setDiff(path, value, [callback])` > * `path` Model path to set -> * `value` Value to be set if not strictly equal to the current value +> * `value` Value to be set only if the current value is not strictly equal (`===`) to the current value > * `previous` Returns the value that was set at the path previously > `model.setDiffDeep(path, value, [callback])` @@ -48,9 +67,9 @@ As well as a synchronous interface, model mutators have an optional callback wit ## Object methods -> `id = model.add(path, object, [callback])` -> * `path` Model path to set -> * `object` A document to add. If the document has an `id` property, it will be set at that value underneath the path. Otherwise, an id from `model.id()` will be set on the object first +> `id = model.add(collectionName, object, [callback])` +> * `collectionName` Name of the collection to add an object to +> * `object` A document to add. If the document has an `id` property, it will be used as the new object's key in the collection. Otherwise, an UUID from `model.id()` will be set on the object first > * `id` Returns `object.id` > `model.setEach(path, object, [callback])` @@ -120,3 +139,49 @@ The string methods support collaborative text editing, and Derby uses string met > * `path` Model path to a string > * `index` Starting character index of the string at which to remove > * `howMany` Number of characters to remove + +## Confirming mutations + +In backend code or certain kinds of frontend code, it's often necessary to know when the mutation is actually committed, and handling errors from specific mutations can be useful. There are a couple ways of doing so: + +- **Callback function** + - All mutator methods take a final optional `callback` parameter, `(error?: Error) => void`. + - The callback is called with no error when the mutation is successfully commited to the database, and it's called with an error if the mutation failed to commit. +- **Promise API** _(since racer@1.1.0)_ + - Each mutator method has a promise-based version `___Promised()`. For example, `set(path, value)` has `setPromised(path, value)`. + - The promise is resolved when the mutation is successfully commited to the database, and it's rejected with an error if the mutation failed to commit. + - Most mutator promises are `Promise`, resolving with no value. The one exception is `addPromised`, which resolves with the string `id` of the new document. + +Even when using these, the mutation is still synchronously applied to the local model, so that the UI responds immediately even if a user has a high latency connection or is currently disconnected. + +```js +// Callback API +model.set('note-1.title', 'Hello world', (error) => { + if (error) { + return handleError(error); + } + console.log('Update successful!'); +}); +// Promise API +try { + await model.setPromised('note-1.title', 'Hello world'); +} catch (error) { + return handleError(error); +} +console.log('Update successful!'); +``` + +### `whenNothingPending()` + +In rare situations, such as backend batch-update code, you might be makes many synchronous-style mutations without callbacks/promises, and you want to know when ALL pending mutations, fetches, and subscribes on a model are finished. + +`whenNothingPending()` can do that, but keep in mind these warnings: +* It can end up waiting on unrelated operations issued by other code using the same model, so only use it if you're sure the model isn't actively being used elsewhere. +* Performance can be bad if there are tons of documents and queries in the model. +* You do _not_ need to call this prior to `model.close()`, since that already waits internally for pending requests to finish. + +> `model.whenNothingPending(callback)` +> * `callback` - `() => void` - Optional callback, called when the model has no more pending mutations, fetches, or subscribes. + +> `nothingPendingPromise = model.whenNothingPendingPromised()` +> * Returns a `Promise` that resolves when the model has no more pending mutations, fetches, or subscribes. The promise will never be rejected. diff --git a/docs/models/paths.md b/docs/models/paths.md index 3c6c7914..ed85403e 100644 --- a/docs/models/paths.md +++ b/docs/models/paths.md @@ -6,41 +6,62 @@ parent: Models # Paths -All model operations happen on paths which represent nested JSON objects. These paths must be globally unique within a particular database and Redis journal. +The model's data can be thought of as a JSON object tree. + +A path is a string with dot-separated segments, referring to a node (sub-object or value) within the tree. Each segment represents a property lookup within an object or array. Array indexes are 0-based like in JavaScript. For example, the model data: ```js { - title: 'Fruit store', - fruits: [ - { name: 'banana', color: 'yellow' }, - { name: 'apple', color: 'red' }, - { name: 'lime', color: 'green' } - ] + _page: { + currentStorefrontId: 'storefront-1', + }, + storefronts: { + 'storefront-a': { + id: 'storefront-a', + title: 'Fruit store', + fruits: [ + { name: 'banana', color: 'yellow' }, + { name: 'apple', color: 'red' }, + { name: 'lime', color: 'green' } + ] + } + } } ``` -Would have paths like `title`, `fruits.1`, and `fruits.0.color`. Any path segment that is a number must be an index of an array. +Would have paths like: +- `'_page'`, referring to the object `{ currentStorefrontId: 'storefront-1' }` +- `'storefronts.storefront-a.title'`, referring to the title of "storefront-a" +- `'storefronts.storefront-a.fruits.0'`, referring to the first fruit object in "storefront-a" + +From the data root, the first level of properties are collection names, in this case `'_page'` and `'storefront-a'`. These can have special meanings, as described in the next section. -> **WARNING** If you want to use an id value that is a number as a path segment, be careful to prefix this with another character, such as `_` before setting it. Otherwise, you will accidentally create a gigantic array and probably run out of memory. For example, use a path like: `items._1239182389123.name` and never `items.1239182389123.name`. +> **WARNING** If you want to use a number as a path segment, be careful to prefix this before setting it. Otherwise, you will accidentally create a gigantic array and probably run out of memory. For example, use a path like: `items.id_1239182389123.name` and never `items.1239182389123.name`. ## Local and remote collections -Collection names (i.e. the first path segment) that start with an underscore (`_`) or dollar sign (`$`) are local to a given model and are not synced. All paths that start with another character are remote, and will be synced to servers and other clients via ShareJS. Collections that begin with dollar signs are reserved for use by Racer, Derby, or extensions, and should not be used for application data. +Collection names (i.e. the first path segment) that start with an underscore (`_`) are local and are not synced to the database. Data written to local collections during server-side rendering _is_ available in the browser, but that data isn't shared with other servers or clients. + +Collection names that begin with dollar signs (`$`) are special local collections reserved for use by Racer, Derby, or extensions, and should not be used for application data. -Almost all non-synced data within an application should be stored underneath the `_page` local collection. This enables Derby to automatically cleanup as the user navigates between pages. Right before rendering a new page, Derby calls `model.destroy('_page')`, which removes all data, references, event listeners, and reactive functions underneath the `_page` collection. If you have some data that you would like to be maintained between page renders, it can be stored underneath a different local collection. This is useful for setting data on the server, such as setting `_session.userId` in authentication code. However, be very careful when storing local data outside of `_page`, since bleeding state between pages is likely to be a source of unexpected bugs. +Collection names not prefixed with those special characters are considered remote collections, and will be synced to the server and other clients via ShareDB. + +Almost all non-synced data within an application should be stored underneath the `_page` local collection, which Derby to automatically cleans up when the user navigates between pages. Right before rendering a new page, Derby calls `model.destroy('_page')`, which removes all data, references, event listeners, and reactive functions underneath the `_page` collection. + +If you have some local data that you would like to be maintained between page renders, it can be stored underneath a different local collection. This is useful for setting data on the server, such as setting `_session.userId` in authentication code. However, be very careful when storing local data outside of `_page`, since bleeding state between pages is likely to be a source of unexpected bugs. ## Scoped models Scoped models provide a more convenient way to interact with commonly used paths. They support the same methods, and they provide the path argument to accessors, mutators, event methods, and subscription methods. Also, wherever a path is accepted in a racer method, a scoped model can typically be used as well. > `scoped = model.at(subpath)` -> * `subpath` The relative reference path to set. The path is appended if called on a scoped model +> * `subpath` A relative path starting from the current model's path > * `scoped` Returns a scoped model -> `scoped = model.scope([path])` -> * `path` *(optional)* The absolute reference path to set, or the root path by default. This will become the scope even if called on a scoped model. May be called without a path to get a model scoped to the root +> `scoped = model.scope([absolutePath])` +> * `absolutePath` *(optional)* An absolute path from the root of the model data, or the root path by default. This will become the scope even if called on a scoped model. May be called without a path to get a model scoped to the root > * `scoped` Returns a scoped model > `scoped = model.parent([levels])` @@ -59,21 +80,29 @@ Scoped models provide a more convenient way to interact with commonly used paths > * `segment` Returns the last segment for the reference path. Useful for getting indices, ids, or other properties set at the end of a path ```js -room = model.at('_page.room'); +const roomModel = model.at('_page.room'); // These are equivalent: -room.at('name').set('Fun room'); -room.set('name', 'Fun room'); +roomModel.at('name').set('Fun room'); +roomModel.set('name', 'Fun room'); // Logs: {name: 'Fun room'} -console.log(room.get()); +console.log(roomModel.get()); // Logs: 'Fun room' -console.log(room.get('name')); +console.log(roomModel.get('name')); + +// Use model.scope(absolutePath) to refer to things outside a model's subtree. +class MyComponent extends Component { + init() { + // In a component, `this.model` is the component's "private" scoped model + const roomModel = this.model.scope('_page.room'); + } +} ``` -## GUIDs +## UUIDs Models provide a method to create globally unique ids. These can be used as part of a path or within mutator methods. -> `guid = model.id()` -> * `guid` Returns a globally unique identifier that can be used for model operations +> `uuid = model.id()` +> * `uuid` Returns a globally unique identifier that can be used for model operations diff --git a/docs/models/queries.md b/docs/models/queries.md index c1a7ca79..6640cad0 100644 --- a/docs/models/queries.md +++ b/docs/models/queries.md @@ -6,21 +6,25 @@ parent: Models # Queries -Racer can fetch or subscribe to queries based on a model value or a database-specific query. When fetching or subscribing to a query, all of the documents associated with that query are also fetched or subscribed. +Racer can fetch or subscribe to queries based a database-specific query. -> `query = model.query(collectionName, path)` -> * `collectionName` The name of a collection from which to get documents -> * `path` A model path whose value contains a documentId or an array of documentIds +When fetching or subscribing to a query, all of the documents associated with that query's results are also individually loaded into the model, as if they were fetched/subscribed. + +First, create a Query object. > `query = model.query(collectionName, databaseQuery)` > * `collectionName` The name of a collection from which to get documents > * `databaseQuery` A query in the database native format, such as a MonogDB query -# MongoDB query format - -The `sharedb-mongo` adapter supports most MongoDB queries that you could pass to the Mongo `find()` method. See the [Mongo DB query documentation](https://docs.mongodb.org/manual/core/read-operations/#read-operations-query-document) and the [query selectors reference](https://docs.mongodb.org/manual/reference/operator/#query-selectors). Supported MongoDB cursor methods must be passed in as part of the query. `$sort` should be used for sorting, and skips and limits should be specified as `$skip` and `$limit`. There is no `findOne()` equivalent—use `$limit: 1` instead. +Next, to actually run the query, it needs to be subscribed or fetched. For details on subscribing vs fetching, see the ["Loading data into a model" documentation](./backends#loading-data-into-a-model). -Note that projections, which are used to limit the fields that a query returns, may not be defined in the query. Please refer to the [guide on using projections](https://github.com/derbyparty/derby-faq/tree/master/en#i-dont-need-all-collections-fields-in-a-browser-how-to-get-only-particular-fields-collections-projection), which you can follow if you only want specific fields of a document transferred to the browser. +- Query objects have subscribe and fetch methods: + - Callback API - `query.subscribe(callback)` and `query.fetch(callback)`. The callback `(error?: Error) => void` is called when the query data is successfully loaded into the model or when the query encounters an error. + - Promise API - `query.subscribePromised()` and `query.fetchPromised()`. They return a `Promise` that is resolved when the the query data is successfully loaded into the model, or is rejected when the query encounters an error. +- The general `model.subscribe` and `model.fetch` methods also accept query objects, which is useful to execute multiple queries in parallel. + - Callback API - `model.subscribe([query1, query2, ...], callback)` and `model.fetch([query1, query2, ...], callback)` + - Promise API _(since racer@1.1.0)_ - `model.subscribePromised([query1, query2, ...])` and `model.fetchPromised([query1, query2, ...])` + - See ["Loading data into a model" documentation](./backends#loading-data-into-a-model) for more details. ## Query results @@ -32,3 +36,72 @@ After a query is subscribed or fetched, its results can be returned directly via > `scoped = query.ref(path)` > * `path` Local path at which to create an updating refList of the queries results > * `scoped` Returns a model scoped to the path at which results are output + +## Examples + +These examples use the MongoDB query format, as sharedb-mongo is the most mature DB adapter for ShareDB. Adjust the query expressions as needed based on your DB adapter. + +### Callback API + +```js +const notesQuery = model.query('notes', { creatorId: userId }); + +// Frontend code usually subscribes. +// Subscribing to multiple things in parallel reduces the number of round-trips. +model.subscribe([notesQuery, `users.${userId}`], (error) => { + if (error) { + return handleError(error); + } + // Add a reference to the query results to get automatic UI updates. + // A view can use these query results with {{#root._page.notesQueryResults}}. + notesQuery.ref('_page.notesQueryResults'); + // Controller code can get the results either with the query or with the ref. + console.log(notesQuery.get()); + console.log(model.get('_page.notesQueryResults')); + // Documents from the results are also loaded into the model individually. + model.get(`notes.${notesQuery[0].id}`); +}); + +// Backend-only code usually only needs to fetch. +notesQuery.fetch((error) => { + if (error) { + return handleError(error); + } + console.log(notesQuery.get()); +}); +``` + +### Promise API + +_(since racer@1.1.0)_ + +```js +const notesQuery = model.query('notes', { creatorId: userId }); + +// Frontend code usually subscribes. +// Subscribing to multiple things in parallel reduces the number of round-trips. +try { + await model.subscribePromised([notesQuery, `users.${userId}`]); +} catch (error) { + return handleError(error); +} +// Add a reference to the query results to get automatic UI updates. +// A view can use these query results with {{#root._page.notesQueryResults}}. +notesQuery.ref('_page.notesQueryResults'); +// Controller code can get the results either with the query or with the ref. +console.log(notesQuery.get()); +console.log(model.get('_page.notesQueryResults')); + +// Backend-only code usually only needs to fetch. +try { + await notesQuery.fetchPromised(); +} catch (error) { + console.log(notesQuery.get()); +} +``` + +## MongoDB query format + +The `sharedb-mongo` adapter supports most MongoDB queries that you could pass to the Mongo `find()` method. See the [Mongo DB query documentation](https://docs.mongodb.org/manual/core/read-operations/#read-operations-query-document) and the [query selectors reference](https://docs.mongodb.org/manual/reference/operator/#query-selectors). Supported MongoDB cursor methods must be passed in as part of the query. `$sort` should be used for sorting, and skips and limits should be specified as `$skip` and `$limit`. There is no `findOne()` equivalent—use `$limit: 1` instead. + +Note that projections, which are used to limit the fields that a query returns, may not be defined in the query. Please refer to the [guide on using projections](https://github.com/derbyparty/derby-faq/tree/master/en#i-dont-need-all-collections-fields-in-a-browser-how-to-get-only-particular-fields-collections-projection), which you can follow if you only want specific fields of a document transferred to the browser. diff --git a/docs/models/reactive-functions.md b/docs/models/reactive-functions.md index ddfa7d21..1a311586 100644 --- a/docs/models/reactive-functions.md +++ b/docs/models/reactive-functions.md @@ -8,21 +8,22 @@ parent: Models Reactive functions provide a simple way to update a computed value whenever one or more objects change. While model events respond to specific model methods and path patterns, reactive functions will be re-evaluated whenever any of their inputs or nested properties change in any way. -Reactive functions may be run any number of times, so they should be [pure functions](https://en.wikipedia.org/wiki/Pure_function). In other words, they should always return the same results given the same input arguments, and they should be side effect free. By default, the inputs to the function are retrieved directly from the model, so be sure not to modify any object or array input arguments. For example, slice an array input before you sort it. The output of the model function is deep cloned by default. +Reactive functions may be run any number of times, so they should be [pure functions](https://en.wikipedia.org/wiki/Pure_function). In other words, they should always return the same results given the same input arguments, and they should be side effect free. + +By default, the inputs to the function are retrieved directly from the model, so be sure not to modify any input arguments in-place. For example, if you need to sort an array, make a copy with `array.slice()` and sort the copy. The output of the reactive function is deep cloned by default, so you don't have to worry about incidental output modifications affecting the inputs. To execute a model function, you then call `model.start()` or `model.evaluate()`. +* `start()` immediately evaluates the function, sets the returned value to the output path, + and also sets up event listeners that continually re-evaluate the function and update the + output whenever any of its input paths are changed. * `evaluate()` runs a function once and returns the result. -* `start()` also sets up event listeners that continually re-evaluate the - function whenever any of its input or output paths are changed. > ``` > value = model.start(path, inputPaths, [options], fn) -> value = model.evaluate(inputPaths, [options], fn) > ``` > ``` -> // Legacy (racer <= 0.9.5) +> // Legacy (racer <= 0.9.5) > value = model.start(path, inputPaths..., [options], fn) -> value = model.evaluate(inputPaths..., [options], fn) > ``` > > * `path` - _string \| ChildModel_ - The output path at which to set the value, @@ -31,14 +32,12 @@ To execute a model function, you then call `model.start()` or `model.evaluate()` > will be retrieved from the model and passed to the function as inputs > * `options` - _Object_ (optional) > * `copy` - Controls automatic deep copying of the inputs and output of the -> function. _Model#evaluate never deep-copies output, since the return -> value is not set onto the model._ +> function. > - `'output'` (default) - Deep-copy the return value of the function > - `'input'` - Deep-copy the inputs to the function > - `'both'` - Deep-copy both inputs and output > - `'none'` - Do not automatically copy anything -> * `mode` - The `model.set*` method to use when setting the output. _This has -> no effect in Model#evaluate._ +> * `mode` - The `model.set*` method to use when setting the output. > - `'diffDeep'` (default) - Do a recursive deep-equal comparison on old > and new output values, attempting to issue fine-grained ops on subpaths > where possible. @@ -77,6 +76,27 @@ To execute a model function, you then call `model.start()` or `model.evaluate()` > defensive and check inputs' existence before using them. > * Return `value` - The initial value computed by the function +> ``` +> value = model.evaluate(inputPaths, [options], fn) +> ``` +> ``` +> // Legacy (racer <= 0.9.5) +> value = model.evaluate(inputPaths..., [options], fn) +> ``` +> +> * `path` - _string \| ChildModel_ - The output path at which to set the value, +> keeping it updated as input paths change +> * `inputPaths` - _Array_ - One or more paths whose values +> will be retrieved from the model and passed to the function as inputs +> * `options` - _Object_ (optional) +> * `copy` - Controls automatic deep copying of the inputs to the function. +> - `'input'` - Deep-copy the inputs to the function +> - `'none'` - Do not automatically copy anything (default behavior) +> * `fn` - _Function \| string_ - A function or the name of a function defined +> via `model.fn()` +> * See the `model.start` docs above for details. +> * Return `value` - The value returned by the function + > `model.stop(path)` > * `path` The path at which the output should no longer update. Note that the value is not deleted; it is just no longer updated @@ -84,7 +104,7 @@ In DerbyJS, `model.start()` functions should typically be established in the `in ```js MyComponent.prototype.init = function(model) { - model.start('total', 'first', 'second', function sum(x, y) { + model.start('total', ['first', 'second'], function sum(x, y) { return (x || 0) + (y || 0); }); }; diff --git a/docs/models/refs.md b/docs/models/refs.md index fb446ff2..3b4e9f7a 100644 --- a/docs/models/refs.md +++ b/docs/models/refs.md @@ -6,7 +6,7 @@ parent: Models # References -References make it possible to write business logic and templates that interact with the model in a general way. They redirect model operations from a reference path to the underlying data, and they set up event listeners that emit model events on both the reference and the actual object's path. +Model references work like [symlinks in filesystems](https://en.wikipedia.org/wiki/Symbolic_link), redirecting model operations from a reference path to the underlying data, and they set up event listeners that emit model events on both the reference and the actual object's path. References must be declared per model, since calling `model.ref` creates a number of event listeners in addition to setting a ref object in the model. When a reference is created or removed, a `change` model event is emitted. References are not actually stored in the model data, but they can be used from getter and setter methods as if they are. @@ -59,7 +59,7 @@ Racer also supports a special reference type created via `model.refList`. This t > `model.removeRefList(path)` > * `path` The location at which to remove the reference -Note that if objects are inserted into a refList without an `id` property, a unique id from [`model.id()`](paths#guids) will be automatically added to the object. +Note that if objects are inserted into a refList without an `id` property, a unique id from [`model.id()`](paths#uuids) will be automatically added to the object. ```js // refLists should consist of objects with an id matching