← Previous	↑	Next →
Routes and shell	Go to index	Logbook stores

Data model and access

The backbone of most apps is the data model, and how it's accessed and updated.

Revisiting the specs of the simple example app we see that only one logbook is required per logged-in user, and a logbook consists of a flat list of time-ordered entries. This is a fairly simple and common data model for many apps and can be modeled as a single "table" (or "collection", in Firestore) of data items differentiated per user.

This list of entries is stored in Firestore in a single collection called entries. Each entry is associated with a user via a userId field, which matches their Firebase Auth user ID. We then use security rules to control access — i.e. to ensure that a user can only retrieve and modify their own entries. We access all data from the frontend (so no server-side access in this case), using the Firebase JavaScript SDK, where we convert data from Firestore into strongly typed objects using TypeScript, within an Angular service that wraps all access to Firestore.

We'll cover how this is then used to enable the logbook feature, later in a separate document.

✅ Pattern

Whilst Firestore is not a relational database (like PostgreSQL or MySQL), it's still a good idea to start from the principles of relational database modelling. In particular: breaking down your domain model into separate database tables connected by foreign keys (known as "normalization"). Once you have this, you can then consider how to denormalize the data to better fit Firestore's document-based model and to optimize for the queries you need to make.

In the case of this simple example app, we don't need to denormalize the data as the data model is already flat and simple enough.

Firestore data model

Tip

Firestore calls its data items "documents" and they are stored in "collections". This is similar to the "rows" in a "table" of a relational database, but with some key differences. It's worth reading up on the Firestore data model to understand these differences.

Important

Technically, Firestore is a schemaless database, which essentially means that you can store anything you want in any document, with differing data types and minimal database-level validation. The only constraints come from the security rules and your application-level code (frontend and backend).

This makes it very flexible, and can make for higher scalability, but also means that you need to be careful to ensure that your data is consistent and that your application code is robust. Not all data fields in an app require a highly stringent schema or set of constraints, so it's always a trade-off and depends on the specific use case.

Caution

It's important to understand how Firestore, in particular, is billed to ensure you keep control of your costs (if that matters to you). Firestore charges for individual reads, writes, deletes, storage, transfer, etc. and the costs can add up quickly if you're not careful.

For a good initial understanding see the Firestore pricing page.

🧠 Design decision
We've chosen to store all entries — for all users — in one Firestore collection, controlling access using security rules. Another approach could've been to store a user's entries in a subcollection of a "user" or "profile" parent document (where each user has one of these parent documents, and everything in the subcollections of that parent belong to that user).

The core of the data model for the simple example app is:

erDiagram
    ENTRY {
        string userId
        Timestamp timestamp
        string title
        string text
        string category "can be `null`"
    }

Loading

userId is the Firebase Auth "User UID" for the logged in user.
timestamp is a Firestore Timestamp type, which we set to be "server-generated" when creating entries.
title and text are both string fields and make up the actual content of the logbook entry.
We'll look at the category field in a dedicated section, below.

✅ Pattern
Never store the ID of the document in the document itself, as this will duplicate the ID that Firestore auto-generates (or you explicitly set) and has the risk of getting out of sync. Instead, use the Firestore document ID to reference the document and merge it into the data object on client-side when fetching data (see below).

Firestore security rules

Given the design decision to store all entries in a single collection, we first set up the security rules to ensure proper access control:

angular-and-firebase-simple-example-app/firebase/firestore.rules

Lines 4 to 19 in 62002e6

    
             match /databases/{database}/documents { 
        
               function isAuthed() { 
        
                 return request.auth != null; 
        
               } 
        
               function matchesAuthedUser(userId) { 
        
                 return isAuthed() && request.auth.uid == userId; 
        
               } 
        
               match /entries/{document=**} { 
        
                 allow read: if matchesAuthedUser(resource.data.userId); 
        
                 allow create: if matchesAuthedUser(request.resource.data.userId); 
        
                 allow update: if matchesAuthedUser(request.resource.data.userId) && matchesAuthedUser(resource.data.userId); 
        
                 allow delete: if matchesAuthedUser(resource.data.userId); 
        
               } 
        
             }

This is a special "domain specific language" (DSL) that Firestore uses to define access control rules in the firebase/firestore.rules file (docs).
isAuthed() and matchesAuthedUser(userId) are helper functions we've defined to allow easy reuse in multiple rules.
We use the special request.auth.uid field to check the ID of the user making that Firestore operation.
The match /entries/{document=**} line is a "wildcard" rule that applies to all documents in, and subcollections of, the entries collection.
For the read and delete operations we ask Firestore to check the userId field of the requested document (the resource.data.userId field).
- This ensures that a user can only read or delete their own entries.
For the create operation we ask Firestore to check the userId field of the data being written (the request.resource.data.userId field).
- This ensures that a user can only create entries with their own userId.
For the update operation we ask Firestore to check both the userId field of the data being written (the request.resource.data.userId field) and the userId field of the existing document (the resource.data.userId field).
- This ensures that a user can only update entries with their own userId and that they can't change the userId field of an existing entry.

We add some fairly extensive tests in firebase/test/firestore/firestore-rules.spec.ts to ensure that these work as expected. These tests run against an instance of the local emulator, which is started and stopped as part of the pnpm test command.

Firestore indexes

Important

By default, Firestore will create indexes for all fields in a collection. It's important to be aware of this as it can lead to unnecessary costs, since each index counts towards Firestore storage costs.

In firebase/firestore.indexes.json we:

Add a compound index on the userId and timestamp fields, as we're always filtering entries by userId and ordering by timestamp.
Add a compound index on the userId, category, and timestamp fields, as we're filtering entries by userId and category, and ordering by timestamp.
Disable the automatic single-field indexes on the title and text fields, as we don't need to filter or order by these.

Note

Sometimes, you may already know you need a compound index, but this may take time to learn and build an intuition for. No worries, as Firebase will error and tell you when you need to add a compound index, with a link to the Firebase Console where you can create the index. But note that we highly recommend you don't use the Console to do this and instead add the index to the firestore.indexes.json file and then deploy it using pnpm firebase deploy -P live --only firestore:indexes.

Indexes may take some time to build — you can see the status of these in the Firebase Console.

Firestore client-side access

TypeScript types

Both Angular and the Firebase JavaScript SDK have first class support for TypeScript — Angular was one of the first frameworks to get on the TypeScript bandwagon, ever since it's first stable release in 2016.

When loading entries from Firestore we want to assume they take a particular shape — the EntryDoc type — which the TypeScript (and VS Code) tooling can then use to look for type errors and provide code completion.

angular-and-firebase-simple-example-app/app/src/app/shared/models.ts

Lines 15 to 23 in 62002e6

    
           export type EntryDoc = Readonly< 
        
             WithId & { 
        
               userId: string; 
        
               timestamp: Timestamp; 
        
               title: string; 
        
               text: string; 
        
               category: string | null; 
        
             } 
        
           >;

This uses the Readonly TypeScript utility type to mark the whole object as readonly.
We build on the WithId type from the firebase/common/models.ts file, which adds the id field to the object.
- We're using TypeScript's intersection operator (&) to combine the WithId type with the rest of the fields, to make up the EntryDoc type.
- We'll see below how this id field is populated.
The rest of the fields conform directly to the data model defined earlier.

✅ Pattern
Use the `Doc` postfix for the ending of all type names that correspond to a document loaded from a collection from Firestore. This is purely a naming convention and makes it easier to see when we're referring to an actual Firestore document versus regular data / objects.

✅ Pattern
Mark types as `Readonly` when you know for sure they should never be mutated once loaded. This works well for data that comes from a database and is only ever read from and displayed. It's a good way to ensure that you don't accidentally mutate the data and cause state bugs.

In the same file we also define a special NewOrUpdatedEntryInput type for the data we send to Firestore when creating or updating an entry:

angular-and-firebase-simple-example-app/app/src/app/shared/models.ts

Line 25 in 62002e6

    
           export type NewOrUpdatedEntryInput = Pick<EntryDoc, 'title' | 'text' | 'category'>;

Here we're picking a subset of the fields from the EntryDoc type — we only want to allow the user to set or update these fields.

🧠 Design decision

It's possible to lock down the setting or updating of particular fields using security rules (essentially, server-side validation). For example, we could define a rule to prevent the timestamp field from being updated by the user (as we only want it to be set at creation, on the server). We do restrict this via the frontend code and user interface, but right now the user could technically make a direct Firestore call (e.g. in the web page console) to update the timestamp field.

The same applies to validating the presence of fields — this could be achieved using security rules, but for now we rely on client-side validation only.

We deem these to be acceptable risks for this particular app, but it's something you would need to take into account for your particular use cases.

Important

One thing we can't risk is a user setting the userId field to something other than their own user ID (whether when creating a new entry or updating an existing one). This is why we have made sure to add security rules to enforce this, as well as a test suite to check this in the local emulator.

Firestore access service

The EntriesService is where we wrap all Firestore access to the entries collection.

✅ Pattern
Wrap all Firebase access into dedicated Angular services that can be injected (and also mocked in tests). For services that wrap Firestore or Realtime Database place them in the `data/db` subfolder of the feature (or shared) folder.

✅ Pattern
Always mark these data access services as global (i.e. `@Injectable({ providedIn: 'root' })`) as the Angular compiler is smart enough to know to optimise the bundling and only put the code into the chunk that needs it (and thus lazy load it). This works well for these data services as they are almost always consumed by higher level services (like stores).

Let's walk through this service:

First, we inject the Firestore client using the helper provided from the base template:

angular-and-firebase-simple-example-app/app/src/app/logbook/data/db/entries.service.ts

Line 29 in 62002e6

readonly #firestore = injectFirestore();

Next, we define a special converter object that the Firestore JavaScript library understands. We also define a reference to the collection (which we use in the actual data access methods):

angular-and-firebase-simple-example-app/app/src/app/logbook/data/db/entries.service.ts

Lines 31 to 51 in 62002e6

    
           readonly #docConverter: FirestoreDataConverter<EntryDoc> = { 
        
             toFirestore: (obj) => { 
        
               // We need to remove the `id` field before persisting the document 
        
               const { id, ...rest } = obj; 
        
               return rest; 
        
             }, 
        
             fromFirestore: (snapshot, options) => { 
        
               const data = snapshot.data(options); 
        
               const obj = { 
        
                 id: snapshot.id, 
        
                 ...data, 
        
               } as EntryDoc; 
        
               return obj; 
        
             }, 
        
           }; 
        
           readonly #collectionRef = collection(this.#firestore, 'entries').withConverter( 
        
             this.#docConverter, 
        
           );

Firestore converters are a first class way to convert Firestore document data into strongly typed objects, and back. A converter is an object that conforms to the FirestoreDataConverter type, which requires two methods:

A fromFirestore method that takes a Firestore document's data and converts it into an EntryDoc object, which will be used by Firestore data retrieval methods.
A toFirestore method that takes a partial EntryDoc object (i.e. anything that confirms partially to its shape) and converts it into data that can be used to create or update a Firestore document.

Tip

It's possible to perform data validation in these converter methods to further check and constrain what goes in and out of the database (though only client-side, here).

The collection reference we define then uses this converter. So anything that builds off this reference will automatically have the converter applied.

Finally, we have the actual data access and update methods in the service (which wrap the Firestore JavaScript library methods):

getEntryDocs$ returns a long-lived observable of the entries for the specified user, ordered by timestamp.
- Any realtime updates are emitted as new values in the observable. So any updates made to the underlying data (e.g. via the other methods in this service) will trigger a new emission of the entries list in the observable.
- Pagination and filtering parameters are passed in to scope the query appropriately. The private method #buildQuery takes these parameters and constructs an appropriate Firestore query object.
  - The most important query parameter is the userId, which MUST be applied to all queries (the security rules won't even allow a fetch without it).
  - We'll discuss how pagination and filtering work in later documents.
createEntryDoc$ and updateEntryDoc$ are used to create and update entries, respectively.
deleteEntryDoc$ is used to delete an entry.

✅ Pattern

As much as possible use an explicit return type for functions. For example, all the public functions mentioned above explicitly define their return type in the function declaration (in this case, all Observable<SomeType>). This is to declare intent and allow the compiler to catch any potential issues for you (where you expect the function to return one type but it actually ends up returning another).

Note

Both Firestore and Realtime Database provide the ability to fetch data and then listen for any new changes. This allows us to build realtime applications, as we show you in this example app.

For Firestore, it's important to be aware of the costs of running these long-lived listeners since Firestore charges for a read per document. This is where pagination is very important, as we'll cover in a later chapter.

Here is an example data flow between the methods to show how they interact together (time-ordered from top to bottom):

sequenceDiagram
  participant Consumer
  participant EO as Entries observable (long-lived)
  participant A as getEntryDocs$
  participant B as createEntryDoc$
  participant C as updateEntryDoc$
  participant D as deleteEntryDoc$
  Consumer->>A: Call
  activate A
  A-->>EO: Return
  deactivate A
  EO-->>Consumer: #160;
  Consumer->>B: Call
  activate B
  B-->>Consumer: Observable that emits only once
  deactivate B
  EO-->>Consumer: Updated list of entries
  Consumer->>C: Call
  activate C
  C-->>Consumer: Observable that emits only once
  deactivate C
  EO-->>Consumer: Updated list of entries
  Consumer->>D: Call
  activate D
  D-->>Consumer: Observable that emits only once
  deactivate D
  EO-->>Consumer: Updated list of entries

Loading

As you can see, reads are distinctly separated from updates (loosely based on the command-query separation (CQS) principle) and we rely on the realtime nature of Firestore to keep the original observable returned from the getEntryDocs$ method up to date with any relevant changes.

🧠 Design decision

Even with the advent of signals, RxJS-based stream processing is still an important tool in our toolbox as it is a powerful way to work with changes over time, and has very good interoperability with Angular's signals. We use it extensively in the example apps to manage these streams of data from Firestore and Realtime Database (and more).

To bridge the gap between the Firebase JavaScript SDK and RxJS we use rxfire — a third-party library that provides a set of functions that wrap the SDK and return observables. We, in turn, alias these functions in the relevant Firebase helper module (e.g. app/src/app/shared/firebase/firestore.ts) so we can centralize these functions and do minor things like add a $ at the end of the function name (to denote that the function returns an observable).

✅ Pattern

All methods of these data access services should return an observable, so that it's easy to plug into the state management lifecycle and chain other operations. This is the case even if the method does an operation that only triggers once and finishes. This gives us a common and predictable interface.

Add a $ to the end of any function name, class field or variable that returns or contains an observable. This makes it easy to quickly see when something is expected to be an observable (and is a common convention used in the Angular / RxJS world).

✅ Pattern

Use the verbs create, update and delete to name any function and variable names that operate on Firestore and Realtime Database documents and objects. This makes it easy to quickly see when we're dealing with Firestore or Realtime Database operations.

You’re of course free (and encouraged) to use whatever terminology you want in the user interface (UI) to best describe to end users what actions are available / taking place — this naming suggestion is purely at a code consistency level.

Caution

It is vital to unsubscribe from any long-lived observables once they are no longer needed. This is a common problem that leads to memory leaks and unexpected behavior in Angular applications.

In this tech stack we use techniques like proper use of RxJS flattening operators (e.g. switchMap), takeUntilDestroyed and NgRx SignalStore's rxMethod to ensure that observables are cleaned up (and underlying resources disconnected) once no longer needed.

Modelling the `category` field

The category field (of the EntryDoc) is modelled as a string field that can contain a null value.

We want to allow the following cases:

An entry with no category (i.e. null).
An entry with a category, from a predefined list of categories.

As explicitly specified in the spec, this predefined list of categories can be updated by an admin without requiring code changes. I.e. they are stored somewhere editable only by an admin, and pulled into the app at runtime, where they are used to populate the category option in the entry form and also in the filter options. Also, we ensure these categories are updated in realtime when an admin updates the list.

🧠 Design decision

We've chosen to store these predefined categories in Firebase's Realtime Database, under a special config object that can be read by anyone but never updated from the client-side, only from the Firebase Console directly (or server-side).

We make sure to use a very simple data structure, so that it’s hard to mess up, since the Firebase Console only provides a simplistic editing approach with no extra data and schema validations (that you might be used to in a proper content management system).

Caution

All edits you make in the Realtime Database editor in the Firebase Console are saved automatically and will be sent to any connected clients that are listening out for changes.

The config object in the Realtime Database is structured like this:

{
  "config": {
    "categories": {
      "Friends": "",
      "Fun": "",
      "General": "",
      "Hobbies": "",
      "Other": "",
      "Personal": "",
      "Reading": "",
      "Work": ""
    }
  }
}

Note that these are just example categories, and you can put whatever you want within the categories object in your running instance. If you do run this simple example app locally then make sure you populate some categories in the local emulator instance of the Realtime Database.

Tip

During local development, you can access an editor for the local emulator instance of the Realtime Database by going to http://127.0.0.1:4000/database in your browser when the emulator is running.

Because Realtime Database doesn't natively support arrays we have to model the categories as an object with keys and empty string values. We then flatten this object into an array of strings in the client-side code.

Warning

An important caveat whenever making configurable data like this available for users to choose from is that you should never substantially edit or delete an existing value, as existing logbook entries may already use that same category, and since we're using this list to populate the filter options (more on this in a separate document) it would be confusing for users if a category they've used in the past suddenly changed or disappeared.

Note

We could’ve implemented a more elaborate data structure for these categories, with a unique ID, description, CSS class names (e.g. for color customization), etc. But then it would be more error prone to make edits via the Firebase Console (as you can’t have data validation there). You would need to build a dedicated admin panel, or use a proper content management system (CMS), to make those edits. And at that point it’s worth reconsidering how best to implement an app-wide config system, with proper versioning etc. (or even using Firebase's Remote Config, though that’s not currently supported in local emulators).

We then have some very basic security rules to allow anyone to read but no one to write (from the client-side):

angular-and-firebase-simple-example-app/firebase/database.rules.json

Lines 1 to 8 in 62002e6

    
           { 
        
             "rules": { 
        
               "config": { 
        
                 ".read": true, 
        
                 ".write": false 
        
               } 
        
             } 
        
           }

We add some tests in firebase/test/rtdb/rtdb-rules.spec.ts to ensure that these work as expected.

Finally, we have a service that wraps access to this config object (and thus the underlying categories): ConfigService. The key line in this service is where we flatten the categories object into an array of strings:

angular-and-firebase-simple-example-app/app/src/app/logbook/data/db/config.service.ts

Lines 29 to 32 in 62002e6

    
           // We expect a Record<string, string> for the `categories` key. 
        
           // Which we need to convert to a flat list. 
        
           // eslint-disable-next-line @typescript-eslint/no-unsafe-argument, @typescript-eslint/no-unsafe-member-access 
        
           const categories = Object.keys(data.categories || {});

Next we look at how we build on top of these data access services to drive state management and app logic using stores.

← Previous	↑	Next →
Routes and shell	Go to index	Logbook stores

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

3.data-model-and-access.md

3.data-model-and-access.md

Data model and access

Firestore data model

Firestore security rules

Firestore indexes

Firestore client-side access

TypeScript types

Firestore access service

Modelling the `category` field

	match /databases/{database}/documents {
	function isAuthed() {
	return request.auth != null;
	}

	function matchesAuthedUser(userId) {
	return isAuthed() && request.auth.uid == userId;
	}

	match /entries/{document=**} {
	allow read: if matchesAuthedUser(resource.data.userId);
	allow create: if matchesAuthedUser(request.resource.data.userId);
	allow update: if matchesAuthedUser(request.resource.data.userId) && matchesAuthedUser(resource.data.userId);
	allow delete: if matchesAuthedUser(resource.data.userId);
	}
	}

	export type EntryDoc = Readonly<
	WithId & {
	userId: string;
	timestamp: Timestamp;
	title: string;
	text: string;
	category: string \| null;
	}
	>;

	readonly #docConverter: FirestoreDataConverter<EntryDoc> = {
	toFirestore: (obj) => {
	// We need to remove the `id` field before persisting the document
	const { id, ...rest } = obj;
	return rest;
	},
	fromFirestore: (snapshot, options) => {
	const data = snapshot.data(options);

	const obj = {
	id: snapshot.id,
	...data,
	} as EntryDoc;

	return obj;
	},
	};

	readonly #collectionRef = collection(this.#firestore, 'entries').withConverter(
	this.#docConverter,
	);

	// We expect a Record<string, string> for the `categories` key.
	// Which we need to convert to a flat list.
	// eslint-disable-next-line @typescript-eslint/no-unsafe-argument, @typescript-eslint/no-unsafe-member-access
	const categories = Object.keys(data.categories \|\| {});

Files

3.data-model-and-access.md

Latest commit

History

3.data-model-and-access.md

File metadata and controls

Data model and access

Firestore data model

Firestore security rules

Firestore indexes

Firestore client-side access

TypeScript types

Firestore access service

Modelling the category field

Modelling the `category` field