From 0d3a8dcd84ba7b32ae0196549bb10c244be38da2 Mon Sep 17 00:00:00 2001 From: Natalie Weizenbaum Date: Wed, 19 Jul 2023 18:36:50 -0700 Subject: [PATCH 1/3] Write implementation documentation Also rename Dispatcher to CompilationDispatcher for clarity --- CONTRIBUTING.md | 11 + lib/src/README.md | 230 ++++++++++++++++++ lib/src/ast/css/README.md | 50 ++++ lib/src/ast/sass/README.md | 34 +++ lib/src/ast/selector/README.md | 24 ++ lib/src/embedded/README.md | 28 +++ ...tcher.dart => compilation_dispatcher.dart} | 19 +- lib/src/embedded/host_callable.dart | 6 +- lib/src/embedded/importer/base.dart | 6 +- lib/src/embedded/importer/file.dart | 5 +- lib/src/embedded/importer/host.dart | 5 +- lib/src/embedded/isolate_dispatcher.dart | 5 +- lib/src/embedded/logger.dart | 6 +- lib/src/embedded/protofier.dart | 4 +- lib/src/embedded/value.dart | 6 +- lib/src/extend/README.md | 35 +++ lib/src/functions/README.md | 24 ++ lib/src/io/README.md | 17 ++ lib/src/node/README.md | 58 +++++ lib/src/parse/README.md | 32 +++ lib/src/value/README.md | 13 + lib/src/visitor/README.md | 15 ++ 22 files changed, 604 insertions(+), 29 deletions(-) create mode 100644 lib/src/README.md create mode 100644 lib/src/ast/css/README.md create mode 100644 lib/src/ast/sass/README.md create mode 100644 lib/src/ast/selector/README.md create mode 100644 lib/src/embedded/README.md rename lib/src/embedded/{dispatcher.dart => compilation_dispatcher.dart} (95%) create mode 100644 lib/src/extend/README.md create mode 100644 lib/src/functions/README.md create mode 100644 lib/src/io/README.md create mode 100644 lib/src/node/README.md create mode 100644 lib/src/parse/README.md create mode 100644 lib/src/value/README.md create mode 100644 lib/src/visitor/README.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 934a0576a..b01d899ca 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -9,6 +9,7 @@ Want to contribute? Great! First, read this page. * [Synchronizing](#synchronizing) * [File Headers](#file-headers) * [Release Process](#release-process) +* [Package Structure](#package-structure) ## Before You Contribute @@ -233,3 +234,13 @@ few things to do before pushing that tag: You *don't* need to create tags for packages in `pkg`; that will be handled automatically by GitHub actions. + +## Package Structure + +The structure of the Sass package is documented in README.md files in most +directories under `lib/`. This documentation is intended to help contributors +quickly build a basic understanding of the structure of the compiler and how its +various pieces fit together. [`lib/src/README.md`] is a good starting point to get +an overview of the compiler as a whole. + +[`lib/src/README.md`]: lib/src/README.md diff --git a/lib/src/README.md b/lib/src/README.md new file mode 100644 index 000000000..65f332f9b --- /dev/null +++ b/lib/src/README.md @@ -0,0 +1,230 @@ +# The Sass Compiler + +* [Life of a Compilation](#life-of-a-compilation) + * [Late Parsing](#late-parsing) + * [Early Serialization](#early-serialization) +* [JS Support](#js-support) +* [APIs](#apis) + * [Importers](#importers) + * [Custom Functions](#custom-functions) + * [Loggers](#loggers) +* [Built-In Functions](#built-in-functions) +* [`@extend`](#extend) + +This is the root directory of Dart Sass's private implementation libraries. This +contains essentially all the business logic defining how Sass is actually +compiled, as well as the APIs that users use to interact with Sass. There are +two exceptions: + +* [`../../bin/sass.dart`] is the entrypoint for the Dart Sass CLI (on all + platforms). While most of the logic it runs exists in this directory, it does + contain some logic to drive the basic compilation logic and handle errors. All + the most complex parts of the CLI, such as option parsing and the `--watch` + command, are handled in the [`executable`] directory. Even Embedded Sass runs + through this entrypoint, although it gets immediately gets handed off to [the + embedded compiler]. + + [`../../bin/sass.dart`]: ../../bin/sass.dart + [`executable`]: executable + [the embedded compiler]: embedded/README.md + +* [`../sass.dart`] is the entrypoint for the public Dart API. This is what's + loaded when a Dart package imports Sass. It just contains the basic + compilation functions, and exports the rest of the public APIs from this + directory. + + [`../sass.dart`]: ../sass.dart + +Everything else is contained here, and each file and some subdirectories have +their own documentation. But before you dive into those, let's take a look at +the general lifecycle of a Sass compilation. + +## Life of a Compilation + +Whether it's invoked through the Dart API, the JS API, the CLI, or the embedded +host, the basic process of a Sass compilation is the same. Sass is implemented +as an AST-walking [interpreter] that operates in roughly three passes: + +[interpreter]: https://en.wikipedia.org/wiki/Interpreter_(computing) + +1. **Parsing**. The first step of a Sass compilation is always to parse the + source file, whether it's SCSS, the indented syntax, or CSS. The parsing + logic lives in the [`parse`] directory, while the abstract syntax tree that + represents the parsed file lives in [`ast/sass`]. + + [`parse`]: parse/README.md + [`ast/sass`]: ast/sass/README.md + +2. **Evaluation**. Once a Sass file is parsed, it's evaluated by + [`visitor/async_evaluate.dart`]. (Why is there both an async and a sync + version of this file? See [Synchronizing] for details!) The evaluator handles + all the Sass-specific logic: it resolves variables, includes mixins, executes + control flow, and so on. As it goes, it builds up a new AST that represents + the plain CSS that is the compilation result, which is defined in + [`ast/css`]. + + [`visitor/async_evaluate.dart`]: visitor/async_evaluate.dart + [Synchronizing]: ../../CONTRIBUTING.md#synchronizing + [`ast/css`]: ast/css/README.md + + Sass evaluation is almost entirely linear: it begins at the first statement + of the file, evaluates it (which may involve evaluating its nested children), + adds its result to the CSS AST, and then moves on to the second statement. On + it goes until it reaches the end of the file, at which point it's done. The + only exception is module resolution: every Sass module has its own compiled + CSS AST, and once the entrypoint file is done compiling the evaluator will go + back through these modules, resolve `@extend`s across them as necessary, and + stitch them together into the final stylesheet. + + SassScript, the expression-level syntax, is handled by the same evaluator. + The main difference between SassScript and statement-level evaluation is that + the same SassScript values are used during evaluation _and_ as part of the + CSS AST. This means that it's possible to end up with a Sass-specific value, + such as a map or a first-class function, as the value of a CSS declaration. + If that happens, the Serialization phase will signal an error when it + encounters the invalid value. + +3. **Serialization**. Once we have the CSS AST that represents the compiled + stylesheet, we need to convert it into actual CSS text. This is done by + [`visitor/serialize.dart`], which walks the AST and builds up a big buffer of + the resulting CSS. It uses [a special string buffer] that tracks source and + destination locations in order to generate [source maps] as well. + + [`visitor/serialize.dart`]: visitor/serialize.dart + [a special string buffer]: util/source_map_buffer.dart + [source maps]: https://web.dev/source-maps/ + +There's actually one slight complication here: the first and second pass aren't +as separate as they appear. When one Sass stylesheet loads another with `@use`, +`@forward`, or `@import`, that rule is handled by the evaluator and _only at +that point_ is the loaded file parsed. So in practice, compilation actually +switches between parsing and evaluation, although each individual stylesheet +naturally has to be parsed before it can be evaluated. + +### Late Parsing + +Some syntax within a stylesheet is only parsed _during_ evaluation. This allows +authors to use `#{}` interpolation to inject Sass variables and other dynamic +values into various locations, such as selectors, while still allowing Sass to +parse them to support features like nesting and `@extend`. The following +syntaxes are parsed during evaluation: + +* [Selectors](parse/selector.dart) +* [`@keyframes` frames](parse/keyframe_selector.dart) +* [Media queries](parse/media_query.dart) (for historical reasons, these are + parsed before evaluation and then _reparsed_ after they've been fully + evaluated) + +### Early Serialization + +There are also some cases where the evaluator can serialize values before the +main serialization pass. For example, if you inject a variable into a selector +using `#{}`, that variable's value has to be converted to a string during +evaluation so that the evaluator can then parse and handle the newly-generated +selector. The evaluator does this by invoking the serializer _just_ for that +specific value. As a rule of thumb, this happens anywhere interpolation is used +in the original stylesheet, although there are a few other circumstances as +well. + +## JS Support + +One of the main benefits of Dart as an implementation language is that it allows +us to distribute Dart Sass both as an extremely efficient stand-alone executable +_and_ an easy-to-install pure-JavaScript package, using the dart2js compilation +tool. However, properly supporting JS isn't seamless. There are two major places +where we need to think about JS support: + +1. When interfacing with the filesystem. None of Dart's IO APIs are natively + supported on JS, so for anything that needs to work on both the Dart VM _and_ + Node.js we define a shim in the [`io`] directory that will be implemented in + terms of `dart:io` if we're running on the Dart VM or the `fs` or `process` + modules if we're running on Node. (We don't support IO at all on the browser + except to print messages to the console.) + + [`io`]: io/README.md + +2. When exposing an API. Dart's JS interop is geared towards _consuming_ JS + libraries from Dart, not producing a JS library written in Dart, so we have + to jump through some hoops to make it work. This is all handled in the [`js`] + directory. + + [`js`]: js/README.md + +## APIs + +One of Sass's core features is its APIs, which not only compile stylesheets but +also allow users to provide plugins that can be invoked from within Sass. In +both the JS API, the Dart API, and the embedded compiler, Sass provides three +types of plugins: importers, custom functions, and loggers. + +### Importers + +Importers control how Sass loads stylesheets through `@use`, `@forward`, and +`@import`. Internally, _all_ stylesheet loads are modeled as importers. When a +user passes a load path to an API or compiles a stylesheet through the CLI, we +just use the built-in [`FilesystemImporter`] which implements the same interface +that we make available to users. + +[`FilesystemImporter`]: importer/filesystem.dart + +In the Dart API, the importer root class is [`importer/async_importer.dart`]. +The JS API and the embedded compiler wrap the Dart importer API in +[`importer/node_to_dart`] and [`embedded/importer`] respectively. + +[`importer/async_importer.dart`]: importer/async_importer.dart +[`importer/node_to_dart`]: importer/node_to_dart +[`embedded/importer`]: embedded/importer + +### Custom Functions + +Custom functions are defined by users of the Sass API but invoked by Sass +stylesheets. To a Sass stylesheet, they look like any other built-in function: +users pass SassScript values to them and get SassScript values back. In fact, +all the core Sass functions are implemented using the Dart custom function API. + +Because custom functions take and return SassScript values, that means we need +to make _all_ values available to the various APIs. For Dart, this is +straightforward: we need to have objects to represent those values anyway, so we +just expose those objects publicly (with a few `@internal` annotations here and +there to hide APIs we don't want users relying on). These value types live in +the [`value`] directory. + +[`value`]: value/README.md + +Exposing values is a bit more complex for other platforms. For the JS API, we do +a bit of metaprogramming in [`node/value`] so that we can return the +same Dart values we use internally while still having them expose a JS API that +feels native to that language. For the embedded host, we convert them to and +from a protocol buffer representation in [`embedded/value.dart`]. + +[`node/value`]: node/value/README.md +[`embedded/value.dart`]: embedded/value.dart + +### Loggers + +Loggers are the simplest of the plugins. They're just callbacks that are invoked +any time Dart Sass would emit a warning (from the language or from `@warn`) or a +debug message from `@debug`. They're defined in: + +* [`logger.dart`](logger.dart) for Dart +* [`node/logger.dart`](node/logger.dart) for Node +* [`embedded/logger.dart`](embedded/logger.dart) for the embedded compiler + +## Built-In Functions + +All of Sass's built-in functions are defined in the [`functions`] directory, +including both global functions and functions defined in core modules like +`sass:math`. As mentioned before, these are defined using the standard custom +function API, although in a few cases they use additional private features like +the ability to define multiple overloads of the same function name. + +[`functions`]: functions/README.md + +## `@extend` + +The logic for Sass's `@extend` rule is particularly complex, since it requires +Sass to not only parse selectors but to understand how to combine them and when +they can be safely optimized away. Most of the logic for this is contained +within the [`extend`] directory. + +[`extend`]: extend/README.md diff --git a/lib/src/ast/css/README.md b/lib/src/ast/css/README.md new file mode 100644 index 000000000..bdd012bcd --- /dev/null +++ b/lib/src/ast/css/README.md @@ -0,0 +1,50 @@ +# CSS Abstract Syntax Tree + +This directory contains the abstract syntax tree that represents a plain CSS +file generated by Sass compilation. It differs from other Sass ASTs in two major +ways: + +1. Instead of being created by [a parser], it's created by [the evaluator] as it + traverses the [Sass AST]. + + [a parser]: ../../parse/README.md + [the evaluator]: ../../visitor/async_evaluate.dart + [Sass AST]: ../sass/README.md + +2. Because of various Sass features like `@extend` and at-rule hoisting, the CSS + AST is mutable even though all other ASTs are immutable. + +**Note:** the CSS AST doesn't have its own representation of declaration values. +Instead, declaration values are represented as [`Value`] objects. This does mean +that a CSS AST can be in a state where some of its values aren't representable +in plain CSS (such as maps)—in this case, [the serializer] will emit an error. + +[`Value`]: ../../value/README.md +[the serializer]: ../../visitor/serialize.dart + +## Mutable and Immutable Views + +Internally, the CSS AST is mutable to allow for operations like hoisting rules +to the root of the AST and updating existing selectors when `@extend` rules are +encountered. However, because mutability poses a high risk for "spooky action at +a distance", we limit access to mutating APIs exclusively to the evaluator. + +We do this by having an _unmodifiable_ interface (written in this directory) for +each CSS AST node which only exposes members that don't modify the node in +question. The implementations of those interfaces, which _do_ have modifying +methods, live in the [`modifiable`] directory. We then universally refer to the +immutable node interfaces except specifically in the evaluator, and the type +system automatically ensures we don't accidentally mutate anything we don't +intend to. + +[`modifiable`]: modifiable + +(Of course, it's always possible to cast an immutable node type to a mutable +one, but that's a very clear code smell that a reviewer can easily identify.) + +## CSS Source Files + +A lesser-known fact about Sass is that it actually supports _three_ syntaxes for +its source files: SCSS, the indented syntax, and plain CSS. But even when it +parses plain CSS, it uses the Sass AST rather than the CSS AST to represent it +so that parsing logic can easily be shared with the other stylesheet parsers. diff --git a/lib/src/ast/sass/README.md b/lib/src/ast/sass/README.md new file mode 100644 index 000000000..415a4b6ae --- /dev/null +++ b/lib/src/ast/sass/README.md @@ -0,0 +1,34 @@ +# Sass Abstract Syntax Tree + +This directory contains the abstract syntax tree that represents a Sass source +file, regardless of which syntax it was written in (SCSS, the indented syntax, +or plain CSS). The AST is constructed recursively by [a parser] from the leaf +nodes in towards the root, which allows it to be fully immutable. + +[a parser]: ../../parse/README.md + +The Sass AST is broken up into three categories: + +1. The [statement AST], which represents statement-level constructs like + variable assignments, style rules, and at-rules. + + [statement AST]: statement + +2. The [expression AST], which represents SassScript expressions like function + calls, operations, and value literals. + + [expression AST]: exprssion + +3. Miscellaneous AST nodes that are used by both statements and expressions or + don't fit cleanly into either category that live directly in this directory. + +The Sass AST nodes are processed (usually from the root [`Stylesheet`]) by [the +evaluator], which runs the logic they encode and builds up a [CSS AST] that +represents the compiled stylesheet. They can also be transformed back into Sass +source using the `toString()` method. Since this is only ever used for debugging +and doesn't need configuration or full-featured indentation tracking, it doesn't +use a full visitor. + +[`Stylesheet`]: statement/stylesheet.dart +[the evaluator]: ../../visitor/async_evaluate.dart +[CSS AST]: ../css/README.md diff --git a/lib/src/ast/selector/README.md b/lib/src/ast/selector/README.md new file mode 100644 index 000000000..721644977 --- /dev/null +++ b/lib/src/ast/selector/README.md @@ -0,0 +1,24 @@ +# Selector Abstract Syntax Tree + +This directory contains the abstract syntax tree that represents a parsed CSS +selector. This AST is constructed recursively by [the selector parser]. It's +fully immutable. + +[the selector parser]: ../../parse/selector.dart + +Unlike the [Sass AST], which is parsed from a raw source string before being +evaluated, the selector AST is parsed _during evaluation_. This is necessary to +ensure that there's a chance to resolve interpolation before fully parsing the +selectors in question. + +[Sass AST]: ../sass/README.md + +Although this AST doesn't include any SassScript, it _does_ include a few +Sass-specific constructs: the [parent selector] `&` and [placeholder selectors]. +Parent selectors are resolved by [the evaluator] before it hands the AST off to +[the serializer], while placeholders are omitted in the serializer itself. + +[parent selector]: parent.dart +[placeholder selectors]: placeholder.dart +[the evaluator]: ../../visitor/async_evaluate.dart +[the serializer]: ../../visitor/serialize.dart diff --git a/lib/src/embedded/README.md b/lib/src/embedded/README.md new file mode 100644 index 000000000..b892e4818 --- /dev/null +++ b/lib/src/embedded/README.md @@ -0,0 +1,28 @@ +# Embedded Sass Compiler + +This directory contains the Dart Sass embedded compiler. This is a special mode +of the Dart Sass command-line executable, only supported on the Dart VM, in +which it uses stdin and stdout to communicate with another endpoint, the +"embedded host", using a protocol buffer-based protocol. See [the embedded +protocol specification] for details. + +[the embedded protocol specification]: https://github.com/sass/sass/blob/main/spec/embedded-protocol.md + +The embedded compiler has two different levels of dispatchers for handling +incoming messages from the embedded host: + +1. The [`IsolateDispatcher`] is the first recipient of each packet. It decodes + the packets _just enough_ to determine which compilation they belong to, and + forwards them to the appropriate compilation dispatcher. It also parses and + handles messages that aren't compilation specific, such as `VersionRequest`. + + [`IsolateDispatcher`]: isolate_dispatcher.dart + +2. The [`CompilationDispatcher`] fully parses and handles messages for a single + compilation. Each `CompilationDispatcher` runs in a separate isolate so that + the embedded compiler can run multiple compilations in parallel. + + [`CompilationDispatcher`]: compilation_dispatcher.dart + +Otherwise, most of the code in this directory just wraps Dart APIs to +communicate with their protocol buffer equivalents. diff --git a/lib/src/embedded/dispatcher.dart b/lib/src/embedded/compilation_dispatcher.dart similarity index 95% rename from lib/src/embedded/dispatcher.dart rename to lib/src/embedded/compilation_dispatcher.dart index f6d6a3e0f..cd6ca91fc 100644 --- a/lib/src/embedded/dispatcher.dart +++ b/lib/src/embedded/compilation_dispatcher.dart @@ -29,7 +29,7 @@ final _outboundRequestId = 0; /// A class that dispatches messages to and from the host for a single /// compilation. -class Dispatcher { +class CompilationDispatcher { /// The channel of encoded protocol buffers, connected to the host. final StreamChannel _channel; @@ -47,13 +47,13 @@ class Dispatcher { /// A completer awaiting a response to an outbound request. /// - /// Since each [Dispatcher] is only running a single-threaded compilation, it - /// can only ever have one request outstanding. + /// Since each [CompilationDispatcher] is only running a single-threaded + /// compilation, it can only ever have one request outstanding. Completer? _outstandingRequest; - /// Creates a [Dispatcher] that sends and receives encoded protocol buffers - /// over [channel]. - Dispatcher(this._channel, this._compilationId) + /// Creates a [CompilatoinDispatcher] that sends and receives encoded protocol + /// buffers over [channel]. + CompilationDispatcher(this._channel, this._compilationId) : _compilationIdVarint = serializeVarint(_compilationId); /// Listens for incoming `CompileRequests` and runs their compilations. @@ -93,7 +93,8 @@ class Dispatcher { var response = await _compile(request); _send(OutboundMessage()..compileResponse = response); success = true; - // Each Dispatcher runs a single compilation and then closes. + // Each CompilationDispatcher runs a single compilation and then + // closes. _channel.sink.close(); case InboundMessage_Message.canonicalizeResponse: @@ -273,8 +274,8 @@ class Dispatcher { if (_outstandingRequest != null) { throw StateError( - "Dispatcher.sendRequest() can't be called when another request is " - "active."); + "CompilationDispatcher.sendRequest() can't be called when another " + "request is active."); } return (_outstandingRequest = Completer()).future; diff --git a/lib/src/embedded/host_callable.dart b/lib/src/embedded/host_callable.dart index bb1770ea4..cd34ec98e 100644 --- a/lib/src/embedded/host_callable.dart +++ b/lib/src/embedded/host_callable.dart @@ -7,7 +7,7 @@ import 'dart:cli'; import '../callable.dart'; import '../exception.dart'; -import 'dispatcher.dart'; +import 'compilation_dispatcher.dart'; import 'embedded_sass.pb.dart'; import 'function_registry.dart'; import 'protofier.dart'; @@ -21,8 +21,8 @@ import 'utils.dart'; /// the name defined in the [signature]. /// /// Throws a [SassException] if [signature] is invalid. -Callable hostCallable( - Dispatcher dispatcher, FunctionRegistry functions, String signature, +Callable hostCallable(CompilationDispatcher dispatcher, + FunctionRegistry functions, String signature, {int? id}) { late Callable callable; callable = Callable.fromSignature(signature, (arguments) { diff --git a/lib/src/embedded/importer/base.dart b/lib/src/embedded/importer/base.dart index 9fad62360..e51a4d0ca 100644 --- a/lib/src/embedded/importer/base.dart +++ b/lib/src/embedded/importer/base.dart @@ -5,14 +5,14 @@ import 'package:meta/meta.dart'; import '../../importer.dart'; -import '../dispatcher.dart'; +import '../compilation_dispatcher.dart'; /// An abstract base class for importers that communicate with the host in some /// way. abstract class ImporterBase extends Importer { - /// The [CompileDispatcher] to which to send requests. + /// The [CompilationDispatcher] to which to send requests. @protected - final Dispatcher dispatcher; + final CompilationDispatcher dispatcher; ImporterBase(this.dispatcher); diff --git a/lib/src/embedded/importer/file.dart b/lib/src/embedded/importer/file.dart index e2b2bad40..a2195da81 100644 --- a/lib/src/embedded/importer/file.dart +++ b/lib/src/embedded/importer/file.dart @@ -6,7 +6,7 @@ import 'dart:cli'; import '../../importer.dart'; -import '../dispatcher.dart'; +import '../compilation_dispatcher.dart'; import '../embedded_sass.pb.dart' hide SourceSpan; import 'base.dart'; @@ -22,7 +22,8 @@ class FileImporter extends ImporterBase { /// The host-provided ID of the importer to invoke. final int _importerId; - FileImporter(Dispatcher dispatcher, this._importerId) : super(dispatcher); + FileImporter(CompilationDispatcher dispatcher, this._importerId) + : super(dispatcher); Uri? canonicalize(Uri url) { if (url.scheme == 'file') return _filesystemImporter.canonicalize(url); diff --git a/lib/src/embedded/importer/host.dart b/lib/src/embedded/importer/host.dart index 743cdda95..bea30b210 100644 --- a/lib/src/embedded/importer/host.dart +++ b/lib/src/embedded/importer/host.dart @@ -6,7 +6,7 @@ import 'dart:cli'; import '../../importer.dart'; -import '../dispatcher.dart'; +import '../compilation_dispatcher.dart'; import '../embedded_sass.pb.dart' hide SourceSpan; import '../utils.dart'; import 'base.dart'; @@ -16,7 +16,8 @@ class HostImporter extends ImporterBase { /// The host-provided ID of the importer to invoke. final int _importerId; - HostImporter(Dispatcher dispatcher, this._importerId) : super(dispatcher); + HostImporter(CompilationDispatcher dispatcher, this._importerId) + : super(dispatcher); Uri? canonicalize(Uri url) { // ignore: deprecated_member_use diff --git a/lib/src/embedded/isolate_dispatcher.dart b/lib/src/embedded/isolate_dispatcher.dart index 78d340997..86ae609d5 100644 --- a/lib/src/embedded/isolate_dispatcher.dart +++ b/lib/src/embedded/isolate_dispatcher.dart @@ -12,7 +12,7 @@ import 'package:protobuf/protobuf.dart'; import 'package:stream_channel/isolate_channel.dart'; import 'package:stream_channel/stream_channel.dart'; -import 'dispatcher.dart'; +import 'compilation_dispatcher.dart'; import 'embedded_sass.pb.dart'; import 'util/explicit_close_transformer.dart'; import 'util/proto_extensions.dart'; @@ -218,7 +218,8 @@ void _isolateMain(SendPort sendPort) { var (compilationSendPort, compilationId) = initialMessage; var compilationChannel = IsolateChannel.connectSend(compilationSendPort); - var success = await Dispatcher(compilationChannel, compilationId).listen(); + var success = + await CompilationDispatcher(compilationChannel, compilationId).listen(); if (!success) channel.sink.close(); }); } diff --git a/lib/src/embedded/logger.dart b/lib/src/embedded/logger.dart index c94e52dc1..a2cab5fae 100644 --- a/lib/src/embedded/logger.dart +++ b/lib/src/embedded/logger.dart @@ -8,14 +8,14 @@ import 'package:stack_trace/stack_trace.dart'; import '../logger.dart'; import '../utils.dart'; -import 'dispatcher.dart'; +import 'compilation_dispatcher.dart'; import 'embedded_sass.pb.dart' hide SourceSpan; import 'utils.dart'; /// A Sass logger that sends log messages as `LogEvent`s. class EmbeddedLogger implements Logger { - /// The [Dispatcher] to which to send events. - final Dispatcher _dispatcher; + /// The [CompilationDispatcher] to which to send events. + final CompilationDispatcher _dispatcher; /// Whether the formatted message should contain terminal colors. final bool _color; diff --git a/lib/src/embedded/protofier.dart b/lib/src/embedded/protofier.dart index 9096c8b9b..52ab6472d 100644 --- a/lib/src/embedded/protofier.dart +++ b/lib/src/embedded/protofier.dart @@ -3,7 +3,7 @@ // https://opensource.org/licenses/MIT. import '../value.dart'; -import 'dispatcher.dart'; +import 'compilation_dispatcher.dart'; import 'embedded_sass.pb.dart' as proto; import 'embedded_sass.pb.dart' hide Value, ListSeparator, CalculationOperator; import 'function_registry.dart'; @@ -16,7 +16,7 @@ import 'utils.dart'; /// custom function call. class Protofier { /// The dispatcher, for invoking deprotofied [Value_HostFunction]s. - final Dispatcher _dispatcher; + final CompilationDispatcher _dispatcher; /// The IDs of first-class functions. final FunctionRegistry _functions; diff --git a/lib/src/embedded/value.dart b/lib/src/embedded/value.dart index 1b08f7b83..2a798d24c 100644 --- a/lib/src/embedded/value.dart +++ b/lib/src/embedded/value.dart @@ -3,7 +3,7 @@ // https://opensource.org/licenses/MIT. import '../value.dart'; -import 'dispatcher.dart'; +import 'compilation_dispatcher.dart'; import 'embedded_sass.pb.dart' as proto; import 'embedded_sass.pb.dart' hide Value, ListSeparator; import 'function_registry.dart'; @@ -83,8 +83,8 @@ proto.ListSeparator _protofySeparator(ListSeparator separator) { /// /// The [functions] tracks the IDs of first-class functions so that they can be /// deserialized to their original references. -Value deprotofyValue( - Dispatcher dispatcher, FunctionRegistry functions, proto.Value value) { +Value deprotofyValue(CompilationDispatcher dispatcher, + FunctionRegistry functions, proto.Value value) { // Curry recursive calls to this function so we don't have to keep repeating // ourselves. deprotofy(proto.Value value) => deprotofyValue(dispatcher, functions, value); diff --git a/lib/src/extend/README.md b/lib/src/extend/README.md new file mode 100644 index 000000000..e27304b6b --- /dev/null +++ b/lib/src/extend/README.md @@ -0,0 +1,35 @@ +# `@extend` Logic + +This directory contains most of the logic for running Sass's `@extend` rule. +This rule is probably the most complex corner of the Sass language, since it +involves both understanding the semantics of selectors _and_ being able to +combine them. + +The high-level lifecycle of extensions is as follows: + +1. When [the evaluator] encounters a style rule, it registers its selector in + the [`ExtensionStore`] for the current module. This applies any extensions + that have already been registered, then returns a _mutable_ + `Box` that will get updated as extensions are applied. + + [the evaluator]: ../visitor/async_evaluate.dart + [`ExtensionStore`]: extension_store.dart + +2. When the evaluator encounters an `@extend`, it registers that in the current + module's `ExtensionStore` as well. This updates any selectors that have + already been registered with that extension, _and_ updates the extension's + own extender (the selector that gets injected when the extension is applied, + which is stored along with the extension). Note that the extender has to be + extended separately from the selector in the style rule, because the latter + gets redundant selectors trimmed eagerly and the former does not. + +3. When the entrypoint stylesheet has been fully executed, the evaluator + determines which extensions are visible from which modules and adds + extensions from one store to one another accordingly using + `ExtensionStore.addExtensions()`. + +Otherwise, the process of [extending a selector] as described in the Sass spec +matches the logic here fairly closely. See `ExtensionStore._extendList()` for +the primary entrypoint for that logic. + +[extending a selector]: https://github.com/sass/sass/blob/main/spec/at-rules/extend.md#extending-a-selector diff --git a/lib/src/functions/README.md b/lib/src/functions/README.md new file mode 100644 index 000000000..967023f82 --- /dev/null +++ b/lib/src/functions/README.md @@ -0,0 +1,24 @@ +# Built-In Functions + +This directory contains the standard functions that are built into Sass itself, +both those that are available globally and those that are available only through +built-in modules. Each of the files here exports a corresponding +[`BuiltInModule`, and most define a list of global functions as well. + +[`BuiltInModule`]: ../module/built_in.dart + +There are a few functions that Sass supports that aren't defined here: + +* The `if()` function is defined directly in the [`functions.dart`] file, + although in most cases this is actually parsed as an [`IfExpression`] and + handled directly by [the evaluator] since it has special behavior about when + its arguments are evaluated. The function itself only exists for edge cases + like `if(...$args)` or `meta.get-function("if")`. + + [`functions.dart`]: ../functions.dart + [`IfExpression`]: ../ast/sass/expression/if.dart + [the evaluator]: ../visitor/async_evaluate.dart + +* Certain functions in the `sass:meta` module require runtime information that's + only available to the evaluator. These functions are defined in the evaluator + itself so that they have access to its private variables. diff --git a/lib/src/io/README.md b/lib/src/io/README.md new file mode 100644 index 000000000..08306bfc9 --- /dev/null +++ b/lib/src/io/README.md @@ -0,0 +1,17 @@ +# Input/Output Shim + +This directory contains an API shim for doing various forms of IO across +different platforms. Dart chooses at compile time which of the three files to +use: + +* `interface.dart` is used by the Dart Analyzer for static checking. It defines + the "expected" interface of the other two files, although there aren't strong + checks that their interfaces are exactly the same. + +* `vm.dart` is used by the Dart VM, and defines IO operations in terms of the + `dart:io` library. + +* `js.dart` is used by JS platforms. On Node.js, it will use Node's `fs` and + `process` APIs for IO operations. On other JS platforms, most IO operations + won't work at all, although messages will still be emitted with + `console.log()` and `console.error()`. diff --git a/lib/src/node/README.md b/lib/src/node/README.md new file mode 100644 index 000000000..c569e674c --- /dev/null +++ b/lib/src/node/README.md @@ -0,0 +1,58 @@ +# JavaScript API + +This directory contains Dart Sass's implementation of the Sass JS API. Dart's JS +interop support is primarily intended for _consuming_ JS libraries from Dart, so +we have to jump through some hoops in order to effectively _produce_ a JS +library with the desired API. + +JS support has its own dedicated entrypoint in [`../js.dart`]. The [`cli_pkg` +package] ensures that when users load Dart Sass _as a library_, this entrypoint +is run instead of the CLI entrypoint, but otherwise it's up to us to set up the +library appropriately. To do so, we use JS interop to define an [`Exports`] +class that is in practice implemented by a CommonJS-like[^1] `exports` object, +and then assign various values to this object. + +[`../js.dart`]: ../js.dart +[`cli_pkg` package]: https://google.com/google/dart_cli_pkg +[`Exports`]: exports.dart + +[^1]: It's not _literally_ CommonJS because it needs to run directly on browsers + as well, but it's still an object named `exports` that we can hang names + off of. + +## Value Types + +The JS API value types pose a particular challenge from Dart. Although every +Dart class is represented by a JavaScript class when compiled to JS, Dart has no +way of specifying what the JS API of those classes should be. What's more, in +order to make the JS API as efficient as possible, we want to be able to pass +the existing Dart [`Value`] objects as-is to custom functions rather than +wrapping them with JS-only wrappers. + +[`Value`]: ../value.dart + +To solve the first problem problems, in [`reflection.dart`] we use JS interop to +wrap the manual method of defining a JavaScript class. We use this to create a +JS-specific class for each value type, with all the JS-specific methods and +properties defined by Sass's JS API spec. However, while normal JS constructors +just sets some properties on `this`, our constructors for these classes return +Dart `Value` objects instead. + +[`reflection.dart`]: reflection.dart + +"But wait," I hear you say, "those `Value` objects aren't instances of the new +JS class you've created!" This is where the deep magic comes in. Once we've +defined our class with its phony constructor, we create a single Dart object of +the given `Value` subclass and _edit its JavaScript prototype chain_ to include +the new class we just created. Once that's done, all the Dart value types will +have exactly the right JS API (including responding correctly to `instanceof`!) +and the constructor will now correctly return an instance of the JS class. + +## Legacy API + +Dart Sass also supports the legacy JS API in the [`legacy`] directory. This hews +as close as possible to the API of the old `node-sass` package which wrapped the +old LibSass implementation. It's no longer being actively updated, but we still +need to support it at least until the next major version release of Dart Sass. + +[`legacy`]: legacy diff --git a/lib/src/parse/README.md b/lib/src/parse/README.md new file mode 100644 index 000000000..87fc48b02 --- /dev/null +++ b/lib/src/parse/README.md @@ -0,0 +1,32 @@ +# Sass Parser + +This directory contains various parsers used by Sass. The two most relevant +classes are: + +* [`Parser`]: The base class of all other parsers, which includes basic + infrastructure, utilities, and methods for parsing common CSS constructs that + appear across multiple different specific parsers. + + [`Parser`]: parser.dart + +* [`StylesheetParser`]: The base class specifically for the initial stylesheet + parse. Almost all of the logic for parsing Sass files, both statement- and + expression-level, lives here. Only places where individual syntaxes differ + from one another are left abstract or overridden by subclasses. + + [`StylesheetParser`]: stylesheet.dart + +All Sass parsing is done by hand using the [`string_scanner`] package, which we +use to read the source byte-by-byte while also tracking source span information +which we can then use to report errors and generate source maps. We don't use +any kind of parser generator, partly because Sass's grammar requires arbitrary +backtracking in various places and partly because handwritten code is often +easier to read and debug. + +[`string_scanner`]: https://pub.dev/packages/string_scanner + +The parser is simple recursive descent. There's usually a method for each +logical production that either consumes text and returns its corresponding AST +node or throws an exception; in some cases, a method (conventionally beginning +with `try`) will instead consume text and return a node if it matches and return +null without consuming anything if it doesn't. diff --git a/lib/src/value/README.md b/lib/src/value/README.md new file mode 100644 index 000000000..e23644722 --- /dev/null +++ b/lib/src/value/README.md @@ -0,0 +1,13 @@ +# Value Types + +This directory contains definitions for all the SassScript value types. These +definitions are used both to represent SassScript values internally and in the +public Dart API. They are usually produced by [the evaluator] as it evaluates +the expression-level [Sass AST]. + +[the evaluator]: ../visitor/async_evaluate.dart +[Sass AST]: ../ast/sass/README.md + +Sass values are always immutable, even internally. Any changes to them must be +done by creating a new value. In some cases, it's easiest to make a mutable +copy, edit it, and then create a new immutable value from the result. diff --git a/lib/src/visitor/README.md b/lib/src/visitor/README.md new file mode 100644 index 000000000..da951508c --- /dev/null +++ b/lib/src/visitor/README.md @@ -0,0 +1,15 @@ +# Visitors + +This directory contains various types that implement the [visitor pattern] for +[various ASTs]. A few of these, such as [the evaluator] and [the serializer], +implement critical business logic for the Sass compiler. Most of the rest are +either small utilities or base classes for small utilities that need to run over +an AST to determine some kind of information about it. Some are even entirely +unused within Sass itself, and exist only to support users of the [`sass_api`] +package. + +[visitor pattern]: https://en.wikipedia.org/wiki/Visitor_pattern +[various ASTs]: ../ast +[the evaluator]: async_evaluate.dart +[the serializer]: serialize.dart +[`sass_api`]: https://pub.dev/packages/sass_api From c58730be7f340836cdda4b97980efd940c04c45f Mon Sep 17 00:00:00 2001 From: Natalie Weizenbaum Date: Tue, 3 Oct 2023 15:33:09 -0700 Subject: [PATCH 2/3] Code review --- lib/src/functions/README.md | 2 +- lib/src/js/README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/src/functions/README.md b/lib/src/functions/README.md index 967023f82..6aa87f6b4 100644 --- a/lib/src/functions/README.md +++ b/lib/src/functions/README.md @@ -3,7 +3,7 @@ This directory contains the standard functions that are built into Sass itself, both those that are available globally and those that are available only through built-in modules. Each of the files here exports a corresponding -[`BuiltInModule`, and most define a list of global functions as well. +[`BuiltInModule`], and most define a list of global functions as well. [`BuiltInModule`]: ../module/built_in.dart diff --git a/lib/src/js/README.md b/lib/src/js/README.md index c569e674c..918534f6f 100644 --- a/lib/src/js/README.md +++ b/lib/src/js/README.md @@ -13,7 +13,7 @@ class that is in practice implemented by a CommonJS-like[^1] `exports` object, and then assign various values to this object. [`../js.dart`]: ../js.dart -[`cli_pkg` package]: https://google.com/google/dart_cli_pkg +[`cli_pkg` package]: https://github.com/google/dart_cli_pkg [`Exports`]: exports.dart [^1]: It's not _literally_ CommonJS because it needs to run directly on browsers From bacba6a49b8c72ac55f65a7dffc087225165958c Mon Sep 17 00:00:00 2001 From: Natalie Weizenbaum Date: Fri, 6 Oct 2023 14:48:36 -0700 Subject: [PATCH 3/3] Code review --- lib/src/ast/css/README.md | 6 ++++-- lib/src/js/README.md | 6 +++--- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/lib/src/ast/css/README.md b/lib/src/ast/css/README.md index bdd012bcd..e75c3ff5c 100644 --- a/lib/src/ast/css/README.md +++ b/lib/src/ast/css/README.md @@ -26,8 +26,10 @@ in plain CSS (such as maps)—in this case, [the serializer] will emit an error. Internally, the CSS AST is mutable to allow for operations like hoisting rules to the root of the AST and updating existing selectors when `@extend` rules are -encountered. However, because mutability poses a high risk for "spooky action at -a distance", we limit access to mutating APIs exclusively to the evaluator. +encountered. However, because mutability poses a high risk for "spooky [action +at a distance]", we limit access to mutating APIs exclusively to the evaluator. + +[action at a distance]: https://en.wikipedia.org/wiki/Action_at_a_distance_(computer_programming) We do this by having an _unmodifiable_ interface (written in this directory) for each CSS AST node which only exposes members that don't modify the node in diff --git a/lib/src/js/README.md b/lib/src/js/README.md index 918534f6f..d6cb699d2 100644 --- a/lib/src/js/README.md +++ b/lib/src/js/README.md @@ -31,11 +31,11 @@ wrapping them with JS-only wrappers. [`Value`]: ../value.dart -To solve the first problem problems, in [`reflection.dart`] we use JS interop to -wrap the manual method of defining a JavaScript class. We use this to create a +To solve the first problem, in [`reflection.dart`] we use JS interop to wrap the +manual method of defining a JavaScript class. We use this to create a JS-specific class for each value type, with all the JS-specific methods and properties defined by Sass's JS API spec. However, while normal JS constructors -just sets some properties on `this`, our constructors for these classes return +just set some properties on `this`, our constructors for these classes return Dart `Value` objects instead. [`reflection.dart`]: reflection.dart