0.8.0 Major Update

Major Features

• Objects can be defined with aliases in configs. This means that the same object can be configured multiple times with different arguments in the same config. https://aepsych.org/docs/configs
• model.predict() now accepts a TransformedPosteriors argument to transform posteriors for the prediction (an alternate path for the probit transform for binary classification models)
• Every model and acquisition function support GPU
• Strategy buffers are always parsed to real strategy objects from databases for ease of use
• Implement IndependentGPsModel and corresponding generator. Supports arbitrary combination of models to support active learning of multiple outcomes. Each model models its own outcome independently but they all share input data. The IndependentOptimizeAcqfGenerator takes an OptimizeAcqfGenerator for each model in the IndependentGPsModel and greedily selects points when asked to generate a point. Each generator.gen() call generates a point for each model. Example here: https://github.com/facebookresearch/aepsych/blob/main/configs/independent_gps_example.ini
• Implemented Mixed Likelihood Constraints for GPClassificationModels. This allows the user to define points in the search space that they expect to take a certain classification probability (e.g., points with 0 magnitude to have 50% classification probability). Users can also define a subspace to apply a constraint, which will be added as a grid of points defining the constraint. See example config: https://github.com/facebookresearch/aepsych/blob/main/configs/mixed_likelihood_constraints_example.ini
• Added a utility to combine databases. This can be used in CLI using aepsych_database -db output.db --combine directory/to_look_for_dbs --exclude specific/databases.db --exclude exclude/another/database.db. This is also accessible in scripts from server/utils.py.

Minor features

• AEPsych server logs are now colored in stdout
• Multi-outcome tell messages are now supported directly for multi-outcome experiments. The server API is documented here: https://aepsych.org/docs/server_overview
• Lookahead acquisition functions use lazy tensors to save on memory
• Marginal likelihood class can be modified for variational GPs during intialization
• Added a new inducing point allocator: FixedPlusAllocator. A variation of the FixedAllocator that allows the user to add specific points that will always be used as inducing points on top of the inducing points generated from any other allocator (essentially an allocator wrapper).
• Add F1 score to LSE benchmarks.
• Add read_only mode for databases that can be invoked from the server CLI or within scripts, a temporary copy of the database will be created and used leaving the originally completely untouched.
• Exporting a dataframe/csv from a database can now include extra data
• Added support for unloading extensions, runs a specific _unload() function for teardown alongside unregistering the extension from the cache.

API Changes

• All variational model subclass from VariationalGP instead of GPClassificationModel (including GPClassificationModel itself). The model class hierarchy in general has been cleaned up to more strictly subclass from a single parent.
• Server-client message handling of shapes is now more consistent where 2D inputs are always expected to be a single trial single stimuli. Further nested (3D) inputs are expected for multi stimuli or multi trial inputs. In 3D cases, the first dimension indicates trials, the second dimension indicates dimensions, and the third dimension indicates stimuli. See #660 for details.
• The can_model server message has been removed and folded into get_info
• All message handlers return dictionaries now and thus the server will always respond to messages with jsons, the python and unity clients are updated to expect this
• Acquisition functions for OptimizeAcqfGenerator is strictly searched for in the OptimizeAcqfGenerator section in configs (and never in strategy anymore).
• All classes use the ConfigurableMixin API now, this means that it should be easier to implement new classes to be used in configs and all classes should more consistently be searching for options in configs.
• Moved out many methods related to querying a model out of strategy and prefer them to be standalone utility functions (functions are found in model/utils.py)
• Removed support for very old configs
• Removed remnants of Ax backend
• Strategy moved to its own submodule
• Removed unused classes to lower maintenance burden
  • Classes related to MonotonicRejectionGP (model, generator, acqf, factory)
  • Classed related to DerivativeGP (model, rejection sampler, partial_grad factory)
  • Multitask regression models (IndependentMultitaskGPRModel, MultitaskGPRModel)

Client updates

• Python client now has a method for every possible message to the server
• Python and Unity client updated to match the latest API
• Matlab client updates to come

Testing and CI

• Tests largely overhauled to be more reliable and faster
• Github Workflow lint job always runs everything when testing
• Github Workflows run nightly
• Added a test/ci mode for AEPsych used for testing that silences most warnings and loggers that are irrelevant for testing, this also raises exceptions for unexpected warnings
• Python client is now part of Github Workflow CI
• Github Workflow CI strictly tests for lint errors using ruff

Important Bug Fixes

• Data moves devices more reliably for GPU ops
• Data and bounds are consistently stored in raw parameter space instead of transformed space everywhere except for within transform wrapped objects
• Strategies now look at their generator to be sure that it can generate limited points
• All server relevant warnings now are sent to the log instead of being bare warnings