-
Notifications
You must be signed in to change notification settings - Fork 53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[native_assets_cli] HookConfigBuilder.checksum
should exclude some keys
#1803
Comments
It should exclude the
How often does the C compiler config change? Does it matter at all? It's the safest thing to do to re-run the hook in a clean slate if the config (anything in the config modulo If there are expensive operations that one wants to cache across different configs, then users have the Also in the future, if we add metadata support (maybe it's still there?) we'd run it from clean state if metadata changed. If we don't do this, then again the hook itself needs to deal with this internally somehow, remember last metadata new metadata and magically know what needs to be invalidated. Basically it's a trade-off of how much the system takes care of caching/invalidation vs the individual hooks. I think the way it's now is a reasonable sweet spot: For a typical developer they will
This particular one we could explicitly exclude, yes. Currently it will only use a new directory if the set of assets changes (or the file uris in the assets), but not if the assets have different content. Though as bugs like #1412 show reusing the same dir leads to bugs. So this is a "safer" solution. I'm open to excluding that one.
Exactly, so we don't benefit from an old populated out directory (normally we benefit because we have an incremental build, but if api levels change we have to compile things from scratch anyway).
That's not a bug as it causes no trouble. Also, a tool like flutter does generate them in a deterministic order (for same config it will always generate them in the same order and the checksum will therefore be the same) - only if the flutter SDK was updated may it change the order, which isn't a problem. If we want to keep it sorted (and we really don't have to) we should do it in one place instead of assuming N different places in the codebase happen to remember to sort the maps, I've filed #1650 for this some time ago.
Or leave it like it is :)
Or leave it like it is :) |
With that kind of logic we should also run from a clean state if the file system changed. Basically preventing hooks from doing any kind of internal caching when being reinvoked. If we move meta data to be files it's the same spiel, if the contents of the metadata change, we reinvoke the hook. It's up to the hook to inspect whether files changed and to do internal caching if they want. Whether metadata is in the config or in a file shouldn't change whether caching is reinvoked or not.
With that kind of logic, why do we even try to provide them the same dir on subsequent invocations of We discussed at length to preserve the outdir in #1593 and #1375. If we go the direction of pushing users to use the Okay, with that kind of argument, target OS versions should also influence it. (I guess I just convinced myself to include everything.)
We're taking a checksum of an unordered collection and depend on the order for the checksum, that's just plain wrong. We can address it by #1650 indeed. Okay, closing this issue as WAI. (Which means metadata caching semantics changes if we switch to files. #1251) The checksums being influenced by order can be solved as part of #1650. |
@dcharkes I'm just expressing my own view. If there's good reasons we should do it differently, we should re-open and discuss more! IMHO we should optimize the case where the That being said, I am a little concerned of us creating lots of directories that users eventually need to |
Side note:
It's actually an ordered collection not an unordered collection: Our maps guarantee iteration order being insertion order (that's why deterministic code building a json will iterate keys in the same order as they were inserted and produce the same hash) |
That would be a second benefit of including less of the JSON into the checksum indeed. Presumably, the asset-specific-config-provider would know which parts of the config would cause unnecessary creation of new output directors. But I'm not entirely convinced of this yet.
(Right, we don't have the equivalent of |
The refactoring in #1643 does a blanket checksum over the whole json representation of the config, this leads to the build-directory changing and cache disappearing for various config elements that should not have this effect:
linkConfig.assetsForLinking
[native_assets_builder] KeepoutputDirectory
ifLinkConfig.assets
changes #1746config.codeConfig.cCompiler
config.codeConfig.targetIOSVersion
,.targetMacOSVersion
,.targetAndroidNdkApi
If any of these change, we're likely not interested in any previous invocations of the same hook with the previous versions. (Note for
config.codeConfig.targetIOSSdk
we do actually want to cache the simulator and device separately, because you'd actually switch between building those in a Flutter project.)I worked around
linkConfig.assetsForLinking
by applyingsetupLinkConfig
after computing the hash, but that doesn't work forsetupCodeConfig
, because some parts should be taken into account for the hash while others don't.Possible solutions:
json
and ajsonForChecksum
.listOfPathsForChecksum
Side note, currently the checksum is dependent on the order in which the separate parts are added. This is a bug, the maps in the JSON should be regarded as inherently unsorted.
Possible solutions:
cc @mkustermann
The text was updated successfully, but these errors were encountered: