Always call 'getGeneratorContent' for posts in watch mode #98

rdipardo · 2021-09-01T22:58:35Z

Since 0a7c669, Generator.GeneratorEvaluator.evaluate loads a generator script from a cache:

Lines 180 to 191 in 0a7c669

    
               let evaluate (fsi : FsiEvaluationSession) (siteContent : SiteContents) (generatorPath : string) (projectRoot: string) (page: string)  = 
        
                   let ok, generator = generatorCache.TryGetValue(generatorPath) 
        
                   let generator = 
        
                       if ok then 
        
                           generator 
        
                       else 
        
                           let generator = 
        
                               getGeneratorContent fsi generatorPath 
        
                               |> Result.bind (compileExpression >> Ok) 
        
                           generatorCache.AddOrUpdate(generatorPath, generator, fun key value -> value) |> ignore 
        
                           generator

But reusing the post.fsx generator crashes fornax in watch mode; the siteContent variable is empty the second time around:

[23:09:34] multiple files generated in 118ms
[23:09:35] '/home/rob/dev/Fornax/src/Fornax.Template/_public/about.html' generated in 476ms
[23:09:35] '/home/rob/dev/Fornax/src/Fornax.Template/_public/contact.html' generated in 504ms
Model+SiteContents has 7 posts
[23:09:36] '/home/rob/dev/Fornax/src/Fornax.Template/_public/posts/post6.html' generated in 537ms
Model+SiteContents has 7 posts
[23:09:37] '/home/rob/dev/Fornax/src/Fornax.Template/_public/posts/post.html' generated in 504ms
Model+SiteContents has 7 posts
[23:09:38] '/home/rob/dev/Fornax/src/Fornax.Template/_public/posts/post2.html' generated in 505ms
Model+SiteContents has 7 posts
[23:09:39] '/home/rob/dev/Fornax/src/Fornax.Template/_public/posts/post4.html' generated in 520ms
Model+SiteContents has 7 posts
[23:09:40] '/home/rob/dev/Fornax/src/Fornax.Template/_public/posts/post5.html' generated in 533ms
Model+SiteContents has 7 posts
[23:09:41] '/home/rob/dev/Fornax/src/Fornax.Template/_public/posts/post3.html' generated in 509ms
[23:09:41] '/home/rob/dev/Fornax/src/Fornax.Template/_public/js/sampleJsFile.js' generated in 91ms
[23:09:41] '/home/rob/dev/Fornax/src/Fornax.Template/_public/images/avatar.jpg' generated in 96ms
[23:09:41] '/home/rob/dev/Fornax/src/Fornax.Template/_public/images/bulma.png' generated in 95ms
[23:09:41] '/home/rob/dev/Fornax/src/Fornax.Template/_public/images/favicon.png' generated in 89ms
[23:09:41] '/home/rob/dev/Fornax/src/Fornax.Template/_public/style/style.css' generated in 100ms
Model+SiteContents has 7 posts
[23:09:42] '/home/rob/dev/Fornax/src/Fornax.Template/_public/posts/subdir/post3.html' generated in 2ms
Generation time: 00:00:32.8188529
[23:09:42] Watch mode started. Press any key to exit.
[23:09:42 INF] Smooth! Suave listener started in 166.701ms with binding 127.0.0.1:8080

[... edit post.md ...]

[23:09:50] Changes detected: /home/rob/dev/Fornax/src/Fornax.Template/posts/post.md
[23:09:53] multiple files generated in 638ms
[23:09:53] '/home/rob/dev/Fornax/src/Fornax.Template/_public/about.html' generated in 1ms
[23:09:53] '/home/rob/dev/Fornax/src/Fornax.Template/_public/contact.html' generated in 1ms
Model+SiteContents has 0 posts
An unexpected error happend: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.Collections.Generic.KeyNotFoundException: An index satisfying the predicate was not found in the collection.
   at Microsoft.FSharp.Collections.SeqModule.Find[T](FSharpFunc`2 predicate, IEnumerable`1 source) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 677
   at FSI_0022.Post.generate'(SiteContents ctx, String page)
   at lambda_method21(Closure , Unit , SiteContents , String , String )
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Generator.EvaluatorHelpers.helper@56(Object next, FSharpList`1 args) in /home/rob/dev/Fornax/src/Fornax/Generator.fs:line 66
   at Generator.EvaluatorHelpers.invokeFunction(Object f, IEnumerable`1 args) in /home/rob/dev/Fornax/src/Fornax/Generator.fs:line 70
   at [email protected](Object generator) in /home/rob/dev/Fornax/src/Fornax/Generator.fs:line 195
   at System.Runtime.CompilerServices.RuntimeHelpers.DispatchTailCalls(IntPtr callersRetAddrSlot, IntPtr callTarget, IntPtr retVal)
   at Generator.GeneratorEvaluator.evaluate(FsiEvaluationSession fsi, SiteContents siteContent, String generatorPath, String projectRoot, String page) in /home/rob/dev/Fornax/src/Fornax/Generator.fs:line 190
   at Generator.generate(FsiEvaluationSession fsi, Config cfg, SiteContents siteContent, String projectRoot, String page) in /home/rob/dev/Fornax/src/Fornax/Generator.fs:line 333
   at Generator.action@1-3(String projectRoot, FsiEvaluationSession fsi, Config config, SiteContents sc, String filePath) in /home/rob/dev/Fornax/src/Fornax/Generator.fs:line 496
   at Generator.generateFolder(String projectRoot, Boolean isWatch) in /home/rob/dev/Fornax/src/Fornax/Generator.fs:line 495
   at Fornax.guardedGenerate@226(String cwd, Unit unitVar0) in /home/rob/dev/Fornax/src/Fornax/Fornax.fs:line 228
Finished (Failed) 'TestTemplate' in 00:00:43.2120491

As long as getGeneratorContent is called in the rebuild cycle, file watching works as intended:

issue-96-patch-demo.mp4

This patch makes evaluate aware of watch mode(*) so that getGeneratorContent can be called on Markdown pages after a file change event.

Fixes #96
Closes #84 (specifically #84 (comment))

(*) In practice, by propagating the value of isWatch via Generator.generateFolder -> Generator.generate -> Generator.GeneratorEvaluator.evaluate.

Because Markdown files are never handled by Generator.runOnceGenerators, there evaluate is called with isWatch hard-coded as false, so the former behaviour hasn't changed.

Generator.GeneratorEvaluator.evaluateMultiple is also unchanged because it doesn't use the cache:

Fornax/src/Fornax/Generator.fs

Lines 212 to 215 in abd3baa

    
               let evaluateMultiple (fsi : FsiEvaluationSession) (siteContent : SiteContents) (generatorPath : string) (projectRoot: string) (page: string)  = 
        
                   getGeneratorContent fsi generatorPath 
        
                   |> Result.bind (fun ft -> 
        
                       let generator = compileExpression ft

This could mean sacrificing some of the efficiency that 0a7c669 was aiming for, but IMHO a really efficient file watcher would only rebuild the changed file, not the entire website.

Actual performance will vary: the video above was captured on a Win 8.1-era notebook with a 900 MHz AMD processor

Fixes a regression introduced in 0a7c669

bigjonroberts · 2022-02-25T18:42:47Z

I ran into this same issue. I'd like to test out a build from this branch, but I'm unsure if there is a quick/easy way to try it out? It looks like dotnet tool install supports a configfile setting to point to another nuget feed and adjust other settings there. Do builds from here get published somewhere as pre-releases?

rdipardo · 2022-03-05T14:05:52Z

@bigjonroberts,

I'd like to test out a build from this branch, but I'm unsure if there is a quick/easy way to try it out?

I'm not sure why, but for me the latest stable fornax doesn't crash anymore!
Maybe a debug build configuration was the source of the problem all along 🤔

rdipardo · 2022-03-07T23:43:01Z

I still don't know why the stable release works fine when development builds continue to crash.

At least I'm sure it's not the build configuration: the complete invocation of the Build and Publish targets contain property:Configuration=Release:

$ FAKE_SDK_RESOLVER_CUSTOM_DOTNET_PATH=/usr/share/dotnet dotnet fake build -t TestTemplate

The last restore is still up to date. Nothing left to do. run TestTemplate Building project with version: LocalBuild Shortened DependencyGraph for Target TestTemplate: <== TestTemplate <== Publish <== Build <== Restore <== Clean The running order is: Group - 1 Clean Group - 2 Restore Group - 3 Build Group - 4 Publish Group - 5 TestTemplate Starting target 'Clean' Finished (Success) 'Clean' in 00:00:00.0177219 Starting target 'Restore' Starting task 'DotNet:restore' "/usr/bin/mono" --version (In: false, Out: true, Err: true) /home/rob/repos/dotnet/Fornax> "dotnet" msbuild /version /nologo (In: false, Out: true, Err: true) 17.1.0.7609 /home/rob/repos/dotnet/Fornax> "dotnet" restore "" /bl:/tmp/tmpkAQDnQ.tmp.binlog (In: false, Out: false, Err: false) /usr/share/dotnet/sdk/6.0.200/MSBuild.dll -nologo -maxcpucount -target:Restore -verbosity:m /bl:/tmp/tmpkAQDnQ.tmp.binlog ./Fornax.sln Determining projects to restore... All projects are up-to-date for restore. Finished (Success) 'DotNet:restore' in 00:00:12.3751631 Finished (Success) 'Restore' in 00:00:12.3853960 Starting target 'Build' Starting task 'DotNet:build' /home/rob/repos/dotnet/Fornax> "dotnet" build "" --configuration Release /bl:/tmp/tmp0r2MRE.tmp.binlog (In: false, Out: false, Err: false) Microsoft (R) Build Engine version 17.1.0+ae57d105c for .NET Copyright (C) Microsoft Corporation. All rights reserved. /usr/share/dotnet/sdk/6.0.200/MSBuild.dll -consoleloggerparameters:Summary -maxcpucount -property:Configuration=Release -restore -verbosity:m /bl:/tmp/tmp0r2MRE.tmp.binlog ./Fornax.sln Determining projects to restore... All projects are up-to-date for restore. Fornax.Core -> /home/rob/repos/dotnet/Fornax/src/Fornax.Core/bin/Release/netstandard2.0/Fornax.Core.dll Fornax.Core.UnitTests -> /home/rob/repos/dotnet/Fornax/test/Fornax.Core.UnitTests/bin/Release/net6.0/Fornax.Core.UnitTests.dll Fornax -> /home/rob/repos/dotnet/Fornax/src/Fornax/bin/Release/net5.0/Fornax.dll Build succeeded. 0 Warning(s) 0 Error(s) Time Elapsed 00:01:24.13 Finished (Success) 'DotNet:build' in 00:01:25.7469377 Finished (Success) 'Build' in 00:01:25.7495507 Starting target 'Publish' Starting task 'DotNet:publish': src/Fornax /home/rob/repos/dotnet/Fornax> "dotnet" publish src/Fornax --configuration Release --output /home/rob/repos/dotnet/Fornax/temp /bl:/tmp/tmp8h4HMY.tmp.binlog (In: false, Out: false, Err: false) Microsoft (R) Build Engine version 17.1.0+ae57d105c for .NET Copyright (C) Microsoft Corporation. All rights reserved.

/usr/share/dotnet/sdk/6.0.200/MSBuild.dll -maxcpucount -property:PublishDir=/home/rob/repos/dotnet/Fornax/temp -property:Configuration=Release -restore -target:Publish -verbosity:m /bl:/tmp/tmp8h4HMY.tmp.binlog src/Fornax/Fornax.fsproj Determining projects to restore... All projects are up-to-date for restore. Fornax.Core -> /home/rob/repos/dotnet/Fornax/src/Fornax.Core/bin/Release/netstandard2.0/Fornax.Core.dll Fornax -> /home/rob/repos/dotnet/Fornax/src/Fornax/bin/Release/net5.0/Fornax.dll Fornax -> /home/rob/repos/dotnet/Fornax/temp/ Finished (Success) 'DotNet:publish' in 00:00:11.5370303 Finished (Success) 'Publish' in 00:00:11.5400999 Starting target 'TestTemplate' templateDir: /home/rob/repos/dotnet/Fornax/src/Fornax.Template/ /home/rob/repos/dotnet/Fornax/src/Fornax.Template/> "dotnet" /home/rob/repos/dotnet/Fornax/temp/Fornax.dll watch (In: false, Out: false, Err: false) [17:41:46] multiple files generated in 610ms [17:41:46] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/about.html' generated in 767ms [17:41:47] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/contact.html' generated in 613ms [17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/posts/post3.html' generated in 632ms [17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/posts/post.html' generated in 2ms [17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/posts/post4.html' generated in 1ms [17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/posts/post6.html' generated in 2ms [17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/posts/post5.html' generated in 2ms [17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/posts/post2.html' generated in 2ms [17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/js/sampleJsFile.js' generated in 117ms [17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/style/style.css' generated in 1ms [17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/images/bulma.png' generated in 0ms [17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/images/avatar.jpg' generated in 0ms [17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/images/favicon.png' generated in 0ms [17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/posts/subdir/post3.html' generated in 1ms Generation time: 00:00:27.6688370 [17:41:48] Watch mode started. Press any key to exit. [17:41:48 INF] Smooth! Suave listener started in 114.535ms with binding 127.0.0.1:8080

$ echo -e "\n\n### More\n" >> src/Fornax.Template/posts/post.md


[17:42:40] Changes detected: /home/rob/repos/dotnet/Fornax/src/Fornax.Template/posts/post.md
[17:42:42] multiple files generated in 635ms
[17:42:42] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/about.html' generated in 1ms
[17:42:42] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/contact.html' generated in 0ms
An unexpected error happend: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.Collections.Generic.KeyNotFoundException: An index satisfying the predicate was not found in the collection.
   at Microsoft.FSharp.Collections.SeqModule.Find[T](FSharpFunc`2 predicate, IEnumerable`1 source) in D:\a\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 677
   at FSI_0022.Post.generate'(SiteContents ctx, String page)
   at lambda_method21(Closure , Unit , SiteContents , String , String )
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Span`1& arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Generator.EvaluatorHelpers.helper@56(Object next, FSharpList`1 args) in /home/rob/repos/dotnet/Fornax/src/Fornax/Generator.fs:line 66
   at Generator.EvaluatorHelpers.invokeFunction(Object f, IEnumerable`1 args) in /home/rob/repos/dotnet/Fornax/src/Fornax/Generator.fs:line 70
   at [email protected](Object generator) in /home/rob/repos/dotnet/Fornax/src/Fornax/Generator.fs:line 195
   at Generator.GeneratorEvaluator.evaluate(FsiEvaluationSession fsi, SiteContents siteContent, String generatorPath, String projectRoot, String page) in /home/rob/repos/dotnet/Fornax/src/Fornax/Generator.fs:line 184
   at Generator.generate(FsiEvaluationSession fsi, Config cfg, SiteContents siteContent, String projectRoot, String page) in /home/rob/repos/dotnet/Fornax/src/Fornax/Generator.fs:line 335
   at Generator.generateFolder(String projectRoot, Boolean isWatch) in /home/rob/repos/dotnet/Fornax/src/Fornax/Generator.fs:line 495
   at Fornax.guardedGenerate@226(String cwd, Unit unitVar0) in /home/rob/repos/dotnet/Fornax/src/Fornax/Fornax.fs:line 228
Finished (Failed) 'TestTemplate' in 00:01:23.6215582

rdipardo · 2022-05-07T20:09:51Z

Closing now that bafe619 has been merged

Always call 'getGeneratorContent' for posts in watch mode

bd61de1

Fixes a regression introduced in 0a7c669

rdipardo closed this May 7, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Always call 'getGeneratorContent' for posts in watch mode #98

Always call 'getGeneratorContent' for posts in watch mode #98

rdipardo commented Sep 1, 2021

bigjonroberts commented Feb 25, 2022

rdipardo commented Mar 5, 2022

rdipardo commented Mar 7, 2022

rdipardo commented May 7, 2022

	let evaluate (fsi : FsiEvaluationSession) (siteContent : SiteContents) (generatorPath : string) (projectRoot: string) (page: string) =

	let ok, generator = generatorCache.TryGetValue(generatorPath)
	let generator =
	if ok then
	generator
	else
	let generator =
	getGeneratorContent fsi generatorPath
	\|> Result.bind (compileExpression >> Ok)
	generatorCache.AddOrUpdate(generatorPath, generator, fun key value -> value) \|> ignore
	generator

	let evaluateMultiple (fsi : FsiEvaluationSession) (siteContent : SiteContents) (generatorPath : string) (projectRoot: string) (page: string) =
	getGeneratorContent fsi generatorPath
	\|> Result.bind (fun ft ->
	let generator = compileExpression ft

Always call 'getGeneratorContent' for posts in watch mode #98

Always call 'getGeneratorContent' for posts in watch mode #98

Conversation

rdipardo commented Sep 1, 2021

bigjonroberts commented Feb 25, 2022

rdipardo commented Mar 5, 2022

rdipardo commented Mar 7, 2022

rdipardo commented May 7, 2022