This happens because on windows listen is always blocking. The common workaround is to fork server in separate thread and use N.close on exit.
Like with this workaround:
import Network.Socket qualified as N
withGrpc :: Server -> (RunningServer -> IO a) -> IO a
withGrpc handlers act =
forkServer def config serv $ \server -> do
sock <- atomically $ getInsecureSocket server
act server `finally` N.close sock