Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dune.3.6.1 crashes when I try to cross-compile a little project with Esperanto #6843

Closed
dinosaure opened this issue Jan 8, 2023 · 2 comments · Fixed by #6958 or ocaml/opam-repository#23349

Comments

@dinosaure
Copy link

Currently, I tried to upgrade [esperanto][espereanto] with Cosmopolitan v2.2 and try to improve [lwt][lwt] to support Cosmopolitan C library. My current situation is:

  1. I use the toolchain esperanto available here: Cosmopolitan v2.2 dinosaure/esperanto#28
  2. I try to use the lwt version available here: Allow the Esperanto support for lwt ocsigen/lwt#945
  3. I'm currently on OCaml 4.14.1, I use dune.3.6.1 and ocamlfind.1.9.5
  4. for the static linking, I use opam-monorepo.0.3.4 to bring dependencies

Let's try a simple to compile a simple program:

$ opam pin add -y https://github.com/dinosaure/esperanto.git#cosmopolitan-v2.2
$ opam pin add -yn lwt https://github.com/dinosaure/lwt.git#esperanto
$ opam install dune ocamlfind opam-monorepo
$ cat >dune-workspace <<EOF
(lang dune 2.0)
(context (default))
(context
 (default
  (name esperanto)
  (toolchain esperanto)
  (host default)))
EOF
$ cat >cat.ml <<EOF
open Lwt.Infix

let rec full_write fd buf off len =
  Lwt_unix.write fd buf off len >>= fun len' ->
  if len - len' > 0
  then full_write fd buf (off + len') (len - len')
  else Lwt.return_unit

let tmp = Bytes.create 0x1000

let rec cat () =
  Lwt.catch begin fun () ->
  Lwt_unx.read Lwt_unix.stdin tmp 0 (Bytes.length tmp) >>= fun len ->
  match len with
  | 0 -> Lwt.return_unit
  | len -> full_write Lwt_unix.stdout tmp 0 len >>= cat
  end @@ function
  | End_of_file -> Lwt.return_unit
  | exn -> raise exn

let () = Lwt_main.run (cat ())
EOF
$ cat >cat.opam <<EOF
opam-version: "2.0"
name:         "cat"
maintainer:   [ "Romain Calascibetta <[email protected]>" ]
authors:      [ "Romain Calascibetta <[email protected]>" ]
homepage:     "https://github.com/dinosaure/esperanto"
bug-reports:  "https://github.com/dinosaure/esperanto/issues"
dev-repo:     "git+https://github.com/dinosaure/esperanto"
doc:          "https://dinosaure.github.io/esperanto/"
license:      "MIT"
synopsis:     "The cat.com tool produced by esperanto"
description:  "The cat.com tool produced by esperanto"

build: [
  [ "dune" "build" "-p" name "-j" jobs ]
]
install:  [
  [ "dune" "install" "-p" name ] {with-test}
]

depends: [
  "ocaml"           {>= "4.12.0"}
  "dune"            {>= "2.8.0"}
  "lwt"
]
EOF
$ opam monorepo lock
$ opam monorepo pull
$ dune build -x esperanto ./cat.exe
$ dune build -x esperanto ./cat.exe
Warning: No dune-project file has been found. A default one is assumed but
the project might break when dune is upgraded. Please create a dune-project
file.
Hint: generate the project file with: $ dune init project <name>
File "src/dune_rules/context.ml", line 755, characters 20-26:
File "src/dune_rules/context.ml", line 755, characters 20-26: Assertion
failed
Raised at Dune_rules__context.Instantiate.instantiate_impl in file
  "src/dune_rules/context.ml", line 755, characters 20-32
Called from Fiber__core.O.(>>|).(fun) in file "src/fiber/core.ml", line 250,
  characters 36-41
Called from Fiber__scheduler.exec in file "src/fiber/scheduler.ml", line 73,
  characters 8-11

I must not crash.  Uncertainty is the mind-killer. Exceptions are the
little-death that brings total obliteration.  I will fully express my cases. 
Execution will pass over me and through me.  And when it has gone past, I
will unwind the stack along its path.  Where the cases are handled there will
be nothing.  Only I will remain.

As far as I can say, previous versions of dune worked and it seems related to this line:

https://github.com/ocaml/dune/blame/63e6845a89eab969323c88673533271b9e3c0baf/src/dune_rules/context.ml#L754

@emillon emillon changed the title dune.3.6.1 crasche when I try to cross-compile a little project with Esperanto dune.3.6.1 crashes when I try to cross-compile a little project with Esperanto Jan 10, 2023
@emillon
Copy link
Collaborator

emillon commented Jan 19, 2023

Hi. Thanks for the bug report.

previous versions of dune worked

Can you try to bisect the problem (just at the release level) to see which release introduced this regression?

rgrinberg added a commit that referenced this issue Jan 28, 2023
Cross compilation setup causes dune to crash

Signed-off-by: Rudi Grinberg <[email protected]>

<!-- ps-id: 8500a05a-f8fe-403f-863c-a26ba7e57769 -->
@rgrinberg
Copy link
Member

This is a bug, but note that your command line invocation is likely not what you want. You don't need to use -x if you're writing a workspace file. -x is simply a shorthand for defining additional cross compilation targets. Your configuration should work if you just select your esperanto context with dune build --context esperanto.

rgrinberg added a commit that referenced this issue Jan 28, 2023
Cross compilation setup causes dune to crash

Signed-off-by: Rudi Grinberg <[email protected]>
rgrinberg added a commit that referenced this issue Jan 28, 2023
It's possible for a context with targets to be a cross compilation
context for other contexts. Previously, we'd assume that wasn't the
case.

fixes #6843

Signed-off-by: Rudi Grinberg <[email protected]>

<!-- ps-id: e7ed7d4e-717a-402a-8cc2-00b9392d7fbb -->
@rgrinberg rgrinberg added this to the 3.7.0 milestone Jan 28, 2023
rgrinberg added a commit that referenced this issue Jan 28, 2023
It's possible for a context with targets to be a cross compilation
context for other contexts. Previously, we'd assume that wasn't the
case.

fixes #6843

Signed-off-by: Rudi Grinberg <[email protected]>

<!-- ps-id: e7ed7d4e-717a-402a-8cc2-00b9392d7fbb -->
rgrinberg added a commit that referenced this issue Jan 30, 2023
It's possible for a context with targets to be a cross compilation
context for other contexts. Previously, we'd assume that wasn't the
case.

fixes #6843

Signed-off-by: Rudi Grinberg <[email protected]>
emillon added a commit to emillon/opam-repository that referenced this issue Feb 6, 2023
…, dune-rpc, dune-rpc-lwt, dune-private-libs, dune-glob, dune-configurator, dune-build-info, dune-action-plugin and chrome-trace (3.7.0~alpha1)

CHANGES:

- Fix parsing of OCaml errors that contain code excerpts with `...` in them.
  (ocaml/dune#7008, @rgrinberg)

- Pre-emptively clear screen in watch mode (ocaml/dune#6987, fixes ocaml/dune#6884, @rgrinberg)

- Fix cross compilation configuration when a context with targets is itself a
  host of another context (ocaml/dune#6958, fixes ocaml/dune#6843, @rgrinberg)

- Fix parsing of the `<=` operator in *blang* expressions of `dune` files.
  Previously, the operator would be interpreted as `,`. (ocaml/dune#6928, @tatchi)

- Fix `--trace-file` output. Dune now emits a single *complete* event for every
  executed process. Unterminated *async* events are no longer written. (ocaml/dune#6892,
  @rgrinberg)

- Fix preprocessing with `staged_pps` (ocaml/dune#6748, fixes ocaml/dune#6644, @rgrinberg)

- Use colored output with MDX when Dune colors are enabled.
  (ocaml/dune#6462, @MisterDA)

- Make `dune describe workspace` return consistent dependencies for
  executables and for libraries. By default, compile-time dependencies
  towards PPX-rewriters are from now not taken into account (but
  runtime dependencies always are). Compile-time dependencies towards
  PPX-rewriters can be taken into account by providing the
  `--with-pps` flag. (ocaml/dune#6727, fixes ocaml/dune#6486, @esope)

- Print missing newline after `$ dune exec`. (ocaml/dune#6821, fixes ocaml/dune#6700, @rgrinberg,
  @Alizter)

- Fix binary corruption when installing or promoting in parallel (ocaml/dune#6669, fixes
  ocaml/dune#6668, @edwintorok)

- Use colored output with GCC and Clang when compiling C stubs. The
  flag `-fdiagnostics-color=always` is added to the `:standard` set of
  flags. (ocaml/dune#4083, @MisterDA)

- Fix the parsing of decimal and hexadecimal escape literals in `dune`,
  `dune-package`, and other dune s-expression based files (ocaml/dune#6710, @shym)

- Report an error if `dune init ...` would create a "dune" file in a location
  which already contains a "dune" directory (ocaml/dune#6705, @gridbugs)

- Fix the parsing of alerts. They will now show up in diagnostics correctly.
  (ocaml/dune#6678, @rginberg)

- Fix the compilation of modules generated at link time when
  `implicit_transitive_deps` is enabled (ocaml/dune#6642, @rgrinberg)

- Allow `$ dune utop` to load libraries defined in data only directories
  defined using `(subdir ..)` (ocaml/dune#6631, @rgrinberg)

- Format dune files when they are named `dune-file`. This occurs when we enable
  the alternative file names project option. (ocaml/dune#6566, @rgrinberg)

- Move `$ dune ocaml-merlin -dump-config=$dir` to `$ dune ocaml merlin
  dump-config $dir`. (ocaml/dune#6547, @rgrinberg)

- Allow compilation rules to be impacted by `(env ..)` stanzas that modify the
  environment or set binaries. (ocaml/dune#6527, @rgrinberg)

- Coq native mode is now automatically detected by Dune starting with Coq lang
  0.7. `(mode native)` has been deprecated in favour of detection from the
  configuration of Coq. (ocaml/dune#6409, @Alizter)

- Print "Leaving Directory" whenever "Entering Directory" is printed. (ocaml/dune#6149,
  fixes ocaml/dune#138, @cpitclaudel, @rgrinberg)

- Allow `$ dune ocaml dump-dot-merlin` to run in watch mode. Also this command
  shouldn't print "Entering Directory" mesages. (ocaml/dune#6497, @rgrinberg)

- `dune clean` should no longer fail under Windows due to the inability to
  remove the `.lock` file. Also, bring the implementation of the global lock
  under Windows closer to that of Unix. (ocaml/dune#6523, @nojb)

- Remove "Entering Directory" messages for `$ dune install`. (ocaml/dune#6513,
  @rgrinberg)

- Stop passing `-q` flag in `dune coq top`, which allows for `.coqrc` to be
  loaded. (ocaml/dune#6848, fixes ocaml/dune#6847, @Alizter)

- Fix missing dependencies when detecting the kind of C compiler we're using
  (ocaml/dune#6610, fixes ocaml/dune#6415, @emillon)

- Allow `(include_subdirs qualified)` for OCaml projects. (ocaml/dune#6594, fixes ocaml/dune#1084,
  @rgrinberg)

- Accurately determine merlin configuration for all sources selected with
  `copy#` and `copy_files#`. The old heuristic of looking for a module in
  parent directories is removed (ocaml/dune#6594, @rgrinberg)

- Fix inline tests with *js_of_ocaml* and whole program compilation mode
  enabled (ocaml/dune#6645, @hhugo)

- Fix *js_of_ocaml* separate compilation rules when `--enable=effects`
  ,`--enable=use-js-string` or `--toplevel` is used. (ocaml/dune#6714, ocaml/dune#6828, ocaml/dune#6920, @hhugo)

- Fix *js_of_ocaml* separate compilation in presence of linkall (ocaml/dune#6832, ocaml/dune#6916, @hhugo)

- Remove spurious build dir created when running `dune init proj ...` (ocaml/dune#6707,
  fixes ocaml/dune#5429, @gridbugs)

- Allow `--sandbox` to affect `ocamldep` invocations. Previously, they were
  wrongly marked as incompatible (ocaml/dune#6749, @rgrinberg)

- Validate the command line arguments for `$ dune ocaml top-module`. This
  command requires one positional argument (ocaml/dune#6796, fixes ocaml/dune#6793, @rgrinberg)

- Add a `dune cache size` command for displaying the size of the cache (ocaml/dune#6638,
  @Alizter)

- Add 4.14.0 MSVC to CI (ocaml/dune#6917, @jonahbeckford)

- Fix dependency cycle when installing files to the bin section with
  `glob_files` (ocaml/dune#6764, fixes ocaml/dune#6708, @gridbugs)

- Handle "Too many links" errors when using Dune cache on Windows (ocaml/dune#6993, @nojb)
emillon added a commit to emillon/opam-repository that referenced this issue Feb 17, 2023
…, dune-rpc, dune-rpc-lwt, dune-private-libs, dune-glob, dune-configurator, dune-build-info, dune-action-plugin and chrome-trace (3.7.0)

CHANGES:

- Allow running `$ dune exec` in watch mode (with the `-w` flag). In watch mode,
  `$ dune exec` the executed binary whenever it is recompiled. (ocaml/dune#6966,
  @gridbugs)

- `coqdep` is now called once per theory, instead of one time per Coq
  file. This should significantly speed up some builds, as `coqdep`
  startup time is often heavy (ocaml/dune#7048, @Alizter, @ejgallego)

- Add `map_workspace_root` dune-project stanza to allow disabling of
  mapping of workspace root to `/workspace_root`. (ocaml/dune#6988, fixes ocaml/dune#6929,
  @richardlford)

- Fix handling of support files generated by odoc. (ocaml/dune#6913, @jonludlam)

- Fix parsing of OCaml errors that contain code excerpts with `...` in them.
  (ocaml/dune#7008, @rgrinberg)

- Pre-emptively clear screen in watch mode (ocaml/dune#6987, fixes ocaml/dune#6884, @rgrinberg)

- Fix cross compilation configuration when a context with targets is itself a
  host of another context (ocaml/dune#6958, fixes ocaml/dune#6843, @rgrinberg)

- Fix parsing of the `<=` operator in *blang* expressions of `dune` files.
  Previously, the operator would be interpreted as `<`. (ocaml/dune#6928, @tatchi)

- Fix `--trace-file` output. Dune now emits a single *complete* event for every
  executed process. Unterminated *async* events are no longer written. (ocaml/dune#6892,
  @rgrinberg)

- Fix preprocessing with `staged_pps` (ocaml/dune#6748, fixes ocaml/dune#6644, @rgrinberg)

- Use colored output with MDX when Dune colors are enabled.
  (ocaml/dune#6462, @MisterDA)

- Make `dune describe workspace` return consistent dependencies for
  executables and for libraries. By default, compile-time dependencies
  towards PPX-rewriters are from now not taken into account (but
  runtime dependencies always are). Compile-time dependencies towards
  PPX-rewriters can be taken into account by providing the
  `--with-pps` flag. (ocaml/dune#6727, fixes ocaml/dune#6486, @esope)

- Print missing newline after `$ dune exec`. (ocaml/dune#6821, fixes ocaml/dune#6700, @rgrinberg,
  @Alizter)

- Fix binary corruption when installing or promoting in parallel (ocaml/dune#6669, fixes
  ocaml/dune#6668, @edwintorok)

- Use colored output with GCC and Clang when compiling C stubs. The
  flag `-fdiagnostics-color=always` is added to the `:standard` set of
  flags. (ocaml/dune#4083, @MisterDA)

- Fix the parsing of decimal and hexadecimal escape literals in `dune`,
  `dune-package`, and other dune s-expression based files (ocaml/dune#6710, @shym)

- Report an error if `dune init ...` would create a "dune" file in a location
  which already contains a "dune" directory (ocaml/dune#6705, @gridbugs)

- Fix the parsing of alerts. They will now show up in diagnostics correctly.
  (ocaml/dune#6678, @rginberg)

- Fix the compilation of modules generated at link time when
  `implicit_transitive_deps` is enabled (ocaml/dune#6642, @rgrinberg)

- Allow `$ dune utop` to load libraries defined in data only directories
  defined using `(subdir ..)` (ocaml/dune#6631, @rgrinberg)

- Format dune files when they are named `dune-file`. This occurs when we enable
  the alternative file names project option. (ocaml/dune#6566, @rgrinberg)

- Move `$ dune ocaml-merlin -dump-config=$dir` to `$ dune ocaml merlin
  dump-config $dir`. (ocaml/dune#6547, @rgrinberg)

- Allow compilation rules to be impacted by `(env ..)` stanzas that modify the
  environment or set binaries. (ocaml/dune#6527, @rgrinberg)

- Coq native mode is now automatically detected by Dune starting with Coq lang
  0.7. `(mode native)` has been deprecated in favour of detection from the
  configuration of Coq. (ocaml/dune#6409, @Alizter)

- Print "Leaving Directory" whenever "Entering Directory" is printed. (ocaml/dune#6419,
  fixes ocaml/dune#138, @cpitclaudel, @rgrinberg)

- Allow `$ dune ocaml dump-dot-merlin` to run in watch mode. Also this command
  shouldn't print "Entering Directory" mesages. (ocaml/dune#6497, @rgrinberg)

- `dune clean` should no longer fail under Windows due to the inability to
  remove the `.lock` file. Also, bring the implementation of the global lock
  under Windows closer to that of Unix. (ocaml/dune#6523, @nojb)

- Remove "Entering Directory" messages for `$ dune install`. (ocaml/dune#6513,
  @rgrinberg)

- Stop passing `-q` flag in `dune coq top`, which allows for `.coqrc` to be
  loaded. (ocaml/dune#6848, fixes ocaml/dune#6847, @Alizter)

- Fix missing dependencies when detecting the kind of C compiler we're using
  (ocaml/dune#6610, fixes ocaml/dune#6415, @emillon)

- Allow `(include_subdirs qualified)` for OCaml projects. (ocaml/dune#6594, fixes ocaml/dune#1084,
  @rgrinberg)

- Accurately determine merlin configuration for all sources selected with
  `copy#` and `copy_files#`. The old heuristic of looking for a module in
  parent directories is removed (ocaml/dune#6594, @rgrinberg)

- Fix inline tests with *js_of_ocaml* and whole program compilation mode
  enabled (ocaml/dune#6645, @hhugo)

- Fix *js_of_ocaml* separate compilation rules when `--enable=effects`
  ,`--enable=use-js-string` or `--toplevel` is used. (ocaml/dune#6714, ocaml/dune#6828, ocaml/dune#6920, @hhugo)

- Fix *js_of_ocaml* separate compilation in presence of linkall (ocaml/dune#6832, ocaml/dune#6916, @hhugo)

- Remove spurious build dir created when running `dune init proj ...` (ocaml/dune#6707,
  fixes ocaml/dune#5429, @gridbugs)

- Allow `--sandbox` to affect `ocamldep` invocations. Previously, they were
  wrongly marked as incompatible (ocaml/dune#6749, @rgrinberg)

- Validate the command line arguments for `$ dune ocaml top-module`. This
  command requires one positional argument (ocaml/dune#6796, fixes ocaml/dune#6793, @rgrinberg)

- Add a `dune cache size` command for displaying the size of the cache (ocaml/dune#6638,
  @Alizter)

- Fix dependency cycle when installing files to the bin section with
  `glob_files` (ocaml/dune#6764, fixes ocaml/dune#6708, @gridbugs)

- Handle "Too many links" errors when using Dune cache on Windows (ocaml/dune#6993, @nojb)

- Allow the `cinaps` stanza to set a custom alias. By default, if the alias is
  not set then the cinaps actions will be attached to both `@cinaps` and
  `@runtest` (ocaml/dune#6991, @rgrinberg)

- Add `(using ctypes 0.3)`. When used, paths in `(ctypes)` are interpreted
  relative to where the stanza is defined. (ocaml/dune#6883, fixes ocaml/dune#5325, @emillon)

- Auto-detect `dune-workspace` files as `dune` files in Emacs (ocaml/dune#7061,
  @ilankri)

- Add native support for polling mode on Windows (ocaml/dune#7010, @yams-yams, @nojb)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment