local_zig still downloads from upstream #241

dch · 2021-02-10T16:42:49Z

I did a minimal zigler nif https://git.sr.ht/~dch/ziggle using hex published 0.7.2 & zig 0.7.1 & also tried latest master.

zig still downloads from upstream instead of using local version
then blows out spectacularly, I'm not yet clear whats happening but I think we're missing some includes/headers
BTW can you publish the tags you're using to publish to hex, also to the zigler github repo?

Any suggestions?

 mix compile
Compiling 1 file (.ex)

15:54:39.817 [debug] compiling nif for module Ziggle in path /tmp/.zigler_compiler/dev/Elixir.Ziggle

15:54:39.880 [debug] wrote build.zig to /tmp/.zigler_compiler/dev/Elixir.Ziggle/build.zig

== Compilation error in file lib/ziggle.ex ==
** (CompileError)  this zig compiler warning hasn't been incorporated into the parser.
Please file a report at:
https://github.com/ityonemo/zigler/issues
error(compilation): clang failed with stderr: In file included from /tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libunwind/src/libunwind.cpp:12:
In file included from /tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libunwind/include/libunwind.h:18:
In file included from /tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libcxx/include/stdint.h:106:
/tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libcxx/include/__config:247:12: fatal error: 'sys/endian.h' file not found

error(compilation): clang failed with stderr: In file included from /tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libunwind/src/Unwind-seh.cpp:13:
/tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libunwind/src/config.h:16:10: fatal error: 'assert.h' file not found

error(compilation): clang failed with stderr: In file included from /tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libunwind/src/UnwindLevel1.c:21:
/tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/include/inttypes.h:21:15: fatal error: 'inttypes.h' file not found

error(compilation): clang failed with stderr: In file included from /tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libunwind/src/UnwindLevel1-gcc-ext.c:12:
/tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/include/inttypes.h:21:15: fatal error: 'inttypes.h' file not found

error(compilation): clang failed with stderr: In file included from /tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libunwind/src/Unwind-sjlj.c:14:
/tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/include/inttypes.h:21:15: fatal error: 'inttypes.h' file not found

error(compilation): /tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libunwind/src/libunwind.cpp:1:1: unable to build C object: clang exited with code 1

error(compilation): /tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libunwind/src/Unwind-seh.cpp:1:1: unable to build C object: clang exited with code 1

error(compilation): /tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libunwind/src/UnwindLevel1.c:1:1: unable to build C object: clang exited with code 1

error(compilation): /tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libunwind/src/UnwindLevel1-gcc-ext.c:1:1: unable to build C object: clang exited with code 1

error(compilation): /tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libunwind/src/Unwind-sjlj.c:1:1: unable to build C object: clang exited with code 1

error: unable to build libunwind: BuildingLibCObjectFailed
Elixir.Ziggle...The following command exited with error code 1:
/tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/zig build-lib /tmp/.zigler_compiler/dev/Elixir.Ziggle/Elixir.Ziggle.zig --single-threaded --cache-dir /tmp/.zigler_compiler/dev/Elixir.Ziggle/zig-cache --global-cache-dir /home/dch/.cache/zig --name Elixir.Ziggle --version 0.1 -dynamic -target x86_64-freebsd-gnu -isystem /usr/local/lib/erlang23/erts-11.1.7/include -isystem /tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/libc/musl/include -isystem /tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/libc/include/x86_64-freebsd-musl --enable-cache 
error: the following build command failed with exit code 1:
/tmp/.zigler_compiler/dev/Elixir.Ziggle/zig-cache/o/4c955e129d544a6d30f2b514bffab757/build /tmp/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/zig /tmp/.zigler_compiler/dev/Elixir.Ziggle /tmp/.zigler_compiler/dev/Elixir.Ziggle/zig-cache /home/dch/.cache/zig

    (zigler 0.7.2) lib/zig/parser/error.ex:54: Zig.Parser.Error.parse/2
    (zigler 0.7.2) lib/zig/command.ex:36: Zig.Command.compile/2
    (zigler 0.7.2) lib/zig/compiler.ex:102: Zig.Compiler.compilation/2
    (zigler 0.7.2) expanding macro: Zig.Compiler.__before_compile__/1
    lib/ziggle.ex:1: Ziggle (module)

Using a full local zig with the same hack as last time:

$ mkdir -p deps/zigler/zig/zig-freebsd-x86_64-0.7.1 \
  && ln -s /usr/local/bin/zig deps/zigler/zig/zig-freebsd-x86_64-0.7.1/zig \
  && ln -s /usr/local/lib deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib
dch@z01 /u/h/d/s/z/ziggle (main)> tree -Fap deps/zigler/zig
deps/zigler/zig
├── [drwxr-xr-x]  zig-freebsd-x86_64-0.7.1/
│   ├── [lrwxr-xr-x]  lib -> /usr/local/lib/
│   └── [lrwxr-xr-x]  zig -> /usr/local/bin/zig
└── [-rw-r--r--]  zig-freebsd-x86_64-0.7.1.tar.xz

2 directories, 2 files
$  mix do compile, test
Compiling 1 file (.ex)

16:39:49.757 [debug] compiling nif for module Ziggle in path /tmp/.zigler_compiler/dev/Elixir.Ziggle

16:39:49.821 [debug] wrote build.zig to /tmp/.zigler_compiler/dev/Elixir.Ziggle/build.zig
Compiling lib/ziggle.ex (it's taking more than 10s)

== Compilation error in file lib/ziggle.ex ==
** (CompileError)  this zig compiler warning hasn't been incorporated into the parser.
Please file a report at:
https://github.com/ityonemo/zigler/issues
error(compilation): clang failed with stderr: In file included from /usr/local/lib/zig/libunwind/src/Unwind-sjlj.c:14:
/usr/local/lib/zig/include/inttypes.h:21:15: fatal error: 'inttypes.h' file not found

error(compilation): clang failed with stderr: In file included from /usr/local/lib/zig/libunwind/src/Unwind-seh.cpp:13:
/usr/local/lib/zig/libunwind/src/config.h:16:10: fatal error: 'assert.h' file not found

error(compilation): clang failed with stderr: In file included from /usr/local/lib/zig/libunwind/src/UnwindLevel1.c:21:
/usr/local/lib/zig/include/inttypes.h:21:15: fatal error: 'inttypes.h' file not found

error(compilation): clang failed with stderr: In file included from /usr/local/lib/zig/libunwind/src/UnwindLevel1-gcc-ext.c:12:
/usr/local/lib/zig/include/inttypes.h:21:15: fatal error: 'inttypes.h' file not found

error(compilation): clang failed with stderr: In file included from /usr/local/lib/zig/libunwind/src/libunwind.cpp:12:
In file included from /usr/local/lib/zig/libunwind/include/libunwind.h:18:
In file included from /usr/local/lib/zig/libcxx/include/stdint.h:106:
/usr/local/lib/zig/libcxx/include/__config:247:12: fatal error: 'sys/endian.h' file not found

error(compilation): /usr/local/lib/zig/libunwind/src/Unwind-sjlj.c:1:1: unable to build C object: clang exited with code 1

error(compilation): /usr/local/lib/zig/libunwind/src/Unwind-seh.cpp:1:1: unable to build C object: clang exited with code 1

error(compilation): /usr/local/lib/zig/libunwind/src/UnwindLevel1.c:1:1: unable to build C object: clang exited with code 1

error(compilation): /usr/local/lib/zig/libunwind/src/UnwindLevel1-gcc-ext.c:1:1: unable to build C object: clang exited with code 1

error(compilation): /usr/local/lib/zig/libunwind/src/libunwind.cpp:1:1: unable to build C object: clang exited with code 1

error: unable to build libunwind: BuildingLibCObjectFailed
Elixir.Ziggle...The following command exited with error code 1:
/usr/local/bin/zig build-lib /tmp/.zigler_compiler/dev/Elixir.Ziggle/Elixir.Ziggle.zig --single-threaded --cache-dir /tmp/.zigler_compiler/dev/Elixir.Ziggle/zig-cache --global-cache-dir /home/dch/.cache/zig --name Elixir.Ziggle --version 0.1 -dynamic -target x86_64-freebsd-gnu -isystem /usr/local/lib/erlang23/erts-11.1.7/include -isystem /usr/home/dch/src/zig/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/libc/musl/include -isystem /usr/home/dch/src/zig/ziggle/deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/libc/include/x86_64-freebsd-musl --enable-cache 
error: the following build command failed with exit code 1:
/tmp/.zigler_compiler/dev/Elixir.Ziggle/zig-cache/o/200da5397a70b244375bad7e52bbfd99/build /usr/local/bin/zig /tmp/.zigler_compiler/dev/Elixir.Ziggle /tmp/.zigler_compiler/dev/Elixir.Ziggle/zig-cache /home/dch/.cache/zig

    (zigler 0.7.2) lib/zig/parser/error.ex:54: Zig.Parser.Error.parse/2
    (zigler 0.7.2) lib/zig/command.ex:36: Zig.Command.compile/2
    (zigler 0.7.2) lib/zig/compiler.ex:102: Zig.Compiler.compilation/2
    (zigler 0.7.2) expanding macro: Zig.Compiler.__before_compile__/1
    lib/ziggle.ex:1: Ziggle (module)

which is ... equally bad

The text was updated successfully, but these errors were encountered:

dch · 2021-02-10T17:01:07Z

build.zig.log

ityonemo · 2021-02-10T17:53:22Z

aw crap. I'm so sorry. I think what is happening is that I disabled implicit libc and zig doesn't currently have an obvious way to load the libc headers without loading libc. ziglang/zig#7687 and if I'm not mistaken freebsd requires libc on account of it being the official OS syscall entry point (*bsd does not guarantee kernel syscall stability). Do you have any suggestions on how to set up freebsd CI pipelines? I want this to not be a problem moving forward.

ityonemo · 2021-02-10T17:55:53Z

can you try including link_libc: true in your use Zig options and see if that makes it go away?

dch · 2021-02-10T23:53:20Z

per chat, that doesn't seem to help either - it does add this to the generated build.zig

defmodule Ziggle do
  use Zig,
    system_include_dirs: ["/usr/include", "/usr/lib/include"],
    link_libc: true
...

    lib.linkSystemLibrary("c");
    lib.addSystemIncludeDir("/usr/include");
    lib.addSystemIncludeDir("/usr/lib/include");

and shows

error(compilation): clang failed with stderr: In file included from /usr/local/lib/zig/libunwind/src/libunwind.cpp:12:
In file included from /usr/local/lib/zig/libunwind/include/libunwind.h:18:
In file included from /usr/local/lib/zig/libcxx/include/stdint.h:106:
/usr/local/lib/zig/libcxx/include/__config:247:12: fatal error: 'sys/endian.h' file not found

error(compilation): clang failed with stderr: In file included from /usr/local/lib/zig/libunwind/src/Unwind-seh.cpp:13:
/usr/local/lib/zig/libunwind/src/config.h:16:10: fatal error: 'assert.h' file not found

zigster64 · 2021-02-26T03:12:03Z

Reporting the same issue here - will try to fix on my system and post an update if I get it working.

defmodule LiveCounter.Count do
  use GenServer
  use Zig,   local_zig: true, link_libc: true

error

error(compilation): clang failed with stderr: In file included from ...deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libunwind/src/libunwind.cpp:12:
In file included from ...deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libunwind/include/libunwind.h:18:
In file included from ...deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libcxx/include/stdint.h:106:
...deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libcxx/include/__config:247:12: fatal error: 'sys/endian.h' file not found

error(compilation): clang failed with stderr: In file included from ...deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libunwind/src/Unwind-seh.cpp:13:
...deps/zigler/zig/zig-freebsd-x86_64-0.7.1/lib/zig/libunwind/src/config.h:16:10: fatal error: 'assert.h' file not found

...

(file paths shortened above for brevity)

zigster64 · 2021-02-26T14:03:10Z

odd finding : in lib/zig/command.ex. line 18

def compile(compiler, zig_tree) do
  zig_executable = if compiler.module_spec.local_zig do
    Logger.debug("Using local zig")
    System.find_exectuable("zig")
  else
    Logger.debug("Not using local zig")
     ...

Regardless of the setting of local_zig in my app code, the variable isnt getting set by the time it gets to compiler.ex

If I force hack the code to use the system zig, that does seem to work.

Just leaves the weird include errors - having a look at that now

zigster64 · 2021-02-26T14:29:24Z

OK - in lib/zig/builder.ex, which I think has the template for generating the build.zig file

I changed the target definition from
const target = b.standardTargetOptions(<%= to_structdef target %>);

to a generic
const target = b.standardTargetOptions(.{});

And now it compiles without all the include errors.

No idea if this is a fix - Im just hacking, so treat with caution :)

... a few moments later :
With a little bit of fiddling around with my nifs code (Im a complete Elixir noob here, so fumbling around a bit)
and it works !

ityonemo · 2021-09-27T14:27:38Z

this should be resolved now in d1afc54 (note this is currently in devel and will be merged as part of 0.8.1 release)

ityonemo added this to the 0.8.0 milestone Apr 28, 2021

ityonemo closed this as completed Sep 27, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

local_zig still downloads from upstream #241

local_zig still downloads from upstream #241

dch commented Feb 10, 2021

dch commented Feb 10, 2021

ityonemo commented Feb 10, 2021

ityonemo commented Feb 10, 2021

dch commented Feb 10, 2021

zigster64 commented Feb 26, 2021 •

edited

Loading

zigster64 commented Feb 26, 2021

zigster64 commented Feb 26, 2021 •

edited

Loading

ityonemo commented Sep 27, 2021

local_zig still downloads from upstream #241

local_zig still downloads from upstream #241

Comments

dch commented Feb 10, 2021

dch commented Feb 10, 2021

ityonemo commented Feb 10, 2021

ityonemo commented Feb 10, 2021

dch commented Feb 10, 2021

zigster64 commented Feb 26, 2021 • edited Loading

zigster64 commented Feb 26, 2021

zigster64 commented Feb 26, 2021 • edited Loading

ityonemo commented Sep 27, 2021

zigster64 commented Feb 26, 2021 •

edited

Loading

zigster64 commented Feb 26, 2021 •

edited

Loading