Use liba2i #1049
Use liba2i #1049
Conversation
alejandro-colomar
force-pushed
the
liba2i
branch
from
6ce703d to
379cdd6
Compare
|
Adding this dependency as optional would probably help to merge the changes more quickly. I say this mostly because this project is used in many distributions, and getting all of them to include this new dependency library is going to take a while. |
Hmmm, interesting. I'll consider changing it to be optional. Maybe we could have it optional for a few releases, and after that make it unconditional. BTW, would you mind doing the packaging for Fedora/RHEL? :-) |
|
I'll package it for openSUSE. Do you publish a release tarball somewhere? |
Lovely; thanks!! :-)
That's correct, although you will probably prefer to use https://git.kernel.org/pub/scm/libs/liba2i/liba2i.git/. Both are supported, as far as I'm concerned, but
I do publish release tarballs here: Since the current versions are 0.x, you'll find them under the And have requested the creation of a
If you have a script that can watch a git repository for tags, look for tags of the form There's a mailing list: I send an email to the list in every release. You could subscribe to it if you want. (But maybe it gets noisier, so you may want to not subscribe to yet another list.) I can also CC you in those emails. I plan to write a file in the repository with a list of known downstream packages and their websites and maintainers, and CC everyone in that list (unless someone doesn't want to get notified) on every release. Current links: Website | Documentation | Contact | Archives | Bugs | Releases | Git | Contributing |
Yes, I'll take care of it for Fedora, but it'll take me some time because the holiday period is approaching and I am closing a few things before I leave. |
Sure; no problem. I expect to need to fix a few things once some distros give feedback, so you'll have 0.11 when you come back, more polished. |
Don' you have LetsEncrypt or something? :) |
Not sure if some distros would want to add a new dependency to their ring0/base systems though. |
Nope, and don't want to: https://www.alejandro-colomar.es/ssl :) You can access it via http, though. |
It's a quite small lib. I hope they don't mind so much. I was talking to Void Linux, and they were receptive. Let's see how others receive it. |
|
Currently, the build system of liba2i behaves like this: Is that behavior ok? Should I change that? What do downstream distros expect? How does one normally append to the default CFLAGS, and how does one normally overwrite the default CFLAGS? |
I gave extensive feedback on it via email several months ago and I don't feel it was fully reflected upon. All the little things like not being able to use HTTPS and these silly quirks do not help. If it's on git.kernel.org, that helps now, I guess, although using it to "launder trust" is a bit meh. I don't really want to see shadow adopt another library, especially one which has received little review. I also think the bar should be particularly high for a critical package like shadow. I don't see what arguments have been made in favour of the move, let alone strong ones. I already think there's been an uncomfortably-large amount of churn in the input handling in shadow, often inventing new functions which.. are now being moved out into an external library anyway? Not only does this introduce opportunities for bugs to creep in, it makes it challenging for others to interact with the shadow codebase. In other communities, this is called NIH syndrome. There is an implicit cost to inventing your own functions, wrappers, and then a whole library. Further, when looking at the implementations in liba2i, there's a not-insignificant amount of indirection which makes reading it challenging. I don't have a vote, but if I did, I'd cast it against at present. EDIT: And the various preliminary concerns I've had to point out again in void-linux/void-packages#51261 are not helping. |
That's unsual ;) I gave a quick go at packaging and noticed that the build isn't straight forward. I wonder why no standard build system and default installation way was chosen. It would be good to keep the "unusualness" to a minimum. |
|
Hi Sam,
I have reflected upon what I've been able to. Of course, the larger request of "please drop your entire build system and use my favourite one^W^W^W autotools", I haven't been able to address, because I do not know how to use autotools. Even if you were so kind of providing patches for doing that change, I doubt I would be able to maintain my own project if the build system is autotools-based, so we'd have an important problem.
It's my server; my rules. I'm sorry if we disagree on how my server should work.
Since we clearly have a disagreement on my server, I provide an impartial server for commodity. I don't like github, nor any other paid services (github is paid; if we account for the data they steal from you legally, including source code to train their AI; it's more expensive that a paid git server (if those exist at all), actually). I wouldn't mind using sourcehut, but I find cgit more comfortable. Since I have a kernel.org account, and they allow me to do this, I provide it there.
I know you're worried after the XZ events. I don't know how I can make you trust me, since of course a malicious actor would also try the same goal. Also, since Jia Tan introduced his stuff via a build system, and I'm also doing unusual stuff with build systems, I guess it makes sense to be suspicious. You can trace back all of my programming history back to when I was learning to program in StackOverflow. This is my account, if you want to investigate: https://stackoverflow.com/users/6872717/alx-recommends-codidact That's my first online pressence programming, so if you search there for my questions in chronological order, it will hopefully make sense to you and be consistent. I also have no problem to give you personal information about which school I went to in Spain, or where I live, if you want to do a full background check. If you need anything, please ask in private. Also regarding trust: I think my transparency in patches is actually unprecedented. I provide range diffs from the first iteration of a patch until it's merged, to let anyone reviewing see that I didn't introduce any sneaky changes during revision. Anyone is free to audit any of my PRs to find out any problems. I also sign all of my commits, so that I can stand by them when I see them. That also adds some other feature: I can't repudiate a commit of mine (unless I repudiate my own signing key, via a revocation certificate, but that'd need to be justified). My reputation depends on my contributions, so I sign them (commits, and emails). And after your feedback about it, I have increased my transparency even further, as you can see in the commit logs, when cherry-picking commits between projects. If you find anything I do to not be transparent enough, please let me know, and I'll try to improve it.
Feel free to review it as much as you want. I can wait 5 years if that's what you need. But you ghosted me some months ago, which I think is a bit rude from you. I still understand it in context, because I could have just been the next Jia Tan trying to convince you of something, so I understand you not wanting to talk too much with someone asking for feedback about packaging his stuff in the middle of the XZ aftermath. But it still sucks. Please let me know what problems you find in the project, and I will kindly try to solve them, if I can (this excludes, at the moment, switching to autotools, but I can try to make my GNUmakefile-based build system behave very much like any other standard build system). Regarding the library, I provide a comprehensive set of tests, so if you try those tests, or can think of anything I'm missing in my tests, please say so.
Agree.
So, I think it makes sense to add the dependency. I'm ready to wait as long as you need to audit the project.
That churn has found and fixed many bugs. Not only in shadow, BTW. I even found and fixed a bug in the design of strtoi(3) in NetBSD (and consequently in libbsd too) while designing and testing these APIs, which has fixed uncountable bugs in NetBSD and Debian, and probably also in other OSes. I agree churn can be a bad thing, but in this case I think it's quite justified. Moreover, in some cases I've taken almost a year, not just some months, to discuss and iterate over some of the changes. That has helped make them quite good before merging. It's not at all like one has used sed(1) to do a global churny change possibly introducing several bugs in the process.
That sets in stone those APIs, which reduces the maintenance of that part. It means no more churn to those APIs, which should be a good thing?
Sure; but I have a comprehensive set of tests in that library, so it would be harder to introduce bugs there than in shadow. In fact, I had a bug in the shadow version, which had already been fixed in liba2i (see 63297e8).
Really? Do you find it easier to interact with strtol(3)? I don't even know how to use it correctly. And clearly, the authors of shadow didn't either, from what we've seen. I understand some friction when using a new API, because
I know that syndrome. But it is only a valid term when the API (or some usable replacement) is actually invented. In this case, it was actually Not Invented Anywhere. Let me list the invented list:
The fact that I provide strtoi(3) in my library (as a2i_strtoi()), is not a chance for adding bugs, but actually a bug fix. By writing that function, and developing comprehensive tests for it, and running those tests against both my implementation and libbsd's one, I found the design issue in NetBSD's strtoi(3). https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=57828
And a reward. Do you suggest to continue using strtol(3)? If so, I'll let you suggest how it should be used correctly in shadow. I don't, myself.
I think the library is the smallest change. Yeah, it has the build system, and packaging cost, but then you get the reward in the form of being allowed to call those APIs from any other projects. Since the only dependency of liba2i is libc, it should be usable almost everywhere. The library has the lowest cost, and the highest reward. It was indeed when writing my library that the strtoi(3) bug was found. If I had kept it just as a shadow internal library, we would have never found that, and you could be arguing that it was indeed a NIH syndrome by not using libbsd's strtoi(3). Now we know that libbsd's strtoi(3) wasn't as good as it sounded (and anyway, it's not good for typedefs).
I have significantly reduced that since you last looked at it, I think. $ grepc a2s include/
include/a2i/a2i/a2s.h:#define a2s(TYPE, n, s, ...) \
( \
_Generic((void (*)(TYPE, typeof(s))) 0, \
void (*)(signed char, const char *): a2shh_c, \
void (*)(signed char, const void *): a2shh_c, \
void (*)(signed char, char *): a2shh_nc, \
void (*)(signed char, void *): a2shh_nc, \
void (*)(short, const char *): a2sh_c, \
void (*)(short, const void *): a2sh_c, \
void (*)(short, char *): a2sh_nc, \
void (*)(short, void *): a2sh_nc, \
void (*)(int, const char *): a2si_c, \
void (*)(int, const void *): a2si_c, \
void (*)(int, char *): a2si_nc, \
void (*)(int, void *): a2si_nc, \
void (*)(long, const char *): a2sl_c, \
void (*)(long, const void *): a2sl_c, \
void (*)(long, char *): a2sl_nc, \
void (*)(long, void *): a2sl_nc, \
void (*)(long long, const char *): a2sll_c, \
void (*)(long long, const void *): a2sll_c, \
void (*)(long long, char *): a2sll_nc, \
void (*)(long long, void *): a2sll_nc \
)(n, s, __VA_ARGS__) \
)$ grepc -B1 a2shh_c lib/src/
lib/src/a2i/a2i/a2s_.c-A2I_ATTR_ALIAS("a2shh_nc")
lib/src/a2i/a2i/a2s_.c:int a2shh_c(signed char *restrict n, const char *s,
const char **a2i_nullable restrict endp, int base,
signed char min, signed char max);$ grepc -tfd a2shh_nc lib/src/
lib/src/a2i/a2i/a2s_.h:inline int
a2shh_nc(signed char *restrict n, char *s,
char **a2i_nullable restrict endp, int base,
signed char min, signed char max)
{
int status;
*n = a2i_strtoi(s, endp, base, min, max, &status);
if (status != 0)
errno = status;
return -!!status;
}And then you could stop there, since a2i_strtoi() is exactly strtoi(3), which is a relatively well-known API. But if you want to check it in my own implementation: $ grepc -tfd a2i_strtoi lib/src/
lib/src/a2i/strtoi/strtoi/strtoi.h:inline intmax_t
a2i_strtoi(char *s,
char **a2i_nullable restrict endp, int base,
intmax_t min, intmax_t max, int *a2i_nullable restrict status)
{
int errno_saved, st;
char *e;
intmax_t n;
if (endp == NULL)
endp = &e;
if (status == NULL)
status = &st;
if (base != 0 && (base < 2 || base > 36)) {
*status = EINVAL;
return MAX(min, MIN(max, 0));
}
errno_saved = errno;
errno = 0;
n = strtoimax(s, endp, base);
if (*endp == s)
*status = ECANCELED;
else if (errno == ERANGE || n < min || n > max)
*status = ERANGE;
else if (**endp != '\0')
*status = ENOTSUP;
else
*status = 0;
errno = errno_saved;
return MAX(min, MIN(max, n));
}
Understood. If you are interested in making an effort in auditing and packaging liba2i, I'll respect that vote and wait for you to be happy with the library.
I'll try to reply to those there. And some last words: after the XZ events, I've noticed a difference in your attitude to me, which I understand, but doesn't feel nice. I'd like to ask you to please be nice with me, as I never did anything to you (or if I did, I dind't notice, I'm sorry if I did). If you're skeptic about trusting me, that's ok, but please be nice, and let me know if there anything I can improve. I'm certainly not going to be an expert in autotools tomorrow, though, so please don't ask me to be one. Have a lovely day! |
I just don't know what's "usual" behavior, especially, since it's not something written in any standard. When I learned to write Makefiles, I learned from the GNU make documentation. If something is not written there, I didn't learn it. I'm now learning what distros expect, after talking to some packagers that gave me feedback about the library. I'm trying to adjust the build system to that feedback.
That's my intention. In |
|
Do I understand correctly that liba2i is only licensed under LGPG 3.0-or-later ? I see the exception, but I'm not quite clear on whether that limits the license that can be applied to binaries and libraries using shadow. |
The source code and build system is For some reason, I accidentally slipped a different license version in the tests (the And the man pages have the So, you shouldn't have any license problems linking to it: AFAIU, that exception means that you can link to it, and the LGPL won't eat your project. But IANAL. |
|
If anyone else can clarify if that license allows linking in shadow, please manifest yourselves. Cc: @g-branden-robinson |
|
$ cat LICENSES/LGPL-3.0-linking-exception.txt
As a special exception to the GNU Lesser General Public License version 3
("LGPL3"), the copyright holders of this Library give you permission to
convey to a third party a Combined Work that links statically or dynamically
to this Library without providing any Minimal Corresponding Source or
Minimal Application Code as set out in 4d or providing the installation
information set out in section 4e, provided that you comply with the other
provisions of LGPL3 and provided that you meet, for the Application the
terms and conditions of the license(s) which apply to the Application.
What about 4a, c and d?
|
4a) I consider the relevant entry in the build system to be "prominent notice". Maybe we can have a one-or-two-liner comment above it to be more prominent, and more of a notice. 4c) It starts saying 4d is explicitly excluded in the exception. |
And maybe add a one-liner to COPYING. |
For example, glibc is LGPL-2.1, which also has similar text in 6 (2nd paragraph). If shadow can link to glibc, it should be able to link to liba2i. |
|
> $ cat LICENSES/LGPL-3.0-linking-exception.txt As a special exception to the GNU Lesser General Public License version 3 ("LGPL3"), the copyright holders of this Library give you permission to convey to a third party a Combined Work that links statically or dynamically to this Library without providing any Minimal Corresponding Source or Minimal Application Code as set out in 4d or providing the installation information set out in section 4e, provided that you comply with the other provisions of LGPL3 and provided that you meet, for the Application the terms and conditions of the license(s) which apply to the Application.
> What about 4a, c and d?
4a) I consider the relevant entry in the build system to be "prominent notice". Maybe we can have a one-or-two-liner comment above it.
4c) It starts saying `For a Combined Work that displays copyright notices during execution, `. AFAIK, shadow utils don't display copyright notices during execution, so it wouldn't apply.
4d is explicitly excluded in the exception.
Thanks. Sounds like it should be manageable.
|
I only tried to see how to build it, no where I said receptive or not :) |
alejandro-colomar
force-pushed
the
liba2i
branch
2 times, most recently
from
a856120 to
3c81aca
Compare
alejandro-colomar
force-pushed
the
liba2i
branch
from
3c81aca to
fcd4e55
Compare
alejandro-colomar
force-pushed
the
liba2i
branch
from
fcd4e55 to
0a0fb50
Compare
alejandro-colomar
force-pushed
the
liba2i
branch
4 times, most recently
from
a640728 to
81d85e4
Compare
alejandro-colomar
force-pushed
the
liba2i
branch
from
81d85e4 to
55ba0e0
Compare
alejandro-colomar
force-pushed
the
liba2i
branch
from
55ba0e0 to
7b7e71f
Compare
alejandro-colomar
force-pushed
the
liba2i
branch
2 times, most recently
from
0bb5642 to
5db2252
Compare
alejandro-colomar
force-pushed
the
liba2i
branch
from
5db2252 to
1d4f962
Compare
alejandro-colomar
force-pushed
the
liba2i
branch
3 times, most recently
from
2f9a41c to
29f2376
Compare
alejandro-colomar
force-pushed
the
liba2i
branch
from
29f2376 to
f9eeb97
Compare
alejandro-colomar
force-pushed
the
liba2i
branch
from
f9eeb97 to
ab03d24
Compare
alejandro-colomar
force-pushed
the
liba2i
branch
4 times, most recently
from
ebccd91 to
f4b2328
Compare
alejandro-colomar
force-pushed
the
liba2i
branch
4 times, most recently
from
9c7ced1 to
fd1e38a
Compare
Signed-off-by: Alejandro Colomar <[email protected]>
Signed-off-by: Alejandro Colomar <[email protected]>
alejandro-colomar
force-pushed
the
liba2i
branch
from
fd1e38a to
415c95f
Compare
A lot needs to happen before we can merge this, but let's have it here as a draft. Most likely, won't happen before 2025.
Distros should package liba2i before this happens.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1076091
Revisions:
v1b
v1c
v1d
v1e
v1f
v1g
v1h
v1i
v1j
v1k
v1l
v1m
v1n
v1o
v1p
v1q
v1r
v1s
v1t
v1u
v1v
v1w
v1x
v1y
v2
v2b
v2c
v2d
v2d
v2e
v2f