magic autogeneration does not work as expected for ++ and --

[p5pRT]

Migrated from rt.perl.org#43356 (status was 'open')

Searchable as RT43356$

[p5pRT]

From perl@plan9.de

Created by perl@plan9.de

The MAGIC AUTGEBNERATINO section in perldoc overload says​:

  Increment and decrement

  The "++$a" operation can be expressed in a+=1" or "$a+1", and "$a--" in
  terms of "$a-=1" and "$a-1".

This leads me (I might be wrong) to believe that if an overlaoded value
has no "++", "+=" or "+" methods defined, then the fallback would generate the
equivalent to​:

  $value = $value + 1

But this is not the case​:

  use overload
  "0+" => sub { ${$_[0]} },
  "=" => sub { ${$_[0]} },
  fallback => 1;

  my $value = bless \(my $dummy = 1), __PACKAGE__;

  warn ++$value;

This outputs, on my system, the number 6886977 instead of the expected 2.

Using $value = $value + 1 gives me the expected result of "2".

One could argue that $value++ falls back to the non-overloaded way of
doing things, taking the refrence and incrementing it by one, but then I
would still expected $a=$a+1 to work the same or similarly to $a++.

(Or otherwise some hint in the documentation that you have to overload ++
and -- even when you do not have to overload the other numeric operators
might be helpful).

Perl Info

Flags:
    category=core
    severity=low

Site configuration information for perl v5.8.8:

Configured by Marc Lehmann at Sat Jan 20 00:23:48 CET 2007.

Summary of my perl5 (revision 5 version 8 subversion 8) configuration:
  Platform:
    osname=linux, osvers=2.6.17.6, archname=amd64-linux
    uname='linux cerebro 2.6.17.6 #1 smp fri oct 20 19:28:13 cest 2006 x86_64 gnulinux '
    config_args='-Duselargefiles -Dxxxxuse64bitint -Uuse64bitall -Dusemymalloc=n -Dcc=gcc -Dccflags=-DPERL_DONT_CREATE_GVSV_disabled -ggdb -Dcppflags=-DPERL_DONT_CREATE_GVSV_disabled -D_GNU_SOURCE -I/opt/include -Doptimize=-O6 -march=opteron -mtune=opteron -funroll-loops -fno-strict-aliasing -Dcccdlflags=-fPIC -Dldflags=-L/opt/perl/lib -L/opt/lib -Dlibs=-ldl -lm -lcrypt -Darchname=amd64-linux -Dprefix=/opt/perl -Dprivlib=/opt/perl/lib/perl5 -Darchlib=/opt/perl/lib/perl5 -Dvendorprefix=/opt/perl -Dvendorlib=/opt/perl/lib/perl5 -Dvendorarch=/opt/perl/lib/perl5 -Dsiteprefix=/opt/perl -Dsitelib=/opt/perl/lib/perl5 -Dsitearch=/opt/perl/lib/perl5 -Dsitebin=/opt/perl/bin -Dman1dir=/opt/perl/man/man1 -Dman3dir=/opt/perl/man/man3 -Dsiteman1dir=/opt/perl/man/man1 -Dsiteman3dir=/opt/perl/man/man3 -Dman1ext=1 -Dman3ext=3 -Dpager=/usr/bin/less -Uafs -Uusesfio -Uusenm -Uuseshrplib -Dd_dosuid -Dusethreads=undef -Duse5005threads=undef -Duseithreads=undef -Dusemultiplicity=undef -Demail=perl-binary@plan9.de -Dcf_email=perl-binary@plan9.de -Dcf_by=Marc Lehmann -Dlocincpth=/opt/perl/include /opt/include -Dmyhostname=localhost -Dmultiarch=undef -Dbin=/opt/perl/bin -Dusedevel -des'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=define use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-DPERL_DONT_CREATE_GVSV_disabled -ggdb -fno-strict-aliasing -pipe -I/opt/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O6 -march=opteron -mtune=opteron -funroll-loops -fno-strict-aliasing',
    cppflags='-DPERL_DONT_CREATE_GVSV_disabled -D_GNU_SOURCE -I/opt/include -DPERL_DONT_CREATE_GVSV_disabled -ggdb -fno-strict-aliasing -pipe -I/opt/include'
    ccversion='', gccversion='4.1.2 20061115 (prerelease) (Debian 4.1.1-21)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags ='-L/opt/perl/lib -L/opt/lib -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-ldl -lm -lcrypt
    perllibs=-ldl -lm -lcrypt
    libc=/lib/libc-2.3.6.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.3.6'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -L/opt/perl/lib -L/opt/lib -L/usr/local/lib'

Locally applied patches:
    MAINT29832


@INC for perl v5.8.8:
    /root/src/sex
    /opt/perl/lib/perl5
    /opt/perl/lib/perl5
    /opt/perl/lib/perl5
    /opt/perl/lib/perl5
    /opt/perl/lib/perl5
    .


Environment for perl v5.8.8:
    HOME=/root
    LANG (unset)
    LANGUAGE (unset)
    LC_CTYPE=de_DE.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/root/s2:/root/s:/opt/bin:/opt/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11/bin:/usr/games:/usr/local/bin:/usr/local/sbin:/root/src/uunet:.
    PERL5LIB=/root/src/sex
    PERL5_CPANPLUS_CONFIG=/root/.cpanplus/config
    PERLDB_OPTS=ornaments=0
    PERL_BADLANG (unset)
    PERL_UNICODE=EAL
    SHELL=/bin/bash

[p5pRT]

From @Hugmeir

On Sat Jun 23 16​:32​:43 2007, perl@​plan9.de wrote​:

This is a bug report for perl from perl@​plan9.de,
generated with the help of perlbug 1.35 running under perl v5.8.8.

-----------------------------------------------------------------
[Please enter your report here]

The MAGIC AUTGEBNERATINO section in perldoc overload says​:

Increment and decrement

The "++$a" operation can be expressed in a+=1" or "$a+1", and "$a--
" in
terms of "$a-=1" and "$a-1".

This leads me (I might be wrong) to believe that if an overlaoded
value
has no "++", "+=" or "+" methods defined, then the fallback would
generate the
equivalent to​:

$value = $value + 1

But this is not the case​:

use overload
"0+" => sub { ${$_[0]} },
"=" => sub { ${$_[0]} },
fallback => 1;

my $value = bless \(my $dummy = 1), __PACKAGE__;

warn ++$value;

This outputs, on my system, the number 6886977 instead of the expected
2.

Using $value = $value + 1 gives me the expected result of "2".

One could argue that $value++ falls back to the non-overloaded way of
doing things, taking the refrence and incrementing it by one, but then
I
would still expected $a=$a+1 to work the same or similarly to $a++.

(Or otherwise some hint in the documentation that you have to overload
++
and -- even when you do not have to overload the other numeric
operators
might be helpful).

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=core
severity=low
---
Site configuration information for perl v5.8.8​:

Configured by Marc Lehmann at Sat Jan 20 00​:23​:48 CET 2007.

Summary of my perl5 (revision 5 version 8 subversion 8) configuration​:
Platform​:
osname=linux, osvers=2.6.17.6, archname=amd64-linux
uname='linux cerebro 2.6.17.6 #1 smp fri oct 20 19​:28​:13 cest 2006
x86_64 gnulinux '
config_args='-Duselargefiles -Dxxxxuse64bitint -Uuse64bitall
-Dusemymalloc=n -Dcc=gcc -Dccflags=-DPERL_DONT_CREATE_GVSV_disabled
-ggdb -Dcppflags=-DPERL_DONT_CREATE_GVSV_disabled -D_GNU_SOURCE
-I/opt/include -Doptimize=-O6 -march=opteron -mtune=opteron
-funroll-loops -fno-strict-aliasing -Dcccdlflags=-fPIC
-Dldflags=-L/opt/perl/lib -L/opt/lib -Dlibs=-ldl -lm -lcrypt
-Darchname=amd64-linux -Dprefix=/opt/perl
-Dprivlib=/opt/perl/lib/perl5 -Darchlib=/opt/perl/lib/perl5
-Dvendorprefix=/opt/perl -Dvendorlib=/opt/perl/lib/perl5
-Dvendorarch=/opt/perl/lib/perl5 -Dsiteprefix=/opt/perl
-Dsitelib=/opt/perl/lib/perl5 -Dsitearch=/opt/perl/lib/perl5
-Dsitebin=/opt/perl/bin -Dman1dir=/opt/perl/man/man1
-Dman3dir=/opt/perl/man/man3 -Dsiteman1dir=/opt/perl/man/man1
-Dsiteman3dir=/opt/perl/man/man3 -Dman1ext=1 -Dman3ext=3
-Dpager=/usr/bin/less -Uafs -Uusesfio -Uusenm -Uuseshrplib
-Dd_dosuid -Dusethreads=undef -Duse5005threads=undef
-Duseithreads=undef -Dusemultiplicity=undef
-Demail=perl-binary@​plan9.de -Dcf_email=perl-binary@​plan9.de
-Dcf_by=Marc Lehmann -Dlocincpth=/opt/perl/include /opt/include
-Dmyhostname=localhost -Dmultiarch=undef -Dbin=/opt/perl/bin
-Dusedevel -des'
hint=recommended, useposix=true, d_sigaction=define
usethreads=undef use5005threads=undef useithreads=undef
usemultiplicity=undef
useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=define use64bitall=undef uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler​:
cc='gcc', ccflags ='-DPERL_DONT_CREATE_GVSV_disabled -ggdb
-fno-strict-aliasing -pipe -I/opt/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
optimize='-O6 -march=opteron -mtune=opteron -funroll-loops
-fno-strict-aliasing',
cppflags='-DPERL_DONT_CREATE_GVSV_disabled -D_GNU_SOURCE
-I/opt/include -DPERL_DONT_CREATE_GVSV_disabled -ggdb
-fno-strict-aliasing -pipe -I/opt/include'
ccversion='', gccversion='4.1.2 20061115 (prerelease) (Debian
4.1.1-21)', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define,
longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries​:
ld='gcc', ldflags ='-L/opt/perl/lib -L/opt/lib -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib
libs=-ldl -lm -lcrypt
perllibs=-ldl -lm -lcrypt
libc=/lib/libc-2.3.6.so, so=so, useshrplib=false,
libperl=libperl.a
gnulibc_version='2.3.6'
Dynamic Linking​:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -L/opt/perl/lib -L/opt/lib
-L/usr/local/lib'

Locally applied patches​:
MAINT29832

---
@​INC for perl v5.8.8​:
/root/src/sex
/opt/perl/lib/perl5
/opt/perl/lib/perl5
/opt/perl/lib/perl5
/opt/perl/lib/perl5
/opt/perl/lib/perl5
.

---
Environment for perl v5.8.8​:
HOME=/root
LANG (unset)
LANGUAGE (unset)
LC_CTYPE=de_DE.UTF-8
LD_LIBRARY_PATH (unset)
LOGDIR (unset)

PATH=/root/s2​:/root/s​:/opt/bin​:/opt/sbin​:/bin​:/sbin​:/usr/bin​:/usr/sbin​:/usr/X11/bin​:/usr/games​:/usr/local/bin​:/usr/local/sbin​:/root/src/uunet​:.

PERL5LIB=/root/src/sex
PERL5\_CPANPLUS\_CONFIG=/root/\.cpanplus/config
PERLDB\_OPTS=ornaments=0
PERL\_BADLANG \(unset\)
PERL\_UNICODE=EAL
SHELL=/bin/bash

While the code's behavior hasn't changed, overload.pm has a "Minimal Set
of Overloaded Operations" section, which addresses the OP's concerns.
Should this ticket be closed?

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From schmorp@schmorp.de

While the code's behavior hasn't changed, overload.pm has a "Minimal Set
of Overloaded Operations" section, which addresses the OP's concerns.
Should this ticket be closed?

Hmm, the section doesn't seem to be related to the problem at hand, as it is
about having "the complete set of overloaded operations at one's disposal"
which is not the goal of the code in the bugreport.

Maybe you mean this section​:

  Of the conversions, only one of string, boolean or numeric is needed
  because each can be generated from either of the other two.

But I don't see how that would mean that "0+" is not used when the value
is used in a numerical context, which is the reported problem, afaicr.

--
  The choice of a Deliantra, the free code+content MORPG
  -----==- _GNU_ http​://www.deliantra.net
  ----==-- _ generation
  ---==---(_)__ __ ____ __ Marc Lehmann
  --==---/ / _ \/ // /\ \/ / schmorp@​schmorp.de
  -=====/_/_//_/\_,_/ /_/\_\

[p5pRT]

From @Hugmeir

On 5/25/12, Marc Lehmann <schmorp@​schmorp.de> wrote​:

While the code's behavior hasn't changed, overload.pm has a "Minimal Set
of Overloaded Operations" section, which addresses the OP's concerns.
Should this ticket be closed?

Hmm, the section doesn't seem to be related to the problem at hand, as it
is
about having "the complete set of overloaded operations at one's disposal"
which is not the goal of the code in the bugreport.

Maybe you mean this section​:

Of the conversions, only one of string, boolean or numeric is needed
because each can be generated from either of the other two.

But I don't see how that would mean that "0+" is not used when the value
is used in a numerical context, which is the reported problem, afaicr.

Nah, that wasn't that I meant. The Magical Autogeneration section has
charts detailing what gets generated from what, and seemed to me like
the Mininal Set just drove the point further; I don't know why 0+
overloading doesn't autogenerate the other mathematical operators, but
as far as the bug report goes, it's all working as documented.

[p5pRT]

From schmorp@schmorp.de

On Fri, May 25, 2012 at 12​:30​:16PM -0300, Brian Fraser <fraserbn@​gmail.com> wrote​:

But I don't see how that would mean that "0+" is not used when the value
is used in a numerical context, which is the reported problem, afaicr.

Nah, that wasn't that I meant. The Magical Autogeneration section has
charts detailing what gets generated from what, and seemed to me like
the Mininal Set just drove the point further; I don't know why 0+
overloading doesn't autogenerate the other mathematical operators, but
as far as the bug report goes, it's all working as documented.

That's what you claim, but nothing in the documentation supports that,
while the documentation clearly states that implementing 0+ is enough to
get all conversions.

As I wrote before (did you read my mail?) the goal is not to have a full
set of overloaded operations - the goal is to use the built-in ++ and +
operators.

And for that, the documentation section you mentioned clearly says that it
is enough to override 0+ as the other conversion operations are
autogenerated​:

  Of the conversions, only one of string, boolean or numeric is
  needed because each can be generated from either of the other two.

So where in the documentation does it say that ++ ignores overloaded
conversion, but + not? The difference is thta + cna force numeric context,
while ++ must be more careful, but the fact that ++ simply ignores
overloading is clearly not documented anywhere in the overload manpage.

--
  The choice of a Deliantra, the free code+content MORPG
  -----==- _GNU_ http​://www.deliantra.net
  ----==-- _ generation
  ---==---(_)__ __ ____ __ Marc Lehmann
  --==---/ / _ \/ // /\ \/ / schmorp@​schmorp.de
  -=====/_/_//_/\_,_/ /_/\_\

[p5pRT]

From @Hugmeir

On Sat, May 26, 2012 at 1​:15 AM, Marc Lehmann <schmorp@​schmorp.de> wrote​:

On Fri, May 25, 2012 at 12​:30​:16PM -0300, Brian Fraser <fraserbn@​gmail.com>
wrote​:

But I don't see how that would mean that "0+" is not used when the
value
is used in a numerical context, which is the reported problem, afaicr.

Nah, that wasn't that I meant. The Magical Autogeneration section has
charts detailing what gets generated from what, and seemed to me like
the Mininal Set just drove the point further; I don't know why 0+
overloading doesn't autogenerate the other mathematical operators, but
as far as the bug report goes, it's all working as documented.

That's what you claim, but nothing in the documentation supports that,
while the documentation clearly states that implementing 0+ is enough to
get all conversions.

As I wrote before (did you read my mail?) the goal is not to have a full
set of overloaded operations - the goal is to use the built-in ++ and +
operators.

And for that, the documentation section you mentioned clearly says that it
is enough to override 0+ as the other conversion operations are
autogenerated​:

  Of the conversions\, only one of string\, boolean or numeric is
  needed because each can be generated from either of the other two\.

Aren't you misreading that? To phrase it differently, it says "Of the three
conversion operations, you only need to overload one, and the other two
will be generated from that" not "You only need to overload one of the
three conversions to get everything."

[p5pRT]

From schmorp@schmorp.de

On Sat, May 26, 2012 at 01​:37​:36AM -0300, Brian Fraser <fraserbn@​gmail.com> wrote​:

  Of the conversions\, only one of string\, boolean or numeric is
  needed because each can be generated from either of the other two\.

Aren't you misreading that?

No.

To phrase it differently, it says "Of the three conversion operations,
you only need to overload one, and the other two will be generated from
that" not "You only need to overload one of the three conversions to get
everything."

Both readings indicate a bug, because in the example, perl doesn't do any
conversion, and conversions are part of both "all three conversions" and
"everything".

So if the set of operations is "all conversions" or "everything" doesn't
matter, because conversion is part of both sets, and perl uses neither, so
under both interprettaions, perl is buggy.

I think the problem is that, despite me telling you otherwise multiple
times, you think the goal is to have an + or ++ operator overload. I tell
you again​: it is not, the goal is to have the conversion operators (and
not + nor ++ nor any other operator), and the documentation is clear on
this even in the section you refer to (and in multiple other places).

The fact that it works with + should be a dead giveaway that something is
wrong, because + isn't overloaded, yet works as documented. ++ does not.

If the goal were to have an overloaded + or ++, it obviously would have to
be implemented (just think about it for a second - you cannot autogenerate
++ from conversion operators alone), that makes no sense at all.

--
  The choice of a Deliantra, the free code+content MORPG
  -----==- _GNU_ http​://www.deliantra.net
  ----==-- _ generation
  ---==---(_)__ __ ____ __ Marc Lehmann
  --==---/ / _ \/ // /\ \/ / schmorp@​schmorp.de
  -=====/_/_//_/\_,_/ /_/\_\

[p5pRT]

From @rjbs

After reading this documentation over and over, I believe that the
current behavior is the documented behavior... but that the
documentation makes it pretty painful to work this out.

Further, I think it's pretty surprising.

My gut feeling (and I have not given it a lot of careful thought yet) is
that just changing the behavior is likely to be Bad. I'll keep thinking.

On the other hand, improving the documentation should be quite possible,
and Good. I'll have a look at doing that.

[p5pRT]

From schmorp@schmorp.de

On Sat, May 26, 2012 at 05​:38​:04AM -0700, Ricardo SIGNES via RT <perlbug-followup@​perl.org> wrote​:

After reading this documentation over and over, I believe that the
current behavior is the documented behavior... but that the
documentation makes it pretty painful to work this out.

Could you explain your reasoning on why overload does not support partial
overloading? I would say requiring to overload ALL operations to overload one
is neither reasonable, nor documented, nor current practise, and would breka
alot of cpan that worked before.

As such, improving the documentation to enforce a state that isn't reality is
hardly an improvement.

All I get from you guys is "it's correct as documented", ignoring the
parts that I quoted and failing to provide anything in the documentation
that substantiates this.

I think if you really think the documentation says what it doesn'T say
then it should be possible to point out where it says that, and why the
parts where it indicates otherwise are somehow irrelevant.

One could probably document that ++ (because it is both a string and
numerical operator) does not use conversion operators - this is what imho
is the current state of affairs.

Further, I think it's pretty surprising.

Right, it's especially surrpising because it isn't true - all built-in
operators work fine with partially overloaded values (why is this support
there if it is a bug to rely on it).

Furthermore, if all operators would have to be overloaded, then the
conversionoperators make no sense - one would have an if operator, a while
operator, a print operator and so on.

The documentation is pretty clear that partial overloads are o.k., because
it says "*if*" you want the full set of overloaded operations, and nowhere
does it say you need to provide ALL overloaded operations.

And since it's clearly not true, even *if* it woudls ay so, it would
obviously not reflect current code and practise.

My gut feeling (and I have not given it a lot of careful thought yet) is
that just changing the behavior is likely to be Bad. I'll keep thinking.

The real question is whether the ++ and -- behaviour could be synthesized
with the existing overloads. If not, the best way to proeed would be to
document that ++ and -- ignore overloads and thus need always to be
overloaded in addition to conversion operations.

I also don't think that some code relies on the fact that ++ and --
ignores overloaded values and treat references as numerical values.

--
  The choice of a Deliantra, the free code+content MORPG
  -----==- _GNU_ http​://www.deliantra.net
  ----==-- _ generation
  ---==---(_)__ __ ____ __ Marc Lehmann
  --==---/ / _ \/ // /\ \/ / schmorp@​schmorp.de
  -=====/_/_//_/\_,_/ /_/\_\

[p5pRT]

From @cpansprout

On Sat May 26 05​:38​:03 2012, rjbs wrote​:

After reading this documentation over and over, I believe that the
current behavior is the documented behavior... but that the
documentation makes it pretty painful to work this out.

I think the subject of this ticket is misleading. It’s not
autogeneration that is at fault, but fallback.

Let’s take eq as an example. If I have fallback set to undef, then
autogeneration still happens, but not fallback to Perl’s default
behaviour. Since eq cannot be autogenerated from "", but only from cmp,
I’ll get an error if an object has "" but not eq overloading.

If I set fallback to 1, then eq won’t be directly autogenerated from "";
but in falling back to the default eq, which stringifies, "" ends up
being called nonetheless, though through a different route.

It seems that ++ has a bug in it that is stopping it from nummifying
normally, but doing it some special way.

Further, I think it's pretty surprising.

And I hope I have convinced you it is a bug.

What I wrote above is an armchair response, so it may be off. I haven’t
actually looked at the code to confirm.

My gut feeling (and I have not given it a lot of careful thought yet) is
that just changing the behavior is likely to be Bad.

I hope not. :-)

--

Father Chrysostomos