Add support for Value ** Value

[Weikang01]

a = Value(2.0)
b = Value(3.0)
c = a ** b
print(c.data)
c.backward()
print(a.grad)
print(b.grad)

output now:

8.0
12.0
5.545177444479562

[conscell]

@Weikang01
If self.data is negative and other is a Value, this implementation will cause a ValueError: math domain error.
Division is implemented as follows:

def __truediv__(self, other): # self / other
    return self * other**-1

[Weikang01]

@Weikang01 If self.data is negative and other is a Value, this implementation will cause a ValueError: math domain error. Division is implemented as follows:

def __truediv__(self, other): # self / other
    return self * other**-1

Thank you @conscell!

I added support for negative values now! Thank you again for pointing out my problem!

    a = Value(-2.0)
    b = Value(-3.0)
    c = a ** b
    d = c ** a
    e = d / c
    f = 0.001 * e
    print(f.data)
    f.backward()
    print(c.grad)
    print(a.grad)
    print(b.grad)

output:

-0.512
-12.288
3.3686740693400763
1.064674069340076

[conscell]

@Weikang01
In negative case perhaps it makes sense to set gradient to math.nan, something like this:

>>> import torch
>>> a = torch.tensor(-2.0, requires_grad=True)
>>> b = torch.tensor(-3.0, requires_grad=True)
>>> c = a ** b
>>> c.backward()
>>> a.grad
tensor(-0.1875)
>>> b.grad
tensor(nan)

[Weikang01]

Thank you @conscell!
You are absolutely right, the exponent of negative numbers should have no derivatives! Now the code works like this:

a = Value(-2.0)
b = Value(3.0)
c = a ** b
print(c.data)
d = c ** a
print(d.data)
e = d / c  # 0.015625 / -8.0
e.backward()
print(a.grad)
print(b.grad)
print(c.grad)
print(d.grad)

output:

-8.0
0.015625
nan
nan
-0.000732421875
-0.125

and this is the code of pytorch:

a = torch.tensor(-2.0, requires_grad=True)
b = torch.tensor(3.0, requires_grad=True)
c = a ** b
c.retain_grad()
print(c)
d = c ** a
d.retain_grad()
print(d)
e = d / c

e.backward()
print(a.grad)
print(b.grad)
print(c.grad)
print(d.grad)

outputs:

tensor(-8., grad_fn=<PowBackward1>)
tensor(0.0156, grad_fn=<PowBackward1>)
tensor(nan)
tensor(nan)
tensor(-0.0007)
tensor(-0.1250)