Clarify the difference between arrays and tensors

Author: certik

src/math/differential-geometry.rst

@@ -2420,3 +2420,32 @@ for $i=1, 2, 3$ we get:
         2\partial_z u^z \partial_z v^z\right)
         \rho \,\d\rho\, \d \phi\, \d z
         = \int f^z v^z \rho \,\d\rho\, \d \phi\, \d z
+
+Difference Between Tensors and Arrays
+-------------------------------------
+
+Every array can be interpreted as coefficients against some tensor basis in
+some curvilinear space. Then n-n-n arrays are 3D tensors in that space. n-m
+array would be a tensor from one space to another (different dimension), that's
+a more generalized case, but I think it can be done.
+
+Now, if one applies a tensor operation and you feed it a tensor, the result is
+a tensor. Here are the most common operations: TensorContract, TensorProduct,
+TensorAdd, TensorTranspose. If you feed an array to it, it will still work, but
+you'll get an array out of course, not a tensor. But the operations can be done
+on arrays. So we can call these operations tensor operations. In fact the
+"tensor product" is a well known operations and called like that, and it is
+applied to all kinds of things which are not tensors.
+
+Now, ArrayItem (or ArrayIndex/ArrayElement) which indexes into an array is not
+a tensor operation, because an element of a tensor is not a scalar. So that
+must be called ArrayElement. Things like ArrayMaxVal are array operations, or
+ArraySection, since a section of a tensor is not a tensor.
+
+So operations with Tensor in front are "fundamental", and they accept tensors
+and return tensors. Operations with Array in front are just array operations,
+not as fundamental.
+
+Many operations in fortran, such as dot_product, matmul, transpose, +, -, *
+happen to be tensor operations. But other operations such as maxval, sum, etc.
+are not tensor operations.