Implement HyKKT Ruiz Scaling

[adhamsi]

Description

A module ruiz is added implementing the RuizScaler class to be used for HyKKT.

Closes #330.

Proposed changes

Implements RuizScaler and RuizScalerKernelImpl in CPU, HIP, and CUDA. Recreates the original test. RuizScaler methods take in matrix::Csr and vector::Vector object types to set the matrix and vector data and implements the scale method to perform the scaling in-place.

Checklist

• All tests pass. Code tested on
  • CPU backend
  • CUDA backend
  • HIP backend
• Code compiles cleanly with flags -Wall -Wpedantic -Wconversion -Wextra.
• The new code follows Re::Solve style guidelines.
• There are unit tests for the new code.
• The new code is documented.
• The feature branch is rebased with respect to the target branch.

Further comments

[shakedregev]

This is getting a segfault for CUDA. Also, fix merge conflicts.

[shakedregev]

Ruiz tests are working, but let's fix the perm tests that made it in here.

[pelesh]

• It would be good to design more intuitive and easy to verify unit tests.
• More extensive documentation is needed for the class methods.
• Matrix/vector objects probably do not need to be unpacked except for passing input to GPU kernels.

[pelesh]

Arguments of this function should be Re::Solve vectors and matrices. I believe you need to unpack them into raw data arrays only before sending them to GPU kernels.

[adhamsi]

This would mean at the beginning of the wrappers there would be something like

const index_type* hes_i = hes->getRowData();
const index_type* hes_j = hes->getColData();
const real_type* hes_v = hes->getValues();
const index_type* jac_i = jac->getRowData();
const index_type* jac_j = jac->getColData();
const real_type* jac_v = jac->getValues();
const index_type* jac_tr_i = jac_tr->getRowData();
const index_type* jac_tr_j = jac_tr->getColData();
const real_type* jac_tr_v = jac_tr->getValues();

so it would be a bit verbose to put this in every kernel implementation. Currently, the unpacking occurs as soon as the data is passed in to the RuizScaler top-level object so it is done once. Implementing it this way would require the unpacking occur repeatedly for every iteration of scaling.

[pelesh]

Yes, the raw pointers should be accessed only in the kernel call. So calling the kernel should look something like this:

  void RuizScalingKernelsHIP::adaptDiagScale(matrix::Sparse* hes,
                                             // more arguments ...
                                             )
    {
      int block_size = 256;
      int num_blocks = (n_total + block_size - 1) / block_size;
      kernels::adaptDiagScale<<<num_blocks, block_size>>>(hes->getNumRows(), 
                                                          hes->getRowData(),
                                                          hes->getColData(), 
                                                          // more arguments ...
                                                          );
    }

Ideally, matrix/vector objects would be passed into the kernel itself, but CUDA/HIP kernels support only fundamental types and pointers to them.

[pelesh]

Tests like this a typically fragile. Consider designing tests that are more intuitive and easier to verify.

This test essentially checks if the behavior of Ruiz scaling has changed. It says little about Ruiz scaling correctness.

[pelesh]

Having some documentation here as to what is being tested here and how would be helpful.

[shakedregev]

Still need to fix the merge conflicts and compile with -D RESOLVE_USE_ASAN=ON to catch memory leaks. Then run the tests normally. If all of them pass, there's no leaks.

[pelesh]

Adding couple of comments related to my earlier review.

[adhamsi]

Still need to fix the merge conflicts and compile with -D RESOLVE_USE_ASAN=ON to catch memory leaks. Then run the tests normally. If all of them pass, there's no leaks.

Leaks have now been fixed.