ylt simd util feature #940

helintongh · 2025-04-17T09:56:12Z

Why

// in cinatra have function split_sv:
inline std::vector<std::string_view> split_sv(std::string_view s,
                                              std::string_view delimiter) {
  size_t start = 0;
  size_t end = s.find_first_of(delimiter);

  std::vector<std::string_view> output;

  while (end <= std::string_view::npos) {
    output.emplace_back(s.substr(start, end - start));

    if (end == std::string_view::npos)
      break;

    start = end + 1;
    end = s.find_first_of(delimiter, start);
  }

  return output;
}

use simd optimize it.

in the future optimize json and base64 etc by simd.

What is changing

simd dispatch, auto chose simd fucntion.now implement string and string_view split.

Example

#include <ylt/simd_util/simd_str_split.h>

#include "doctest.h"

TEST_CASE("test string_view split") {
  std::string_view sv_tmp =
      "hello world\t127.0.0.1\t1024\twww.yalantinglibs.com";

  auto tokens = ylt::split_sv(sv_tmp, '\t');

  CHECK(tokens[0] == "hello world");
  CHECK(tokens[1] == "127.0.0.1");
  CHECK(tokens[2] == "1024");
  CHECK(tokens[3] == "www.yalantinglibs.com");
}

TEST_CASE("test string split") {
  std::string_view sv_tmp =
      "hello world\t127.0.0.1\t1024\twww.yalantinglibs.com";

  auto tokens = ylt::split_str(sv_tmp, '\t');

  CHECK(tokens[0] == "hello world");
  CHECK(tokens[1] == "127.0.0.1");
  CHECK(tokens[2] == "1024");
  CHECK(tokens[3] == "www.yalantinglibs.com");
}

Refer to the tests path and benchmark path in simd_util.

in neon:

neon better than normal.

in avx2:

It's single header library, when wanna use avx2 add compile option -mavx2,-mbmi.when in arm,do nothing will auto use neon simd.

github-actions · 2025-04-17T10:35:34Z