No SoL config found for device NVIDIA H200.

[acecchini]

Hi,

I have just upgraded jax version to 0.7.2 and now I get a myriad of warning messages with the follwoing format:

W0918 HH:MM:SS.XXXXXX  YYYYYY sol_gpu_cost_model.cc:102] No SoL config found for device: NVIDIA H200. Using default config.

It seems like “SoL” here means Speed-of-Light (an analytical GPU cost/latency model XLA uses to guide scheduling/fusion from my searches).
Is this normal that XLA cannot find such file?

Thanks :)

[golechwierowicz]

Hey,

This should definitely be at a lower level than WARNING, I'll prepare a change shortly to make it less verbose.

It's expected that XLA cannot find the config. For H200s we could not get data internally to model the networking speed so we fallback to H100 numbers. We might underestimate the networking speed a little bit, but in internal tests we figured it's fine.

You can either:

1. Disable the new cost model via export XLA_FLAGS=--xla_gpu_enable_analytical_sol_latency_estimator=false. This will silence model specific warnings.
2. Suppress all warnings by export TF_CPP_MIN_LOG_LEVEL=3.
3. Optionally if you'd like to tinker with the new model, and tailor it to your network we recommend tuning the NIC speed via export XLA_FLAGS=--xla_gpu_analytical_latency_estimator_options='nic_speed_gbps=$YOUR_NIC_SPEED'.

More extensive documentation in OpenXLA webpage is coming soon.

[acecchini]

Hey,

Thanks for taking the time to answer me. For my usecase, I have a single node with 8 H200 connected through NVLink (NVLink speed is 900Gbps), to what value should YOUR_NIC_SPEED be set to?

Thanks!

[golechwierowicz]

The change that lowers verbosity is in. It should be available in the next JAX release.

If you're just running single host workloads, there are probably very little benefits from the new cost model. But in your case you don't have to change anything – for single hosts workloads it should work out of the box, albeit with inconvenient warning(s) in logs.

cc: @frgossen