Skip to content

Conversation

justinrosner
Copy link
Contributor

@justinrosner justinrosner commented Oct 8, 2025

Motivation

The current usage of rocmlir-driver is unintuitive and may lead to frustrating debug sessions. When running the following command: rocmlir-driver -kernel-pipeline=migraphx -host-pipeline=migraphx,highlevel, calling highlevel for the host pipeline also runs the functions marked with kernel (i.e., it is also doing the job of -kernel-pipeline=highlevel). This is incredibly unintuitive, and breaks from the behavior that migraphx, and all the other pipelines have.

This PR fixes the following issue: https://github.com/ROCm/rocMLIR-internal/issues/1983

Technical Details

This PR implements the following changes:

  • Update TosaToRockPass so that when kernel-pipeline=highlevel is run on a function without the kernel attribute, it will error out instead of silently doing nothing (we can safely do this now that the kernel pipeline and host pipeline are separate)
  • Update rocmlir-driver to separate kernel-pipeline=highlevel and host-pipeline=highlevel
    • Previously the host pipeline was making use of runKernelPipeline. I've pulled out the necessary logic into a separate function runHostHighLevelPipeline
    • Now that we have a separate call to the highlevel pipeline for the host path, we no longer need the cpu-only flag
  • Updates to how perfRunner calls rocmlir-driver
  • Updates to outdated LIT tests

Test Plan

  • Nightly CI
  • Weekly CI
    • Manually inspect the results of parameterSweeps and perfRunner to make sure that things are working as expected
  • Run Navi2X LIT tests (that currently aren't run in the CI)

Test Result

  • Nightly CI
  • Weekly CI
  • Navi2X LIT tests

Submission Checklist

isHighLevel, kernelPipelineSet);
kernelPipelineSet);
// Run host high-level pipeline if specified
if (hostPipelineSet.contains("highlevel"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this be kernelPipelineSet.contains("highlevel")? are we running highlevel for the kernel here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if that's the case, maybe we should change the name of the function as well: runHostHighLevelPipeline -> runHighLevelPipeline ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

runHostHighLevelPipeline is only running highlevel for the host path, so we want to keep it as is. If kernelPipelineSet.contains("highlevel") then this will trigger that specific pipeline to be run in runKernelPipeline. This fixes previous behavior where hostPipelineSet.contains("highlevel") would then cause for runKernelPipeline run (i.e., the weird behavior where setting host-pipeline=highlevel would run both the CPU and GPU paths).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand now. However, the logic is hard to follow. IMO it'd be better if we handle them separately instead of if(!kernelPipelineSet.empty() || hostPipelineSet.contains("highlevel")) in line 305. But, that's out of the scope of this PR.

Copy link
Contributor

@pabloantoniom pabloantoniom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, but I think it would benefit a lot from:

  • Throwing errors in the if (!func->hasAttr("kernel")) case
  • Splitting the pipelines execution in 2: runHostPipeline and runKernelPipeline

auto func = getOperation();
if (!func->hasAttr("kernel")) {
return;
llvm::report_fatal_error("func op does not have the kernel attribute");
Copy link
Contributor

@pabloantoniom pabloantoniom Oct 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! But I think this PR is missing more of these.

In my opinion this PR must have a commit that do the same for all occurrences in grep -R 'hasAttr("kernel"))' mlir/lib/.

The pattern:

if(thisShouldNeverHappen)
  return;

is evil. In my opinion we should avoid this pattern at all costs.

@@ -134,8 +131,33 @@ parsePipeline(StringRef pipeline, llvm::SmallDenseSet<StringRef> &pipelineSet,
return success();
}

static LogicalResult runHostHighLevelPipeline(ModuleOp mod) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a step in the right direction. And I know it's quite some work but I think it would be good if we had two functions:

  1. runHostPipeline (for the host only)
  2. runKernelPipeline (for the gpu only)

If you like this it could go into another PR, or maybe here, not sure

return success();
}

static LogicalResult runHostHighLevelPipeline(ModuleOp mod) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Use either m or module instead of mod (I've seen the first 2 quite often, but never mod).


static LogicalResult
runKernelPipeline(StringRef arch, ModuleOp kmod, bool isHighLevel,
runKernelPipeline(StringRef arch, ModuleOp kmod,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Same here, I'd use m or module instead of kmod

bool hasKernels = false;
// Find kernel module, defaults to top module
if (!kernelPipelineSet.empty() || isHighLevel) {
if (!kernelPipelineSet.empty() || hostPipelineSet.contains("highlevel")) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been staring at this for a while and I still don't know what this if is checking for exactly. Is it checking if kernel pipeline was specified or if host pipeline was specified with highlevel? If this is the case I find it weird that we have a lot of code for that specific case.

I know it's not strictly related to the PR but can you think of a concise name and use a boolean variable for this specific case? It would help readability.

@@ -1,4 +1,4 @@
// RUN: rocmlir-driver -kernel-pipeline migraphx,highlevel %s | rocmlir-gen -ph -print-results -rand none -fut test - | \
// RUN: rocmlir-driver --host-pipeline=migraphx,highlevel %s | rocmlir-gen -ph -print-results -rand none -fut test - | \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious: Why = instead of a space? Is it different?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no difference. We use a combination of both methods all over our LIT tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants