ChiselSim integration for inline tests

[tmckay-sifive]

Contributor Checklist

• Did you add Scaladoc to every public function/method?
• Did you add at least one test demonstrating the PR?
• Did you delete any extraneous printlns/debugging code?
• Did you specify the type of improvement?
• Did you add appropriate documentation in docs/src?
• Did you request a desired merge strategy?
• Did you add text to be included in the Release Notes for this change?

Type of Improvement

• Feature (or new API)

Desired Merge Strategy

• Squash: The PR will be squashed and merged (choose this if you have no preference).
• Rebase: You will rebase the PR onto master and it will be merged with a merge commit.

Release Notes

• Add ChiselSim.simulateTests API to simulate inline tests and report results

Reviewer Checklist (only modified by reviewer)

• Did you add the appropriate labels? (Select the most appropriate one based on the "Type of Improvement")
• Did you mark the proper milestone (Bug fix: 3.6.x, 5.x, or 6.x depending on impact, API modification or big change: 7.0)?
• Did you review?
• Did you check whether all relevant Contributor checkboxes have been checked?
• Did you do one of the following when ready to merge:
  • Squash: You/ the contributor Enable auto-merge (squash) and clean up the commit message.
  • Merge: Ensure that contributor has cleaned up their commit history, then merge with Create a merge commit.

[tmckay-sifive]

Depends on these PRs:

• Add command line options to control elaboration of inline tests #4826
• Standardize inline testharness interface to clock, reset, finish, and success #4832
• Track tests in HasTests to support metadata needs #4833
• Remove type parameter from chisel3.experimental.inlinetest.HasTests #4834

[seldridge]

Very cool!

I started to review this then realized there are a bunch of prerequisite PRs.

[seldridge]

It would be better to try to avoid the proliferation of different functions that all have slightly different arguments. This was, IMO, a big mistake in both our external and internal testing flows.

What I would do is move in a direction of adding a parameter with a default value that gives you a type-safe way to choose the tests, e.g., an enum like:

object TestSelection {
  sealed trait Type
  case object All
  case class Exactly(tests: Seq[String])
  case class Glob(glob: String)
}

I'm not proposing the above organization exactly, just something like this. You could see LayerControl for an example of how to organize this.

[seldridge]

The return type of these methods is too low-level. This should just be Unit. Alternatively, there should be a ChiselSim-specific return type that is used for all methods here. I.e., also for simulate and simulateRaw.

[tmckay-sifive]

This has been cleaned up and is ready for review. Note: this branch is based on #4833.

EDIT: For reviewers, probably best to hold off for a bit longer. I put this in draft. I have some cleanups I want to backport to this branch that I made in order to support generating scalatest specs for inline tests.