Skip to content

Question: How is the device topology determined when in_strategy does not match the number of devices? #332

@qnnn

Description

@qnnn

Hi, I'm a beginner and trying to understand how device topology is determined when the in_strategy tuple does not match the actual number of available devices.

For example, if I have 8 devices and use the following sharding configuration:

ops.MatMul().shard(((2, 2), (2, 1)))

What would the resulting device topology mapping look like?
Would mat_1 be mapped to something like device[-1][2][2], and mat_2 to device[-1][2]?

Also, will automatic broadcast happen in this case?

Thanks in advance for any help!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions