[flang][acc] Lower do and do concurrent loops specially in acc regions #149614
+716
−121
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When OpenACC is enabled and Fortran loops are annotated with `acc loop`, they are lowered to `acc.loop` operation. And rest of the contained loops use the normal FIR lowering path. Hovever, the OpenACC specification has special provisions related to contained loops and their induction variable. In order to adhere to this, we convert all valid contained loops to `acc.loop` in order to store this information appropriately. The provisions in the spec that motivated this change (line numbers are from OpenACC 3.4): - 1353 Loop variables in Fortran do statements within a compute construct are predetermined to be private to the thread that executes the loop. - 3783 When do concurrent appears without a loop construct in a kernels construct it is treated as if it is annotated with loop auto. If it appears in a parallel construct or an accelerator routine then it is treated as if it is annotated with loop independent. By valid loops - we convert do loops and do concurrent loops which have induction variable. Loops which are unstructured are not handled.
llvmbot
added
mlir
flang
|
@llvm/pr-subscribers-mlir @llvm/pr-subscribers-openacc Author: Razvan Lupusoru (razvanlupusoru) ChangesWhen OpenACC is enabled and Fortran loops are annotated with Hovever, the OpenACC specification has special provisions related to contained loops and their induction variable. In order to adhere to this, we convert all valid contained loops to The provisions in the spec that motivated this change (line numbers are from OpenACC 3.4):
By valid loops - we convert do loops and do concurrent loops which have induction variable. Loops which are unstructured are not handled. Patch is 37.79 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/149614.diff 5 Files Affected:
diff --git a/flang/include/flang/Lower/OpenACC.h b/flang/include/flang/Lower/OpenACC.h
index af3451023e3df..8b13ce94f7bc4 100644
--- a/flang/include/flang/Lower/OpenACC.h
+++ b/flang/include/flang/Lower/OpenACC.h
@@ -43,6 +43,7 @@ struct ProcedureDesignator;
namespace parser {
struct AccClauseList;
+struct DoConstruct;
struct OpenACCConstruct;
struct OpenACCDeclarativeConstruct;
struct OpenACCRoutineConstruct;
@@ -58,6 +59,7 @@ namespace lower {
class AbstractConverter;
class StatementContext;
+class SymMap;
namespace pft {
struct Evaluation;
@@ -114,14 +116,32 @@ void attachDeclarePostDeallocAction(AbstractConverter &, fir::FirOpBuilder &,
void genOpenACCTerminator(fir::FirOpBuilder &, mlir::Operation *,
mlir::Location);
-int64_t getLoopCountForCollapseAndTile(const Fortran::parser::AccClauseList &);
+/// Used to obtain the number of contained loops to look for
+/// since this is dependent on number of tile operands and collapse
+/// clause.
+uint64_t getLoopCountForCollapseAndTile(const Fortran::parser::AccClauseList &);
+/// Checks whether the current insertion point is inside OpenACC loop.
bool isInOpenACCLoop(fir::FirOpBuilder &);
+/// Checks whether the current insertion point is inside OpenACC compute construct.
+bool isInsideOpenACCComputeConstruct(fir::FirOpBuilder &);
+
void setInsertionPointAfterOpenACCLoopIfInside(fir::FirOpBuilder &);
void genEarlyReturnInOpenACCLoop(fir::FirOpBuilder &, mlir::Location);
+/// Generates an OpenACC loop from a do construct in order to
+/// properly capture the loop bounds, parallelism determination mode,
+/// and to privatize the loop variables.
+/// When the conversion is rejected, nullptr is returned.
+mlir::Operation *genOpenACCLoopFromDoConstruct(
+ AbstractConverter &converter,
+ Fortran::semantics::SemanticsContext &semanticsContext,
+ Fortran::lower::SymMap &localSymbols,
+ const Fortran::parser::DoConstruct &doConstruct,
+ pft::Evaluation &eval);
+
} // namespace lower
} // namespace Fortran
diff --git a/flang/lib/Lower/Bridge.cpp b/flang/lib/Lower/Bridge.cpp
index 5f0783f869bf6..4073b8623e333 100644
--- a/flang/lib/Lower/Bridge.cpp
+++ b/flang/lib/Lower/Bridge.cpp
@@ -2164,10 +2164,35 @@ class FirConverter : public Fortran::lower::AbstractConverter {
/// - structured and unstructured concurrent loops
void genFIR(const Fortran::parser::DoConstruct &doConstruct) {
setCurrentPositionAt(doConstruct);
- // Collect loop nest information.
- // Generate begin loop code directly for infinite and while loops.
Fortran::lower::pft::Evaluation &eval = getEval();
bool unstructuredContext = eval.lowerAsUnstructured();
+
+ // Loops with induction variables inside OpenACC compute constructs
+ // need special handling to ensure that the IVs are privatized.
+ if (Fortran::lower::isInsideOpenACCComputeConstruct(*builder)) {
+ mlir::Operation* loopOp = Fortran::lower::genOpenACCLoopFromDoConstruct(
+ *this, bridge.getSemanticsContext(), localSymbols,
+ doConstruct, eval);
+ bool success = loopOp != nullptr;
+ if (success) {
+ // Sanity check that the builder insertion point is inside the newly
+ // generated loop.
+ assert(
+ loopOp->getRegion(0).isAncestor(
+ builder->getInsertionPoint()->getBlock()->getParent()) &&
+ "builder insertion point is not inside the newly generated loop");
+
+ // Loop body code.
+ auto iter = eval.getNestedEvaluations().begin();
+ for (auto end = --eval.getNestedEvaluations().end(); iter != end; ++iter)
+ genFIR(*iter, unstructuredContext);
+ return;
+ }
+ // Fall back to normal loop handling.
+ }
+
+ // Collect loop nest information.
+ // Generate begin loop code directly for infinite and while loops.
Fortran::lower::pft::Evaluation &doStmtEval =
eval.getFirstNestedEvaluation();
auto *doStmt = doStmtEval.getIf<Fortran::parser::NonLabelDoStmt>();
@@ -3121,7 +3146,7 @@ class FirConverter : public Fortran::lower::AbstractConverter {
Fortran::lower::pft::Evaluation *curEval = &getEval();
if (accLoop || accCombined) {
- int64_t loopCount;
+ uint64_t loopCount;
if (accLoop) {
const Fortran::parser::AccBeginLoopDirective &beginLoopDir =
std::get<Fortran::parser::AccBeginLoopDirective>(accLoop->t);
@@ -3139,7 +3164,7 @@ class FirConverter : public Fortran::lower::AbstractConverter {
if (curEval->lowerAsStructured()) {
curEval = &curEval->getFirstNestedEvaluation();
- for (int64_t i = 1; i < loopCount; i++)
+ for (uint64_t i = 1; i < loopCount; i++)
curEval = &*std::next(curEval->getNestedEvaluations().begin());
}
}
diff --git a/flang/lib/Lower/OpenACC.cpp b/flang/lib/Lower/OpenACC.cpp
index 51eb33dec186b..950b02501751a 100644
--- a/flang/lib/Lower/OpenACC.cpp
+++ b/flang/lib/Lower/OpenACC.cpp
@@ -35,6 +35,7 @@
#include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h"
#include "mlir/IR/MLIRContext.h"
#include "mlir/Support/LLVM.h"
+#include "llvm/ADT/ScopeExit.h"
#include "llvm/ADT/STLExtras.h"
#include "llvm/Frontend/OpenACC/ACC.h.inc"
#include "llvm/Support/CommandLine.h"
@@ -2138,6 +2139,168 @@ static void determineDefaultLoopParMode(
}
}
+// Extract loop bounds, steps, induction variables, and privatization info
+// for both DO CONCURRENT and regular do loops
+static void processDoLoopBounds(
+ Fortran::lower::AbstractConverter &converter,
+ mlir::Location currentLocation, Fortran::lower::StatementContext &stmtCtx,
+ fir::FirOpBuilder &builder,
+ const Fortran::parser::DoConstruct &outerDoConstruct,
+ Fortran::lower::pft::Evaluation &eval,
+ llvm::SmallVector<mlir::Value> &lowerbounds,
+ llvm::SmallVector<mlir::Value> &upperbounds,
+ llvm::SmallVector<mlir::Value> &steps,
+ llvm::SmallVector<mlir::Value> &privateOperands,
+ llvm::SmallVector<mlir::Value> &ivPrivate,
+ llvm::SmallVector<mlir::Attribute> &privatizationRecipes,
+ llvm::SmallVector<mlir::Type> &ivTypes,
+ llvm::SmallVector<mlir::Location> &ivLocs,
+ llvm::SmallVector<bool> &inclusiveBounds,
+ llvm::SmallVector<mlir::Location> &locs, uint64_t loopsToProcess) {
+ assert(loopsToProcess > 0 && "expect at least one loop");
+ locs.push_back(currentLocation); // Location of the directive
+ Fortran::lower::pft::Evaluation *crtEval = &eval.getFirstNestedEvaluation();
+ bool isDoConcurrent = outerDoConstruct.IsDoConcurrent();
+
+ if (isDoConcurrent) {
+ locs.push_back(converter.genLocation(
+ Fortran::parser::FindSourceLocation(outerDoConstruct)));
+ const Fortran::parser::LoopControl *loopControl =
+ &*outerDoConstruct.GetLoopControl();
+ const auto &concurrent =
+ std::get<Fortran::parser::LoopControl::Concurrent>(loopControl->u);
+ if (!std::get<std::list<Fortran::parser::LocalitySpec>>(concurrent.t)
+ .empty())
+ TODO(currentLocation, "DO CONCURRENT with locality spec inside ACC");
+
+ const auto &concurrentHeader =
+ std::get<Fortran::parser::ConcurrentHeader>(concurrent.t);
+ const auto &controls =
+ std::get<std::list<Fortran::parser::ConcurrentControl>>(
+ concurrentHeader.t);
+ for (const auto &control : controls) {
+ lowerbounds.push_back(fir::getBase(converter.genExprValue(
+ *Fortran::semantics::GetExpr(std::get<1>(control.t)), stmtCtx)));
+ upperbounds.push_back(fir::getBase(converter.genExprValue(
+ *Fortran::semantics::GetExpr(std::get<2>(control.t)), stmtCtx)));
+ if (const auto &expr =
+ std::get<std::optional<Fortran::parser::ScalarIntExpr>>(
+ control.t))
+ steps.push_back(fir::getBase(converter.genExprValue(
+ *Fortran::semantics::GetExpr(*expr), stmtCtx)));
+ else // If `step` is not present, assume it is `1`.
+ steps.push_back(builder.createIntegerConstant(
+ currentLocation, upperbounds[upperbounds.size() - 1].getType(), 1));
+
+ const auto &name = std::get<Fortran::parser::Name>(control.t);
+ privatizeIv(converter, *name.symbol, currentLocation, ivTypes, ivLocs,
+ privateOperands, ivPrivate, privatizationRecipes,
+ isDoConcurrent);
+
+ inclusiveBounds.push_back(true);
+ }
+ } else {
+ for (uint64_t i = 0; i < loopsToProcess; ++i) {
+ const Fortran::parser::LoopControl *loopControl;
+ if (i == 0) {
+ loopControl = &*outerDoConstruct.GetLoopControl();
+ locs.push_back(converter.genLocation(
+ Fortran::parser::FindSourceLocation(outerDoConstruct)));
+ } else {
+ auto *doCons = crtEval->getIf<Fortran::parser::DoConstruct>();
+ assert(doCons && "expect do construct");
+ loopControl = &*doCons->GetLoopControl();
+ locs.push_back(converter.genLocation(
+ Fortran::parser::FindSourceLocation(*doCons)));
+ }
+
+ const Fortran::parser::LoopControl::Bounds *bounds =
+ std::get_if<Fortran::parser::LoopControl::Bounds>(&loopControl->u);
+ assert(bounds && "Expected bounds on the loop construct");
+ lowerbounds.push_back(fir::getBase(converter.genExprValue(
+ *Fortran::semantics::GetExpr(bounds->lower), stmtCtx)));
+ upperbounds.push_back(fir::getBase(converter.genExprValue(
+ *Fortran::semantics::GetExpr(bounds->upper), stmtCtx)));
+ if (bounds->step)
+ steps.push_back(fir::getBase(converter.genExprValue(
+ *Fortran::semantics::GetExpr(bounds->step), stmtCtx)));
+ else // If `step` is not present, assume it is `1`.
+ steps.push_back(builder.createIntegerConstant(
+ currentLocation, upperbounds[upperbounds.size() - 1].getType(), 1));
+
+ Fortran::semantics::Symbol &ivSym =
+ bounds->name.thing.symbol->GetUltimate();
+ privatizeIv(converter, ivSym, currentLocation, ivTypes, ivLocs,
+ privateOperands, ivPrivate, privatizationRecipes);
+
+ inclusiveBounds.push_back(true);
+
+ if (i < loopsToProcess - 1)
+ crtEval = &*std::next(crtEval->getNestedEvaluations().begin());
+ }
+ }
+}
+
+static mlir::acc::LoopOp
+buildACCLoopOp(Fortran::lower::AbstractConverter &converter,
+ mlir::Location currentLocation,
+ Fortran::semantics::SemanticsContext &semanticsContext,
+ Fortran::lower::StatementContext &stmtCtx,
+ const Fortran::parser::DoConstruct &outerDoConstruct,
+ Fortran::lower::pft::Evaluation &eval,
+ llvm::SmallVector<mlir::Value> &privateOperands,
+ llvm::SmallVector<mlir::Attribute> &privatizationRecipes,
+ llvm::SmallVector<mlir::Value> &gangOperands,
+ llvm::SmallVector<mlir::Value> &workerNumOperands,
+ llvm::SmallVector<mlir::Value> &vectorOperands,
+ llvm::SmallVector<mlir::Value> &tileOperands,
+ llvm::SmallVector<mlir::Value> &cacheOperands,
+ llvm::SmallVector<mlir::Value> &reductionOperands,
+ llvm::SmallVector<mlir::Type> &retTy, mlir::Value yieldValue,
+ uint64_t loopsToProcess) {
+ fir::FirOpBuilder &builder = converter.getFirOpBuilder();
+
+ llvm::SmallVector<mlir::Value> ivPrivate;
+ llvm::SmallVector<mlir::Type> ivTypes;
+ llvm::SmallVector<mlir::Location> ivLocs;
+ llvm::SmallVector<bool> inclusiveBounds;
+ llvm::SmallVector<mlir::Location> locs;
+ llvm::SmallVector<mlir::Value> lowerbounds, upperbounds, steps;
+
+ // Look at the do/do concurrent loops to extract bounds information.
+ processDoLoopBounds(converter, currentLocation, stmtCtx, builder,
+ outerDoConstruct, eval, lowerbounds, upperbounds, steps,
+ privateOperands, ivPrivate, privatizationRecipes, ivTypes,
+ ivLocs, inclusiveBounds, locs, loopsToProcess);
+
+ // Prepare the operand segment size attribute and the operands value range.
+ llvm::SmallVector<mlir::Value> operands;
+ llvm::SmallVector<int32_t> operandSegments;
+ addOperands(operands, operandSegments, lowerbounds);
+ addOperands(operands, operandSegments, upperbounds);
+ addOperands(operands, operandSegments, steps);
+ addOperands(operands, operandSegments, gangOperands);
+ addOperands(operands, operandSegments, workerNumOperands);
+ addOperands(operands, operandSegments, vectorOperands);
+ addOperands(operands, operandSegments, tileOperands);
+ addOperands(operands, operandSegments, cacheOperands);
+ addOperands(operands, operandSegments, privateOperands);
+ addOperands(operands, operandSegments, reductionOperands);
+
+ auto loopOp = createRegionOp<mlir::acc::LoopOp, mlir::acc::YieldOp>(
+ builder, builder.getFusedLoc(locs), currentLocation, eval, operands,
+ operandSegments, /*outerCombined=*/false, retTy, yieldValue, ivTypes,
+ ivLocs);
+
+ for (auto [arg, value] : llvm::zip(
+ loopOp.getLoopRegions().front()->front().getArguments(), ivPrivate))
+ builder.create<fir::StoreOp>(currentLocation, arg, value);
+
+ loopOp.setInclusiveUpperbound(inclusiveBounds);
+
+ return loopOp;
+}
+
static mlir::acc::LoopOp createLoopOp(
Fortran::lower::AbstractConverter &converter,
mlir::Location currentLocation,
@@ -2150,9 +2313,9 @@ static mlir::acc::LoopOp createLoopOp(
std::nullopt,
bool needEarlyReturnHandling = false) {
fir::FirOpBuilder &builder = converter.getFirOpBuilder();
- llvm::SmallVector<mlir::Value> tileOperands, privateOperands, ivPrivate,
+ llvm::SmallVector<mlir::Value> tileOperands, privateOperands,
reductionOperands, cacheOperands, vectorOperands, workerNumOperands,
- gangOperands, lowerbounds, upperbounds, steps;
+ gangOperands;
llvm::SmallVector<mlir::Attribute> privatizationRecipes, reductionRecipes;
llvm::SmallVector<int32_t> tileOperandsSegments, gangOperandsSegments;
llvm::SmallVector<int64_t> collapseValues;
@@ -2321,107 +2484,6 @@ static mlir::acc::LoopOp createLoopOp(
}
}
- llvm::SmallVector<mlir::Type> ivTypes;
- llvm::SmallVector<mlir::Location> ivLocs;
- llvm::SmallVector<bool> inclusiveBounds;
- llvm::SmallVector<mlir::Location> locs;
- locs.push_back(currentLocation); // Location of the directive
- Fortran::lower::pft::Evaluation *crtEval = &eval.getFirstNestedEvaluation();
- bool isDoConcurrent = outerDoConstruct.IsDoConcurrent();
- if (isDoConcurrent) {
- locs.push_back(converter.genLocation(
- Fortran::parser::FindSourceLocation(outerDoConstruct)));
- const Fortran::parser::LoopControl *loopControl =
- &*outerDoConstruct.GetLoopControl();
- const auto &concurrent =
- std::get<Fortran::parser::LoopControl::Concurrent>(loopControl->u);
- if (!std::get<std::list<Fortran::parser::LocalitySpec>>(concurrent.t)
- .empty())
- TODO(currentLocation, "DO CONCURRENT with locality spec");
-
- const auto &concurrentHeader =
- std::get<Fortran::parser::ConcurrentHeader>(concurrent.t);
- const auto &controls =
- std::get<std::list<Fortran::parser::ConcurrentControl>>(
- concurrentHeader.t);
- for (const auto &control : controls) {
- lowerbounds.push_back(fir::getBase(converter.genExprValue(
- *Fortran::semantics::GetExpr(std::get<1>(control.t)), stmtCtx)));
- upperbounds.push_back(fir::getBase(converter.genExprValue(
- *Fortran::semantics::GetExpr(std::get<2>(control.t)), stmtCtx)));
- if (const auto &expr =
- std::get<std::optional<Fortran::parser::ScalarIntExpr>>(
- control.t))
- steps.push_back(fir::getBase(converter.genExprValue(
- *Fortran::semantics::GetExpr(*expr), stmtCtx)));
- else // If `step` is not present, assume it is `1`.
- steps.push_back(builder.createIntegerConstant(
- currentLocation, upperbounds[upperbounds.size() - 1].getType(), 1));
-
- const auto &name = std::get<Fortran::parser::Name>(control.t);
- privatizeIv(converter, *name.symbol, currentLocation, ivTypes, ivLocs,
- privateOperands, ivPrivate, privatizationRecipes,
- isDoConcurrent);
-
- inclusiveBounds.push_back(true);
- }
- } else {
- int64_t loopCount =
- Fortran::lower::getLoopCountForCollapseAndTile(accClauseList);
- for (unsigned i = 0; i < loopCount; ++i) {
- const Fortran::parser::LoopControl *loopControl;
- if (i == 0) {
- loopControl = &*outerDoConstruct.GetLoopControl();
- locs.push_back(converter.genLocation(
- Fortran::parser::FindSourceLocation(outerDoConstruct)));
- } else {
- auto *doCons = crtEval->getIf<Fortran::parser::DoConstruct>();
- assert(doCons && "expect do construct");
- loopControl = &*doCons->GetLoopControl();
- locs.push_back(converter.genLocation(
- Fortran::parser::FindSourceLocation(*doCons)));
- }
-
- const Fortran::parser::LoopControl::Bounds *bounds =
- std::get_if<Fortran::parser::LoopControl::Bounds>(&loopControl->u);
- assert(bounds && "Expected bounds on the loop construct");
- lowerbounds.push_back(fir::getBase(converter.genExprValue(
- *Fortran::semantics::GetExpr(bounds->lower), stmtCtx)));
- upperbounds.push_back(fir::getBase(converter.genExprValue(
- *Fortran::semantics::GetExpr(bounds->upper), stmtCtx)));
- if (bounds->step)
- steps.push_back(fir::getBase(converter.genExprValue(
- *Fortran::semantics::GetExpr(bounds->step), stmtCtx)));
- else // If `step` is not present, assume it is `1`.
- steps.push_back(builder.createIntegerConstant(
- currentLocation, upperbounds[upperbounds.size() - 1].getType(), 1));
-
- Fortran::semantics::Symbol &ivSym =
- bounds->name.thing.symbol->GetUltimate();
- privatizeIv(converter, ivSym, currentLocation, ivTypes, ivLocs,
- privateOperands, ivPrivate, privatizationRecipes);
-
- inclusiveBounds.push_back(true);
-
- if (i < loopCount - 1)
- crtEval = &*std::next(crtEval->getNestedEvaluations().begin());
- }
- }
-
- // Prepare the operand segment size attribute and the operands value range.
- llvm::SmallVector<mlir::Value> operands;
- llvm::SmallVector<int32_t> operandSegments;
- addOperands(operands, operandSegments, lowerbounds);
- addOperands(operands, operandSegments, upperbounds);
- addOperands(operands, operandSegments, steps);
- addOperands(operands, operandSegments, gangOperands);
- addOperands(operands, operandSegments, workerNumOperands);
- addOperands(operands, operandSegments, vectorOperands);
- addOperands(operands, operandSegments, tileOperands);
- addOperands(operands, operandSegments, cacheOperands);
- addOperands(operands, operandSegments, privateOperands);
- addOperands(operands, operandSegments, reductionOperands);
-
llvm::SmallVector<mlir::Type> retTy;
mlir::Value yieldValue;
if (needEarlyReturnHandling) {
@@ -2430,16 +2492,13 @@ static mlir::acc::LoopOp createLoopOp(
retTy.push_back(i1Ty);
}
- auto loopOp = createRegionOp<mlir::acc::LoopOp, mlir::acc::YieldOp>(
- builder, builder.getFusedLoc(locs), currentLocation, eval, operands,
- operandSegments, /*outerCombined=*/false, retTy, yieldValue, ivTypes,
- ivLocs);
-
- for (auto [arg, value] : llvm::zip(
- loopOp.getLoopRegions().front()->front().getArguments(), ivPrivate))
- builder.create<fir::StoreOp>(currentLocation, arg, value);
-
- loopOp.setInclusiveUpperbound(inclusiveBounds);
+ uint64_t loopsToProcess =
+ Fortran::lower::getLoopCountForCollapseAndTile(accClauseList);
+ auto loopOp = buildACCLoopOp(
+ converter, currentLocation, semanticsContext, stmtCtx, outerDoConstruct,
+ eval, privateOperands, privatizationRecipes, gangOperands,
+ workerNumOperands, vectorOperands, tileOperands, cacheOperands,
+ reductionOperands, retTy, yieldValue, loopsToProcess);
if (!gangDeviceTypes.empty())
loopOp.setGangAttr(builder.getArrayAttr(gangDeviceTypes));
@@ -4891,6 +4950,12 @@ bool Fortran::lower::isInOpenACCLoop(fir::FirOpBuilder &builder) {
return false;
}
+bool Fortran::lower::isInsideOpenACCComputeCo...
[truncated]
|
|
@llvm/pr-subscribers-flang-fir-hlfir Author: Razvan Lupusoru (razvanlupusoru) ChangesWhen OpenACC is enabled and Fortran loops are annotated with Hovever, the OpenACC specification has special provisions related to contained loops and their induction variable. In order to adhere to this, we convert all valid contained loops to The provisions in the spec that motivated this change (line numbers are from OpenACC 3.4):
By valid loops - we convert do loops and do concurrent loops which have induction variable. Loops which are unstructured are not handled. Patch is 37.79 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/149614.diff 5 Files Affected:
diff --git a/flang/include/flang/Lower/OpenACC.h b/flang/include/flang/Lower/OpenACC.h
index af3451023e3df..8b13ce94f7bc4 100644
--- a/flang/include/flang/Lower/OpenACC.h
+++ b/flang/include/flang/Lower/OpenACC.h
@@ -43,6 +43,7 @@ struct ProcedureDesignator;
namespace parser {
struct AccClauseList;
+struct DoConstruct;
struct OpenACCConstruct;
struct OpenACCDeclarativeConstruct;
struct OpenACCRoutineConstruct;
@@ -58,6 +59,7 @@ namespace lower {
class AbstractConverter;
class StatementContext;
+class SymMap;
namespace pft {
struct Evaluation;
@@ -114,14 +116,32 @@ void attachDeclarePostDeallocAction(AbstractConverter &, fir::FirOpBuilder &,
void genOpenACCTerminator(fir::FirOpBuilder &, mlir::Operation *,
mlir::Location);
-int64_t getLoopCountForCollapseAndTile(const Fortran::parser::AccClauseList &);
+/// Used to obtain the number of contained loops to look for
+/// since this is dependent on number of tile operands and collapse
+/// clause.
+uint64_t getLoopCountForCollapseAndTile(const Fortran::parser::AccClauseList &);
+/// Checks whether the current insertion point is inside OpenACC loop.
bool isInOpenACCLoop(fir::FirOpBuilder &);
+/// Checks whether the current insertion point is inside OpenACC compute construct.
+bool isInsideOpenACCComputeConstruct(fir::FirOpBuilder &);
+
void setInsertionPointAfterOpenACCLoopIfInside(fir::FirOpBuilder &);
void genEarlyReturnInOpenACCLoop(fir::FirOpBuilder &, mlir::Location);
+/// Generates an OpenACC loop from a do construct in order to
+/// properly capture the loop bounds, parallelism determination mode,
+/// and to privatize the loop variables.
+/// When the conversion is rejected, nullptr is returned.
+mlir::Operation *genOpenACCLoopFromDoConstruct(
+ AbstractConverter &converter,
+ Fortran::semantics::SemanticsContext &semanticsContext,
+ Fortran::lower::SymMap &localSymbols,
+ const Fortran::parser::DoConstruct &doConstruct,
+ pft::Evaluation &eval);
+
} // namespace lower
} // namespace Fortran
diff --git a/flang/lib/Lower/Bridge.cpp b/flang/lib/Lower/Bridge.cpp
index 5f0783f869bf6..4073b8623e333 100644
--- a/flang/lib/Lower/Bridge.cpp
+++ b/flang/lib/Lower/Bridge.cpp
@@ -2164,10 +2164,35 @@ class FirConverter : public Fortran::lower::AbstractConverter {
/// - structured and unstructured concurrent loops
void genFIR(const Fortran::parser::DoConstruct &doConstruct) {
setCurrentPositionAt(doConstruct);
- // Collect loop nest information.
- // Generate begin loop code directly for infinite and while loops.
Fortran::lower::pft::Evaluation &eval = getEval();
bool unstructuredContext = eval.lowerAsUnstructured();
+
+ // Loops with induction variables inside OpenACC compute constructs
+ // need special handling to ensure that the IVs are privatized.
+ if (Fortran::lower::isInsideOpenACCComputeConstruct(*builder)) {
+ mlir::Operation* loopOp = Fortran::lower::genOpenACCLoopFromDoConstruct(
+ *this, bridge.getSemanticsContext(), localSymbols,
+ doConstruct, eval);
+ bool success = loopOp != nullptr;
+ if (success) {
+ // Sanity check that the builder insertion point is inside the newly
+ // generated loop.
+ assert(
+ loopOp->getRegion(0).isAncestor(
+ builder->getInsertionPoint()->getBlock()->getParent()) &&
+ "builder insertion point is not inside the newly generated loop");
+
+ // Loop body code.
+ auto iter = eval.getNestedEvaluations().begin();
+ for (auto end = --eval.getNestedEvaluations().end(); iter != end; ++iter)
+ genFIR(*iter, unstructuredContext);
+ return;
+ }
+ // Fall back to normal loop handling.
+ }
+
+ // Collect loop nest information.
+ // Generate begin loop code directly for infinite and while loops.
Fortran::lower::pft::Evaluation &doStmtEval =
eval.getFirstNestedEvaluation();
auto *doStmt = doStmtEval.getIf<Fortran::parser::NonLabelDoStmt>();
@@ -3121,7 +3146,7 @@ class FirConverter : public Fortran::lower::AbstractConverter {
Fortran::lower::pft::Evaluation *curEval = &getEval();
if (accLoop || accCombined) {
- int64_t loopCount;
+ uint64_t loopCount;
if (accLoop) {
const Fortran::parser::AccBeginLoopDirective &beginLoopDir =
std::get<Fortran::parser::AccBeginLoopDirective>(accLoop->t);
@@ -3139,7 +3164,7 @@ class FirConverter : public Fortran::lower::AbstractConverter {
if (curEval->lowerAsStructured()) {
curEval = &curEval->getFirstNestedEvaluation();
- for (int64_t i = 1; i < loopCount; i++)
+ for (uint64_t i = 1; i < loopCount; i++)
curEval = &*std::next(curEval->getNestedEvaluations().begin());
}
}
diff --git a/flang/lib/Lower/OpenACC.cpp b/flang/lib/Lower/OpenACC.cpp
index 51eb33dec186b..950b02501751a 100644
--- a/flang/lib/Lower/OpenACC.cpp
+++ b/flang/lib/Lower/OpenACC.cpp
@@ -35,6 +35,7 @@
#include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h"
#include "mlir/IR/MLIRContext.h"
#include "mlir/Support/LLVM.h"
+#include "llvm/ADT/ScopeExit.h"
#include "llvm/ADT/STLExtras.h"
#include "llvm/Frontend/OpenACC/ACC.h.inc"
#include "llvm/Support/CommandLine.h"
@@ -2138,6 +2139,168 @@ static void determineDefaultLoopParMode(
}
}
+// Extract loop bounds, steps, induction variables, and privatization info
+// for both DO CONCURRENT and regular do loops
+static void processDoLoopBounds(
+ Fortran::lower::AbstractConverter &converter,
+ mlir::Location currentLocation, Fortran::lower::StatementContext &stmtCtx,
+ fir::FirOpBuilder &builder,
+ const Fortran::parser::DoConstruct &outerDoConstruct,
+ Fortran::lower::pft::Evaluation &eval,
+ llvm::SmallVector<mlir::Value> &lowerbounds,
+ llvm::SmallVector<mlir::Value> &upperbounds,
+ llvm::SmallVector<mlir::Value> &steps,
+ llvm::SmallVector<mlir::Value> &privateOperands,
+ llvm::SmallVector<mlir::Value> &ivPrivate,
+ llvm::SmallVector<mlir::Attribute> &privatizationRecipes,
+ llvm::SmallVector<mlir::Type> &ivTypes,
+ llvm::SmallVector<mlir::Location> &ivLocs,
+ llvm::SmallVector<bool> &inclusiveBounds,
+ llvm::SmallVector<mlir::Location> &locs, uint64_t loopsToProcess) {
+ assert(loopsToProcess > 0 && "expect at least one loop");
+ locs.push_back(currentLocation); // Location of the directive
+ Fortran::lower::pft::Evaluation *crtEval = &eval.getFirstNestedEvaluation();
+ bool isDoConcurrent = outerDoConstruct.IsDoConcurrent();
+
+ if (isDoConcurrent) {
+ locs.push_back(converter.genLocation(
+ Fortran::parser::FindSourceLocation(outerDoConstruct)));
+ const Fortran::parser::LoopControl *loopControl =
+ &*outerDoConstruct.GetLoopControl();
+ const auto &concurrent =
+ std::get<Fortran::parser::LoopControl::Concurrent>(loopControl->u);
+ if (!std::get<std::list<Fortran::parser::LocalitySpec>>(concurrent.t)
+ .empty())
+ TODO(currentLocation, "DO CONCURRENT with locality spec inside ACC");
+
+ const auto &concurrentHeader =
+ std::get<Fortran::parser::ConcurrentHeader>(concurrent.t);
+ const auto &controls =
+ std::get<std::list<Fortran::parser::ConcurrentControl>>(
+ concurrentHeader.t);
+ for (const auto &control : controls) {
+ lowerbounds.push_back(fir::getBase(converter.genExprValue(
+ *Fortran::semantics::GetExpr(std::get<1>(control.t)), stmtCtx)));
+ upperbounds.push_back(fir::getBase(converter.genExprValue(
+ *Fortran::semantics::GetExpr(std::get<2>(control.t)), stmtCtx)));
+ if (const auto &expr =
+ std::get<std::optional<Fortran::parser::ScalarIntExpr>>(
+ control.t))
+ steps.push_back(fir::getBase(converter.genExprValue(
+ *Fortran::semantics::GetExpr(*expr), stmtCtx)));
+ else // If `step` is not present, assume it is `1`.
+ steps.push_back(builder.createIntegerConstant(
+ currentLocation, upperbounds[upperbounds.size() - 1].getType(), 1));
+
+ const auto &name = std::get<Fortran::parser::Name>(control.t);
+ privatizeIv(converter, *name.symbol, currentLocation, ivTypes, ivLocs,
+ privateOperands, ivPrivate, privatizationRecipes,
+ isDoConcurrent);
+
+ inclusiveBounds.push_back(true);
+ }
+ } else {
+ for (uint64_t i = 0; i < loopsToProcess; ++i) {
+ const Fortran::parser::LoopControl *loopControl;
+ if (i == 0) {
+ loopControl = &*outerDoConstruct.GetLoopControl();
+ locs.push_back(converter.genLocation(
+ Fortran::parser::FindSourceLocation(outerDoConstruct)));
+ } else {
+ auto *doCons = crtEval->getIf<Fortran::parser::DoConstruct>();
+ assert(doCons && "expect do construct");
+ loopControl = &*doCons->GetLoopControl();
+ locs.push_back(converter.genLocation(
+ Fortran::parser::FindSourceLocation(*doCons)));
+ }
+
+ const Fortran::parser::LoopControl::Bounds *bounds =
+ std::get_if<Fortran::parser::LoopControl::Bounds>(&loopControl->u);
+ assert(bounds && "Expected bounds on the loop construct");
+ lowerbounds.push_back(fir::getBase(converter.genExprValue(
+ *Fortran::semantics::GetExpr(bounds->lower), stmtCtx)));
+ upperbounds.push_back(fir::getBase(converter.genExprValue(
+ *Fortran::semantics::GetExpr(bounds->upper), stmtCtx)));
+ if (bounds->step)
+ steps.push_back(fir::getBase(converter.genExprValue(
+ *Fortran::semantics::GetExpr(bounds->step), stmtCtx)));
+ else // If `step` is not present, assume it is `1`.
+ steps.push_back(builder.createIntegerConstant(
+ currentLocation, upperbounds[upperbounds.size() - 1].getType(), 1));
+
+ Fortran::semantics::Symbol &ivSym =
+ bounds->name.thing.symbol->GetUltimate();
+ privatizeIv(converter, ivSym, currentLocation, ivTypes, ivLocs,
+ privateOperands, ivPrivate, privatizationRecipes);
+
+ inclusiveBounds.push_back(true);
+
+ if (i < loopsToProcess - 1)
+ crtEval = &*std::next(crtEval->getNestedEvaluations().begin());
+ }
+ }
+}
+
+static mlir::acc::LoopOp
+buildACCLoopOp(Fortran::lower::AbstractConverter &converter,
+ mlir::Location currentLocation,
+ Fortran::semantics::SemanticsContext &semanticsContext,
+ Fortran::lower::StatementContext &stmtCtx,
+ const Fortran::parser::DoConstruct &outerDoConstruct,
+ Fortran::lower::pft::Evaluation &eval,
+ llvm::SmallVector<mlir::Value> &privateOperands,
+ llvm::SmallVector<mlir::Attribute> &privatizationRecipes,
+ llvm::SmallVector<mlir::Value> &gangOperands,
+ llvm::SmallVector<mlir::Value> &workerNumOperands,
+ llvm::SmallVector<mlir::Value> &vectorOperands,
+ llvm::SmallVector<mlir::Value> &tileOperands,
+ llvm::SmallVector<mlir::Value> &cacheOperands,
+ llvm::SmallVector<mlir::Value> &reductionOperands,
+ llvm::SmallVector<mlir::Type> &retTy, mlir::Value yieldValue,
+ uint64_t loopsToProcess) {
+ fir::FirOpBuilder &builder = converter.getFirOpBuilder();
+
+ llvm::SmallVector<mlir::Value> ivPrivate;
+ llvm::SmallVector<mlir::Type> ivTypes;
+ llvm::SmallVector<mlir::Location> ivLocs;
+ llvm::SmallVector<bool> inclusiveBounds;
+ llvm::SmallVector<mlir::Location> locs;
+ llvm::SmallVector<mlir::Value> lowerbounds, upperbounds, steps;
+
+ // Look at the do/do concurrent loops to extract bounds information.
+ processDoLoopBounds(converter, currentLocation, stmtCtx, builder,
+ outerDoConstruct, eval, lowerbounds, upperbounds, steps,
+ privateOperands, ivPrivate, privatizationRecipes, ivTypes,
+ ivLocs, inclusiveBounds, locs, loopsToProcess);
+
+ // Prepare the operand segment size attribute and the operands value range.
+ llvm::SmallVector<mlir::Value> operands;
+ llvm::SmallVector<int32_t> operandSegments;
+ addOperands(operands, operandSegments, lowerbounds);
+ addOperands(operands, operandSegments, upperbounds);
+ addOperands(operands, operandSegments, steps);
+ addOperands(operands, operandSegments, gangOperands);
+ addOperands(operands, operandSegments, workerNumOperands);
+ addOperands(operands, operandSegments, vectorOperands);
+ addOperands(operands, operandSegments, tileOperands);
+ addOperands(operands, operandSegments, cacheOperands);
+ addOperands(operands, operandSegments, privateOperands);
+ addOperands(operands, operandSegments, reductionOperands);
+
+ auto loopOp = createRegionOp<mlir::acc::LoopOp, mlir::acc::YieldOp>(
+ builder, builder.getFusedLoc(locs), currentLocation, eval, operands,
+ operandSegments, /*outerCombined=*/false, retTy, yieldValue, ivTypes,
+ ivLocs);
+
+ for (auto [arg, value] : llvm::zip(
+ loopOp.getLoopRegions().front()->front().getArguments(), ivPrivate))
+ builder.create<fir::StoreOp>(currentLocation, arg, value);
+
+ loopOp.setInclusiveUpperbound(inclusiveBounds);
+
+ return loopOp;
+}
+
static mlir::acc::LoopOp createLoopOp(
Fortran::lower::AbstractConverter &converter,
mlir::Location currentLocation,
@@ -2150,9 +2313,9 @@ static mlir::acc::LoopOp createLoopOp(
std::nullopt,
bool needEarlyReturnHandling = false) {
fir::FirOpBuilder &builder = converter.getFirOpBuilder();
- llvm::SmallVector<mlir::Value> tileOperands, privateOperands, ivPrivate,
+ llvm::SmallVector<mlir::Value> tileOperands, privateOperands,
reductionOperands, cacheOperands, vectorOperands, workerNumOperands,
- gangOperands, lowerbounds, upperbounds, steps;
+ gangOperands;
llvm::SmallVector<mlir::Attribute> privatizationRecipes, reductionRecipes;
llvm::SmallVector<int32_t> tileOperandsSegments, gangOperandsSegments;
llvm::SmallVector<int64_t> collapseValues;
@@ -2321,107 +2484,6 @@ static mlir::acc::LoopOp createLoopOp(
}
}
- llvm::SmallVector<mlir::Type> ivTypes;
- llvm::SmallVector<mlir::Location> ivLocs;
- llvm::SmallVector<bool> inclusiveBounds;
- llvm::SmallVector<mlir::Location> locs;
- locs.push_back(currentLocation); // Location of the directive
- Fortran::lower::pft::Evaluation *crtEval = &eval.getFirstNestedEvaluation();
- bool isDoConcurrent = outerDoConstruct.IsDoConcurrent();
- if (isDoConcurrent) {
- locs.push_back(converter.genLocation(
- Fortran::parser::FindSourceLocation(outerDoConstruct)));
- const Fortran::parser::LoopControl *loopControl =
- &*outerDoConstruct.GetLoopControl();
- const auto &concurrent =
- std::get<Fortran::parser::LoopControl::Concurrent>(loopControl->u);
- if (!std::get<std::list<Fortran::parser::LocalitySpec>>(concurrent.t)
- .empty())
- TODO(currentLocation, "DO CONCURRENT with locality spec");
-
- const auto &concurrentHeader =
- std::get<Fortran::parser::ConcurrentHeader>(concurrent.t);
- const auto &controls =
- std::get<std::list<Fortran::parser::ConcurrentControl>>(
- concurrentHeader.t);
- for (const auto &control : controls) {
- lowerbounds.push_back(fir::getBase(converter.genExprValue(
- *Fortran::semantics::GetExpr(std::get<1>(control.t)), stmtCtx)));
- upperbounds.push_back(fir::getBase(converter.genExprValue(
- *Fortran::semantics::GetExpr(std::get<2>(control.t)), stmtCtx)));
- if (const auto &expr =
- std::get<std::optional<Fortran::parser::ScalarIntExpr>>(
- control.t))
- steps.push_back(fir::getBase(converter.genExprValue(
- *Fortran::semantics::GetExpr(*expr), stmtCtx)));
- else // If `step` is not present, assume it is `1`.
- steps.push_back(builder.createIntegerConstant(
- currentLocation, upperbounds[upperbounds.size() - 1].getType(), 1));
-
- const auto &name = std::get<Fortran::parser::Name>(control.t);
- privatizeIv(converter, *name.symbol, currentLocation, ivTypes, ivLocs,
- privateOperands, ivPrivate, privatizationRecipes,
- isDoConcurrent);
-
- inclusiveBounds.push_back(true);
- }
- } else {
- int64_t loopCount =
- Fortran::lower::getLoopCountForCollapseAndTile(accClauseList);
- for (unsigned i = 0; i < loopCount; ++i) {
- const Fortran::parser::LoopControl *loopControl;
- if (i == 0) {
- loopControl = &*outerDoConstruct.GetLoopControl();
- locs.push_back(converter.genLocation(
- Fortran::parser::FindSourceLocation(outerDoConstruct)));
- } else {
- auto *doCons = crtEval->getIf<Fortran::parser::DoConstruct>();
- assert(doCons && "expect do construct");
- loopControl = &*doCons->GetLoopControl();
- locs.push_back(converter.genLocation(
- Fortran::parser::FindSourceLocation(*doCons)));
- }
-
- const Fortran::parser::LoopControl::Bounds *bounds =
- std::get_if<Fortran::parser::LoopControl::Bounds>(&loopControl->u);
- assert(bounds && "Expected bounds on the loop construct");
- lowerbounds.push_back(fir::getBase(converter.genExprValue(
- *Fortran::semantics::GetExpr(bounds->lower), stmtCtx)));
- upperbounds.push_back(fir::getBase(converter.genExprValue(
- *Fortran::semantics::GetExpr(bounds->upper), stmtCtx)));
- if (bounds->step)
- steps.push_back(fir::getBase(converter.genExprValue(
- *Fortran::semantics::GetExpr(bounds->step), stmtCtx)));
- else // If `step` is not present, assume it is `1`.
- steps.push_back(builder.createIntegerConstant(
- currentLocation, upperbounds[upperbounds.size() - 1].getType(), 1));
-
- Fortran::semantics::Symbol &ivSym =
- bounds->name.thing.symbol->GetUltimate();
- privatizeIv(converter, ivSym, currentLocation, ivTypes, ivLocs,
- privateOperands, ivPrivate, privatizationRecipes);
-
- inclusiveBounds.push_back(true);
-
- if (i < loopCount - 1)
- crtEval = &*std::next(crtEval->getNestedEvaluations().begin());
- }
- }
-
- // Prepare the operand segment size attribute and the operands value range.
- llvm::SmallVector<mlir::Value> operands;
- llvm::SmallVector<int32_t> operandSegments;
- addOperands(operands, operandSegments, lowerbounds);
- addOperands(operands, operandSegments, upperbounds);
- addOperands(operands, operandSegments, steps);
- addOperands(operands, operandSegments, gangOperands);
- addOperands(operands, operandSegments, workerNumOperands);
- addOperands(operands, operandSegments, vectorOperands);
- addOperands(operands, operandSegments, tileOperands);
- addOperands(operands, operandSegments, cacheOperands);
- addOperands(operands, operandSegments, privateOperands);
- addOperands(operands, operandSegments, reductionOperands);
-
llvm::SmallVector<mlir::Type> retTy;
mlir::Value yieldValue;
if (needEarlyReturnHandling) {
@@ -2430,16 +2492,13 @@ static mlir::acc::LoopOp createLoopOp(
retTy.push_back(i1Ty);
}
- auto loopOp = createRegionOp<mlir::acc::LoopOp, mlir::acc::YieldOp>(
- builder, builder.getFusedLoc(locs), currentLocation, eval, operands,
- operandSegments, /*outerCombined=*/false, retTy, yieldValue, ivTypes,
- ivLocs);
-
- for (auto [arg, value] : llvm::zip(
- loopOp.getLoopRegions().front()->front().getArguments(), ivPrivate))
- builder.create<fir::StoreOp>(currentLocation, arg, value);
-
- loopOp.setInclusiveUpperbound(inclusiveBounds);
+ uint64_t loopsToProcess =
+ Fortran::lower::getLoopCountForCollapseAndTile(accClauseList);
+ auto loopOp = buildACCLoopOp(
+ converter, currentLocation, semanticsContext, stmtCtx, outerDoConstruct,
+ eval, privateOperands, privatizationRecipes, gangOperands,
+ workerNumOperands, vectorOperands, tileOperands, cacheOperands,
+ reductionOperands, retTy, yieldValue, loopsToProcess);
if (!gangDeviceTypes.empty())
loopOp.setGangAttr(builder.getArrayAttr(gangDeviceTypes));
@@ -4891,6 +4950,12 @@ bool Fortran::lower::isInOpenACCLoop(fir::FirOpBuilder &builder) {
return false;
}
+bool Fortran::lower::isInsideOpenACCComputeCo...
[truncated]
|
You can test this locally with the following command:git-clang-format --diff HEAD~1 HEAD --extensions cpp,h -- flang/include/flang/Lower/OpenACC.h flang/lib/Lower/Bridge.cpp flang/lib/Lower/OpenACC.cppView the diff from clang-format here.diff --git a/flang/include/flang/Lower/OpenACC.h b/flang/include/flang/Lower/OpenACC.h
index 8b13ce94f..e974f3d6e 100644
--- a/flang/include/flang/Lower/OpenACC.h
+++ b/flang/include/flang/Lower/OpenACC.h
@@ -124,7 +124,8 @@ uint64_t getLoopCountForCollapseAndTile(const Fortran::parser::AccClauseList &);
/// Checks whether the current insertion point is inside OpenACC loop.
bool isInOpenACCLoop(fir::FirOpBuilder &);
-/// Checks whether the current insertion point is inside OpenACC compute construct.
+/// Checks whether the current insertion point is inside OpenACC compute
+/// construct.
bool isInsideOpenACCComputeConstruct(fir::FirOpBuilder &);
void setInsertionPointAfterOpenACCLoopIfInside(fir::FirOpBuilder &);
@@ -139,8 +140,7 @@ mlir::Operation *genOpenACCLoopFromDoConstruct(
AbstractConverter &converter,
Fortran::semantics::SemanticsContext &semanticsContext,
Fortran::lower::SymMap &localSymbols,
- const Fortran::parser::DoConstruct &doConstruct,
- pft::Evaluation &eval);
+ const Fortran::parser::DoConstruct &doConstruct, pft::Evaluation &eval);
} // namespace lower
} // namespace Fortran
diff --git a/flang/lib/Lower/Bridge.cpp b/flang/lib/Lower/Bridge.cpp
index 80f9a08bd..01eebd0e7 100644
--- a/flang/lib/Lower/Bridge.cpp
+++ b/flang/lib/Lower/Bridge.cpp
@@ -2170,9 +2170,8 @@ private:
// Loops with induction variables inside OpenACC compute constructs
// need special handling to ensure that the IVs are privatized.
if (Fortran::lower::isInsideOpenACCComputeConstruct(*builder)) {
- mlir::Operation* loopOp = Fortran::lower::genOpenACCLoopFromDoConstruct(
- *this, bridge.getSemanticsContext(), localSymbols,
- doConstruct, eval);
+ mlir::Operation *loopOp = Fortran::lower::genOpenACCLoopFromDoConstruct(
+ *this, bridge.getSemanticsContext(), localSymbols, doConstruct, eval);
bool success = loopOp != nullptr;
if (success) {
// Sanity check that the builder insertion point is inside the newly
@@ -2184,7 +2183,8 @@ private:
// Loop body code.
auto iter = eval.getNestedEvaluations().begin();
- for (auto end = --eval.getNestedEvaluations().end(); iter != end; ++iter)
+ for (auto end = --eval.getNestedEvaluations().end(); iter != end;
+ ++iter)
genFIR(*iter, unstructuredContext);
return;
}
diff --git a/flang/lib/Lower/OpenACC.cpp b/flang/lib/Lower/OpenACC.cpp
index 950b02501..5f58ce02c 100644
--- a/flang/lib/Lower/OpenACC.cpp
+++ b/flang/lib/Lower/OpenACC.cpp
@@ -35,8 +35,8 @@
#include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h"
#include "mlir/IR/MLIRContext.h"
#include "mlir/Support/LLVM.h"
-#include "llvm/ADT/ScopeExit.h"
#include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/ScopeExit.h"
#include "llvm/Frontend/OpenACC/ACC.h.inc"
#include "llvm/Support/CommandLine.h"
#include "llvm/Support/Debug.h"
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When OpenACC is enabled and Fortran loops are annotated with
acc loop, they are lowered toacc.loopoperation. And rest of the contained loops use the normal FIR lowering path.Hovever, the OpenACC specification has special provisions related to contained loops and their induction variable. In order to adhere to this, we convert all valid contained loops to
acc.loopin order to store this information appropriately.The provisions in the spec that motivated this change (line numbers are from OpenACC 3.4):
By valid loops - we convert do loops and do concurrent loops which have induction variable. Loops which are unstructured are not handled.