Skip to content

EMSUSD-282 - MayaUsd: build with Qt directly from Maya devkit/runtime #3298

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Sep 22, 2023
Merged

EMSUSD-282 - MayaUsd: build with Qt directly from Maya devkit/runtime #3298

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4805a7c
EMSUSD-282 - MayaUsd: build with Qt directly from Maya devkit/runtime
seando-adsk Jul 19, 2023
22f81d4
EMSUSD-282 - MayaUsd: build with Qt directly from Maya devkit/runtime
seando-adsk Aug 28, 2023
8927d6e
EMSUSD-282 - MayaUsd: build with Qt directly from Maya devkit/runtime
seando-adsk Aug 29, 2023
547740d
EMSUSD-282 - MayaUsd: build with Qt directly from Maya devkit/runtime
seando-adsk Aug 30, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 30 additions & 8 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,27 +156,49 @@ endif()

if(DEFINED QT_LOCATION)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still support setting a custom Qt location, but I think after this PR is merged we should consider deprecating this cmake variable and outputting a warning/error if its used. MayaUsd plugin is required to run from Maya which has Qt, so building/using the Qt from Maya is the proper thing and will make sure there are no incompatible libraries.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After this PR is merged I will fix our internal builds (preflight included) to no longer set this cmake variable. And thus our builds will fall into the else below.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dgovil What do you think about this section where a user is providing a custom built Qt to build MayaUsd with. Should I just remove this section and only use my new section below (currently the else). I can't think of a case where someone would want to build with their own Qt anymore now that we can build with the Qt from Maya devkit. I'm thinking of removing this section and outputting a cmake warning if QT_LOCATION is set, saying that flag is deprecated and not needed anymore.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with you. Having a custom Qt build has enough gotchas that I doubt anyone would use it, especially now that they can use the version inside Maya.

if(NOT DEFINED QT_VERSION)
if(BUILD_WITH_QT_6)
if(BUILD_WITH_QT_6)
set(QT_VERSION "6.5")
else() # Maya 2022, 2023, 2024
set(QT_VERSION "5.15")
endif()
message(STATUS "Setting Qt version to Qt ${QT_VERSION}")
endif()
set(CMAKE_PREFIX_PATH "${QT_LOCATION}")

if(BUILD_WITH_QT_6)
find_package(Qt6 ${QT_VERSION} COMPONENTS Core Gui Widgets REQUIRED)
if(Qt6_FOUND)
message(STATUS "Building with Qt ${QT_VERSION} features enabled.")
endif()
else()
find_package(Qt5 ${QT_VERSION} COMPONENTS Core Gui Widgets REQUIRED)
if(Qt5_FOUND)
message(STATUS "Building with Qt ${QT_VERSION} features enabled.")
endif()
endif()
if(Qt5_FOUND OR Qt6_FOUND)
message(STATUS "Found Qt in custom location, building with Qt ${QT_VERSION} features enabled.")
set(Qt_FOUND TRUE)
else()
message(WARNING "Could not find Qt in custom location: ${QT_LOCATION}. Building with Qt features will be disabled.")
endif()
else()
message(STATUS "QT_LOCATION not set. Building Qt features will be disabled.")
# First look for Qt6 in the Maya devkit.
# The Qt6 archive in the Maya devkit contains everything needed for the normal cmake find_package.
set(CMAKE_PREFIX_PATH "${MAYA_DEVKIT_LOCATION}/Qt")
find_package(Qt6 6.5 COMPONENTS Core Gui Widgets QUIET)
Comment on lines +158 to +159
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Maya devkit has been "fixed" for the switch to Qt6 by including a Qt zip that has all of Qt packaged up for use by cmake builds. All we need to do now is set the CMAKE_PREFIX_PATH and then call the normal find_package.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use QUIET here since this find is allowed to fail, meaning there is no Qt6 and we'll fall below and search for Qt5.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also relies on the new Qt.tar.gz file being extracted this I do in build.py.

if (Qt6_FOUND)
message(STATUS "Found Qt6 in Maya devkit, building with Qt ${Qt6_VERSION} features enabled.")
set(BUILD_WITH_QT_6 ON)
set(Qt_FOUND TRUE)
else()
# If we didn't find Qt6 in Maya devkit, search again, but for Qt5 this time.
# Minimum version required is 5.15 (Maya 2022/2023/2024).
# This will find Qt in the Maya devkit using a custom find package.
# So the version will match Maya we are building for.
find_package(Maya_Qt 5.15 COMPONENTS Core Gui Widgets QUIET)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For older Maya using Qt 5, I took the cmake rules that Dhruv wrote for issue #1070 and created a new find "Maya_Qt" module. This also relies on qt_5*-include.{zip/tar.gz} being extracted which I do in build.py. I again used QUIET here which means if the user building isn't using our build.py and hasn't extracted the qt zip file it will fail to find Qt and will output a warning message (below).

if (Maya_Qt_FOUND)
message(STATUS "Found Qt5 in Maya devkit, building with Qt ${MAYA_QT_VERSION} features enabled.")
set(BUILD_WITH_QT_6 OFF)
set(Qt_FOUND TRUE)
else()
message(WARNING "Could not find Qt in Maya devkit directory: ${MAYA_DEVKIT_LOCATION}. Building with Qt features will be disabled.")
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about warning vs error here? I am open to changing this to error which means you either use our build.py (which automatically extracts qt for you) or you have to manually extract Qt.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think an error would be better here. Now that it can find the Qt from the Maya versions, I don't see a scenario where someone would prefer to build without it (and also can't just modify the file).
I think an error would prevent any quiet discovery issues

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dgovil Thanks I agree. I've removed the QT_LOCATION flag (no need to support building with custom Qt anymore) and output cmake error if Qt cannot be found in Maya devkit.

endif()
endif()
endif()

#------------------------------------------------------------------------------
Expand Down
116 changes: 107 additions & 9 deletions build.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
from __future__ import print_function

from distutils.spawn import find_executable
from glob import glob

import argparse
import contextlib
Expand All @@ -18,6 +19,8 @@
import sys
import distutils.util
import time
import zipfile
import tarfile

############################################################
# Helpers for printing output
Expand Down Expand Up @@ -359,6 +362,99 @@ def RunMakeZipArchive(context):
PrintError("Failed to write to directory {pkgDir} : {exp}".format(pkgDir=pkgDir,exp=exp))
sys.exit(1)

def SetupMayaQt(context):
def haveQtHeaders(rootPath):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Helper function to search (recursively) to find the qt headers (only the 3 components we use). If found then we know the qt zip was extracted.

if os.path.exists(rootPath):
# MayaUsd uses these components from Qt (so at a minimum we must find them).
qtComponentsToFind = ['QtCore', 'QtGui', 'QtWidgets']
# Qt6 includes the entire Qt in a single zip file, which when extracted ends in folder 'Qt'.
startDir = os.path.join(rootPath, 'Qt', 'include') if os.path.exists(os.path.join(rootPath, 'Qt')) else os.path.join(rootPath, 'include')
for root,dirs,files in os.walk(startDir):
if 'qt' not in root.lower() or not files:
continue
if not any(root.endswith(qtComp) for qtComp in qtComponentsToFind):
# Skip any folders that aren't the components we are looking for.
continue

for qtComp in qtComponentsToFind[:]: # Loop over slice copy as we remove items
if qtComp in root and '{comp}version.h'.format(comp=qtComp.lower()) in files:
qtComponentsToFind.remove(qtComp)
PrintInfo('Found {comp} in {dir}'.format(comp=qtComp, dir=root))
break # Once we've found (and removed) a component, move to the next os.walk

if not qtComponentsToFind: # Once we've found them all, we are done.
return True

# The list of directories (in order) that we'll search. This list matches the one
# in FindMayaQt.cmake.
dirsToSearch = [context.devkitLocation]
if 'MAYA_DEVKIT_LOCATION' in os.environ:
dirsToSearch.append(os.path.expandvars('$MAYA_DEVKIT_LOCATION'))
dirsToSearch.append(context.mayaLocation)
if 'MAYA_LOCATION' in os.environ:
dirsToSearch.append(os.path.expandvars('$MAYA_LOCATION'))
Comment on lines +416 to +421
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Various places Qt could be found, but it will most likely always be found in the first entry which is the Maya devkit location.


# Check if the Qt zip file has been extracted (we need the Qt headers).
for dirToSearch in dirsToSearch:
if haveQtHeaders(dirToSearch):
PrintStatus('Found Maya Qt headers in: {dir}'.format(dir=dirToSearch))
return
Comment on lines +423 to +427
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Search thru all the locations and try to find the Qt headers, if we do return - no extra step needed.


# Qt5
# Didn't find Qt headers, so try and extract qt_5.x-include.zip (the first one we find).
qtIncludeArchiveName = 'qt_5*-include.zip' if Windows() else 'qt_5*-include.tar.gz'
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Qt5 we only need to extract the headers.

for dirToSearch in dirsToSearch:
qtZipFiles = glob(os.path.join(dirToSearch, 'include', qtIncludeArchiveName))
if qtZipFiles:
qtZipFile = qtZipFiles[0]
baseDir = os.path.dirname(qtZipFile)
if os.access(baseDir, os.W_OK):
qtZipDirName = os.path.basename(qtZipFile)
qtZipDirName = qtZipDirName.replace('.zip', '').replace('.tar.gz', '')
qtZipDirFolder = os.path.join(baseDir, qtZipDirName)
PrintStatus("Could not find Maya Qt headers.")
PrintStatus(" Extracting '{zip}' to '{dir}'".format(zip=qtZipFile, dir=qtZipDirFolder))
if not os.path.exists(qtZipDirFolder):
os.makedirs(os.path.join(baseDir, qtZipDirFolder))
try:
# We only need certain Qt components so we only extract the headers for those.
if Windows():
archive = zipfile.ZipFile(qtZipFile, mode='r')
files = [n for n in archive.namelist()
if (n.startswith('QtCore/') or n.startswith('QtGui/') or n.startswith('QtWidgets/'))]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm only extracting the headers for the 3 components we use.

else:
archive = tarfile.open(qtZipFile, mode='r')
files = [n for n in archive.getmembers()
if (n.name.startswith('./QtCore/') or n.name.startswith('./QtGui/') or n.name.startswith('./QtWidgets/'))]
archive.extractall(qtZipDirFolder, files)
archive.close()
except zipfile.BadZipfile as error:
PrintError(str(error))
except tarfile.TarError as error:
PrintError(str(error))

# We found and extracted the Qt5 include zip - we are done.
return

# Qt6
# The entire Qt is in a single zip file, which we extract to 'Qt'.
# Then we can simply use find_package(Qt6) on it.
for dirToSearch in dirsToSearch:
# Qt archive has same name on all platforms.
qtArchive = os.path.join(dirToSearch, 'Qt.tar.gz')
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Qt6 extract the entire archive. This one is new and contains everything needed to build with Qt.

if os.path.exists(qtArchive):
qtZipDirFolder = os.path.dirname(qtArchive)
if os.access(qtZipDirFolder, os.W_OK):
PrintStatus("Could not find Maya Qt6.")
PrintStatus(" Extracting '{zip}' to '{dir}'".format(zip=qtArchive, dir=qtZipDirFolder))
try:
archive = tarfile.open(qtArchive, mode='r')
archive.extractall(qtZipDirFolder)
archive.close()
except tarfile.TarError as error:
PrintError(str(error))
return

def BuildAndInstall(context, buildArgs, stages):
with CurrentWorkingDirectory(context.mayaUsdSrcDir):
extraArgs = []
Expand Down Expand Up @@ -389,10 +485,6 @@ def BuildAndInstall(context, buildArgs, stages):
else:
extraArgs.append('-DMAYAUSD_DEFINE_BOOST_DEBUG_PYTHON_FLAG=OFF')

if context.qtLocation:
extraArgs.append('-DQT_LOCATION="{qtLocation}"'
.format(qtLocation=context.qtLocation))

extraArgs += buildArgs
stagesArgs += stages

Expand Down Expand Up @@ -477,7 +569,7 @@ def Package(context):
help="Define Boost Python Debug if your Python library comes with Debugging symbols (default: %(default)s).")

parser.add_argument("--qt-location", type=str,
help="Directory where Qt is installed.")
help="DEPRECATED: Qt is found automatically in Maya devkit.")

parser.add_argument("--build-args", type=str, nargs="*", default=[],
help=("Comma-separated list of arguments passed into CMake when building libraries"))
Expand All @@ -486,7 +578,7 @@ def Package(context):
help=("Comma-separated list of arguments passed into CTest.(e.g -VV, --output-on-failure)"))

parser.add_argument("--stages", type=str, nargs="*", default=['clean','configure','build','install'],
help=("Comma-separated list of stages to execute.(possible stages: clean, configure, build, install, test, package)"))
help=("Comma-separated list of stages to execute. Possible stages: clean, configure, build, install, test, package."))

parser.add_argument("-j", "--jobs", type=int, default=GetCPUCount(),
help=("Number of build jobs to run in parallel. "
Expand Down Expand Up @@ -550,9 +642,11 @@ def __init__(self, args):
self.devkitLocation = (os.path.abspath(args.devkit_location)
if args.devkit_location else None)

# Qt Location
self.qtLocation = (os.path.abspath(args.qt_location)
if args.qt_location else None)
# DEPRECATED: Qt Location
if args.qt_location:
PrintError("--qt-location flag is deprecated as Qt is found automatically in Maya devkit.")
PrintError("If you want to use custom Qt, use flag: --build-args=-DQT_LOCATION=<dir>")
sys.exit(1)

# MaterialX
self.materialxEnabled = args.build_materialx
Expand Down Expand Up @@ -632,6 +726,10 @@ def __init__(self, args):

Print(summaryMsg)

# Make sure Qt from Maya devkit is ready.
if 'configure' in context.stagesArgs:
SetupMayaQt(context)

# BuildAndInstall
if any(stage in ['clean', 'configure', 'build', 'install'] for stage in context.stagesArgs):
StartBuild()
Expand Down
Loading