Skip to content

Commit 29019c9

Browse files
alcroitoalcroito
committed
CMake: Rewrite init-repository using CMake and .sh / .bat scripts
init-repository is now implemented using CMake + .sh / .bat scripts. The intent behind the change is not to require Perl to checkout and build Qt, because it can be troublesome to acquire on Windows and it can also lead to issues during builds due to CMake picking up a Perl distribution-shipped compiler. All previous options were ported over like - module-subset - alternates - etc. A few new options were added: - --resolve-deps / --no-resolve-deps - --optional-deps / --no-optional-deps - --verbose and some other internal ones for testing reasons. The new script does automatic resolving of dependencies based on the depends / recommends keys in .gitmodules unless --no-resolve-deps is passed. So if you configure with --module-subset=qtsvg, the script will also initialize qtbase. If --no-optional-deps is passed, only required dependencies ('depends' ky) will be included and optional dependencies ('recommends' key) will be excluded. The new script now has a new default behavior when calling init-repository a second time with --force, without specifying a --module-subset option. Instead of initializing all submodules, it will just update the existing / previously initialized submodules. It also understands a new module-subset keyword "existing", which expands to the previously initialized submodules, so someone can initialize an additional submodule by calling init-repository -f --module-subset=existing,qtsvg Implementation notes: The overall code flow is init-repository -> cmake/QtIRScript.cmake -> qt_ir_run_main_script -> qt_ir_run_after_args_parsed -> qt_ir_handle_init_submodules (recursive) -> qt_ir_clone_one_submodule with some bells and whistles on the side. The command line parsing is an adapted copy of the functions in qtbase/cmake/QtProcessConfigureArgs.cmake. We can't use those exact functions because qtbase is not available when init-repository is initially called, and force cloning qtbase was deemed undesirable. We also have a new mechanism to detect whether init-repository was previously called. The perl script used the existence of the qtbase submodule as the check. In the cmake script, we instead set a custom marker into the local repo config file. Otherwise the code logic should be a faithful reimplementation of init-repository.pl aside from some small things like logging and progress reporting. The pre-existing git cloning logic in QtTopLevelHelpers was not used because it would not be compatible with the alternates option and I didn't want to accidentally break the pre-existing code. Plus init-repository is a bit opinionated about how it clones and checks out repos. The dependency collection and sorting logic uses the pre-existing code though. See follow up commit about implicitly calling init-repository when qt5/configure is called and the repo was not initialized before. [ChangeLog][General] init-repository was rewritten using CMake. Perl is no longer required to initialize the qt5.git super repo. Task-number: QTBUG-120030 Task-number: QTBUG-122622 Change-Id: Ibc38ab79d3fdedd62111ebbec496eabd64c20d2b Reviewed-by: Alexey Edelev <[email protected]> Reviewed-by: Joerg Bornemann <[email protected]> Reviewed-by: Tor Arne Vestbø <[email protected]>
1 parent 1971e80 commit 29019c9

13 files changed

+2460
-10
lines changed

Diff for: ‎.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,5 @@ CMakeUserPresets.json
88
build
99
build-*
1010
.DS_Store
11+
init-repository.opt
12+
init-repository.opt.in

Diff for: ‎cmake/QtIRCommandLineHelpers.cmake

+369
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,369 @@
1+
# Copyright (C) 2024 The Qt Company Ltd.
2+
# SPDX-License-Identifier: BSD-3-Clause
3+
4+
# This file contains a modified subset of the qtbase/QtProcessConfigureArgs.cmake commands
5+
# with renamed functions, because we need similar logic for init-repository, but
6+
# we can't access qtbase before we clone it.
7+
8+
# Call a function with the given arguments.
9+
function(qt_ir_call_function func)
10+
set(call_code "${func}(")
11+
math(EXPR n "${ARGC} - 1")
12+
foreach(i RANGE 1 ${n})
13+
string(APPEND call_code "\"${ARGV${i}}\" ")
14+
endforeach()
15+
string(APPEND call_code ")")
16+
string(REPLACE "\\" "\\\\" call_code "${call_code}")
17+
if(${CMAKE_VERSION} VERSION_LESS "3.18.0")
18+
set(incfile qt_tmp_func_call.cmake)
19+
file(WRITE "${incfile}" "${call_code}")
20+
include(${incfile})
21+
file(REMOVE "${incfile}")
22+
else()
23+
cmake_language(EVAL CODE "${call_code}")
24+
endif()
25+
endfunction()
26+
27+
# Show an error.
28+
function(qt_ir_add_error)
29+
message(FATAL_ERROR ${ARGV})
30+
endfunction()
31+
32+
# Check if there are still unhandled command line arguments.
33+
function(qt_ir_args_has_next_command_line_arg out_var)
34+
qt_ir_get_unhandled_args(args)
35+
36+
list(LENGTH args n)
37+
if(n GREATER 0)
38+
set(result TRUE)
39+
else()
40+
set(result FALSE)
41+
endif()
42+
set(${out_var} ${result} PARENT_SCOPE)
43+
endfunction()
44+
45+
# Get the next unhandled command line argument without popping it.
46+
function(qt_ir_args_peek_next_command_line_arg out_var)
47+
qt_ir_get_unhandled_args(args)
48+
list(GET args 0 result)
49+
set(${out_var} ${result} PARENT_SCOPE)
50+
endfunction()
51+
52+
# Get the next unhandled command line argument.
53+
function(qt_ir_args_get_next_command_line_arg out_var)
54+
qt_ir_get_unhandled_args(args)
55+
list(POP_FRONT args result)
56+
qt_ir_set_unhandled_args("${args}")
57+
set(${out_var} ${result} PARENT_SCOPE)
58+
endfunction()
59+
60+
# Helper macro to parse the arguments for the command line options.
61+
macro(qt_ir_commandline_option_parse_arguments)
62+
set(options UNSUPPORTED)
63+
set(oneValueArgs TYPE NAME SHORT_NAME ALIAS VALUE DEFAULT_VALUE)
64+
set(multiValueArgs VALUES MAPPING)
65+
cmake_parse_arguments(arg "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})
66+
endmacro()
67+
68+
# We use this to define the command line options that init-repository accepts.
69+
# Arguments
70+
# name - name of the long form option
71+
# e.g. 'module-subset' will parse '--module-subset'
72+
# UNSUPPORTED - mark the option as unsupported in the cmake port of init-repository,
73+
# which means we will fall back to calling the perl script instead
74+
# TYPE - the type of the option, currently we support boolean, string and void
75+
# VALUE - the value to be set for a 'void' type option
76+
# VALUES - the valid values for an option
77+
# MAPPING - currently unused
78+
# SHORT_NAME - an alternative short name flag,
79+
# e.g. 'f' will parse -f for --force
80+
# ALIAS - mark the option as an alias of another option, both will have the
81+
# same value when retrieved.
82+
# DEFAULT_VALUE - the default value to be set for the option when it's not specified
83+
# on the command line
84+
#
85+
# NOTE: Make sure to update the SHORT_NAME code path when adding new options.
86+
function(qt_ir_commandline_option_helper name)
87+
qt_ir_commandline_option_parse_arguments(${ARGN})
88+
89+
set(unsupported_options "${commandline_known_unsupported_options}")
90+
if(arg_UNSUPPORTED)
91+
set(commandline_option_${name}_unsupported
92+
"${arg_UNSUPPORTED}" PARENT_SCOPE)
93+
list(APPEND unsupported_options "${name}")
94+
endif()
95+
set(commandline_known_unsupported_options "${unsupported_options}" PARENT_SCOPE)
96+
97+
set(commandline_known_options
98+
"${commandline_known_options};${name}" PARENT_SCOPE)
99+
100+
set(commandline_option_${name}_type "${arg_TYPE}" PARENT_SCOPE)
101+
102+
if(NOT "${arg_VALUE}" STREQUAL "")
103+
set(commandline_option_${name}_value "${arg_VALUE}" PARENT_SCOPE)
104+
endif()
105+
106+
if(arg_VALUES)
107+
set(commandline_option_${name}_values ${arg_VALUES} PARENT_SCOPE)
108+
elseif(arg_MAPPING)
109+
set(commandline_option_${name}_mapping ${arg_MAPPING} PARENT_SCOPE)
110+
endif()
111+
112+
if(NOT "${arg_SHORT_NAME}" STREQUAL "")
113+
set(commandline_option_${name}_short_name "${arg_SHORT_NAME}" PARENT_SCOPE)
114+
endif()
115+
116+
if(NOT "${arg_ALIAS}" STREQUAL "")
117+
set(commandline_option_${name}_alias "${arg_ALIAS}" PARENT_SCOPE)
118+
endif()
119+
120+
# Should be last, in case alias was specified
121+
if(NOT "${arg_DEFAULT_VALUE}" STREQUAL "")
122+
set(commandline_option_${name}_default_value "${arg_DEFAULT_VALUE}" PARENT_SCOPE)
123+
qt_ir_command_line_set_input("${name}" "${arg_DEFAULT_VALUE}")
124+
endif()
125+
endfunction()
126+
127+
# Defines an option that init-repository understands.
128+
# Uses qt_ir_commandline_option_helper to define both long and short option names.
129+
macro(qt_ir_commandline_option name)
130+
# Define the main option
131+
qt_ir_commandline_option_helper("${name}" ${ARGN})
132+
133+
qt_ir_commandline_option_parse_arguments(${ARGN})
134+
135+
# Define the short name option if it's requested
136+
if(NOT "${arg_SHORT_NAME}" STREQUAL ""
137+
AND "${commandline_option_${arg_SHORT_NAME}_type}" STREQUAL "")
138+
set(unsupported "")
139+
if(arg_UNSUPPORTED)
140+
set(unsupported "${arg_UNSUPPORTED}")
141+
endif()
142+
143+
qt_ir_commandline_option_helper("${arg_SHORT_NAME}"
144+
TYPE "${arg_TYPE}"
145+
ALIAS "${name}"
146+
VALUE "${arg_VALUE}"
147+
VALUES ${arg_VALUES}
148+
MAPPING ${arg_MAPPING}
149+
DEFAULT_VALUE ${arg_DEFAULT_VALUE}
150+
${unsupported}
151+
)
152+
endif()
153+
endmacro()
154+
155+
# Saves the value of a command line option into a global property.
156+
function(qt_ir_command_line_set_input name val)
157+
if(NOT "${commandline_option_${name}_alias}" STREQUAL "")
158+
set(name "${commandline_option_${name}_alias}")
159+
endif()
160+
161+
set_property(GLOBAL PROPERTY _qt_ir_input_${name} "${val}")
162+
set_property(GLOBAL APPEND PROPERTY _qt_ir_inputs ${name})
163+
endfunction()
164+
165+
# Appends a value of a command line option into a global property.
166+
# Currently unused
167+
function(qt_ir_command_line_append_input name val)
168+
if(NOT "${commandline_option_${name}_alias}" STREQUAL "")
169+
set(name "${commandline_option_${name}_alias}")
170+
endif()
171+
172+
get_property(oldval GLOBAL PROPERTY _qt_ir_input_${name})
173+
if(NOT "${oldval}" STREQUAL "")
174+
string(PREPEND val "${oldval};")
175+
endif()
176+
qt_ir_command_line_set_input(${name} "${val}" )
177+
endfunction()
178+
179+
# Checks if the value of a command line option is valid.
180+
function(qt_ir_validate_value opt val out_var)
181+
set(${out_var} TRUE PARENT_SCOPE)
182+
183+
set(valid_values ${commandline_option_${arg}_values})
184+
list(LENGTH valid_values n)
185+
if(n EQUAL 0)
186+
return()
187+
endif()
188+
189+
foreach(v ${valid_values})
190+
if(val STREQUAL v)
191+
return()
192+
endif()
193+
endforeach()
194+
195+
set(${out_var} FALSE PARENT_SCOPE)
196+
list(JOIN valid_values " " valid_values_str)
197+
qt_ir_add_error("Invalid value '${val}' supplied to command line option '${opt}'."
198+
"\nAllowed values: ${valid_values_str}\n")
199+
endfunction()
200+
201+
# Sets / handles the value of a command line boolean option.
202+
function(qt_ir_commandline_boolean arg val nextok)
203+
if("${val}" STREQUAL "")
204+
set(val "yes")
205+
endif()
206+
if(NOT val STREQUAL "yes" AND NOT val STREQUAL "no")
207+
message(FATAL_ERROR
208+
"Invalid value '${val}' given for boolean command line option '${arg}'.")
209+
endif()
210+
qt_ir_command_line_set_input("${arg}" "${val}")
211+
endfunction()
212+
213+
# Sets / handles the value of a command line string option.
214+
function(qt_ir_commandline_string arg val nextok)
215+
if(nextok)
216+
qt_ir_args_get_next_command_line_arg(val)
217+
218+
if("${val}" MATCHES "^-")
219+
qt_ir_add_error("No value supplied to command line options '${arg}'.")
220+
endif()
221+
endif()
222+
qt_ir_validate_value("${arg}" "${val}" success)
223+
if(success)
224+
qt_ir_command_line_set_input("${arg}" "${val}")
225+
endif()
226+
endfunction()
227+
228+
# Sets / handles the value of a command line void option.
229+
# This is an option like --force, which doesn't take any arguments.
230+
# Currently unused
231+
function(qt_ir_commandline_void arg val nextok)
232+
if(NOT "${val}" STREQUAL "")
233+
qt_i_add_error("Command line option '${arg}' expects no argument ('${val}' given).")
234+
endif()
235+
if(DEFINED commandline_option_${arg}_value)
236+
set(val ${commandline_option_${arg}_value})
237+
endif()
238+
if("${val}" STREQUAL "")
239+
set(val yes)
240+
endif()
241+
qt_ir_command_line_set_input("${arg}" "${val}")
242+
endfunction()
243+
244+
# Reads the command line arguments from the optfile_path.
245+
function(qt_ir_get_raw_args_from_optfile optfile_path out_var)
246+
file(STRINGS "${optfile_path}" args)
247+
set(${out_var} "${args}" PARENT_SCOPE)
248+
endfunction()
249+
250+
# Reads the optfile_path, iterates over the given command line arguments,
251+
# sets the input for recongized options.
252+
#
253+
# IGNORE_UNKNOWN_ARGS tells the function not to fail if it encounters an unknown
254+
# option, needed when the script is called from the configure script with
255+
# configure-only-known options.
256+
function(qt_ir_process_args_from_optfile optfile_path)
257+
set(options IGNORE_UNKNOWN_ARGS)
258+
set(oneValueArgs "")
259+
set(multiValueArgs "")
260+
cmake_parse_arguments(arg "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})
261+
262+
qt_ir_get_raw_args_from_optfile("${optfile_path}" configure_args)
263+
qt_ir_set_unhandled_args("${configure_args}")
264+
265+
while(1)
266+
qt_ir_args_has_next_command_line_arg(has_next)
267+
if(NOT has_next)
268+
break()
269+
endif()
270+
qt_ir_args_get_next_command_line_arg(arg)
271+
272+
# parse out opt and val
273+
set(nextok FALSE)
274+
if(arg MATCHES "^--?(disable|no)-(.*)")
275+
set(opt "${CMAKE_MATCH_2}")
276+
set(val "no")
277+
elseif(arg MATCHES "^--([^=]+)=(.*)")
278+
set(opt "${CMAKE_MATCH_1}")
279+
set(val "${CMAKE_MATCH_2}")
280+
elseif(arg MATCHES "^--(.*)")
281+
set(nextok TRUE)
282+
set(opt "${CMAKE_MATCH_1}")
283+
unset(val)
284+
elseif(arg MATCHES "^-(.*)")
285+
set(nextok TRUE)
286+
set(opt "${CMAKE_MATCH_1}")
287+
unset(val)
288+
else()
289+
qt_ir_add_error("Invalid command line parameter '${arg}'.")
290+
endif()
291+
292+
set(type "${commandline_option_${opt}_type}")
293+
294+
if("${type}" STREQUAL "")
295+
if(NOT arg_IGNORE_UNKNOWN_ARGS)
296+
qt_ir_add_error("Unknown command line option '${arg}'.")
297+
else()
298+
message(DEBUG "Unknown command line option '${arg}'. Ignoring.")
299+
continue()
300+
endif()
301+
endif()
302+
303+
if(NOT COMMAND "qt_ir_commandline_${type}")
304+
qt_ir_add_error("Unknown type '${type}' for command line option '${opt}'.")
305+
endif()
306+
qt_ir_call_function("qt_ir_commandline_${type}" "${opt}" "${val}" "${nextok}")
307+
endwhile()
308+
endfunction()
309+
310+
# Shows help for the command line options.
311+
function(qt_ir_show_help)
312+
set(help_file "${CMAKE_CURRENT_LIST_DIR}/QtIRHelp.txt")
313+
if(EXISTS "${help_file}")
314+
file(READ "${help_file}" content)
315+
message("${content}")
316+
endif()
317+
318+
message([[
319+
General Options:
320+
-help, -h ............ Display this help screen
321+
]])
322+
endfunction()
323+
324+
# Gets the unhandled command line args.
325+
function(qt_ir_get_unhandled_args out_var)
326+
get_property(args GLOBAL PROPERTY _qt_ir_unhandled_args)
327+
set(${out_var} "${args}" PARENT_SCOPE)
328+
endfunction()
329+
330+
# Sets the unhandled command line args.
331+
function(qt_ir_set_unhandled_args args)
332+
set_property(GLOBAL PROPERTY _qt_ir_unhandled_args "${args}")
333+
endfunction()
334+
335+
# Gets the unsupported options that init-repository.pl supports, but the cmake port does
336+
# not support.
337+
function(qt_ir_get_unsupported_options out_var)
338+
set(${out_var} "${commandline_known_unsupported_options}" PARENT_SCOPE)
339+
endfunction()
340+
341+
# Get the value of a command line option.
342+
function(qt_ir_get_option_value name out_var)
343+
if(NOT "${commandline_option_${name}_alias}" STREQUAL "")
344+
set(name "${commandline_option_${name}_alias}")
345+
endif()
346+
347+
get_property(value GLOBAL PROPERTY _qt_ir_input_${name})
348+
set(${out_var} "${value}" PARENT_SCOPE)
349+
endfunction()
350+
351+
# Set the value of a command line option manually.
352+
function(qt_ir_set_option_value name value)
353+
if(NOT "${commandline_option_${name}_alias}" STREQUAL "")
354+
set(name "${commandline_option_${name}_alias}")
355+
endif()
356+
357+
qt_ir_command_line_set_input("${name}" "${value}")
358+
endfunction()
359+
360+
# Get the value of a command line option as a cmakke flag option, to be passed
361+
# to functions that use cmake_parse_arguments.
362+
function(qt_ir_get_option_as_cmake_flag_option cli_name cmake_option_name out_var)
363+
qt_ir_get_option_value("${cli_name}" bool_value)
364+
set(cmake_option "")
365+
if(bool_value)
366+
set(cmake_option "${cmake_option_name}")
367+
endif()
368+
set(${out_var} "${cmake_option}" PARENT_SCOPE)
369+
endfunction()

0 commit comments

Comments
 (0)