Comma required at the end of \pgfkeys

[ilyaza]

Brief outline of the bug

Without comma at the end of \pgfkeys, the MWE produces:

  ! Extra }, or forgotten \endgroup.

(probably due to \unskip being redefined to become closing-brace). Checked with tl2022 (in unknown state) and tl2023 (in pristine state).

Background: I have a large (a few K-LoC) system of macros based on PGF/TiKZ in use for about a decade, and a few aspects of its operation are quite flaky. (In total, it is probably several man-days lost on trying to debug this.)

This bug seems to be the root cause of ALL the flakiness I observe.

Example of a similar trace

In this shortened form, the traces of the runs with/without the comma are very different. However, in the process of isolation there was one version where the diffs contained a small standalone fragment (paraphrased below by removing irrelevant part). Essentially, at end of the argument-to-\pgfkeys, \unskip is redefined to become close-brace:

@@ -500161,753 +500240,55 @@ edef \pgfkeyscurrentkey {\pgfkeyscurrent
 mpty \else \pgfkeys@add@path@as@needed \pgfkeys@spdef \pgfkeyscurrentvalue {#2}
 \ifx \pgfkeyscurrentvalue \pgfkeysnovalue@text \pgfkeysifdefined {\pgfkeyscurre
 ntkey /.@def}{\pgfkeysgetvalue {\pgfkeyscurrentkey /.@def}{\pgfkeyscurrentvalue
  }} {}\fi \ifx \pgfkeyscurrentvalue \pgfkeysvaluerequired \def \pgf@marshal {\p
 gfkeysvalueof {/errors/value required/.@cmd}}\expandafter \pgf@marshal \expanda
 fter {\pgfkeyscurrentkey }{}\pgfeov \else \pgfkeys@case@one \fi \fi 
 #1<-every scope/.try
 #2<-\pgfkeysnovalue 
 #3<-
 
 \pgfkeys@spdef #1#2->\futurelet \pgfkeys@possiblespace \pgfkeys@sp@a #2\pgfkeys
 @stop \pgfkeys@stop  \pgfkeys@stop \relax #1
 #1<-\pgfkeyscurrentkey 
 #2<-every scope/.try
 {\futurelet}
-{changing \unskip=\unskip}
-{into \unskip=end-group character }}
-{\hfil}
-{end-group character }}
-! Extra }, or forgotten \endgroup.
-<recently read> }
-                 
-<template> \unskip \hfil }
-                          \hskip \tabcolsep \endtemplate 
-\pgfkeys@spdef ...uturelet \pgfkeys@possiblespace 
-                                                  \pgfkeys@sp@a #2\pgfkeys@s...
-
-\pgfkeys@unpack ...s@spdef \pgfkeyscurrentkey {#1}
-                                                  \edef \pgfkeyscurrentkey {...
-
-\pgfkeys@@normal ...pgfkeysnovalue =\pgfkeys@stop 
-                                                  \pgfkeys@parse 
-\pgfkeys@@qset ...aultpath {#2/}\pgfkeys@parse #3,
-                                                  \pgfkeys@mainstop \def \pg...
-
-\scope ...y@groupfalse \tikzset {every scope/.try}
-                                                  \tikz@collect@scope@anims ...
-
-\tikz@picture ...{.5}\tikz@installcommands \scope 
-                                                  [every picture,#1]\iftikz@...
-
-\tikz@opt [#1]->\tikzpicture [#1]
-                                 \pgfutil@ifnextchar \bgroup {\tikz@ }{\tikz...
-
-\\Tcircled ...vevmode \tikz [baseline=(char.base)]
-                                                  {\node [circle,#1,inner se...
-
-\NUMcc ... #1>0 \circledAsDivisor {\NUMc {#2}{#3}}
-                                                  \else \NUMc {#2}{#3}\fi 
-\joinedRowsInTable ...}\NUMcc 00{14}&\NUMcc 11{13}
-                                                  &\NUMcc 00{12}&\NUMcc 01{1...
-l.145      \joinedRowsInTable
-                             
-? s
+{changing \pgfkeys@possiblespace=undefined}
+{into \pgfkeys@possiblespace=the letter e}
 
 \pgfkeys@sp@a ->\ifx \pgfkeys@possiblespace \pgfkeys@sptoken \expandafter \pgfk
 eys@sp@b \else \expandafter \pgfkeys@sp@b \expandafter  \fi 
 {\ifx: (level 2) entered on line 145}
 {false}
 {\else: \ifx (level 2) entered on line 145}
 {\expandafter}
 {\expandafter}
 {\fi: \ifx (level 2) entered on line 145}
 
 \pgfkeys@sp@b  #1 \pgfkeys@stop ->\pgfkeys@sp@c #1
 #1<-every scope/.try\pgfkeys@stop \pgfkeys@stop 
 
 \pgfkeys@sp@c #1\pgfkeys@stop #2\relax #3->\pgfkeys@temptoks {#1}\edef #3{\the 
 \pgfkeys@temptoks }

Minimal working example (MWE)

\documentclass{article}
\usepackage{tikz}

\begin{document}
{%    \tracingall  \tracingstacklevels=0
     \pgfkeys{/joinerDefault/.initial=&}%   With comma at end, works.  Otherwise \unskip is redefined to close-brace, breaking:
     \begin{tabular}{c}
         \leavevmode\tikz \node {12};
     \end{tabular}
}
\end{document}

[muzimuzhi]

 \pgfkeys@spdef #1#2->\futurelet \pgfkeys@possiblespace \pgfkeys@sp@a #2\pgfkeys
 @stop \pgfkeys@stop  \pgfkeys@stop \relax #1
 #1<-\pgfkeyscurrentkey 
 #2<-every scope/.try
 {\futurelet}
-{changing \unskip=\unskip}
-{into \unskip=end-group character }}
-{\hfil}
-{end-group character }}
-! Extra }, or forgotten \endgroup.
-<recently read> }
-                 
-<template> \unskip \hfil }
-                          \hskip \tabcolsep \endtemplate 
+{changing \pgfkeys@possiblespace=undefined}
+{into \pgfkeys@possiblespace=the letter e}

It seems the \futurelet used by \tikzset{every scope/.try} in \tikz@scope@env interfered with the tabular preamble.

BTW tested on overleaf.com, your example gave the same error in each of the TeX Live versions ranging from 2014 to 2025. So I assume this is a long-existing issue.

[muzimuzhi]

This is the culprit:

• after \pgfkeys{/joinerDefault/.initial=&} (no extra trailing comma), \pgfkeys@possiblespace is let to &, with catcode 4 (alignment tab)
• after \pgfkeys{/joinerDefault/.initial=&,} (with extra trailing comma), \pgfkeys@possiblespace is undefined

Then, when \pgfkeys@possiblespace is found in a tabular environment, although it should have been used as an internal temp macro, but is treated as an alignment tab character (due to its catcode), thus the low-level tabular error.

For allowing use of catcode 4 tokens in tabulars (directly in a cell), a non-general workaround is to wrap the material in a group of braces. A typical fix is to surround the material in between brace tricks \ifnum`{=0\fi<material with &>\ifnum`}=0\fi. See TeX-SX question "Showcase of brace tricks: }, \egroup, \iffalse{\fi}, etc." for more info. expl3 provides such brace tricks as two functions \group_align_safe_begin: and \group_align_safe_end:.

A simplified example, which loads no packages:

\documentclass{article}

\begin{document}
\let\myAmpersand=&
\ifcat\myAmpersand&T\else F\fi % leaves "T"

% works
\begin{tabular}{c}
  \leavevmode
  \UseName{group_align_safe_begin:}
    \let\myAmpersand=x
  \UseName{group_align_safe_end:}
\end{tabular}

% errors
% ! Extra alignment tab has been changed to \cr.
\begin{tabular}{c}
  \let\myAmpersand=x
\end{tabular}
\end{document}

Although the error raised is different, the cause is the same as that in OP's example.

[muzimuzhi]

To apply the brace-trick fix to \tikz and tikzpicture environment, it seems to me pgf-aliased brace-trick commands can be added to \pgfscope and \endpgfscope.

Update: Only needed by \tikz, because the environment form already enclose content in a real group.

Theoretically, many more tikz and pgf commands which can be used alone, may need a similar patch, especially those which take a list of pgfkeys key-value pairs.

\documentclass{article}
\usepackage{pgfkeys}

\begin{document}
% errors
% ! Argument of \pgfkeys@@normal has an extra }.
\begin{tabular}{c}
  \pgfkeys{keyA/.initial=&, keyB}
\end{tabular}
\end{document}

[muzimuzhi]

Oh your example is also sensitive to the \leavevmode, which, to my limited knowledge to TeX tables, I don't understand why.

[ilyaza]

I do not follow your logic. The error is not

! Extra alignment tab has been changed to \cr.

The error I observe is not with the ampersand catcode 4, but with the close-brace catcode (2 IIRC).

[ilyaza]

BTW, while \let\ampersand=* inside tabular leads to the error above, doing

         \expandafter\let\csname ampersand\endcsname=*%

is safe. I have no clue why.

In particular, all the temporary macros which may be contaminated like this during parsing in \pgfkeys can be safely undone like above at the end of processing the keys.

[ilyaza]

Oups, cannot reproduce my preceding claim any more. Doing it inside tabular does not hide catcode=4 from the scanner… Still:

! Extra alignment tab has been changed to \cr.