CUPS does not auto-discover IPP printers for temporary queues

[tillkamppeter]

I am using CUPS from this repository here at OpenPrinting, in an Ubuntu 20.10 environment with Avahi 0.8. cups-browsed is not running, also no CUPS Snap, the driverless utility used is of cups-filters 1.28.5.

I have several local IPP print services running: ipp-usb (with HP OfficeJet Pro 8730 connected to USB), the PostScript Printer Application (with printers "test", "test2", "test3", "test4", "test5", and "xxx") and ippeveprinter (started with ippeveprinter -p 8001 evetest).

The HP OfficeJet Pro 8730 is also connected to the network, so its IPP print service appears this way, too.

The running CUPS daemon (not the CUPS Snap) has several local CUPS queues.

Output of lpstat -e:

$ lpstat -e
authtest
drvlessfax
faxtest
HP-OfficeJet-Pro-8730
ippusbtest
office-local
OfficeJet-Pro-8730
pappl-test
pappl-test-e
Printer
Printer-HPLIP
snap-label
snap-test
test-tray2

The output shows only my permanent CUPS queues, no entries from the three local IPP print services.

$ lpstat -l -e
authtest permanent ipp://localhost/printers/authtest /dev/null
drvlessfax permanent ipp://localhost/printers/drvlessfax ipps://HP%20OfficeJet%20Pro%208730%20%5B08C229%5D._ipps._tcp.local/
faxtest permanent ipp://localhost/printers/faxtest ipp://localhost:60001/ipp/faxout
HP-OfficeJet-Pro-8730 permanent ipp://localhost/printers/HP-OfficeJet-Pro-8730 ipps://HP%20OfficeJet%20Pro%208730%20%5B08C229%5D._ipps._tcp.local/
ippusbtest permanent ipp://localhost/printers/ippusbtest ipp://HP%20OfficeJet%20Pro%208730%20%5B08C229%5D%20(USB)._ipp._tcp.local/
office-local permanent ipp://localhost/printers/office-local ipp://Office%20Printer._ipp._tcp.local/
OfficeJet-Pro-8730 permanent ipp://localhost/printers/OfficeJet-Pro-8730 ipps://HP%20OfficeJet%20Pro%208730%20%5B08C229%5D._ipps._tcp.local/
pappl-test permanent ipp://localhost/printers/pappl-test ipp://till-x1yoga.local:8000/ipp/print/Inkjet%20Printer
pappl-test-e permanent ipp://localhost/printers/pappl-test-e ipp://Inkjet%20Printer._ipp._tcp.local/
Printer permanent ipp://localhost/printers/Printer ipps://HP%20OfficeJet%20Pro%208730%20%5B08C229%5D._ipp._tcp.local/
Printer-HPLIP permanent ipp://localhost/printers/Printer-HPLIP hp:/usb/HP_OfficeJet_Pro_8730?serial=CN783F60W1
snap-label permanent ipp://localhost/printers/snap-label ipp://Label%20Printer._ipp._tcp.local/
snap-test permanent ipp://localhost/printers/snap-test ipp://HP%20OfficeJet%20Pro%208730%20%5B08C229%5D._ipp._tcp.local/
test-tray2 permanent ipp://localhost/printers/test-tray2 ipp://localhost:631/printers/test-tray
$ lpstat -l -e | grep -v permanent

Here one also sees that all entries in the output are "permanent", none "temporary".

So no temporary queues are listed for the local IPP services, which is the issue I want to report here.

More info:

$ ippfind
ipp://till-x1yoga.local:631/printers/OfficeJet-Pro-8730
ipp://till-x1yoga.local:631/printers/HP-OfficeJet-Pro-8730
ipp://HP18602408C229.local:631/ipp/print
ipp://till-x1yoga.local:60000/ipp/print
ipp://till-x1yoga.local:631/printers/faxtest
ipp://till-x1yoga.local:631/printers/Printer-HPLIP
ipp://till-x1yoga.local:631/printers/pappl-test-e
ipp://till-x1yoga.local:631/printers/office-local
ipp://till-x1yoga.local:631/printers/Printer
ipp://till-x1yoga.local:631/printers/authtest
ipp://till-x1yoga.local:631/printers/drvlessfax
ipp://till-x1yoga.local:8001/ipp/print
ipp://till-x1yoga.local:631/printers/ippusbtest
ipp://till-x1yoga.local:631/printers/pappl-test
ipp://till-x1yoga.local:631/printers/snap-label
ipp://till-x1yoga.local:631/printers/snap-ps
ipp://till-x1yoga.local:8000/ipp/print/test
ipp://till-x1yoga.local:631/printers/test-tray2
ipp://till-x1yoga.local:8000/ipp/print/test2
ipp://till-x1yoga.local:8000/ipp/print/test3
ipp://till-x1yoga.local:8000/ipp/print/test4
ipp://till-x1yoga.local:8000/ipp/print/test5
ipp://till-x1yoga.local:8000/ipp/print/xxx
$ driverless
ipp://HP%20OfficeJet%20Pro%208730%20%5B08C229%5D%20(USB)._ipp._tcp.local/
ipps://HP%20OfficeJet%20Pro%208730%20%5B08C229%5D._ipps._tcp.local/
ipps://evetest._ipps._tcp.local/
ipps://test._ipps._tcp.local/
ipps://test2._ipps._tcp.local/
ipps://test3._ipps._tcp.local/
ipps://test4._ipps._tcp.local/
ipps://test5._ipps._tcp.local/
ipps://xxx._ipps._tcp.local/

ippfind and driverless list all these services correctly, so they are correctly DNS-SD registered and CUPS should list them as destinations for temporary queues.

[tillkamppeter]

This seems to be a timeout issue. If I do the following change (set timeout for all DNS-SD-advertised printers to be found from 250 msec to 5 sec)

diff --git a/cups/dest.c b/cups/dest.c
index 2017792a7..c91ea9c0f 100644
--- a/cups/dest.c
+++ b/cups/dest.c
@@ -57,7 +57,7 @@
 #endif /* __APPLE__ */
 
 #if defined(HAVE_DNSSD) || defined(HAVE_AVAHI)
-#  define _CUPS_DNSSD_GET_DESTS 250     /* Milliseconds for cupsGetDests */
+#  define _CUPS_DNSSD_GET_DESTS 5000     /* Milliseconds for cupsGetDests */
 #  define _CUPS_DNSSD_MAXTIME  50      /* Milliseconds for maximum quantum of time */
 #else
 #  define _CUPS_DNSSD_GET_DESTS 0       /* Milliseconds for cupsGetDests */

All printers appear in the output of lpstat -e, but lpstat -e takes 5 sec to complete.

[tillkamppeter]

I can also print to the Printer Application with CUPS, using a command like

lp -d test3 ~/ghostscript/testfiles/CityMap.pdf

only the command takes several seconds for sending this small job.

[michaelrsweet]

I'm guessing this is probably libcups trying to filter out local queues. I can update it to look at the TXT record and only exclude CUPS queues.

[tillkamppeter]

It seems not to filter out queues at all. The output of lpstat -e with the longer timeout looks like this:

$ lpstat -e
authtest
drvlessfax
drvlessfax_till_x1yoga
faxtest
HP-OfficeJet-Pro-8730
HP_OfficeJet_Pro_8730_08C229_
HP_OfficeJet_Pro_8730_08C229_till_x1yoga
HP_OfficeJet_Pro_8730_08C229_USB_
HP_OfficeJet_Pro_8730_fully_driverless_till_x1yoga
HP_OfficeJet_Pro_8730_till_x1yoga
HPLIP_Test_till_x1yoga
Inkjet_Printer_till_x1yoga
ippusbtest
ippusbtest_till_x1yoga
office-local
Office_Printer_till_x1yoga
OfficeJet-Pro-8730
pappl-test
pappl-test-e
pappl_test_till_x1yoga
Printer
Printer-HPLIP
Printer_till_x1yoga
snap-label
snap-test
snap_label_till_x1yoga
test
test-tray2
test2
test3
test4
test5
test_tray_till_x1yoga
xxx

You see that the queue of the PostScript Printer Application show up now (I did not have ippeveprinter running this time).
Also the two IPP incarnations (network and IPP-over-USB) of my HP OfficeJet Pro 8730 show (HP_OfficeJet_Pro_8730_08C229_ and HP_OfficeJet_Pro_8730_08C229_USB_ appear.
Each CUPS queue apears twice, once directly (for example Printer) and once picked up through DNS-SD (for example Printer_till_x1yoga and all the others ending with _till_x1yoga), the latter should be suppressed.

With the original timeout setting all DNS-SD-discovered entries disappear for me, only having the direct CUPS queue entries remaining.

So there are two problems:

• DNS-SD discovery takes too long
• Suppressing of local CUPS queues does not work

Note also that not all local services should be suppressed, only local CUPS queues (safest is to compare UIID with direct CUPS queue entries, this I do in cups-browsed).

[michaelrsweet]

OK, so I did some initial testing on macOS and it works as expected with mDNSResponder. Now testing on Ubuntu 20.04 (what my XPS13 is loaded with at the moment) to see how things work with Avahi 0.8...

[michaelrsweet]

OK, my XPS13 (again, running Ubuntu 20.04) is able to list my local printers. If I run ippeveprinter in another window it appears as expected, and if I terminate it, it disappears from the list as expected.

As for the duplicate queues, I checked the code and we are looking for local (permanent/temporary) queues with the same name as the auto-generated queue name from libcups - I'm guessing your permanent queues (created with the driverless utility) aren't following the same naming conventions... I think we have the printer-uuid information accessible to the browse callbacks so that we can eliminate dupes that way as well.

We definitely cannot increase the timeout to 5 seconds - that is way too long. The most we can justify is 1 second, and it should return faster than that if we stop getting data updates before then... FWIW, the Bonjour spec calls for a 100ms response time for queries/browsing...

[tillkamppeter]

The CUPS queues are all manually created (cups-browsed is NOT running and "driverless" does not create queues) for each manually created (and so permanent) CUPS queue, for example my queue "Printer", lpstat -e lists an extra queue with "_till_x1yoga" (the host name of my machine is "till-x1yoga") added, for "Printer" being "Printer_x1_yoga", Here you see it in the output lpstst -l -e with "Printer" being "permanent" and "Printer_till_x1yoga" being "network" (temporary), same for all the other "permanent" queues:

$ lpstat -l -e
authtest permanent ipp://localhost/printers/authtest /dev/null
drvlessfax permanent ipp://localhost/printers/drvlessfax ipps://HP%20OfficeJet%20Pro%208730%20%5B08C229%5D._ipps._tcp.local/
drvlessfax_till_x1yoga network none ipps://drvlessfax%20%40%20till-x1yoga._ipps._tcp.local/cups
faxtest permanent ipp://localhost/printers/faxtest ipp://localhost:60001/ipp/faxout
HP-OfficeJet-Pro-8730 permanent ipp://localhost/printers/HP-OfficeJet-Pro-8730 ipps://HP%20OfficeJet%20Pro%208730%20%5B08C229%5D._ipps._tcp.local/
HP_OfficeJet_Pro_8730_08C229_ network none ipps://HP%20OfficeJet%20Pro%208730%20%5B08C229%5D._ipps._tcp.local/
HP_OfficeJet_Pro_8730_08C229_till_x1yoga network none ipps://HP%20OfficeJet%20Pro%208730%20%5B08C229%5D%20%40%20till-x1yoga._ipps._tcp.local/cups
HP_OfficeJet_Pro_8730_08C229_USB_ network none ipp://HP%20OfficeJet%20Pro%208730%20%5B08C229%5D%20(USB)._ipp._tcp.local/
HP_OfficeJet_Pro_8730_fully_driverless_till_x1yoga network none ipps://HP%20OfficeJet%20Pro%208730%20(fully%20driverless)%20%40%20till-x1yoga._ipps._tcp.local/cups
HP_OfficeJet_Pro_8730_till_x1yoga network none ipps://HP%20OfficeJet%20Pro%208730%20%40%20till-x1yoga._ipps._tcp.local/cups
HPLIP_Test_till_x1yoga network none ipps://HPLIP%20Test%20%40%20till-x1yoga._ipps._tcp.local/cups
Inkjet_Printer_till_x1yoga network none ipps://Inkjet%20Printer%20%40%20till-x1yoga._ipps._tcp.local/cups
ippusbtest permanent ipp://localhost/printers/ippusbtest ipp://HP%20OfficeJet%20Pro%208730%20%5B08C229%5D%20(USB)._ipp._tcp.local/
ippusbtest_till_x1yoga network none ipps://ippusbtest%20%40%20till-x1yoga._ipps._tcp.local/cups
office-local permanent ipp://localhost/printers/office-local ipp://Office%20Printer._ipp._tcp.local/
Office_Printer_till_x1yoga network none ipps://Office%20Printer%20%40%20till-x1yoga._ipps._tcp.local/cups
OfficeJet-Pro-8730 permanent ipp://localhost/printers/OfficeJet-Pro-8730 ipps://HP%20OfficeJet%20Pro%208730%20%5B08C229%5D._ipps._tcp.local/
pappl-test permanent ipp://localhost/printers/pappl-test ipp://till-x1yoga.local:8000/ipp/print/Inkjet%20Printer
pappl-test-e permanent ipp://localhost/printers/pappl-test-e ipp://Inkjet%20Printer._ipp._tcp.local/
pappl_test_till_x1yoga network none ipps://pappl-test%20%40%20till-x1yoga._ipps._tcp.local/cups
Printer permanent ipp://localhost/printers/Printer ipps://HP%20OfficeJet%20Pro%208730%20%5B08C229%5D._ipp._tcp.local/
Printer-HPLIP permanent ipp://localhost/printers/Printer-HPLIP hp:/usb/HP_OfficeJet_Pro_8730?serial=CN783F60W1
Printer_till_x1yoga network none ipps://Printer%20%40%20till-x1yoga._ipps._tcp.local/cups
snap-label permanent ipp://localhost/printers/snap-label ipp://Label%20Printer._ipp._tcp.local/
snap-test permanent ipp://localhost/printers/snap-test ipp://HP%20OfficeJet%20Pro%208730%20%5B08C229%5D._ipp._tcp.local/
snap_label_till_x1yoga network none ipps://snap-label%20%40%20till-x1yoga._ipps._tcp.local/cups
test network none ipps://test._ipps._tcp.local/
test-tray2 permanent ipp://localhost/printers/test-tray2 ipp://localhost:631/printers/test-tray
test2 network none ipps://test2._ipps._tcp.local/
test3 network none ipps://test3._ipps._tcp.local/
test4 network none ipps://test4._ipps._tcp.local/
test5 network none ipps://test5._ipps._tcp.local/

This means that DNS-SD records from CUPS queues are picked up and temporary queue entries with a queue name composed of the original queue name and the network host name of the server (should this not be "localhost"?) are created.
These temporary queue entries need to get suppressed, in a way that the entries of other local services, like Printer Applications and ipp-usb, do not go away. I suggest using UUIDs here, as this works perfectly for me to solve the same problem in cups-browsed.
For the timeout issue I fear that this is a bug in Avahi and I do not want to wait another 3 years to get it fixed.
So one question: Is the mDNSResponder which Apple uses free software? Is it well maintained? Would it make sense for Linux distros to switch to it?

[tillkamppeter]

I have done a test now: On my Ubuntu 20.10 I have installed the Avahi packages (0.7) of 20.04 and now I do not need to extend the timeout any more, Avahi is answering quick enough to list all the printers and I can print out of CUPS on the PostScript Printer Application, no need of cups-browsed, nor of manually creating a CUPS queue, all works as intended now.
But note that the Avahi downgrade only solves the timeout problem. The temporary queue entries for the permanent CUPS queues are still there. So the UUID checking is still needed.

[tillkamppeter]

The "driverless" utility is also much faster after the downgrade ...

[tillkamppeter]

Another hint for the Avahi timing issue. In the release notes of Avah 0.8 it is written:

Notable Changes: [...] avahi-daemon: Delay sending results on an object for 10ms in an attempt to
give clients enough time to subscribe to signals from the new object after
receiving it's path in response so the New call. See "API Changes" for more
info

API Changes: The avahi-core API and D-Bus API have implemented a new API where
a call to the "New" method can now be split into a "Prepare" and then "Start"
method for some objects. The previous "New" API is still fully supported and
there is no intention to deprecate it.

This change affects the the following objects: AsyncAddressResolver,
AsyncHostNameResolver, AsyncServiceResolver, DomainBrowser, RecordBrowser,
ServiceBrowser, ServiceTypeBrowser

This is because the D-Bus implementation in some languages would only bind to
signals of an object after it was created and had received the new object's
path. This led to such languages missing the initial results sent between the
time the object was created and it had setup a filter to receive it's signals.
This primarily occured in languages that create dynamic bindings for D-Bus
objects using introspection such as Python. The avahi-client C api was not
affected as it globally binds to all avahi signals without specifying
individual object paths and still makes use of the V1 API.

The v2 Prepare/Start API is available under the new
org.freedesktop.Avahi.Server2 D-Bus interface and also has corresponding
avahi_s_* calls for users of the embedded avahi-core library.

The old org.freedesktop.Avahi.Server interface is still supported and there is
no intention to remove this API. Additionally this problem has also been solved
for old clients by adding a very small 10ms delay before we start sending
results to give the client time to bind to the signals which should silently
fix the issue in most cases without introducing a noticable or impactful delay.

Clients implementing the new org.freedesktop.Avahi.Server2 D-Bus interface will
not work with older Avahi daemons. It is suggested that clients may wish to
either check for and fallback to the older API version, or continue to use the
OLD API and rely on the 10ms timer to resolve the issue.

Could the 10ms delay mentioned here causing the problem and should CUPS perhaps use the new API here then?

[michaelrsweet]

Till,

10ms shouldn't make a difference, and we are using the higher-level client API which should insulate us from these differences. I will clone and upgrade one of my VMs to 20.10 so I can duplicate this issue and come up with a fix.