Not working

[pavpanchekha]

I got this issue over email a year ago. Well, I'm obviously not working on, let's leave it as an issue instead. It's a pity, it's one of the best bug reports I've ever received:

---

Hi. I just discovered dropbox.el. I'm having issues getting /db:
to actually call the underlying mechanism, though calling
dropbox-handle-directory-files does actually see files.

I'm running "Emacs 25.2.2 (x86_64-pc-linux-gnu, GTK+ Version 3.22.21)
of 2017-09-22, modified by Debian" under Ubuntu 18.04.3 LTS.

To install dropbox.el, I first had to retrieve "request" from [1].
I byte-compiled and loaded just request.el from that package, then
byte-compiled dropbox.el from yours [2] and loaded that. I then setq'd
dropbox-access-token to the token I got when I created the app on Dropbox.

[1] https://github.com/tkf/emacs-request
[2] https://github.com/pavpanchekha/dropbox.el

After reading the only open issue in your Github (re Helm #7), I saw the
clue about first calling dropbox-connect, which either isn't in your
README or I missed it somewhere, so I also called that after some
previous attempts didn't work, but it doesn't seem to have helped.

Instead, if I try something like C-x d /db: or (directory-files "/db:"),
I get:

ssh: Could not resolve hostname db: Name or service not known^M"

so clearly it's trying to connect to a host called "db" instead of the
actual Dropbox API endpoints.

*Messages* has:

Dropbox’ing operation substitute-in-file-name for (/db:)
... returning /db:
Dropbox’ing operation substitute-in-file-name for (/db:)
... returning /db:
Dropbox’ing operation substitute-in-file-name for (/db:)
... returning /db:
Dropbox’ing operation file-name-as-directory for (/db:)
... returning /db:
Dropbox’ing operation directory-file-name for (/db:)
... returning /db:
Dropbox’ing operation expand-file-name for (/db: nil)
Tramp: Opening connection for db using scp...
Tramp: Sending command ‘exec ssh   -o ControlMaster=auto -o
ControlPath='tramp.%C' -o ControlPersist=no -e none db’
Tramp: Waiting for prompts from remote shell...
Tramp failed to connect.  If this happens repeatedly, try
    ‘M-x tramp-cleanup-this-connection’
Tramp: Waiting for prompts from remote shell...failed
Tramp: Opening connection for db using scp...failed
tramp-file-name-handler: Tramp failed to connect.  If this happens
repeatedly, try
    ‘M-x tramp-cleanup-this-connection’

file-name-handler-alist is:

(("\\`/db:" . dropbox-handler)
("\\(?:\\.dz\\|\\.txz\\|\\.xz\\|\\.lzma\\|\\.lz\\|\\.g?z\\|\\.\\(?:tgz\\|svgz\\|sifz\\)\\|\\.tbz2?\\|\\.bz2\\|\\.Z\\)\\(?:~\\|\\.~[-[:alnum:]:#@^._]+\\(?:~[[:digit:]]+\\)?~\\)?\\'"
. jka-compr-handler) ("\\.\
gpg\\(~\\|\\.~[0-9]+~\\)?\\'" . epa-file-handler) ("\\`/[^/]*\\'"
. tramp-completion-file-name-handler) ("\\`/[^/|:][^/|]*:"
. tramp-file-name-handler) ("\\`/:" . file-name-non-special))

I tried again via emacs -q so it would not run my init,
and loaded request.elc, dropbox.elc, and setq'd the token.
No change in behavior.

What am I missing?

P.S. My *Compile Log* buffer had this in it:

Leaving directory ‘~/’
^L
Compiling file /home/foner/request.el at Mon Dec 30 15:33:56 2019
Entering directory ‘/home/foner/’
^L
Compiling file /home/foner/dropbox.el at Mon Dec 30 15:34:04 2019

In dropbox-handler:
dropbox.el:237:40:Warning: reference to free variable
‘dropbox-handler-alist’

In dropbox-handle-file-selinux-context:
dropbox.el:521:8:Warning: function ‘dropbox-handle-file-selinux-context’
    defined multiple times in this file

In dropbox-handle-verify-visited-file-modtile:
dropbox.el:533:26:Warning: assignment to free variable ‘newmetadata’
dropbox.el:534:60:Warning: reference to free variable ‘newmetadata’

In dropbox-handle-copy-directory:
dropbox.el:721:26:Warning: reference to free variable ‘file’

In end of data:
dropbox.el:885:1:Warning: the following functions are not known to be
defined: list*,
    move-file

P.P.S. Next-older Emacs I have available easily is Emacs 24.3.1,
in Ubuntu 14.04.6, but that didn't know about request either and
might also be missing the json package, IIRC, so I stopped testing
there and tried a machine with Emacs 25 on it instead.

Thanks again!

[pavpanchekha]

More from email:

---

I'll try to update the README, and maybe add error messages, to make clear
the need to call dropbox-connect. As for your actual problem, the issue
may be the backtick in the /db:: entry in your file-handler-alist, or
it might be the behavior of expand-file-name. I'll try it on a fresh
Emacs instance and see what happens.

I've already tried removing the backtick, which is clearly there for
good reasons anyway. I haven't looked at e-f-n yet. I figured I can
step through the entire chain of how Emacs knows this is a foreign FS
reference, but that I'd ask first if this was an obvious issue or if
this had only been tested on older/newer Emacs versions than mine.
(But Emacs 24.3 doesn't have alist-get, hence trying also under 25
because I assumed it was written for a newer version, but if it's
seven years old, maybe not. :)

[pavpanchekha]

Author is a better developer than me:

---

I poked at this tonight. The issue is that tramp tries to grab just
about everything. In particular, if one or both of these conses [1] are
in file-name-handler-alist, tramp will (a) try to visit a server called
"db" and (b) reorder the list (!) to put the dropbox handler near the
end [2]. What I don't understand is how this worked in just about any
Emacs, because tramp has been there for a long time---it claims it was
included as part of Emacs since Emacs 22.1, which was released in June
2007.

It's a little unpredictable what happens with [1]'s conses, btw, because
tramp -keeps screwing around with that list- because it's got a zillion
autoloads and its handlers keep paranoically modifying the list back to
the way it think it's supposed to be. Removing just t-a-f-n-h worked,
but then a test case w/everything included failed (as I expected) but
then removing just t-a-f-n-h again -didn't- work (and kept putting
itself back) because t-f-n-h kept mucking around with the list, etc.
So debugging this is really annoying, because tramp keeps mutating
that list behind your back and you have to check it -every- time.

[1] ("\/[^/|:][^/|]*:" . tramp-autoload-file-name-handler) ("\\/[^/|:][^/|]*:" . tramp-file-name-handler)

[2] Original file-name-handler-alist:

(("\\`/db:" . dropbox-handler) ("\\.gpg\\(~\\|\\.~[0-9]+~\\)?\\'" . epa-file-handler) ("\\(?:\\.dz\\|\\.txz\\|\\.xz\\|\\.lzma\\|\\.lz\\|\\.g?z\\|\\.\\(?:tgz\\|svgz\\|sifz\\)\\|\\.tbz2?\\|\\.bz2\\|\\.Z\\)\\(?:~\\|\\.~[-[:alnum:]:#@^._]+\\(?:~[[:digit:]]+\\)?~\\)?\\'" . jka-compr-handler) ("\\`/[^/]*\\'" . tramp-completion-file-name-handler) ("\\`/[^/|:][^/|]*:" . tramp-autoload-file-name-handler) ("\\`/:" . file-name-non-special))

List after tramp gets through with it:

(("\\(?:\\.dz\\|\\.txz\\|\\.xz\\|\\.lzma\\|\\.lz\\|\\.g?z\\|\\.\\(?:tgz\\|svgz\\|sifz\\)\\|\\.tbz2?\\|\\.bz2\\|\\.Z\\)\\(?:~\\|\\.~[-[:alnum:]:#@^._]+\\(?:~[[:digit:]]+\\)?~\\)?\\'" . jka-compr-handler) ("\\.gpg\\(~\\|\\.~[0-9]+~\\)?\\'" . epa-file-handler) ("\\`/[^/]*\\'" . tramp-completion-file-name-handler) ("\\`/[^/|:][^/|]*:" . tramp-file-name-handler) ("\\'/db:" . dropbox-handler) ("\\`/:" . file-name-non-special))

Tramp also obnoxiously seems to have various other list reorderings
hardcoded into it, which seems like a pretty big abstraction violation
but was obviously a kluge to not break decompression etc; see dolist at
the end of this:

(defun tramp-register-file-name-handlers ()
  "Add Tramp file name handlers to `file-name-handler-alist'."
  ;; Remove autoloaded handlers from file name handler alist.  Useful,
  ;; if `tramp-syntax' has been changed.
  (dolist (fnh '(tramp-file-name-handler
                 tramp-completion-file-name-handler
                 tramp-autoload-file-name-handler))
    (let ((a1 (rassq fnh file-name-handler-alist)))
      (setq file-name-handler-alist (delq a1 file-name-handler-alist))))
  ;; Add the handlers.
  (add-to-list 'file-name-handler-alist
               (cons tramp-file-name-regexp 'tramp-file-name-handler))
  (put 'tramp-file-name-handler 'safe-magic t)
  (add-to-list 'file-name-handler-alist
               (cons tramp-completion-file-name-regexp
                     'tramp-completion-file-name-handler))
  (put 'tramp-completion-file-name-handler 'safe-magic t)
  ;; If jka-compr or epa-file are already loaded, move them to the
  ;; front of `file-name-handler-alist'.
  (dolist (fnh '(epa-file-handler jka-compr-handler))
    (let ((entry (rassoc fnh file-name-handler-alist)))
      (when entry
        (setq file-name-handler-alist
              (cons entry (delete entry file-name-handler-alist)))))))

For my use case, I can
perhaps just nuke all of the tramp entries. But I don't understand how
this worked on any other Emacs, either, given that tramp is deeply wired
into it and probably has been for a long time.

[A slightly different dropbox prefix syntax than "/db:", such as maybe
"\db:" or "^db" or whatever would also bypass this problem, but I note
the comment in your source saying that various regexps have "/db:"
hardcoded into them as well, so this might be an invasive change, though
at first glance it just looks like you could replace every occurrence of
the hardcoded string "/db:" with dropbox-prefix and it would just work.]

Basically, your choice of syntax dramatically conflicts with tramp,
which assumes that -anything- starting with / and having a colon after
that is fair game. (It just occurred to me that this implies one
-could- use your dropbox code as a custom handler to tramp, which would
then require that one configure that handler for the "db" host and then
let tramp handle everything. But since I have basically never use
tramp anyway and especially for my single-use weirdo iPad case, for me
it's easier just to disable tramp by editing file-name-handler-alist. :)

P.S. It also looks like you should just put (dropbox-connect) at the
bottom of the file, so it just runs upon load, and then having to
manually call dropbox-connect wouldn't be necessary; all it's doing
is adding to file-name-handler-alist, and you could even make that
idempotent by using add-to-list instead of (setf .. (cons ..))

[pavpanchekha]

In summary, if only all of us could have such wonderful users with such thorough bug reports. It's unlikely I'll get to this any time soon, since with teaching, writing a book, and my students I have to prioritize my time. But it'd be nice if this worked again.