Description

The Journalist Interface and API return different subsets of Sources. Under certain circumstances, the same journalist user will see more sources listed in a SecureDrop Client session than they will see in a simultaneous Journalist Interface session.

Recommended approach

1. Reconcile (or factor out?) the filters used by the Journalist Interface's index() and the Journalist API's get_all_sources()
2. Bonus: Check whether loaddata.py::record_source_interaction() can use session.commit() as usual
3. Bonus: Tests!

Steps to Reproduce

On the Server: Not an exact reproduction; this is my long-running test instance.

sdadmin@app:~$ sudo apt-cache policy securedrop-app-code
securedrop-app-code:
  Installed: 2.5.0+focal
  Candidate: 2.5.0+focal
  Version table:
 *** 2.5.0+focal 500
        500 https://apt.freedom.press/ focal/main amd64 Packages
        100 /var/lib/dpkg/status

Previously:

www-data@app:~/securedrop$ ./loaddata.py --seed 1547 \
    --journalist-count 0 \
    --source-count 3500 \
    --messages-per-source 1 \
    --replies-per-source 1

Then I used the Journalist Interface to Select All sources and Delete Selected.

Expected Behavior

On the Client: No sources are listed.

Actual Behavior

On the Client:

user@sd-app:~/securedrop-client$ sudo apt-cache policy securedrop-client
securedrop-client:
  Installed: 0.8.1+bullseye
  Candidate: 0.8.1+bullseye
  Version table:
 *** 0.8.1+bullseye 500
        500 https://apt.freedom.press/ bullseye/main amd64 Packages
        100 /var/lib/dpkg/status

Comments

On the Server:

sdadmin@app:~$ sudo -u www-data sqlite3 /var/lib/securedrop/db.sqlite "SELECT * FROM sources;"
1|eb2df246-c2d6-46dd-af81-83608411870f|3IGB6CI2DKHC2OGLJTSZYND6SOFGAQNMV2D5BCMXNR52PPV5UKXED5BCSMLKNN5M5UQTZPTB5VPJQBZMJYN34DXGE5GYIHMMOXZT7EQ=|politic paperweight||0|0|
255|db09f881-3b8f-4dec-bdf6-ab0bb715ea86|4PBQ3ZMPOJTKRRZZHGBCAX74GHOBINT6LHET3YOSO4Q77XGMMF3M5RZZVNYTO7DSEDUAWBUT36DOYCHM6O3QDMV7NVLYBPTJEO7HZEQ=|blissful obverse||0|0|
3756|b6280f51-ad5f-448c-8697-45e70cd417c5|IJNT2TRBQNT4ZHXSKRHCA44EAEPBOY76I3IXGQNBAGQVL2R7TZ5UHLAFLPUDM2A6UNTH2IUXWVMZ5WQHO7V3DX6BHZBROD2D5XWSTEA=|odds-on interruption||1|0|

And sure enough, compare:

• The Journalist Interface filters on last_updated != None:
  

  securedrop/securedrop/journalist_app/main.py

  Lines 101 to 106 in 6170370

  	# Query for sources without stars, along with their unread
  	# submission counts.
  	unstarred = (
  	db.session.query(Source, unread_stmt.c.num_unread)
  	.filter_by(pending=False, deleted_at=None)
  	.filter(Source.last_updated.isnot(None))

• The Journalist API does not:
  

  securedrop/securedrop/journalist_app/api.py

  Lines 124 to 126 in 6170370

  	@api.route("/sources", methods=["GET"])
  	def get_all_sources() -> Tuple[flask.Response, int]:
  	sources = Source.query.filter_by(pending=False, deleted_at=None).all()