Remove encoded characters from file name

[dionisislolas]

What

Remove encoded characters from file name
Fix for the issue with the visualisations zip files that was containing files with spaces in the name

How to review

Changes are ok and test pass.

Run the static-files-with-auth stack.
Create a collection with a dataset that is going to the lockbox
Add to the same collection also a visualisation that contains a file with spaces in the visualisations zip file
Collection should get approved and published successfully

Who can review

Anyone

[franmoore05]

Can "myFile" be renamed to something like "decodedFile" please

[dionisislolas]

This is now addressed.

[franmoore05]

Could "filename" be changed to lower camel case please e.g. "fileName"

[dionisislolas]

This is now addressed.

[franmoore05]

I think there are several things we could change in code to improve how lockbox 2.0 works. We are still sending all the files in the collection to be checked and processed which we don't need to do, and this is what is introducing the errors. Some suggestions to change the code are as follows:

Within our if statement to check if the upload new endpoint is enabled (https://github.com/ONSdigital/zebedee/blob/develop/zebedee-cms/src/main/java/com/github/onsdigital/zebedee/model/approval/ApproveTask.java#L167), we could do something like the following:

List<String> reviewedList = collection.getReviewed().uris().stream().filter(i -> i.contains("datasets")).collect(Collectors.toList());
       
String whitelist = getDatasetWhitelist();
List<String> allowed = Arrays.asList(whitelist.split(","));

reviewedList = reviewedList.stream().filter(j -> allowed.stream().anyMatch(j::contains)).collect(Collectors.toList()); 

This would allow us to send a reduced list through to the uploadNewEndpoint if we change the parameters to be a List rather than needing to send the whole collection. Also, the uploadNewEndpoint is not required as it doesn't do anything other than forward on to another function, so we could send that list straight to uploadWhitelistedFiles which would then have the signature uploadWhitelistedFiles(CollectionReader collectionReader, List urisToCheck, String description).

The description string parameter has also been added in place of using the collection to get the description to add to the file path. Sending through only the files required means we no longer need to handle any special characters or spaces within the filenames as we should be doing as little with those as possible.

Also, if we move the findCorrectDatasetVersion function call within this if statement https://github.com/ONSdigital/zebedee/blob/develop/zebedee-cms/src/main/java/com/github/onsdigital/zebedee/model/approval/ApproveTask.java#L367 and pass in the file name then it will resolve the bug with the incorrect version number of the non-timeseries datasets. findCorrectDatasetVersion will then have to be updated to include an additional check like this if (uri.contains("previous") && uri.contains(fileName)) to ensure the correct version is returned for the specific file, but it seems to work ok.

[franmoore05]

We will probably need a null check on the reviewedList to ensure no null pointer exceptions