Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IUS-2451] handle archive files in Dataset .zip download #129

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion app/controllers/hyrax/data_sets_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -412,7 +412,7 @@ def zip_download
end

def zip_download_enabled?
true
Settings.zip_download_enabled
end

# end zip download operations
Expand Down
2 changes: 2 additions & 0 deletions app/helpers/deepblue/export_files_helper.rb
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ def self.export_file_sets( target_dir:,
file = file_set.files_to_file
if file.nil?
Rails.logger.warn "#{log_prefix} file_set.id #{file_set.id} files[0] is nil"
elsif file_set.archive_file?
# exclude from .zip
else
target_file_name = file_set.label
# fix possible issues with target file name
Expand Down
2 changes: 1 addition & 1 deletion app/models/archive_file.rb
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ def log_denied_attempt!(request_hash: {}, update_only: true)

# bypasses status in job file via checking directly
def downloaded?
File.exist?(local_path)
File.exist?(local_path.to_s)
end

def staged?
Expand Down
1 change: 1 addition & 0 deletions app/models/file_set.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ class FileSet < ActiveFedora::Base
include ::Deepblue::MetadataBehavior
include ::Deepblue::ProvenanceBehavior
include ::Deepblue::EmbargoableBehavior
include ::Datacore::PresentsArchiveFile

before_destroy :provenance_before_destroy_file_set

Expand Down
2 changes: 1 addition & 1 deletion app/presenters/concerns/datacore/presents_archive_file.rb
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ module PresentsArchiveFile

# archive files bypass fedora storage
def archive_file?
mime_type.match(/^message\/external-body\;.*access-type=URL/).present?
mime_type.to_s.match(/^message\/external-body\;.*access-type=URL/).present?
end

# needs to pass through archive_file in case any / to %2F encoding happened there
Expand Down
8 changes: 1 addition & 7 deletions app/presenters/hyrax/data_set_presenter.rb
Original file line number Diff line number Diff line change
Expand Up @@ -216,14 +216,8 @@ def total_file_size_human_readable
human_readable( total_file_size )
end

# begin zip download

def zip_download_enabled?
true
Settings.zip_download_enabled
end

# end zip download

end

end
18 changes: 17 additions & 1 deletion app/views/hyrax/base/_download_panel.html.erb
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps line 8 should say, "Some files associated with this dataset " etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revised language in new 4th commit

Original file line number Diff line number Diff line change
@@ -1,6 +1,21 @@
<% if Hyrax.config.download_files && (@presenter.zip_download_enabled? || @presenter.try(:globus_enabled?)) %>
<div class="panel panel-default panel-download">
<div class="panel-heading">
<h2>Download All Files (To download individual files, select them in the “Files” panel above)</h2>
<h2>Download All Files</h2>
<% if @presenter.member_presenters.all?(&:archive_file?) %>

<h3>All files in this dataset must be individually downloaded. This zip download will include the dataset metadata export, only.</h3>
<% elsif @presenter.member_presenters.any?(&:archive_file?) %>
<h3>Some files in this dataset must be individually downloaded, and will not be included in the zip download:</h3>
<ul>
<% @presenter.member_presenters.select(&:archive_file?).each do |archive_file| %>
<li><%= link_to archive_file.title&.first, hyrax.download_path(archive_file) %></li>
<% end %>
</ul>
<% else %>
<h3>All files in this dataset will be included in the zip download.</h3>
<% end %>
<h3>Files can be downloaded invididually in the "Files" panel above.</h3>
</div>
<div class="panel-body">
<% if Hyrax.config.download_files %>
Expand Down Expand Up @@ -83,3 +98,4 @@
<% end %>
</div>
</div> <!--End Download panel-->
<% end %>
2 changes: 1 addition & 1 deletion config/application.rb
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ class Application < Rails::Application
Dir.mkdir config.globus_prep_dir unless Dir.exist? config.globus_prep_dir
end
config.globus_enabled = true && Dir.exist?( config.globus_download_dir ) && Dir.exist?( config.globus_prep_dir )
config.base_file_name = "DeepBlueData_"
config.base_file_name = "DataCORE_"
config.globus_base_url = 'https://app.globus.org/file-manager?origin_id=99d8c648-a9ff-11e7-aedd-22000a92523b&origin_path=%2Fdownload%2F'
config.globus_restart_all_copy_jobs_quiet = true
config.globus_debug_delay_per_file_copy_job_seconds = 0
Expand Down
2 changes: 2 additions & 0 deletions config/settings.yml
Original file line number Diff line number Diff line change
Expand Up @@ -197,3 +197,5 @@ ldap:
tree: 'ldap_tree'
account_ou: 'ldap_account_ou'
group_ou: 'ldap_group_ou'

zip_download_enabled: true
4 changes: 2 additions & 2 deletions spec/jobs/globus_restart_all_job_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
describe GlobusRestartAllJob, "GlobusJob globus_enabled: :true", globus_enabled: :true do # rubocop:disable RSpec/DescribeMethod

let( :globus_dir ) { Pathname "/tmp/deepbluedata-globus" }
let( :target_name ) { "DeepBlueData_Restart_All" }
let( :target_name ) { "DataCORE_Restart_All" }
# let( :target_name_prep_dir ) { "#{GlobusJob.server_prefix(str:'_')}#{target_name}" }
let( :globus_prep_dir ) { globus_dir.join 'prep' }
let( :job_complete_file ) { globus_prep_dir.join ".test.restarted.#{target_name}" }
Expand All @@ -29,7 +29,7 @@
end
let( :log_prefix ) { "Globus: globus_restart_all_job " }
let( :globus_era_file ) { GlobusJob.era_token }
let( :target ) { "DeepBlueData_" }
let( :target ) { "DataCORE_" }
let( :id00 ) { "id000" }
let( :file00 ) { "#{globus_prep_dir}/.development.lock.#{target}#{id00}" }
let( :dir00 ) { "#{globus_prep_dir}/development_#{target}#{id00}" }
Expand Down