Cannot view screenshots of failed tests

[craig-jamieson]

New Issue Checklist

• Updated fastlane-plugin-test_center to the latest version
• I read the README.md
• I reviewed the example(s) for the action(s) I am using
• I have removed any sensitive data such as passwords, authentication tokens, or anything else I do not want to world to see
• I have reviewed the Discussions forum to see my question has already been addressed.

Issue Description

Normally on the CI when our UI tests have finished it copies the test results file from the derived data into the test artifacts and zips everything together so we can download the results and view any screenshots etc, however, since installing the plugin-test_center the zip folder is empty, can you see any reason why this would be happening?

begin
      multi_scan(
        workspace: "xcworkspace",
        scheme: "Stage",
        try_count: 2,
        result_bundle: true,
        parallel_testrun_count: 3,
        clean: clean,
        configuration: "Stage Debug",
        destination: "platform=iOS Simulator,OS=#{IOS_VERSION},id=CE009200-82BE-4C99-94C5-B97705398B6C",
        reset_simulator: true,
        include_simulator_logs: false,
        skip_slack: true,
        code_coverage: true,
        only_testing: ["UITests"],
        derived_data_path: DERIVED_DATA_FOLDER,
        output_directory: TEST_RESULTS_FOLDER,
        output_files: "report.html,junit.xml"
        )
    rescue => ex
      report_result = build_failing_tests_report(report_identifier: "Report ID", test_results_folder: TEST_RESULTS_FOLDER)
      total_tests_count = 0
      total_failed_tests = 0
      for failing_test_suite in report_result
        if failing_test_suite[:failures] == "0"
          next
        end
        e2e_error_message = ""
        class_name = ""
        total_tests_count += failing_test_suite[:tests].to_i
        total_failed_tests += failing_test_suite[:failures].to_i
        for test_summary in failing_test_suite[:total_tests]
          e2e_error_message += "#{test_summary[:test_name]} \n "
          class_name = test_summary[:class_name]
        end
        branch = git_branch
        final_error_message += "#{class_name} - #{e2e_error_message} \n"
      end
      final_error_message = "Branch - #{branch} \n \n #{final_error_message}"
      tests_failed = true
    end
copy_artifacts(
  target_path: "artifacts/test-results",
  artifacts: ["#{TEST_RESULTS_FOLDER}/*.*"]
  )

copy_artifacts(
  target_path: "e2e_tests/test-results/archive",
  artifacts: ["#{DERIVED_DATA_FOLDER}/Logs/Test/*.xcresult"]
  )

zip(
  path: "e2e_tests/test-results/archive",
  output_path: "artifacts/test-results/archivedResults.zip",
  verbose: false
  )

clear_derived_data(derived_data_path: DERIVED_DATA_FOLDER)</pre>

Environment

platform :ios do
  before_all do
    # Allow up to 6 attempts for starting xcodebuild
    ENV["FASTLANE_XCODEBUILD_SETTINGS_RETRIES"] = "5"
# Prepare CircleCi configuration, including a usable Keychain
setup_circle_ci

# This uses the .xcode-version file in the project directory to ensure we
# use a consistent Xcode version to ship and on the CI to test.
# If upgrading to a new Xcode, change the `.xcode-version` file.
ensure_xcode_version

end
import("./Fastfiles/Setup.rb")
import("./Fastfiles/Testing.rb")
import("./Fastfiles/Release.rb")
import("./Fastfiles/EmergencyRelease.rb")
import("./Fastfiles/Utilities.rb")
import("./Fastfiles/Build.rb")
CircleCI exclusive lanes. Only imported if IS_CI_MACHINE
if ENV["IS_CI_MACHINE"] == "TRUE"

import("./Fastfiles/CircleCI.rb")

end

end
Path resolvers
def build_output_path

File.expand_path("../#{BUILD_OUTPUT_FOLDER}")

end
def ipa_file_path

File.join(build_output_path, IPA_FILENAME)

end
def dsyms_file_path

File.join(build_output_path, DSYM_FILENAME)

end
</details>

<details><summary>`./fastlane/Appfile`</summary>

```ruby
app_identifier "io.b2a.BankProd" # The bundle identifier of your app
apple_id "itc@monzo.com" # Your Apple email address

# For more information about the Appfile, see:
#     https://docs.fastlane.tools/advanced/#appfile

fastlane gems

Gem	Version	Update-Status
fastlane	2.188.0	🚫 Update available

Loaded fastlane plugins:

Plugin	Version	Update-Status
fastlane-plugin-aws_s3	2.0.2	🚫 Update available
fastlane-plugin-firebase_app_distribution	0.2.9	🚫 Update available
fastlane-plugin-versioning	0.4.5	🚫 Update available
fastlane-plugin-test_center	3.15.3	✅ Up-To-Date

Loaded gems

Gem	Version
did_you_mean	1.3.0
bundler	2.1.4
rake	13.0.6
CFPropertyList	3.0.3
concurrent-ruby	1.1.9
i18n	1.8.10
minitest	5.14.4
thread_safe	0.3.6
tzinfo	1.2.9
activesupport	5.2.6
public_suffix	4.0.6
addressable	2.8.0
httpclient	2.8.3
json	2.5.1
algoliasearch	1.27.5
rubyzip	2.3.2
apktools	0.7.4
artifactory	3.0.15
atomos	0.1.3
aws-eventstream	1.1.1
aws-partitions	1.479.0
aws-sigv4	1.2.4
jmespath	1.4.0
aws-sdk-core	3.117.0
aws-sdk-kms	1.44.0
aws-sdk-s3	1.96.2
babosa	1.0.4
claide	1.0.3
colored2	3.1.2
cork	0.3.0
nap	1.1.0
open4	1.3.4
claide-plugins	0.9.2
fuzzy_match	2.0.4
netrc	0.11.0
ffi	1.15.3
ethon	0.14.0
typhoeus	1.4.0
cocoapods-core	1.10.1
cocoapods-deintegrate	1.0.4
cocoapods-downloader	1.4.0
cocoapods-plugins	1.0.0
cocoapods-search	1.0.0
cocoapods-trunk	1.5.0
cocoapods-try	1.2.0
escape	0.0.4
fourflusher	2.3.1
gh_inspector	1.1.3
molinillo	0.6.6
ruby-macho	1.4.0
nanaimo	0.3.0
rexml	3.2.5
xcodeproj	1.20.0
cocoapods	1.10.1
colored	1.2
colorize	0.8.1
highline	2.0.3
commander	4.6.0
faraday-em_http	1.0.0
faraday-em_synchrony	1.0.0
faraday-excon	1.1.0
faraday-httpclient	1.0.1
faraday-net_http	1.0.1
faraday-net_http_persistent	1.2.0
faraday-patron	1.0.0
multipart-post	2.0.0
ruby2_keywords	0.0.5
faraday	1.5.1
faraday-http-cache	2.2.0
rchardet	1.8.0
git	1.8.1
kramdown	2.3.1
kramdown-parser-gfm	1.1.0
no_proxy_fix	0.1.2
sawyer	0.8.2
octokit	4.21.0
unicode-display_width	1.7.0
terminal-table	1.8.0
danger	8.2.3
thor	0.20.3
danger-swiftlint	0.24.5
declarative	0.0.20
digest-crc	0.6.4
unf_ext	0.0.7.7
unf	0.1.4
domain_name	0.5.20190701
dotenv	2.7.6
emoji_regex	3.2.2
excon	0.85.0
http-cookie	1.0.4
faraday-cookie_jar	0.0.7
faraday_middleware	1.0.0
fastimage	2.2.4
jwt	2.2.3
memoist	0.16.2
multi_json	1.15.0
os	1.1.1
signet	0.15.0
googleauth	0.16.2
mini_mime	1.1.0
trailblazer-option	0.1.1
uber	0.1.0
representable	3.1.1
retriable	3.1.2
webrick	1.7.0
google-apis-core	0.4.1
google-apis-androidpublisher_v3	0.9.0
google-apis-playcustomapp_v1	0.5.0
google-apis-iamcredentials_v1	0.6.0
google-apis-storage_v1	0.6.0
google-cloud-env	1.5.0
google-cloud-errors	1.1.0
google-cloud-core	1.6.0
google-cloud-storage	1.34.1
mini_magick	4.11.0
naturally	2.2.1
plist	3.6.0
security	0.1.3
simctl	1.6.8
terminal-notifier	2.0.0
tty-screen	0.8.1
tty-cursor	0.7.1
tty-spinner	0.9.3
word_wrap	1.0.0
rouge	2.0.7
xcpretty	0.3.0
xcpretty-travis-formatter	1.0.1
mime-types-data	3.2021.0704
mime-types	3.3.1
fastlane-plugin-aws_s3	2.0.2
fastlane-plugin-firebase_app_distribution	0.2.9
trainer	0.9.1
xctest_list	1.2.1
fastlane-plugin-test_center	3.15.3
fastlane-plugin-versioning	0.4.5
mini_portile2	2.6.1
racc	1.5.2
nokogiri	1.12.5
xcode-build-times	0.1.1
xcode-install	2.6.2

[lyndsey-ferguson]

Are you stating that the derived data directory is empty?

[lyndsey-ferguson]

Can you include the logs using the --verbose flag? This may help determine when that directory is being emptied.

[craig-jamieson]

@lyndsey-ferguson I'm having trouble using the verbose flag on our pipeline, but I've attached the logs from one of the runs, without using multi_scan we currently:

• Set the derived data for the ui tests
• Copy the test results file from the derived data into the test artefacts, and zip the xcresult file into a archivedResults file

However, now the archivedResults folder is empty, I think that perhaps the derived data file path is not being read correctly when using multi_scan

0.txt

[lyndsey-ferguson]

I noticed this in the logs: #179 can you try not using CircleCI (GitHub Actions maybe), or not setting parallel runners?

[craig-jamieson]

When I don't set parallel runners I can view the screenshots as expected, however, I haven't had an issue with the simulators being cloned on CircleCI, it's only when viewing the archivedResults

[lyndsey-ferguson]

It will be a while before I can look at this. Perhaps the collation of test results is wiping out the screenshots, can you add collate_reports: false as one of the options for multi_scan?

[lyndsey-ferguson]

I want to apologize that I have not responded to this issue. A family health issue is requiring me to narrow my focus on the essentials and I don't have the time I need to focus on the plugin more than a minute. If you can get some other contributors to this project to help with this, I would consider merging a resulting PR.