Release version 2.4.0

Features

• Added a more GDPR compliant banner for the hosting

Improvements

• Upgrade Dobly module to use the validator system
• Improved legibility of checks
• Applied limiting value for Period and Adaptation Set counts as well

Bugfixes

• Fixed invalid svrl: xml parsing form schematron output #713 #745
• Fixes for null element in MPD implementation #715
• Fixed propagation of segment analysis output to frontend
• Fixes for schematron assertions which were previously silenced #597
• Fixed allowed framerates for DVB #734

Release version 3.0.0-alpha

After a period of quiet but steady progress, we are happy to announce the next alpha version of the Conformance tool.

After starting the migration to PHP8, we took the decision to re-implement both the core patterns and some initial modules of the tool in Laravel, which allows for a more well-organized and more legible code base at the same time.

As this comprises a re-implementation over a period of multiple months, with this initial release we only feature the highlights.

Highlights

• New Interface
  • We have revised the interface, both to make it more dynamic and to better show what is being checked. After filling in a manifest URL, you can now enable/disable modules in the interface, switching up which checks actually get run in a much more straightforward way.
• Segment downloading
  • We have moved the downloading of the segments into a background process. As long as the segments are still being downloaded, a progress bar appears to inform you of the progress. All manifest checks remain functional at this point, segment checks - obviously - check only what has been downloaded already. Once segment downloading is complete, just click on any segment tab to load the latest results.
• Persistent downloads
  • Both the manifest and the segments get downloaded and analysed once per session. Sessions lasts for 6 hours starting at the first manifest analysis, giving plenty of time to retrieve whatever results you might need. If you want to force a re-download, you will have to clear the session with the red button next to the URL, and re-input the URL for now.
• Re-organisation of statement results
  • In order to make the results easier to process, we now have a two-step organisation: first by Manifest or Segment level, followed by the specification. This makes it easier to see whether any issue with the stream might arise from the Manifest or from the Segmented data, rather than having to figure that out from the wording of the statements
• Spec Versioning
  • This version introduces a new level of information, where the relevant version of the spec gets added to the spec itself. All statements that have been re-implemented, but could not be matched to a specification / version for any reason have been marked LEGACY.
  • Legacy checks are either:
    • Statements that have been removed from the specification
    • Statements that no longer apply (e.g. the spec now supports different values from those in the corresponding PHP check).
• Specification-relative dependencies
  • As an in-progress feature, we are working for dependency level statements between different specifications. Enabling a module that has dependency statements related to it, will also enable the dependent module automatically, and copy over the result of the corresponding section. A dependency that is only enabled for this reason can be detected by the green outline of the specification, rather than a full enabled status.
• Skip reasons
  • Starting with this version, we strive to let all module components register the statements they check in advance, combined with a reason why this check might be skipped. Statements that do not get checked are marked with this reason and a state of 'SKIP', and allow for spotting elements of the stream that you might have expected to be enabled.
  • 'SKIP' state statements count as a 'PASS' for module conformance.
• Modular analysis tools
  • With this version we also re-implement all statements to make use of the validator system we introduced before. In a future version this will allow you to more easily plug your own stream validation software into the tool.
  • For now streams can only be validated with the MP4Box tool, but the ISOSegmentValidator will be ported in a future version as well.

DASH-IF Conformance Tools v2.3.0

Features

• Added pluggable validator system
• Analysis of CTAWave 5005A specification
• Added basic processing statistics to UI
• Added limiting of segment downloads
• Added support for analysing updates to live manifests
• Analysis of TS 103 285 v1.4.1 - Section 4.8.3 (DVB updates for live streams)

Improvements

• Upgrade the project to PHP 8.x
• Upgrade the project to use Ubuntu 24.04
• Removed 'smart' segment downloading in favor of robustness

Bugfixes

• Allow non-zero start time for 'static' manifests #594
• Limit the amount of segments per representation to 5 as default #702
• Fixed calculation of stream framerate
• Whitelisted some false negative errors reported by the ISOSegmentValidator #705
• Retrieve content type from manifest instead of stream analysis
• Divergence between CLI and Rest API #707
• Fixed a lot of suppressed PHP warnings and Fatal errors as uncovered by #707

DASH-IF Conformance Tools v2.2.3

What's Changed

• Fix multiple bugs when checking for elements in the DVB testsuite by @dsilhavy in #678
• Fix error message for the DVB UTCTiming element check by @dsilhavy in #679
• Adjust CI/CD jobs to reflect new EC2 server config by @dsilhavy in #682
• Ignore empty lines from segment validation output to remove trailing check mark by @dsilhavy in #683
• Fix usage of wrong global variables for saving codec information by @dsilhavy in #685
• Fix count for contentProtection elements in DVB test by @dsilhavy in #686
• Remove doctype tag from test result output to avoid creating an invalid JSON by @dsilhavy in 5faeee7

DASH-IF Conformance Tools v2.2.2

Bugfixes

• Fix accessing the audio sample description. by @dsilhavy in #659
• Add support for segments with multiple moof/mdat boxes. by @dsilhavy in #660
• Fix a division by zero that lead to tests failing by @dsilhavy in #661
• Fix output of minimumBufferTime in ISO Validator. Should be a float by @dsilhavy in #663
• ISOSegmentValidator: remove error saying that 'isom' can only be comp… by @rbouqueau in #665
• fix: use URL instatiation to validate url string by @cadubentzen in #664
• Fix accessing the wrong attribute for the NAL unit type by @dsilhavy in #668
• Save presentation profile regardless of conformance in previous tests… by @dsilhavy in #674
• Add pasp and btrt validation for "hev1","hvc1" by @dsilhavy in #675

DASH-IF Conformance Tools v2.2.1

Features

#654 Adds a state symbol for all test messages @Phencys

Bugfixes

#654 Fix issues in the CTA-WAVE testsuite leading to wrong parsing of media profiles and invalid period timing @Phencys @dsilhavy
#646 Fix bugs causes by wrong variable names and missing references in the HbbTV testsuite @dsilhavy
#650 Remove a false check for matrix coefficients leading to a wrong message @Phencys
#651 Allow new logger initialization with set module and hook, used for low latency @Phencys
#655 Fix a bug when merging higher level attributes into lower level attributes leading to missing information in the Representations @dsilhavy
#647 Use HTML links in "About" and "FAQ" markdown of landing page @dsilhavy

DASH-IF Conformance Tools v2.2.0

Features

#625 Refactor phase 2: Various bugfixes, optimization, removal of unused code @Phencys
#610 Implemented a new frontend in plain JavaScript utilizing the new REST interface @FritzHeiden
#620 Setup Github action to verify correct execution of CLI for each pull request @FritzHeiden
#622 ISO Segment Validator Refactoring Part 1 @rbouqueau
#626 ISO Segment Validator Refactoring Part 2 @rbouqueau
#631 Add output of segment validation to JSOn result. @dsilhavy
#635 #639 Enable text input and file upload @Phencys @FritzHeiden
#641 Add new flag to disable/enable detailed segment validation output. Reduces the size of the resulting .json @dsilhavy

Improvements

#630 Add missing license files @dsilhavy
#629 Adjust functional test scripts to include right dependencies based on #625 @dsilhavy
#633 Update xsd to 5th edition @Phencys
#642 Add profiles for autodetection of CMAF and CTA-WAVE @dsilhavy
#643 Change wording of segment related checks @dsilhavy

Bugfixes

#618 Fix parsing error of resulting JSON in new UI @FritzHeiden
#637 Fix a bug where stderr output would always be considered empty @Phencys
#640 Fix segment validation results not displaying properly @FritzHeiden

DASH-IF Conformance Tools v2.1.0

Features

#610 Add a new conformance frontend written in plain Javascript. The new frontend uses the new REST Interface to trigger testruns and derive test results. It is automatically deployed to a separate folder and available here: https://beta.conformance.dashif.org/Conformance-Frontend-JCCP/#home @FritzHeiden

Improvements

#616 Added a 30 second timeout / 60 second kill timeout on the segment validator, to prevent it from running indefinitely. @Phencys
#616 Added a category "health" checks to the logger module, to provide information about various issues with the segment validator such as invalid XML files and timeouts @Phencys

DASH-IF Conformance Tools v2.0.0

Features

#548 Add a Command Line Interface to run the tests directly from the command line instead of using the web interface. Multiple options available, see documentation @Phencys
#606 Add a REST interface to trigger conformance tests and @Phencys
#547 Support for running the Conformance Tools in Docker @aldafu
Link Implement a dashboard for evaluation of the conformance validation and to derive line, method, function, class coverage for each of the testruns. Serves as an essential tool to identify parts of the code that are never executed due to missing test content and implementation errors. @dsilhavy

Refactoring

#548, #585, #606 Move all the tests of the existing modules to dedicated .php files and include them using module.php approach. Note this was a major refactoring step influencing the whole existing codebase. @Phencys
#548 Move to a mono repository instead of including the required modules as Git submodules of the main repository. @Phencys
#548 Implement a dedicated module.php for each of the submodules. Includes of the individual test files are done in these module files. @Phencys

Improvements

#548 Default verdict to PASS instead of a null value @Phencys
#548 Add .gitignore file @Phencys
Link Show commit hash on landing page @aldafu
#591 Add support for DVB DASH 2017 @wschidol
#606 Give an empty entry for hooks where no checks have been run. @Phencys
#606 Move Dolby validation to separate module @Phencys
#606 Add additional unit tests for CTA WAVE testcases @Phencys

Bugfixes

#606 Fix multiple typo errors leading to tests not being executed @Phencys
#606 Remove files that are not being used @Phencys
#606 Remove issues with case-insensitive filesystems @Phencys

Rationalize Programming Languages

#606 Removed Python as a dependency @Phencys
#606 Move to php 7.4, prepare migration to php 8 @Phencys

CI/CD

#581 Add workflow to automatically deploy to beta and staging environment. Results are available at https://beta.conformance.dashif.org/ and https://staging.conformance.dashif.org/. Internally, the different versions are hosted on the same EC2 instance and run in dedicated Docker containers. @aldafu
#584 Add support for automatic creation of doxygen documentation. Results are automatically uploaded to https://conformance.dashif.org @dsilhavy
#582 Add support for automatic linting @aldafu
#586 Add support for using phpunit to run the unit tests and generate HTML code coverage report. Results are automatically uploaded to https://conformance.dashif.org/unit-tests @dsilhavy
DASH-IF CTA-WAVE HbbTV Add support for running manifest and segment validation for CTA WAVE, DASH-IF and HbbTV testvectors. Note that for HbbTV the segments are not publicly available, only automatic MPD validation is performed. @aldafu

Documentation

Link Add installation and setup guide including Troubleshooting Section @dsilhavy
Link Add usage guide previously available in pdf format @dsilhavy @aldafu
Link Establish development principles including Git branching strategy, pull request handling and documentation on Github actions @dsilhavy
Link Add documentation how to use the command line interface @dsilhavy
Link Add documentation how to release a new version of the tools @dsilhavy
Link Collect a list of relevant specifications @dsilhavy
Link Collect a list of relevant testvector URLs @dsilhavy
[Link](https://github.com/Dash-Industry-Forum/DASH-IF-Conformance/wiki/REST-Interface] Document the usage of the REST interface @dsilhavy