6x Current Status

6x Current Status

Purpose of this chapter is to present results of Audiveris 6.0 prototype, now that we have reached a first integration of page classifier into Audiveris and a demo of patch classifier.

Dichterliebe example

Score input is available here.

This score is part of Audiveris standard examples (by curtesy of Michael Good). It is a 2-page score, digitally rendered (perhaps by Finale?) and thus of perfect quality..

• Page #1, 275 annotations reported, with 6 errors:

  • Dot in "1." title mistaken for articStaccatoBelow
  • Dot in "Mai" title mistaken for articStaccatoAbove
  • keyboardPedalPed "Ped" mark not detected
  • flag16thDown mistaken for flag32ndDown
  • flag16thDown mistaken for flag8thDown
  • "8" text mistaken for fingering2

• Page #2, 361 annotations reported, with 12 errors:

  • accidentalSharp mistaken for accidentalNatural (3 times)
  • flag8thUp mistaken for graceNoteAppoggiaturaStemUp
  • flag8thDown mistaken for flag16thDown
  • flag16thDown mistaken for flag8thDown (2 times)
  • "le" text mistaken for fingering3
  • "20" text mistaken for fingering2
  • fermataBelow not detected
  • keyboardPedalPed "Ped" mark not detected
  • keyboardPedalUp "*" mark not detected

Total: 18 mistakes on a population of about 639 symbols gives an error rate at 2.8%.
We can consider this result as pretty good. By comparison, Audiveris 5.1 exhibits 0 errors on the same input.

It is worth noting that, while symbol detection is OK, the accuracy of bounding boxes is rather poor. So much that several detected flags were initially rejected by the OMR engine because their left side was abscissa-wise too far from the related stem. We had to relax several OMR engine checks to get these flags accepted. See the case below:

Chula example

Score input is available here.

This score is part of Audiveris examples, it's a scan of a real-world printed score, of good quality though not digitally rendered, rather simple.

• 214 annotations reported with 18 errors
  • timeSig4 not detected (2 times)
  • rest16th mistaken for rest64th
  • repeatDot mistaken for articStaccatoAbove (3 times)
  • repeatDot mistaken for augmentationDot (2 times)
  • repeatDot not detected
  • rest8th mistaken for rest16th (2 times)
  • pair of repeatDot's mistaken for flag32ndDown
  • keyFlat mistaken for accidentalFlat (2 times)

18 errors for about 217 symbols gives an error rate of 8.3%, not so good. (Audiveris 5.1 gives 3 errors (1.4%) on the same input)

Note that a case of overlapping symbols (a flag and a rest) is well detected. See the snapshot below. Unfortunately this rest8th was mistaken for a rest16th, but it's worth noting that this case could never be resolved by Audiveris 5.x engine.

Here below are the outputs of the patch classifier, for the flag and for the rest. They are both correct:

Wrong classification can lead to invalid key signature, as follows, where one of the keyFlat symbols was mistaken for an accidentalFlat:

Another problem is the non-detection on the two timeSig4 symbols, see below. This resulted in invalid times signatures (numerator without denominator)

What should the OMR engine do when it encounters such invalid configurations? As of this writing, the "confidence" attribute provided with any detected symbol comes with the 1.0 value, so this information cannot be used to resolve invalid configurations.

Here, we are hitting a limitation of the page classifier. It does detection, segmentation and classification:

• Segmentation results today in poor bounding boxes, but we can more or less live with that.
• Errors in detection and/or classification are more difficult to cope with.

Some errors can be detected by the OMR engine, as is the case for the invalid configurations mentioned above. But then, we cannot really relaunch the page classifier to disambiguate the cases and anyway its outputs are today not valued: it's an "all or nothing" binary situation.

We thus need an extra tool to help the engine on line, and this may be just a simple classifier (either a glyph classifier or preferably a patch classifier since in these invalid configurations the engine knows rather precisely where to look).

Fortunately, here are the patch classifier outputs on the 2 problems listed above: Respectively the mistaken keyFlat and the missing timeSig4 symbols are correctly recognized:

Also, we can compare the behavior of page and patch classifiers on less frequent symbols, as in this horizontal sequence of pedal symbols here below.

The page annotations show a correct keyboardPedalPed (though its bounds are strongly shifted), a keyboardPedalUp mistaken for a restQuarter and a keyboardPedalPed not detected:

On the same sequence, the patch classifier works much better:

Is this to say that the patch classifier is our silver bullet?

Well it has sometimes a weird behavior, the most problematic one relates to black note heads.

Very often, standard black note heads are mistaken for small black note heads. It is so frequent, that we can suspect a bug somewhere.

Here below, we can observe in a horizontal sequence of natural / head / sharp / head symbols, that accidental symbols are well recognized by the patch classifier, but black note heads are not.

In short, we can say that the patch classifier is promising but still needs to be worked upon.

Bach Invention 5 example

Score input is available here.

This score is part of Audiveris examples, it's a scan of a real-world printed score, moderately complex but with a non-uniform illumination.

This example is interesting to test whether the page classifier should be run on the initial (gray) image or on the binarized (black and white) image. By default, the classifiers run on the binary image but this can easily be changed by setting the option org.audiveris.omr.sheet.Picture.classifiersUseInitial to true.

The result is clear:

• On the initial (gray) image, 447 symbols are detected, almost every staff header (clef and key) is left blank of symbols
• On the binary (B&W) image, 490 symbols are detected, most staff headers are correctly detected.

This is the reason why the binary image has been chosen as the default input for 6.0 classifiers.