[Annex 2] Summary of Evaluation Test Results

1. Summary of Robustness Test Results

• Several enterprises scored highly on robustness tests for both CCI and CMI. This outcome, and the results of the audibility tests indicate that the technology level of digital watermarks for music have moved beyond the research and experiment stage and have reached a level of practical utilization.
• Many participating enterprises faired well in "D/A - A/D transition", which is a security requirement peculiar to digital watermarks, and one of the most important robustness requirements. On the other hand, for "down sampling", "amplitude compression", "time and pitch compression / decompression" and "altered number of channels", the results were polarized as some enterprises scored highly while others did not.
• Many participating enterprises faired well in tests on overall robustness against compression. However, results for tests on robustness against high levels of compression (especially for "RealAudio" which is a streaming transmission) were generally poor.
• Aside from a few exceptions, good results could not be obtained on tests for robustness against compression for broadcasting in a broadcasting environment for AM broadcasting. However, for FM and PCM broadcasting, several enterprises scored highly.

2. Summary of Audibility Test Results

• Watermarks from several enterprises went undetected by many of the Golden Ears and Silver Ears for all of the assignment compositions.
• The level of difficulty in embedding watermarks, when judged from the rate of detection by the listeners, was greatest for distinctive acoustic compositions, then classical music, and finally popular music. The level of detection in popular music for enterprises that received certification was extremely low.
• Enterprises that scored highly in audibility tests were also proven capable in acoustic science analysis (frequency analysis, etc.) as well.

3. Others (Processing Time)

• Differences in time required to embed watermarks per one minute of music were insignificant. However, for the extracting of the watermarks, some enterprises required only several seconds to process one minute of music, while others required much more time.