This document provides a number of sets of confusables mappings for the Khmer script. This data is intended to be used in String Similarity Evaluation [SSE]. Generally, if it is possible to convert a given label into another label by applying the mappings listed in this document to one or more code points or sequences, then the two labels are considered (potentially) confusable.
In String Similarity Evaluation, the first step will be to discover whether two labels can be mapped into each other using these mappings. In a second step, a visual distance metric is determined based on similarity category values. The higher each category is, the more it contributes to the distance. Above some threshold to be determined, the two labels would be considered too distant to be confusable.
This is a DRAFT document released for public comments and not final. For details on how to submit comments see the announcement for public comments on String Similarity Evaluation Data on the ICANN website. https://icann.org/idn
This file presents an overview of all the code points and sequences in and outside the Khmer script that are deemed confusable with at least one other code point or sequence in the script, as well as those that have been defined as variants in the Root Zone LGR [RZ-LGR].
In the HTML format, this summary is contained in the “Repertoire” table. For each listed item in the table, there are one or more links to applicable Confusables Sets.
The data that define the mapping between code points and sequences, including the degree of their similarity or confusability is collected in Confusables Sets which are symmetric and transitive. They also include all variant mappings defined in the [RZ-LGR].
Each Confusables Set is presented in the same format as a “Variant Set” in the RZ-LGR, even though only some of the mappings are actually defined as variant mappings. Those are marked appropriately. The reason confusables and variants must be reviewed and processed together is that two labels may be converted into each other by applying a mix of confusable and variant mappings. Note that labels that only differ by the application of variant mappings alone would have already been identified using the RZ-LGR.
Each mapping has a string similarity category value listed in the “Similarity Category” column. The category shows the degree of similarity, with 1 being (practically or fully) identical, and higher values indicating lesser degrees of confusability. Unless otherwise categorized a mapping is considered “Distinct” (5) and this value is not explicitly listed.
Each mapping has an associated “Reason” code that is a highlighted part of the Comment column.
| Reason Code | Explanation |
|---|---|
| Capitalization | The similarity is between two uppercase forms or between an uppercase and unrelated lowercase form. |
| Confusing | A majority of users familiar with the script of either target, source or both will confuse the pair unless viewed side-by-side. |
| Fallback | One of the code points is a common fallback representation for the other. |
| Font | The confusability depends on font choice. |
| Handwriting | The confusability is caused or strengthened by users being familiar with a shape, even though it is normally only used in handwriting. |
| Other | None of the other reason code applies. In particular, it applies when the category is homograph (1) for a mapping that exceptionally has not been defined as variant. NOTE: This code MUST be accompanied by a comment explaining the nature of the confusability and reasoning for choosing the assigned category value. |
| Underlining | The presence of underlining makes these confusable. This is typically the case when the distinction between two characters rests primarily on a mark below or a descender that might be obscured when underlined. |
| Variant | The mapping is defined as a variant in the [RZ-LGR]. NOTE: the category value is set to 1 by definition. |
| Special values | |
| Not_confusing | Mappings that are marked with Reason code “not_confusing” are listed here for reference only. They will be ignored in screening or label distance calculations. |
| Imposed | This Reason code is automatically generated when a mapping has to be imposed to make a confusables set transitive after Variants have been injected. NOTE:All imposed mappings are assigned category 4 by default. This value may need to be replaced with one of the other category values using an explicit assignment based on the input received during public comment. |
A mapping with Reason code Not_confusing is included here primarily for reference to document that it was considered by the expert team but explicitly resolved as not confusing despite some weak similarity. These mappings are flagged with a symbolic context “excluded-similarity” in the “context” column. This context ensures that while such mappings are documented they are ignored in processing and do not interact with other mappings that are not so marked.
All variant mappings injected from the RZ-LGR will have a Reason code of Variant, but also will have a category value of 1 which reflects their strong confusability. However, category 1 is not limited to variants; any other pair that appears (practically or fully) identical may also be given a similarity category of 1, but then with a different Reason code in the “Comment” column. Code points added as targets of injected variant may have the term “Variant” in their comment. After variants have been injected, the mixed confusables/variant sets are made transitive again, which imposes additional confusables.
In some cases, transitive confusables sets with the "excluded-similarity" context include mappings based on variant pairs. Despite the context, the Reason code remains Variant and the Similarity Category remains 1.
Context rules defined for injected RZ-LGR code points and variants are copied from the RZ-LGR, as are any other items and attributes referenced in or required by those rules.
String similarity considers some sequences that are not part of the [RZ-LGR]. Where all members are code points from the [RZ-LGR], they are annotated as such. This verification ignores any context constraints, it does not establish whether all sequences would indeed be valid for their RZ script.
Reviewers are kindly requested to confirm the proposed degree of similarity as expressed in the category, or suggest improved values. In particular, reviewers are requested to indicate any Imposed mappings that should not receive category 4 by default. Suggested changes in review should be limited to data presented in the list of “Confusables Sets”. If any mappings should have been or should not have been defined as variants in the RZ-LGR, reviewers should note this with a reason for why they believe the mapping should be reclassified.
The data presented here are based on data collected by experts in the script. However, the data were acquired as a list of one-directional mappings. For presentation and later processing these have been mechanically completed into fully symmetric and transitive sets. Note that symmetric mappings have the same category value for forward and reverse direction, but transitive mappings show the value Imposed instead.
For more details on the methodology common to all scripts, and the special methodology used for Han ideographs, please see [101].