| Reference LGR for language: Arabic (ar) | lgr-second-level-arabic-rsp-full-variant-language-25oct24-en |
|---|
This document is mechanically formatted from the above XML file for the LGR. It provides additional summary data and explanatory text. The XML file remains the sole normative specification of the LGR.
| Date | 2024-10-25 |
|---|---|
| LGR Version | 3 (Second Level Reference LGR) |
| Language | ar (Arabic Language) |
| Unicode Version | 11.0.0 |
Description
RSP FULL VARIANTS LGR
This file has been modified from the cited reference LGR by mechanically injecting the complete (maximal) set of out-of-repertoire code points and cross-repertoire variants from the Common LGR, including any that are imposed by the required symmetry and transitivity of the maximal variant set. These additions are marked with "(injected)" in comments.
Any other LGR elements, such as character classes, context and WLE rules required for processing with the augmented LGR are also imported.
Any existing LGR elements that might pose a possible conflict when the LGR is used with the Common LGR for cross-repertoire variant resolution are clearly marked.
Note that existing annotations, comments and descriptions have been retained largely unchanged. Other than this notice, no effort has been made to reflect the presence of any of the added elements or the purpose and intended use of this label generation file.
To distinguish any generated annotations from those that were part of the original content of the LGR, they use color.
Created using: lgr-second-level-arabic-language-25oct24-en and lgr-second-level-common-25oct24-en
Label Generation Rules for the Arabic Language
Overview
This document specifies a set of Label Generation Rules (LGR) for the Arabic language for the second level domain or domains identified above. The ultimate starting point for the development of this LGR can be found in the related Root Zone LGR [RZ-LGR-Arab]. Note that while it is the Arabic Script Root Zone LGR that forms the starting point, the LGR defined here covers the Arabic Language. The format of this file follows [RFC 7940]. This LGR is adapted from the “Reference LGR for the Second Level for the Arabic Language” [Ref-LGR-ar-Arab], for details, see Change History below.
For details and additional background on the Arabic script, see “Proposal for Arabic Script Root Zone LGR”, Version 3.4, 18 November 2015 [Proposal-Arabic]. Additional work was done by a TF-AIDN group to create extensions appropriate to the second level and these are incorporated in this LGR. (See “Methodology” below).
The document contains certain provisions for customization. (See “Customizing actions for International Reachability” below.)
Standalone LGR: This LGR is designed to be used in a zone that does not cater to IDNs other than those valid under this LGR. This LGR lacks features that would allow its use in the context of another LGR in the same zone, and it may contain other features incompatible with such use.
Repertoire
The repertoire is a subset of the one described in Section 3.2 of [Proposal-Arabic] and only includes the 36 code points for letters in active everyday general use in the Arabic language. The repertoire is a subset of [Unicode 11.0.0]. For details, see Section 3.2 in [Proposal-Arabic]. (The proposal cited has been adopted for the Arabic script portion of the Root Zone LGR.)
For the second level, the repertoire has been augmented with the HYPHEN-MINUS, and two sets of decimal digits:
- European (common) digits
- Arabic-Indic digits
for a total of 57 repertoire elements.
This LGR does not include combining marks or code point sequences. All combining marks have been excluded for these reasons:
- First, they can significantly overproduce and would require additional rules to contain them effectively, complicating the design.
- Second, even where they are required for some languages, they are optional for others.
- Third, this also circumvents the issue regarding duplication between some precomposed code points and combining sequences raised by [IAB].
Any code points outside the Arabic Language repertoire that are targets for out-of-repertoire variants would be included here only if the variant is listed in this file. In this case they are identified as a reflexive (identity) variant of type “out-of-repertoire-var”. Whether or not they are listed, they do not form part of the repertoire.
Repertoire Listing: Each code point or range is tagged with the script or scripts with which the code point is used and one or more other character categories. For each repertoire element, one or more references document sufficient justification for inclusion in the repertoire; see the “References” below. For code points in the repertoire, comments identify the language using the code point.
Correlation Between Repertoire and Bidi Rules
Because this LGR contains only a subset of PVALID code points, the Bidi rules expressed in Section 2 of RFC 5893 [160] (Bidi Rule) can be simplified to two constraints:
- Prevent digits (whether European, Arabic-Indic, or Extended-Arabic-Indic) from starting a label.
- In any label, allow only digits from one the sets (whether European, Arabic-Indic, or Extended-Arabic-Indic).
Variants
This LGR is designed for use in zones shared by more than one language. For details, see Section 3, “Use of Multiple Reference LGRs in the Same Zone” in [Level-2-Overview]. Where appropriate, cross-language variants have been defined to mutually exclude labels from different languages that could otherwise be substituted by the users. Once a label has been delegated for one language, any of its variant labels consisting entirely of cross-language variants would be blocked. Any label containing at least one code point that is not a cross-language variant would be considered distinct. Because of that, even large numbers of defined cross-language variants generally do not lead to a high percentage of labels experiencing a collision.
Digit Variants: All Arabic digits are treated as semantic variants of the corresponding common (ASCII) digits. By transitivity, they are also semantic variants of any native digits in scripts that also include the common digits. Such variant relations are deemed to exist implicitly by transitivity but are not listed explicitly in each reference LGR. Instead, if needed, they are applied by using the Common LGR in label processing.
Because of the restriction on leading digits, most labels would contain a unique code point in addition to any digits, thus no variant labels would actually result from any pro-forma cross-script variants.
To keep digit variant sets manageable in zones where multiple scripts are present, no attempt has been made at identifying cross-script variants among digits of different numeric value or between a digit in one script and a letter in another, such as between digit zero and Latin letter ‘o’. Other mechanisms may be required to prevent homograph labels.
Variant Disposition: This LGR includes “blocked” and “allocatable” variants, assigned according to Section 4, “Final recommendation of variants for Top Level Domains (TLDs)” in [Proposal-Arabic]. These recommendations balance the desire to minimize the number of possible allocatable variants with the need to keep the definition of variants simple. See also the comments given in the listing.
For the second level, three additional variant types are added: “optionally-allocatable”, “activated”, and “optionally-activated”. They are used to improve user acceptance. The rationale for these new types is provided below. Note that some of these optional variants map to code points that are not declared as part of the repertoire. If such optional variants are enabled, the reflexive “out-of-repertoire-var” mappings for the targets of these mappings must be removed, lest the generated variants fail to validate.
Methodology for Defining Variants
In generating the variant tables, a number of concepts have been used:
When building a variant table, for each code point in the Arabic language repertoire, a full study was conducted across the whole Arabic script in order to identify all possible variants corresponding to the code point. This was done for more protection to the registry’s TLD-space and to minimize the overhead of re-studying when the registry decides to support more than one language from the Arabic script. Hence, conducting this variant study across the whole Arabic script provides an IDN registry with the following benefits:
- Protection to the registry name space regardless of the supported languages.
- Doing the work for a language one time. No need to re-study the code-point’s relationship each time a new language (or new set of code point) is supported by the registries.
- Flexibility to add more language as they become ready without affecting currently supported languages.
- Consistency with the principle of identifying all possible variants without neglecting or overlooking any similarities without documenting it.
One of the main principles for the stability of the Internet and IDNs is that the end user should be able to reach a website connected to his/her domain name regardless of location. Additionally, an end user reads and types website addresses based on his/her language alphabet and whatever is available in his/her keyboard. Therefore, in order to enforce this principle the input devices (language table) that the user may use to reach a domain name (based on the user location) should be carefully considered when defining variants. Otherwise, it may cause a reachability problem and reduce the user acceptance. For example, if someone registered the domain name “مكة” (all characters from the Arabic language) and a user try to reach the website connected to this domain name from an Internet café or airport, say, in Pakistan. He/she will not be able to reach that website unless if the variant “مکۃ” (Urdu variant) is already allocated and activated. Thus, variants need to be studied from both similarity point of view (by language community) and reachability point of view (based on input devices used by other language communities).
Consistency is a very important concept in generating variants. Regardless of the selected applied-for label the list of generated allocatable variants should be the same. As we are dealing with normal users at the SLD, their 1st choice (applied-for) label might not be the one that will be used by the internet community. Therefore, a registry should provide the registrant the ability to “correct” his/her choice if he/she was not successful with the first try. For example, the word “Internet” is written in most Arab countries in North Africa as (أنترنت) while it is written in other Arab countries as (إنترنت), while end users often write it as (انترنت). Therefore, if someone registered “أنترنت”, he/she should be able to enable “إنترنت” or “انترنت”. Additionally, many words have two correct ways to write them, for example, both “آدم” and “أدم” are widely used for the same name Adam. Hence, if someone registered “أدم” he/she should be able to enable “آدم” or “ادم”. This is achieved by making variants allocatable in at least one direction in the LGR.
Even though we are constructing an LGR at a language level, because we are addressing international reachability, we need to prevent mixing between code points from different language tables while generating a valid variant. Otherwise, some of permuted variants will be composed of characters (code points) that are not part of a single language or they are not easily available for a single input device (keyboard). Therefore, from a practical and realistic point of view, and to significantly minimize the number of allocatable variants and maximize the number of blocked variants, it blocking these unrealistic variants represents best practice.
Based on the principle expressed in the previous item, code points not part of the Arabic language repertoire have been added to variant sets to facilitate international reachability. Many of these out-of-repertoire code points are involved in at least one variant mapping of type “activated”, “optionally-activated”, or “optionally-allocatable. The dispositions related to the extended variants may be optionally modified by a registry by editing the appropriate action elements. See the section on “Customizing Actions for International Reachability” for details of how this feature can be made mandatory or disabled.
Note: Please note that the set of supported languages used for implementing the above concepts (namely international reachability and no mixing between languages) is a first version. The set may be updated in the future by adding new languages whenever they become ready.
The specification of variants in this LGR follows the guidelines in [RFC 8228].
Character Classes
This proposal defines the following named character classes:
- common-digits — ASCII digits: U+0030 0 to U+0039 9
- arabic-indic-digits — Arabic-Indic digits used with Arabic: U+0660 ٠ to U+0669 ٩
- extended arabic-indic-digits — Arabic-Indic digits used with other languages: U+06F0 ۰ to U+06F9 ۹
- arabic-language — code points used in pure Arabic language labels
- pashto-language — code points used in pure Pashto language labels
- persian-language — code points used in pure Persian language labels
- urdu-language — code points used in pure Urdu language labels
- dual-joining — joining-type D
- right-joining — joining-type R
- joins-to-the right — union of joining-type R and joining-type D
Whole Label Evaluation (WLE) and Context Rules
Common Rules
By default, the LGR includes the rules and actions to implement the following restrictions mandated by the IDNA protocol. They are marked with ⍟.
- Hyphen Restrictions — restrictions on the allowable placement of hyphens (no leading/ending hyphen and no hyphen in positions 3 and 4). These restrictions are described in Section 4.2.3.1 of RFC 5891 [150]. They are implemented here as context rule on U+002D (-) HYPHEN-MINUS.
- Leading Combining Marks — restrictions on the allowable placement of combining marks (no leading combining mark). This rule is described in Section 4.2.3.2 of RFC 5891 [150].
Right-to-Left Rules
- leading-digit — restrictions on the allowable placement of digits in a right-to-left context (no leading digit), (see section 2.1 of RFC 5893 [160]); implemented here as a context rule on each digit.