Label Generation Rules for the Arabic Script

Overview

This document specifies a set of Label Generation Rules (LGR) for the Arabic script for the second level domain or domains identified above. The starting point for the development of this LGR can be found in the related Root Zone LGR [RZ-LGR-Arab]. The format of this file follows [RFC 7940]. This LGR is adapted from the “Reference LGR for the Second Level for the Arabic Script” [Ref-LGR-und-Arab], for details, see Change History below.

For details and additional background on the Arabic script, see TF-AIDN, “Proposal for Arabic Script Root Zone LGR”, Version 3.4, 18 November 2015 [Proposal-Arabic]. Additional work was done by a TF-AIDN group to create extensions appropriate to the second level and these are incorporated in this LGR. (See “Methodology” below).

The document contains certain provisions for customization. (See “Customizing actions for International Reachability” below.)

Repertoire

The repertoire includes the 128 code points for letters used by languages that are actively written in the Arabic script. The repertoire is a subset of [Unicode 11.0.0]. For details, see Section 3.2 in [Proposal-Arabic]. (The proposal cited has been adopted for the Arabic script portion of the Root Zone LGR.)

For the second level, the repertoire has been augmented with the HYPHEN-MINUS, and three sets of decimal digits:

• European (common) digits
• Arabic-Indic digits
• Extended Arabic-Indic digits

for a total of 159 repertoire elements.

The repertoire excludes code points for which TF-AIDN was unable to find sufficient evidence of use (see Appendix F in [Proposal-Arabic]).

This LGR does not include combining marks or code point sequences. All combining marks have been excluded for these reasons:

• First, they can significantly overproduce and would require additional rules to contain them effectively, complicating the design.
• Second, even where they are required for some languages, they are optional for others.
• Third, this also circumvents the issue regarding duplication between some precomposed code points and combining sequences raised by [IAB].

Repertoire Listing: Each code point or range is tagged with the script or scripts with which the code point is used and one or more other character categories. For each repertoire element, one or more references document sufficient justification for inclusion in the repertoire; see the “References” below. Comments identify the principal languages or orthographies using the code point. Languages are given with their [EGIDS] level.

Note: use of standard Arabic code points by other languages is not marked.

Correlation Between Repertoire and Bidi Rules

Because this LGR contains only a subset of PVALID code points, the Bidi rules expressed in Section 2 of RFC 5893 [160] (Bidi Rule) can be simplified to two constraints:

• Prevent digits (whether European, Arabic-Indic, or Extended-Arabic-Indic) from starting a label.
• In any label, allow only digits from one the sets (whether European, Arabic-Indic, or Extended-Arabic-Indic).

Variants

The variants defined in this LGR are limited to those required for use in zones not shared with any other script. However, because it does not share cross-script variants with other script LGRs, this LGR can be mixed easily with other LGRs in the same zone.

For details on the methodology used to assign in-script variants for letters, see “Methodology for Defining Variants” below.

Digit Variants: All Arabic digits are treated as semantic variants of the corresponding common (ASCII) digits. By transitivity, they are also semantic variants of any native digits in scripts that also include the common digits. Such variant relations are deemed to exist implicitly by transitivity but are not listed explicitly in each reference LGR. Instead, if needed, they are applied by using the Common LGR in label processing.

Because of the restriction on leading digits, most labels would contain a unique code point in addition to any digits, thus no variant labels would actually result from any pro-forma cross-script variants.

To keep digit variant sets manageable in zones where multiple scripts are present, no attempt has been made at identifying cross-script variants among digits of different numeric value or between a digit in one script and a letter in another, such as between digit zero and Latin letter ‘o’. Other mechanisms may be required to prevent homograph labels.

Variant Disposition: This LGR includes “blocked” and “allocatable” variants, assigned according to Section 4, “Final recommendation of variants for Top Level Domains (TLDs)” in [Proposal-Arabic]. These recommendations balance the desire to minimize the number of possible allocatable variants with the need to keep the definition of variants simple. See also the comments given in the listing.

For the second level, three additional variant types are added: “optionally-allocatable”, “activated”, and “optionally-activated”. They are used to improve user acceptance. The rationale for these new types is provided below.

Methodology for Defining Variants

In generating the variant tables, a number of concepts have been used:

• One of the main principles for the stability of the Internet and IDNs is that the end user should be able to reach a website connected to his/her domain name regardless of location. Additionally, an end user reads and types website addresses based on his/her language alphabet and whatever is available in his/her keyboard. Therefore, in order to enforce this principle the input devices (language table) that the user may use to reach a domain name (based on the user location) should be carefully considered when defining variants. Otherwise, it may cause a reachability problem and reduce the user acceptance. For example, if someone registered the domain name “مكة” (all characters from the Arabic language) and a user try to reach the website connected to this domain name from an Internet café or airport, say, in Pakistan. He/she will not be able to reach that website unless if the variant “مکۃ” (Urdu variant) is already allocated and activated. Thus, variants need to be studied from both similarity point of view (by language community) and reachability point of view (based on input devices used by other language communities).

• Consistency is a very important concept in generating variants. Regardless of the selected applied-for label the list of generated allocatable variants should be the same. As we are dealing with normal users at the SLD, their 1st choice (applied-for) label might not be the one that will be used by the internet community. Therefore, a registry should provide the registrant the ability to “correct” his/her choice if he/she was not successful with the first try. For example, the word “Internet” is written in most Arab countries in North Africa as (أنترنت) while it is written in other Arab countries as (إنترنت), while end users often write it as (انترنت). Therefore, if someone registered “أنترنت”, he/she should be able to enable “إنترنت” or “انترنت”. Additionally, many words have two correct ways to write them, for example, both “آدم” and “أدم” are widely used for the same name Adam. Hence, if someone registered “أدم” he/she should be able to enable “آدم” or “ادم”. This is achieved by making variants allocatable in at least one direction in the LGR.

• Based on the principle expressed in the previous item, some code points are involved in at least one variant mapping of type “activated”, “optionally-activated”, or “optionally-allocatable. The dispositions related to these mappings may be optionally modified by a registry by editing the appropriate action elements. See the section on “Customizing Actions for International Reachability” for details of how this feature can be made mandatory or disabled.

The specification of variants in this LGR follows the guidelines in [RFC 8228].

Character Classes

This proposal defines the following named character classes:

• common-digits — ASCII digits: U+0030 0 to U+0039 9
• arabic-indic-digits — Arabic-Indic digits used with Arabic: U+0660 ٠ to U+0669 ٩
• extended arabic-indic-digits — Arabic-Indic digits used with other languages: U+06F0 ۰ to U+06F9 ۹

Whole Label Evaluation (WLE) and Context Rules

Common Rules

By default, the LGR includes the rules and actions to implement the following restrictions mandated by the IDNA protocol. They are marked with ⍟.

• Hyphen Restrictions — restrictions on the allowable placement of hyphens (no leading/ending hyphen and no hyphen in positions 3 and 4). These restrictions are described in Section 4.2.3.1 of RFC 5891 [150]. They are implemented here as context rule on U+002D (-) HYPHEN-MINUS.
• Leading Combining Marks — restrictions on the allowable placement of combining marks (no leading combining mark). This rule is described in Section 4.2.3.2 of RFC 5891 [150].

Right-to-Left Rules

• leading-digit — restrictions on the allowable placement of digits in a right-to-left context (no leading digit), (see section 2.1 of RFC 5893 [160]); implemented here as a context rule on each digit.
• digit-mixing — no mixing between different digit sets (European, Arabic-Indic, and Extended-Arabic-Indic digits) is allowed (see section 2.4 of RFC 5893 [160]); implemented here as a WLE rule with associated action.

Default Actions

This LGR includes the default actions for LGRs as well as the action needed to invalidate labels with misplaced combining marks. They are marked with ⍟. For a description see [RFC 7940].

Arabic-specific Rules

The following rules aim at reducing the allocation of redundant labels.

• No connected ALEF MAKSURA in the Arabic language — restriction on having ALEF MAKSURA (0649) before a right joining or dual joining code point.
• Restricting language mixing in the generated variants — restriction on mixing code points from different languages using the Arabic scripts. This is implemented by not mixing the first group of letters (U+0629 ة, U+0643 ك, U+0647 ه, and U+0649 ى) mostly used only in the Arabic language, with a second group of letters (U+06A9 ک, U+06AA ڪ, U+06BE ھ, U+06C1 ہ, U+06C2 ۂ, U+06C3 ۃ, and U+06CC ی) not commonly used in the Arabic language.

In addition, this LGR includes WLE rules and actions specific to the Arabic script found in Section 5, “Whole Label Evaluation (WLE) rules” in [Proposal-Arabic]. As specified, the rules and actions serve to prevent the mixing of two variants of the same code point within the same label. This reduces overproduction of variant labels. The rules are listed here with the numbers given in Table 17 in [Proposal-Arabic]. See also the comments given for each rule or action. Note that that WLE Rules 1, 2, 3, and 8 part of that Table 17 are omitted here because they are superseded by the rule restricting language mixing described above.

• no-mix-heh-goal — WLE Rule 4: do not mix Arabic letters HEH and HEH GOAL in the same label
• no-mix-heh-goal-ae — WLE Rule 5: do not mix Arabic letters HEH GOAL and AE in the same label
• no-mix-heh-ae — WLE Rule 6: do not mix Arabic letters HEH and AE in the same label
• no-mix-heh-doachashmee — WLE Rule 7: do not mix Arabic letters HEH and HEH DOACHASHMEE in the same label
• no-mix-noon-with-three-dots-above-yeh-with-three-dots-below — WLE Rule 9: do not mix Arabic letters NOON WITH THREE DOTS ABOVE and YEH WITH THREE DOTS BELOW in the same label
• no-mix-peh-noon-with-three-dots-above — WLE Rule 10: do not mix Arabic letters PEH and NOON WITH THREE DOTS ABOVE in the same label
• no-mix-feh-with-dot-moved-below — WLE Rule 11:do not mix Arabic letters FEH and FEH WITH DOT MOVED BELOW in the same label
• no-mix-qaf-with-dot-above — WLE Rule 12: do not mix Arabic letters QAF and QAF WITH DOT ABOVE in the same label
• no-mix-feh-qaf-with-dot-above — WLE Rule 13: do not mix Arabic letters FEH and QAF WITH DOT ABOVE in the same label
• no-mix-kaf-with-ring-gaf — WLE Rule 14: do not mix Arabic letters KAF WITH RING and GAF in the same label
• no-mix-kaf-with-ring-keheh-with-three-dots-above — WLE Rule 15: do not mix Arabic letters KAF WITH RING and KEHEH WITH THREE DOTS ABOVE
• no-mix-gaf-keheh-with-three-dots-above — WLE Rule 16: do not mix Arabic letters GAF and KEHEH WITH THREE DOTS ABOVE in the same label

Arabic Script-specific Actions

The LGR includes several actions that invalidate a label violating the constraints expressed by the Arabic-specific rules. In addition, the three non-default variant mapping types used in this LGR trigger actions that result in variant label dispositions as follows:

• if at least one variant is in {activated}, force delegation,
• if at least one variant is in {optionally-activated}, by default the label is allocatable, but the LGR can be customized to force delegation for these variant labels by selecting a different disposition in the corresponding action,
• if at least one variant is in {optionally-allocatable}, by default the label is blocked, but the LGR can be customized to make the variant label allocatable instead, by selecting a different disposition in the corresponding action.

Variant mappings of type “activated” lead to a disposition of “activated”, which implies means the variant label should be allocated. This variant type is used here for the variant mappings between the three sets of digits. Activated variants are handled by the first action above, and are not affected by the optional customizations described below.

Customizing Actions for International Reachability

As described above, this LGR is designed so that full “international reachability” is optionally available. It can easily be enabled with a small customization.

The disposition for the second of the actions listed above is by default set to “allocatable” which makes the variant available to registrants that would like to support international reachability. Some registry operators may desire to make support for such variants mandatory, in which case the disposition (disp=“allocatable” in the XML) for the relevant action could be changed from “allocatable” to “activated” without further changes to the LGR.

The disposition for the last of the actions listed above is by default set to “blocked” which reserves the variants of type “optionally-allocatable”. Some registry operators may desire to support such variants as allocatable, in which case the disposition (disp=“blocked” in the XML) for the relevant action could be changed from “blocked” to “allocatable” without further changes to the LGR.

Most of the code points involved in labels resulting in “activated” or “allocatable” actions are subject to “no-mix” rules to decrease the number of activated or allocatable variants. This is especially a concern for the “activated” action which results in forced delegation.

If optional variant labels are all made allocatable, a significantly large number of allocatable variant labels can potentially be created. Keeping the allocatable variant labels manageable is advised by SSAC in their [SAC060] report. One way to contain such variant labels, as suggested by the Arabic script community, is to allow only the variant labels which are usable and still block those which may be redundant, e.g. cannot be typed from a single keyboard supporting a given language. For this purpose, the no language mixing rule is part of this LGR.

Furthermore, applying this customization for variant sets involving code points which do not have “no-mix” should be done with caution. This is especially true for the variant set including the ALEF characters, for which an external policy should be established to ensure that only a strictly limited number of variant labels are allocated. It may not be possible to create a general policy that determine which of the variants from among a potential large variant pool are indeed suitable; therefore, the selection may have to be left as a choice to the registrant, but within a strict cap.

Methodology and Contributors

The LGR in this document has been adapted from the corresponding Reference LGR for the Second Level. The Second Level Reference LGR for the Arabic Script has been edited by Michel Suignard and Asmus Freytag, based on extensive community input. LGR is based on the Root Zone LGR for Arabic and relies on information contained or referenced therein, see [RZ-LGR-Arab]. Suitable extensions for the second level have been applied according to the [Guidelines] and with community input. The original proposal for an Arabic Script Root Zone LGR [Proposal-Arabic] that this LGR is based on, was developed by the Task Force for Arabic Script IDNs [TF-AIDN]. The specific features related to the International Reachability aspect were drafted by a TF-AIDN sub-group (namely in alphabetical order: Abdalmonem Tharwat Galila, Abdeslam Nasri, Abdulaziz Al-Zoman, Abdulrahman Alghadir, Hazem Hezzah, Nabil Benamar, Raed Alfayez, Tarik Merghani). In addition, Sarmad Hussain provided editorial support.

For more information on methodology and contributors to the underlying Root Zone LGR, see Section 2 and Appendix H in [Proposal-Arabic], as well as [RZ-LGR-Overview].

Changes from Version Dated 22 April 2021

The formulation of the no-mix rule for languages in Version 1 was found to be defective and has been corrected. Without the correction, the LGR would disallow labels with characters from the common subset (those neither in group 1 or group 2). The result of the fix is to allow such labels as expected, while continuing to prohibit labels containing code points from both group 1 and group 2 in the same label.

Changes from Version Dated 11 January 2022

Unicode Version has been updated.

Changes from Version Dated 25 October 2024

This version modifies the Second Level Reference LGR for the Arabic Script [Ref-LGR-und-Arab] by injecting the complete (maximal) set of cross-repertoire variants, including any imposed by transitivity. Injected variants are marked by special comments. Any character classes, context and WLE rules required for testing are also imported. In all other respects, description or annotations have not been updated to reflect the injected variants. Any LGR elements that might pose a possible conflict when the LGR is used with the Common LGR for cross-repertoire variant resolution are clearly marked. All modifications are carried out mechanically; there may be an occasional context rule or similar that is injected because its definition relates to some repertoire element even though it can never be triggered by any label valid under the modified LGR.

References

The following general references are cited in this document:

[EGIDS]

Lewis and Simons, “EGIDS: Expanded Graded Intergenerational Disruption Scale,” documented in [SIL-Ethnologue] and summarized here:
https://en.wikipedia.org/wiki/Expanded_Graded_Intergenerational_Disruption_Scale_(EGIDS)

[Guidelines]

ICANN, “Guidelines for Developing Reference LGRs for the Second Level”, (Los Angeles, California: ICANN, 27 May 2020),
https://www.icann.org/en/system/files/files/lgr-guidelines-second-level-27may20-en.pdf

[IAB]

Internet Architecture Board (IAB), “IAB Statement on Identifiers and Unicode 7.0.0”
https://www.iab.org/documents/correspondence-reports-documents/2015-2/iab-statement-on-identifiers-and-unicode-7-0-0/

[MSR-5]

Integration Panel, “Maximal Starting Repertoire — MSR-5 Overview and Rationale”, 24 June 2021,
https://www.icann.org/en/system/files/files/msr-5-overview-24jun21-en.pdf

[Proposal-Arabic]

TF-AIDN, “Proposal for Arabic Script Root Zone LGR”, Version 3.4, 18 November 2015
https://www.icann.org/en/system/files/files/arabic-lgr-proposal-18nov15-en.pdf

[RFC 6365]

Hoffman, P. and J. Klensin, “Terminology Used in Internationalization in the IETF”, BCP 166, RFC 6365, DOI 10.17487/RFC6365, September 2011,
https://www.rfc-editor.org/info/rfc6365

[RFC 7940]

Davies, K. and A. Freytag, “Representing Label Generation Rulesets Using XML”, RFC 7940, August 2016,
https://www.rfc-editor.org/info/rfc7940

[RFC 8228]

A. Freytag, “Guidance on Designing Label Generation Rulesets (LGRs) Supporting Variant Labels”, RFC 8228, August 2017,
https://www.rfc-editor.org/info/rfc8228

[RZ-LGR-Overview]

Integration Panel, “Root Zone Label Generation Rules (RZ LGR-5): Overview and Summary”, 26 May 2022 (PDF),
https://www.icann.org/sites/default/files/lgr/rz-lgr-5-overview-26may22-en.pdf

[RZ-LGR-Arab]

ICANN, Root Zone Label Generation Rules for the Arabic Script (und-Arab), 26 May 2022 (XML)
https://www.icann.org/sites/default/files/lgr/rz-lgr-5-arabic-script-26may22-en.xml

[SAC060]

ICANN Security and Stability Advisory Committee (SSAC), “SSAC Comment on Examining the User Experience Implications of Active Variant TLDs Report”, 23 July 2013,
https://www.icann.org/en/system/files/files/sac-060-en.pdf

[SIL-Ethnologue]

David M. Eberhard, Gary F. Simons & Charles D. Fennig (eds.). 2021. Ethnologue: Languages of the World, Twenty fourth edition. Dallas, Texas: SIL International. Online version available as
https://www.ethnologue.com

[TF-AIDN]

Blog, “Task Force for Arabic Script IDNs”
https://community.icann.org/display/MES/Task+Force+on+Arabic+Script+IDNs
workspace:
https://community.icann.org/display/MES/TF-AIDN+Work+Space

[Ref-LGR-und-Arab]

ICANN, Second Level Reference Label Generation Rules for the Arabic Script (und-Arab), 25 October 2024 (XML)
https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-arabic-script-25oct24-en.xml
non-normative HTML presentation:
https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-arabic-script-25oct24-en.html

[Unicode 11.0.0]

The Unicode Consortium. The Unicode Standard, Version 11.0.0, (Mountain View, CA: The Unicode Consortium, 2018. ISBN 978-1-936213-19-1)
https://www.unicode.org/versions/Unicode11.0.0/

For references consulted particularly in designing the repertoire for the Arabic Script for the second level please see details in the Table of References below.

References [0] to [12] refer to the Unicode Standard versions in which the corresponding code points were initially encoded. References [100] and above correspond to sources justifying the inclusion of the corresponding code points. Entries in the table may have multiple source reference values. In the listing of whole label evaluation and context rules, references [150] and [160] indicate the source for common rules.