Kerning feature for Malayalam fonts

At SMC, we’ve been continuously working on improving the fonts for Malayalam – by updating to newer opentype standard (mlm2), adding new glyphs, supporting new Unicode points, fixing shaping issues, reducing complexity and the compiled font size, involving new contributors etc.

Recently, out of scratching my own itch, I decided that it is high time to fix the annoyance that combination of Virama(U+D04D ് ) with quote marks (‘ ” ‘ ’ “ ” etc) used to overlap into an ugly amalgam in all our fonts. Usually Virama(് ) connects/combines two consonants which makes all 3 into a new glyph – for example സ+്+ന is shaped into a new glyph സ്ന (Note that you need a traditional orthography font installed to see the distinction in this example. Many of them are available here) . The root of the problem is that sometimes when Virama(് ) appears individually in a word such as “സ്വപ്‌‌നം” it connects two consonants പ and ന, it is positioned above the x height of most glyphs and it shall not have much left and right bearing to avoid ugly spacing between the consonants പ and ന.  Because of small side bearings, in fact of the negative right bearing (് protrudes beyond the right bearing) when a quote mark follows it, quote mark gets a little juxtaposed into Virama glyph and renders rather bad. The issue is quite prominent when you professionally typeset a book or article in Malayalam using XeTeX or SILE.

Fontforge’s tools made it easy to write opentype lookup rules for horizontal pair kerning to allow more space between Virama(് ) and quote marks. You can see the before and after effect of the change with Rachana font in the screenshot.

Rachana-kerning-before-and-after

Update 26/03/2016: Many applications already support the kerning feature out of the box, including Firefox, SILE and VLC (3.0.0-git for subtitles) that I have tested, but many still need support, for instance LibreOffice, Kwrite etc. Here is a screenshot of VLC (3.0.0-git) taking kerning rules into account while displaying Malayalam subtitle.

vlc-subtitle-kerning

Other fonts like AnjaliOldLipi, Meera and Chilanka also got this feature and those will be available with the new release in the pipeline. I have plans to expand this further to use with post-base vowels of വ(്വ) and യ(്യ) with abundant stacked glyphs that Malayalam has.

Advertisements

Paratype PT Serif and PT Mono fonts are now available in Fedora

Paratype has a set of nice Latin/pan-Cyrillic typefaces including sans-serif, serif and monospace fonts. The sans-serif typeface, PT Sans, released in 2010 has been part of Fedora for a long time and it is the default font for Cyrillic/Russian. It is a nice font for display in desktop, documents and web.

In 2011, PT Serif and PT Mono were added to the collection. They both are nice looking and very good quality fonts. All the fonts are also made available under OFL (Open Font License) and all it needed was someone to package them for Fedora. Something in my todo list for a long time, couple of weeks ago I have leveraged the spec file of paratype-pt-sans-fonts and packaged the serif and monospaced fonts. Paratype distributes the source tar balls separately for each set and Fedora mandates to create individual packages in such cases. Thanks to the review and comments from Fedora Fonts-SIG, especially Parag Nemade, two new font packages – paratype-pt-serif-fonts and paratype-pt-mono-fonts are now in Fedora repositories.

Obligatory screenshot of both fonts:

Paratype-PT-Serif-Mono-fonts

Malayalam opentype specification – part 1

This post is a promised followup from last November documenting intricacies of opentype specification for Indic languages, specifically for Malayalam. There is an initiative to document similar details in the IndicFontbook, this series might make its way into it. A Malayalam unicode font supporting traditional orthography is required to correctly display most of the examples described in this article, some can be obtained from here.

Malayalam has a complex script, which in general means the shape and position of glyphs are determined in relation with other surrounding glyphs, for example a single glyph can be formed out of a combination of independent glyphs in a specific sequence forming a conjunct. Take an example: ക + ്‌ ‌+ ത + ്‌ + ര => ക്ത്ര in traditional orthography. Note that in almost all the cases glyph shaping and positioning change such as this example is due to the involvement of Virama diacritic ” ്‌ “. The important rules on glyph forming are:

  1. When Virama is used to combine two Consonants, it usually forms a Conjunct, such as ക + ്‌ ‌+ ത => ക്ത. This is known as C₁ conjoining as a half form of first consonant is joined with second consonant.
  2. The notable exceptions to point 1 are when the followed Consonants are either of യ, ര, ല, വ. In those cases, they form the ‘Mark’ shapes of യ, ര, ല, വ =>  ്യ, ്ര,  ്ല,  ്വ. This is known as C₂ conjoining as a modified form of second consonant is attached to the first consonant.
  3. When Virama is used to combine a Consonant with Vowel, the Vowel forms a Vowel Mark => such as ാ, ി, ീ.

Opentype organizes these glyph forming and shaping logic by a sequence of ‘Lookup tables (or rules)’ to be defined in the font. The first part gives an overview of the relevant lookup rules used for glyph processing by shaping engine such as Harfbuzz or Uniscribe.

Only those opentype features applicable for Malayalam are discussed. The features (or lookups) are applied in the following order:

  1. akhn (Akhand – used for conjuncts like ക്ക, ക്ഷ, ല്ക്ക, യ്യ, വ്വ, ല്ല etc)
  2. pref (Pre-base form – used for pre base form of Ra –  ്‌ + ര =   ്ര)
  3. blwf (Below base form – used for below base form of La – virama+La – ്‌ + ല =  ്ല)
  4. half (Half form – Not used in mlm2 spec by Rachana and Meera, but used in mlym spec and might be useful later. For now, ignore)
  5. pstf (Post base form – used for post base forms of Ya and Va – ്‌ +യ =  ്യ, ്‌ + വ = ്വ. Note that  യ്യ & വ്വ are under akhn rule)
  6. pres (Pre-base substitution – mostly used for ligatures involving pref Ra – like ക്ര, പ്ര, ക്ത്ര, ഗ്ദ്ധ്ര  etc)
  7. blws (Below base substitution – used for ligatures involving blwf La – like ക്ല, പ്ല, ത്സ്ല etc. Note that  ല്ല is under akhn rule)
  8. psts (Post base substitution – used for ligatures involving post base Matras – like കു, ക്കൂ, മൃ etc)
  9. abvm (Above base Mark  positioning – used for dot Reph – ൎ)

Last 3 forms (pres, blws, psts) are presentation forms, they have lower priority in the glyph formation. They usually form the large number of secondary glyphs. The final one (abvm) is not a GSUB (glyph substitution lookup) but a GPOS (glyph position lookup) – this is used to position dotreph correctly above the glyphs.

  • akhn: Use this for conjuncts (കൂട്ടക്ഷരങ്ങള്‍) like ക്ക, ട്ട, ണ്ണ, ക്ഷ, യ്യ, വ്വ, ല്ല, മ്പ. This rule has the highest priority, so akhn glyphs won’t be broken by the shaping engine.
  • pref: Used only for pre-base form of Ra ര –  ്ര
  • blwf: Used only for below base form of La ല –  ്ല
  • pstf: Used for the post base forms of Ya, Va യ, വ – ്യ, ്വ
  • pres: One of the presentation forms, mostly used for ligatures/glyphs with pref Ra ര – like ക്ര, പ്ര, ക്ത്ര, ഗ്ദ്ധ്ര etc. This could also used together with the ‘half’ forms in certain situations, but that is for later.
  • blws: Used for ligatures/glyphs with blwf La ല – like ക്ല, പ്ല, ത്സ്ല etc.
  • psts: Used by a large number of ligatures/glyphs due to the post base Matras (ു,ൂ,ൃ etc) – like  കു, ക്കൂ, മൃ etc. Other Matras (ാ,ി,ീ,േ,ൈ,ൈ,ൊ,ോ,ൌ,ൗ) are implicitly handled by the shaping engine based on their Unicode properties (pre-base, post-base etc) as they don’t form a different glyph together with a consonant – there is no need to define lookup rules for those matras in the font.

I will discuss these lookup rules and how they fit in the glyph shaping sequence with detailed examples in next episodes.

(P.S: WordPress tells me I started this blog 7 years ago on this day. How time flies.)

SMC Malayalam fonts updated versions released

The Unicode fonts for Malayalam maintained by Swathanthra Malayalam Computing were last updated almost 2 years ago. They all were supporting just the v1 Indic opentype spec. But there were rendering problems with the fonts under Harfbuzz.

I was fortunate to attend the Open Source Language Summit 2012 (last year!) organized by Wikimedia Foundation and Red Hat (thank you, guys!) where many of the Indic language experts came together to work on issues at hand. The 2-days workshop helped me greatly to get much more insight into fonts, opentype spec and Harfbuzz in general. Since then I have been spending a lot of effort in updating and fixing the Malayalam fonts and also testing git snapshots of Harfbuzz and reporting issues to harfbuzz development list.

In the meantime, Harfbuzz matured enough and fixed many rendering issues. Thanks to the last Udupi hackfest by Behdad and Jonathan Kew, all known issues with Malayalam shaping has been addressed. And we were busy updating the fonts, opentype lookup rules and fixing bugs to work with old shapers (old pango, Qt, ICU Layout Engine, Windows XP) as well as the new ones (Harfbuzz, Uniscribe, Adobe). The v1 Indic opentype spec was a mess due to ‘undesirable’ Halant reordering (Consonant+Halant forms were ligated while it should have been Halant+Consonant). It has caused a lot of grief on the font developers and shaping engine developers side. With the v2 spec (mlm2 script tag for Malayalam), this has been changed and there is no need to perform Halant shifting anymore by shaping engines. I was leading the effort of porting to mlm2 spec of Malayalam fonts. We could port only Meera and Rachana for now, and RaghuMalayalam taken care by a few sed scripts.

During the 12th anniversary celebrations of Swathanthra Malayalam Computing group, the new version of fonts (5.1 supporting old shapers and 6.0 supporting new shapers) were released. See the email to smc-discuss for details. Remaining fonts also need to be updated, there is interest from community to collaborate on that. The new release will show up in Fedora 20.

In the process, I have learned quite some intricacies of the Indic opentype spec and would try to document them in a series of posts.

smc-fonts (Meera) updated

Meera is the default font for Malayalam in Fedora. Lately, a few bugs causing wrong rendering of some complex conjuncts were identified and reported – 1, 2.  There was another bug reported in RedHat bugzilla. As Unicode 5.1 is not supported by smc-fonts, glyph Ⓡ was being displayed for Atomic Chillu letters. To make the font Unicode complying, it was suggested to remove this.

Yesterday a new release of Meera fixing all these bugs was made available by Suresh, and I’ve uploaded a new upstream source file at Savannah repository. Then the RPMs were rebuilt for rawhide, which can be found here.

And the ChangeLog reads:

* The glyphs(an R inside a circle) at unallocated Unicode code points are removed.
* Wrong glyph for 'th1s1r3u1' (ത്സ്രു) fixed.
* Breaking up of conjunct 'l3k1k1' (ല്ക്ക) fixed.
* Shaping issues for 'r3cil'+'l3l3' (ര്‍ല്ല) context fixed.

Thanks to Suresh, Ani Peter, Santhosh Thottingal and Pravin Satpute !