On data encoding and complex text shaping

As part of the historical move of Janayugom news paper migrating into a completely libre software based workflow, Kerala Media Academy organized a summit on self-reliant publishing on 31-Oct-2019. I was invited to speak about Malayalam Unicode fonts.

The summit was inaugurated by Fahad Al-Saidi of the Scribus fame, who was instrumental in implementing complex text layout (CTL). Prior to the talks, I got to meet the team who made it possible to switch Janayugom’s entire publishing process on to free software platform — Kubuntu based ThengOS, Scribus for page layout, Inkspace for vector graphics, GIMP for raster graphics, CMYK color profiling for print, new Malayalam Unicode fonts with traditional orthography etc. It was impressive to see that entire production fleet was transformed, team was trained and the news paper is printed every day without delay.

I also met Fahad later and pleasantly surprised to realize that he already knows me from open source contributions. We had a productive discussion about Scribus.

My talk was on data encoding and text shaping in Unicode Malayalam. The publishing industry in Malayalam is at large still trapped in ASCII which causes numerous issues now, and many are still not aware of Unicode and its advantages. I tried to address that in my presentation with examples — so the preface of my talk filled half of the session; while the second half focused on font shaping. Many in the industry seems to be aware of Unicode and traditional Malayalam orthography can be used in computers now; but many in the academia still has not realized it — evident from the talk of the moderator of the discussion, who is director of the school of Indian languages. There was a lively discussion with the audience in the Q&A session. After the talk, a number of people gave me feedback and requested the slides be made available.

Slides on data encoding and complex text shaping are available under CC-BY-NC license here.

Meera font updated to fix issue with InDesign

I have worked to make sure that fonts maintained at SMC work with mlym (Pango/Qt4/Windows XP era) opentype specification as well as mlm2 (Harfbuzz/Windows Vista+ era) specification, in the same font. These have also been tested in the past (2016ish) with Adobe softwares which use their own shaping engine (they use neither Harfbuzz nor Uniscribe; but there are plans to use Harfbuzz in the future — the internet tells me).

Some time ago, I received reports that typesetting articles in Adobe InDesign using Meera font has some serious issues with Chandrakkala/Halant positioning in combination with conjuncts.

When the Savmruthokaram/Chandrakkala ് (U+0D4D) follows a consonant or conjunct, it should be placed at the ‘right shoulder’ of the consonant/conjunct. But in InDesgin (CC 2019), it appears incorrectly on the ‘left shoulder’. This incorrect rendering is highlighted in figure below.

Wrong chandrakkala position before consonant in InDesign.

The correct rendering should have Chandrakkala appearing at the right of ത as in figure below.

Correct chandrakkala position after consonant.

This issue manifested only in Meera, but not in other fonts like Rachana or Uroob. Digging deeper, I found that only Meera has Mark-to-Base positioning GPOS lookup rule for Chandrakkala. This was done (instead of adjusting leftt bearing of the Chandrakkala glyph) to appear correctly on the ‘right shoulder’ of consonant. Unfortunately, InDesign seems to get this wrong.

To verify, shaping involving the Dot Reph ൎ (U+0D4E) (which is also opentype engineered as Mark-to-Base GPOS lookup) is checked. And sure enough, InDesign gets it wrong as well.

Dot Reph position (InDesign on left, Harfbuzz/Uniscribe on right)

The issue has been worked around by removing the GPOS lookup rules for Chandrakkala and tested with Harfbuzz, Uniscribe and InDesign. I have tagged a new version 7.0.2 of Meera and it is available for download from SMC website. As this issue has affected many users of InDesign, hopefully this update brings much joy to them to use Meera again. Windows/InDesign users make sure that previous versions of the font are uninstalled before installing this version.

Sundar — a new traditional orthography ornamental font for Malayalam

There is a dearth of good Unicode fonts for Malayalam script. Most publishing houses and desktop publishing agencies still rely on outdated ASCII era fonts. This not only causes issues with typesetting using present technologies, it makes the ‘document’ or ‘data’ created using these fonts and tools absolutely useless — because the ‘document/data’ is still Latin, not Malayalam.

Rachana Institute of Typography (rachana.org.in) has designed and published a new traditional orthography ornamental Unicode font for Malayalam script, for use in headings, captions and titles. It is named after Sundar, who was a relentless advocate of open fonts, open standards and open publishing. He dreamed of making available several good quality Malayalam fonts, particularly created by Narayana Bhattathiri with his unique calligraphic and typographic signature, freely and openly to the users. The font is licensed under OFL.

The font follows traditional orthography for Malayalam, rather than the unpleasing reformed orthography which was solely introduced due to the technical limitations of typewriters in the ’70s. Such restrictions do not apply to computers and present technology, so it is possible to render the classic beauty of Malayalam script using Unicode and Opentype technologies.

‘Sundar’ is designed by K.H. Hussain — known for his work on Rachana and Meera fonts which comes pre-installed with most Linux distributions; and Narayana Bhattathiri — known for his beautiful calligraphy and lettering in Malayalam script. Graphic engineers of STM Docs (stmdocs.in) did the vectoring and glyph creation. Yours truly took care of the Opentype feature programming. The font can be freely downloaded from rachana.org.in.

The source code of ‘Sundar’, licensed under OFL is available at https://gitlab.com/rit-fonts/Sundar.