Adventures in upgrading to Fedora 27/28 using ‘dnf system-upgrade’

[This post was drafted on the day Fedora 27 released, about half a year ago, but was not published. The issue bit me again with Fedora 28, so documenting it for referring next time.]

UPDATE: The issue occurred in Fedora 28 because I had exclude=grub2-tools in /etc/dnf/dnf.conf which is the reason error “nothing provides grub2-tools” was coming up. Removing that previously added and then forgotten line fixes the issue with updating grub2 packages.

With fedup and subsequently dnf improving the upgrade experience of Fedora for power users, last few system upgrades have been smooth, quiet, even unnoticeable. That actually speaks volumes of the maturity and user friendliness achieved by these tools.

Upgrading from Fedora 25 to 26 was so event-less and smooth (btw: I have installed and used every version of Fedora from its inception and the default wallpaper of Fedora 26 was the most elegant of them all!).

With that, on the release day I set out to upgrade the main workstation from Fedora 26 to 27 using dnf system-upgrade as documented. Before downloading the packages, dnf warned that upgrade cannot be done because of package dependency issues with grub2-efi-modules and grub2-tools.

Things go wrong!

I simply removed both the offending packages and their dependencies (assuming were probably installed for the grub2-breeze-theme dependency, but grub2-tools actually provides grub2-mkconfig) and proceeded with dnf upgrade --refresh and dnf system-upgrade download --refresh --releasever=27. If you are attempting this, don’t remove the grub2 packages yet, but read on!

Once the download and check is completed, running dnf system-upgrade reboot will cause the system reboot to upgrade target and actual upgrade happen.

Except, I was greeted with EFI MOK (Machine Owner Key) screen on reboot. Now that the grub2 bootloader is broken thanks to the removal of grub2-efi-modules and other related packages, a recovery must be attempted.

Rescue

It is important to have a (possibly EUFI enabled) live media where you can boot from. Boot into the live media and try to reinstall grub. Once booted in, mount the root filesystem under /mnt/sysimage, and EFI boot partition at /mnt/sysimage/boot/efi. Then chroot /mnt/sysimage and try to reinstall grub2-efi-x64 and shim packages. If there’s no network connectivity, don’t despair, nmcli is to your rescue. Connect to wifi using nmcli device wifi connect <ssid> password <wifi_password>. Generate the boot configuration using grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg followed by actual install grub2-install --target=x86_64-efi /dev/sdX (the –target option ensures correct host installation even if the live media is booted via legacy BIOS). You may now reboot and proceed with the upgrade.

But this again failed at the upgrade stage because of grub package clash that dnf warned earlier about.

Solution

Once booted into old installation, take a backup of the /boot/ directory, remove the conflicting grub related packages, and copy over the backed up /boot/ directory contents, especially /boot/efi/EFI/fedora/grubx64.efi. Now rebooting (using dnf system-upgrade reboot) had  the grub contents intact and the upgrade worked smoothly.

For more details on the package conflict issue, follow this bug.

Advertisements

Sundar — a new traditional orthography ornamental font for Malayalam

There is a dearth of good Unicode fonts for Malayalam script. Most publishing houses and desktop publishing agencies still rely on outdated ASCII era fonts. This not only causes issues with typesetting using present technologies, it makes the ‘document’ or ‘data’ created using these fonts and tools absolutely useless — because the ‘document/data’ is still Latin, not Malayalam.

Rachana Institute of Typography (rachana.org.in) has designed and published a new traditional orthography ornamental Unicode font for Malayalam script, for use in headings, captions and titles. It is named after Sundar, who was a relentless advocate of open fonts, open standards and open publishing. He dreamed of making available several good quality Malayalam fonts, particularly created by Narayana Bhattathiri with his unique calligraphic and typographic signature, freely and openly to the users. The font is licensed under OFL.

The font follows traditional orthography for Malayalam, rather than the unpleasing reformed orthography which was solely introduced due to the technical limitations of typewriters in the ’70s. Such restrictions do not apply to computers and present technology, so it is possible to render the classic beauty of Malayalam script using Unicode and Opentype technologies.

‘Sundar’ is designed by K.H. Hussain — known for his work on Rachana and Meera fonts which comes pre-installed with most Linux distributions; and Narayana Bhattathiri — known for his beautiful calligraphy and lettering in Malayalam script. Graphic engineers of STM Docs (stmdocs.in) did the vectoring and glyph creation. Yours truly took care of the Opentype feature programming. The font can be freely downloaded from rachana.org.in.

The source code of ‘Sundar’, licensed under OFL is available at https://gitlab.com/rit-fonts/Sundar.

Switching Raspbian to Pixel desktop

Official Raspbian images based on Debian Stretch by default has the Pixel desktop environment and will login new users to it. But if you have had a Raspbian installation with another DE (such as LXDE), here are the steps to install and login to the Pixel desktop.

apt-get install raspberrypi-ui-mods
sed -i 's/^autologin-user=pi/#autologin-user=pi/' /etc/lightdm/lightdm.conf
update-alternatives --set x-session-manager /usr/bin/startlxde-pi
sed -i 's/^Session=.*/Session=lightdm-xsession/' ${USER}/.dmrc

Make sure the user’s ‘.dmrc’ file is updated with the new startlxde-pi session as that is where lightdm login manager looks to decide which desktop should be launched.

Improvement in converting video/audio files with VLC

VLC Media Player has the ability to convert video/audio files into various formats it supports, since a long time. There is a dedicated “Convert/Save” menu for converting single or multiple files at once into a different format, with limited ‘editing’ features such as specifying a start time, caching options etc. It is quite useful for basic editing/cropping of multimedia files.

As an example, one of the easiest ways to create a custom iPhone ringtone is to create a “.m4r” (AAC format) file exactly 40 seconds long. It is a matter of selecting your favourite music file and doing a “Convert/Save” with appropriate “Profile”. A “Profile” specifies the video/audio encoding to be used, which can be easily customized by selecting different audio and video codecs.

The options “Caching time”, “Play another media synchronously” (think adding different sound track to a video clipping) and a “Start time” etc can be specified under “Show more options” button and even more advanced functionality is available by making use of the “Edit Options” line. Internally, all the options specified at this line are passed to the converter.

There was one thing lacking in this “Convert/Save” dialog though – there was no possibility to specify a “Stop Time” akin to the “Start Time”, in the GUI (although it can be manually specified in the “Edit Options”, but you need to calculate the time in milliseconds). VLC 2.x series convert looks like as follows – notice the lack of “Stop time”:

Being bugged by this minor annoyance, I set out to add the missing “Stop-time” functionality. Going through the codebase of VLC, it was relieving to see that the converter backend already supports “:stop-time=” option (akin to “:start-time=”). It was then a matter of adding “Stop Time” to the GUI and properly updating the “Edit Options” when user changes the value.

A working patch was then sent to vlc-devel mailing list for review and feedback. After 5 rounds of review and constructive feedback from Filip Roséen the code was cleaned up (including existing code) which is now committed to the master branch. This functionality should be available to users in the upcoming 3.0 release. Screenshot below:

 

Convert iPhone contacts to vCard

On a recent troubleshooting attempt, I lost all the contacts in my Android phone. It had also received a recent update which took away the option to import contacts from another phone via bluetooth.
I still had some contacts in the old iPhone, but now that mass transfer via bluetooth is gone, it was a question of manually sending each contact in vCard format to the Android phone. That means I should probably find a less dreadful way to get the contacts back.

Here is one way to extract contacts en-masse from iPhone into popular vCard format. The contact and address details in iOS are stored by AddressBook application in a file named ‘AddressBook.sqlitedb’ which is an sqlite database. The idea is to open this database using sqlite, extract the details from a couple of tables and convert the entries into vCard format.

Disclaimer: the iPhone is an old 3GS running iOS 6 and it is jailbroken. If you attempt this, your mileage would vary. Required tools/softwares are usbmuxd (especially libusbmuxd-utils) and sqlite, with the prerequisite that openssh server is running on the jailbroken iPhone.

1. Connect iPhone via USB cable to the Linux machine. Run iproxy 2222 22 to connect to the openssh server running on the jailbroken phone. iproxy comes with libusbmuxd-utils package.
2. Copy the addressbook sqlite database from phone:scp -P 2222 mobile@localhost:/var/mobile/Library/AddressBook/AddressBook.sqlitedb .Instead of steps 1 and 2 above, it might be possible to copy this file using Nautilus (gvfs-afc) or Dolphin (kio_afc) file manager, although I’m not sure if the file is accessible.
3. Extract the contact and address details from the sqlite db (based on this forum post):sqlite3 -cmd ".out contacts.txt" AddressBook.sqlitedb "select ABPerson.prefix, ABPerson.first,ABPerson.last,ABPerson.organization, c.value as MobilePhone, h.val ue as HomePhone, he.value as HomeEmail, w.value as WorkPhone, we.value as WorkEmail,ABPerson.note from ABPerson left outer join ABMultiValue c on c.record_id = ABPerson.ROWID and c.label = 1 and c.property= 3 left outer join ABMultiValue h on h.record_id = ABPerson.ROWID and h.label = 2 and h.property = 3 left outer join ABMultiValue he on he.record_id = ABPerson.ROWID and he.label = 2 and he.property = 4 left outer join ABMultiValue w on w.record_id = ABPerson.ROWID and w.label = 4 and w.property = 3 left outer join ABMultiValue we on we.record_id = ABPerson.ROWID and we.label = 4 and we.property = 4;"
4. Convert the extracted contact details to vCard format:cat contacts.txt | awk -F\| '{print "BEGIN:VCARD\nVERSION:3.0\nN:"$3";"$2";;;\nFN:"$2" "$3"\nORG:"$4"\nEMAIL;type=INTERNET;type=WORK;type=pref:" $9"\nTEL;type=CELL;type=pref:"$5"\nTEL;TYPE=HOME:"$6"\nTEL;TYPE=WORK:"$8"\nNOTE:"$9"\nEND:VCARD\n"}' > Contacts.vcf
5. Remove the empty content lines if some contacts do not have all the different fields:sed -i '/.*:$/d' Contacts.vcf

Now simply transfer the Contact.vcf file containing all the contact details to Android phone’s storage and import contacts from there.

TeX: Justification with no hyphenation

I’ve been reading a copy of 1984 by George Orwell, published by Fingerprint publishing — a beautifully typeset one. Already into half of Part II, but that’s when I noticed that the book is typeset with full justification sans any hyphenation. Incidentally I was typesetting something else, which probably is the reason I noticed it now. And I wanted to typeset my article the same way, with full justification but no hyphenation.

Biggest strength of TeX is its line, paragraph and page breaking algorithms where hyphenation plays a big part. Thus removing hyphenation means taking away a lot of those advantages. In any case, there are seemingly multiple ways to do it. From the TeX FAQ:

Use hyphenat package with

\usepackage[none]{hyphenat}

or set \hyphenpenalty and \exhyphenpenalty to 10000. These avoided hyphenation, but justification was bad – there were long words extending beyond the right margin. So moved on to next solution:

\hyphenchar\font=-1
\sloppy

This one kept hyphenation to the minimum with justified text, but didn’t fully avoid it. And it would work only for text with current font. \sloppy sets \tolerance and \emergencystretch to large values.

And the last one, which provided full justification with no hyphenation is:

%\tolerance=1
\emergencystretch=\maxdimen
\hyphenpenalty=10000

The \tolerance parameter sets how much badness is allowed, which influence paragraph/line breaking. \emergencystretch is the magical parameter which stretches text over multiple passes to balance the spacing. Setting \hyphenpenalty to infinite value actually has better effect than setting \tolerance.

Kerning feature for Malayalam fonts

At SMC, we’ve been continuously working on improving the fonts for Malayalam – by updating to newer opentype standard (mlm2), adding new glyphs, supporting new Unicode points, fixing shaping issues, reducing complexity and the compiled font size, involving new contributors etc.

Recently, out of scratching my own itch, I decided that it is high time to fix the annoyance that combination of Virama(U+D04D ് ) with quote marks (‘ ” ‘ ’ “ ” etc) used to overlap into an ugly amalgam in all our fonts. Usually Virama(് ) connects/combines two consonants which makes all 3 into a new glyph – for example സ+്+ന is shaped into a new glyph സ്ന (Note that you need a traditional orthography font installed to see the distinction in this example. Many of them are available here) . The root of the problem is that sometimes when Virama(് ) appears individually in a word such as “സ്വപ്‌‌നം” it connects two consonants പ and ന, it is positioned above the x height of most glyphs and it shall not have much left and right bearing to avoid ugly spacing between the consonants പ and ന.  Because of small side bearings, in fact of the negative right bearing (് protrudes beyond the right bearing) when a quote mark follows it, quote mark gets a little juxtaposed into Virama glyph and renders rather bad. The issue is quite prominent when you professionally typeset a book or article in Malayalam using XeTeX or SILE.

Fontforge’s tools made it easy to write opentype lookup rules for horizontal pair kerning to allow more space between Virama(് ) and quote marks. You can see the before and after effect of the change with Rachana font in the screenshot.

Update 26/03/2016: Many applications already support the kerning feature out of the box, including Firefox, SILE and VLC (3.0.0-git for subtitles) that I have tested, but many still need support, for instance LibreOffice, Kwrite etc. Here is a screenshot of VLC (3.0.0-git) taking kerning rules into account while displaying Malayalam subtitle.

Other fonts like AnjaliOldLipi, Meera and Chilanka also got this feature and those will be available with the new release in the pipeline. I have plans to expand this further to use with post-base vowels of വ(്വ) and യ(്യ) with abundant stacked glyphs that Malayalam has.