Okular: another improvement to annotation

Continuing with the addition of line terminating style for the Straight Line annotation tool, I have added the ability to select the line start style also. The required code changes are committed today.

Line annotation with circled start and closed arrow ending.

Currently it is supported only for PDF documents (and poppler version ≥ 0.72), but that will change soon — thanks to another change by Tobias Deiminger under review to extend the functionality for other documents supported by Okular.


Okular: improved PDF annotation tool

Okular, KDE’s document viewer has very good support for annotating/reviewing/commenting documents. Okular supports a wide variety of annotation tools out-of-the-box (enable the ‘Review’ tool [F6] and see for yourself) and even more can be configured (such as the ‘Strikeout’ tool) — right click on the annotation tool bar and click ‘Configure Annotations’.

One of the annotation tools me and my colleagues frequently wanted to use is a line with arrow to mark an indent. Many PDF annotating software have this tool, but Okular was lacking it.

So a couple of weeks ago I started looking into the source code of okular and poppler (which is the PDF library used by Okular) and noticed that both of them already has support for the ‘Line Ending Style’ for the ‘Straight Line’ annotation tool (internally called the TermStyle). Skimming through the source code for a few hours and adding a few hooks in the code, I could add an option to configure the line ending style for ‘Straight Line’ annotation tool. Many line end styles are provided out of the box, such as open and closed arrows, circle, diamond etc.

An option to the ‘Straight Line’ tool configuration is added to choose the line ending style:

New ‘Line Ending Style’ for the ‘Straight Line’ annotation tool.

Here’s the review tool with ‘Open Arrow’ ending in action:

‘Arrow’ annotation tool in action.

Once happy with the outcome, I’ve created a review request to upstream the improvement. A number of helpful people reviewed and commented. One of the suggestions was to add icon/shape of the line ending style in the configuration options so that users can quickly preview what the shape will look like without having to try each one. The first attempt to implement this feature was by adding Unicode symbols (instead of a SVG or internally drawn graphics) and it looked okay. Here’s a screen shot:

‘Line End’ with symbols preview.

But it had various issues — some symbols are not available in Unicode and the localization of these strings without some context would be difficult. So, for now it is decided to drop the symbols.

For now, this feature works only on PDF documents. The patch is committed today and will be available in the next version of Okular.

Meera font updated to fix issue with InDesign

I have worked to make sure that fonts maintained at SMC work with mlym (Pango/Qt4/Windows XP era) opentype specification as well as mlm2 (Harfbuzz/Windows Vista+ era) specification, in the same font. These have also been tested in the past (2016ish) with Adobe softwares which use their own shaping engine (they use neither Harfbuzz nor Uniscribe; but there are plans to use Harfbuzz in the future — the internet tells me).

Some time ago, I received reports that typesetting articles in Adobe InDesign using Meera font has some serious issues with Chandrakkala/Halant positioning in combination with conjuncts.

When the Savmruthokaram/Chandrakkala ് (U+0D4D) follows a consonant or conjunct, it should be placed at the ‘right shoulder’ of the consonant/conjunct. But in InDesgin (CC 2019), it appears incorrectly on the ‘left shoulder’. This incorrect rendering is highlighted in figure below.

Wrong chandrakkala position before consonant in InDesign.

The correct rendering should have Chandrakkala appearing at the right of as in figure below.

Correct chandrakkala position after consonant.

This issue manifested only in Meera, but not in other fonts like Rachana or Uroob. Digging deeper, I found that only Meera has Mark-to-Base positioning GPOS lookup rule for Chandrakkala. This was done (instead of adjusting leftt bearing of the Chandrakkala glyph) to appear correctly on the ‘right shoulder’ of consonant. Unfortunately, InDesign seems to get this wrong.

To verify, shaping involving the Dot Reph ൎ (U+0D4E) (which is also opentype engineered as Mark-to-Base GPOS lookup) is checked. And sure enough, InDesign gets it wrong as well.

Dot Reph position (InDesign on left, Harfbuzz/Uniscribe on right)

The issue has been worked around by removing the GPOS lookup rules for Chandrakkala and tested with Harfbuzz, Uniscribe and InDesign. I have tagged a new version 7.0.2 of Meera and it is available for download from SMC website. As this issue has affected many users of InDesign, hopefully this update brings much joy to them to use Meera again. Windows/InDesign users make sure that previous versions of the font are uninstalled before installing this version.

Smarter tabular editing with Vim

I happen to edit tabular data in LaTeX format quite a bit. Being scientific documents, the table columns are (almost) always left-aligned, even for numbers. That warrants carefully crafted decimal and digit alignment on such columns containing only numbers.

I also happen to edit the text (almost) always in Vim, and just selecting/changing a certain column only is not easily doable (like in a spreadsheet). If there are tens of rows that needs manual digit/decimal align adjustment, it gets even more tedious. There must be another way!

Thankfully, smarter people already figured out better ways (h/t MasteringVim).

With that neat trick, it is much more palatable to look at the tabular data and edit it. Even then, though, it is not possible to search & replace only within a column using Visual Block selection. The Visual Block (^v) sets mark on the column of first row till the column on last row, so any :<','>s/.../.../g would replace any matching text in-between (including any other columns).

To solve that, I’ve figured out another way. It is possible to copy the Visual Block alone and pasting any other content over (though cutting it and pasting would not work as you think). Thus, the plan is:

  • Copy the required column using Visual Block (^v + y)
  • Open a new buffer and paste the copied column there
  • Edit/search & replace to your need in that buffer, so nothing else would be unintentionally changed
  • Select the modified content as Visual Block again, copy/cut it and come back to the main buffer/file
  • Re-select the required column using Visual Block again and paste over
  • Profit!

Here’s a short video of how to do so. I’d love to hear if there are better ways.

Column editing in Vim
Demo of column editing in Vim

Powerline git dirty status without powerline_gitstatus

With git-prompt it is possible to display the dirty state (when a tracked file is modified) by setting the env variable GIT_PS1_SHOWDIRTYSTATE=true. Powerline can display the status of a git repository, such as number of commits ahead/behind, number of modified files etc. using the powerline_gitstatus module. Unfortunately, Fedora doesn’t have it packaged. I did some digging in, and found that there’s colour highlighting for branch_dirty and powerline.segments.common.vcs.branch function (which displays the current branch name) takes 2 parameters  to modify its behaviour. Modify the shell theme /etc/xdg/powerline/themes/shell/default.json under the left segment (because only left works in shell) then as follows:
    {   "function": "powerline.segments.common.vcs.branch",
        "args": {"ignore_statuses": ["U"], "status_colors": true},
        "priority": 20
The branch will now be highlighted if a tracked file is modified (ignore_statuses = ["U"] causes untracked files to be ignored). Clean repository:
Clean repo
Once a tracked file is modified:
Dirty repo

Improvement in converting video/audio files with VLC

VLC Media Player has the ability to convert video/audio files into various formats it supports, since a long time. There is a dedicated “Convert/Save” menu for converting single or multiple files at once into a different format, with limited ‘editing’ features such as specifying a start time, caching options etc. It is quite useful for basic editing/cropping of multimedia files.

As an example, one of the easiest ways to create a custom iPhone ringtone is to create a “.m4r” (AAC format) file exactly 40 seconds long. It is a matter of selecting your favourite music file and doing a “Convert/Save” with appropriate “Profile”. A “Profile” specifies the video/audio encoding to be used, which can be easily customized by selecting different audio and video codecs.

The options “Caching time”, “Play another media synchronously” (think adding different sound track to a video clipping) and a “Start time” etc can be specified under “Show more options” button and even more advanced functionality is available by making use of the “Edit Options” line. Internally, all the options specified at this line are passed to the converter.

There was one thing lacking in this “Convert/Save” dialog though – there was no possibility to specify a “Stop Time” akin to the “Start Time”, in the GUI (although it can be manually specified in the “Edit Options”, but you need to calculate the time in milliseconds). VLC 2.x series convert looks like as follows – notice the lack of “Stop time”:


Being bugged by this minor annoyance, I set out to add the missing “Stop-time” functionality. Going through the codebase of VLC, it was relieving to see that the converter backend already supports “:stop-time=” option (akin to “:start-time=”). It was then a matter of adding “Stop Time” to the GUI and properly updating the “Edit Options” when user changes the value.

A working patch was then sent to vlc-devel mailing list for review and feedback. After 5 rounds of review and constructive feedback from Filip Roséen the code was cleaned up (including existing code) which is now committed to the master branch. This functionality should be available to users in the upcoming 3.0 release. Screenshot below:



Convert iPhone contacts to vCard

On a recent troubleshooting attempt, I lost all the contacts in my Android phone. It had also received a recent update which took away the option to import contacts from another phone via bluetooth.
I still had some contacts in the old iPhone, but now that mass transfer via bluetooth is gone, it was a question of manually sending each contact in vCard format to the Android phone. That means I should probably find a less dreadful way to get the contacts back.

Here is one way to extract contacts en-masse from iPhone into popular vCard format. The contact and address details in iOS are stored by AddressBook application in a file named ‘AddressBook.sqlitedb’ which is an sqlite database. The idea is to open this database using sqlite, extract the details from a couple of tables and convert the entries into vCard format.

Disclaimer: the iPhone is an old 3GS running iOS 6 and it is jailbroken. If you attempt this, your mileage would vary. Required tools/softwares are usbmuxd (especially libusbmuxd-utils) and sqlite, with the prerequisite that openssh server is running on the jailbroken iPhone.

  1. Connect iPhone via USB cable to the Linux machine. Run iproxy 2222 22 to connect to the openssh server running on the jailbroken phone. iproxy comes with libusbmuxd-utils package.
  2. Copy the addressbook sqlite database from phone:scp -P 2222 mobile@localhost:/var/mobile/Library/AddressBook/AddressBook.sqlitedb .Instead of steps 1 and 2 above, it might be possible to copy this file using Nautilus (gvfs-afc) or Dolphin (kio_afc) file manager, although I’m not sure if the file is accessible.
  3. Extract the contact and address details from the sqlite db (based on this forum post):sqlite3 -cmd ".out contacts.txt" AddressBook.sqlitedb "select ABPerson.prefix, ABPerson.first,ABPerson.last,ABPerson.organization, c.value as MobilePhone, h.val ue as HomePhone, he.value as HomeEmail, w.value as WorkPhone, we.value as WorkEmail,ABPerson.note from ABPerson left outer join ABMultiValue c on c.record_id = ABPerson.ROWID and c.label = 1 and c.property= 3 left outer join ABMultiValue h on h.record_id = ABPerson.ROWID and h.label = 2 and h.property = 3 left outer join ABMultiValue he on he.record_id = ABPerson.ROWID and he.label = 2 and he.property = 4 left outer join ABMultiValue w on w.record_id = ABPerson.ROWID and w.label = 4 and w.property = 3 left outer join ABMultiValue we on we.record_id = ABPerson.ROWID and we.label = 4 and we.property = 4;"
  4. Convert the extracted contact details to vCard format:cat contacts.txt | awk -F\| '{print "BEGIN:VCARD\nVERSION:3.0\nN:"$3";"$2";;;\nFN:"$2" "$3"\nORG:"$4"\nEMAIL;type=INTERNET;type=WORK;type=pref:" $9"\nTEL;type=CELL;type=pref:"$5"\nTEL;TYPE=HOME:"$6"\nTEL;TYPE=WORK:"$8"\nNOTE:"$9"\nEND:VCARD\n"}' > Contacts.vcf
  5. Remove the empty content lines if some contacts do not have all the different fields:sed -i '/.*:$/d' Contacts.vcf

Now simply transfer the Contact.vcf file containing all the contact details to Android phone’s storage and import contacts from there.