Skip to content
A scene from Ireland

Edlib

Selections and clipping in The X11 Window System

One of the many tasks when writing an editor for Linux - which I am with my "edlib" project - is to support sending and received content from other applications for copy/paste operations. The X11 protocol provides support this is with "Selections" and a "clipboard" which are documented in the ICCCM - the Inter-Client Communication Conventions Manual. Unfortunately the big-picture of how this should work is poor described, and consequently different applications can behave quite differently and many, in my opinion, get it wrong.

While the selection mechanism can seem confusing and needlessly complex, it actually makes lot of sense and is easy to work with, providing you look at the right way. The important observation is that the "PRIMARY" selection and the "CLIPBOARD" should NOT provide different content. They are really just different interfaces to essentially the same content. Let me explain ....

edlib ideas from elsewhere

There have been a couple of articles on lwn.net (https://lwn.net/Articles/819452/ and https://lwn.net/Articles/832311/) about emacs and how it could be better, which have generated lots of discussion (a really valuable aspect of lwn.net). For some reason this encouraged me to explore more of vim - I've never really got passed the basics that I learned in in the late 80's) and I also stumbled across https://ecc-comp.blogspot.com/2015/05/a-brief-glance-at-how-5-text-editors.html (largely because edlib was mentioned in the hacker-news discussion https://news.ycombinator.com/item?id=11244103). All of this has encouraged me to take a step back and look for a different perspective on some editor issues.

Designing plugins for edlib

As edlib (my Emacs-replacement editor) matures I'm being more adventurous in the functionality I'm adding: spell checker, calculator, difference highlighter. This often involves importing functionality from an external source and making it available within edlib. This has raised the question of how to map an ad hoc interface from some library into the more constrained interfaces supported within edlib. Experience so far suggests that it is always possible, but there are sometimes multiple options and it is worth making the effort to choose carefully.

One of the behaviours of emacs that I want to avoid as I build edlib (my answer to emacs) is modal dialogs. I recently saw mention of Oberon and this link in particular: https://pdfs.semanticscholar.org/d48b/ecdaf5c3d962e2778f804e8c64d292de408b.pdf. It also mentions an aversion of modal dialogues. I don't find myself completely convinced by the example given, but I do support the idea. This is as good an excuse as any to clarify my thoughts on the topic.

Complex documents in edlib

One of my core goals in developing edlib is to allow the display to be programmatically controlled: the content of a document is formatted dynamically as it is displayed, and so can respond to context (e.g. location of "point" can modify appearance more than just by displaying a cursor) and also so that the entire document doesn't need to be rendered into a buffer, just the parts being displayed.

A natural (for me) extension to this idea was the possibility that the source of the display wasn't just one single document - multiple documents could be blended. Various examples have occurred to me, though few have been implemented.

Fewer NULL dereferences in edlib

My recent efforts with edlib have been to get rid of some potential NULL pointer dereference issues. When I first started writing "commands" I assumed that the correct value would always be passed. Of course that isn't very safe unless it is enforced, and is particularly unsafe if I'm going to let users write their own commends in extension languages like python. So more safety is required.

Auditing the code to ensure I'm always being careful enough is fairly boring and error prone, so I wrote a tool to help me. More accurately I extended some existing tools. You can read lots more details in my LWN.net article. At the time I wrote that I still had some unresolved issues. I've resolved enough of those now that I no longer have warnings. Sometimes that is because I used casts to hide things, but a lot of real issues have been addressed. The versions of "sparse" and "smatch" that I am using are in the "safe" branch of the copies of these trees on github.com. So this for smatch and this for sparse.

Doing this involved adding a lot of 'safe' annotations throughout edlib. Hopefully these aren't too confusing. It is nice to have the documentation of intent, and nice to know that a whole class of errors is now impossible.

I had to fix a few bugs in smatch/sparse as well as add new functionality to smatch. I really should post those bug fixes upstream...

A notmuch based email reader

I've been very slack. Sorry. I keep thinking "I should write a blog post about that" when I make some progress with edlib. But I also think "or I could write some more code instead". I do enjoy writing blog posts. But I seem to enjoy the code more.

Consequently, I've lost track of what has happened since my last post. Git tells me that I have made 379 commits since then. I've added "Alt-!" to run a shell command, with a history of recent commands stored in a buffer. I've provided a way for attributes on text to trigger call-backs to display panes, to enabling highlighting of text; and used this to better display matches for the current search. I've broken the "Refresh" event into three separate events, one that updates the sizes of panes, one that can update the position of a document in a pane, and one that redraws the content. And I've fixed lots of bugs and cleaned up lots of code. But the big thing that I've been working on is a notmuch email client.

edlib is back on track...

In the weeks leading up toe linux.conf.au 2016 I approached edlib development as a sprint. I wanted to put together sufficient functionality so that I could present my LCA2016 talk using slides displayed by edlib. I achieved this goal but at some cost. A lot of the development that went into that sprint was fairly hackish and not well thought out. It achieved the immediate goal but wasn't good long term development.

With that cost came benefits. Not just working code but a clearer perspective on what I needed to do and what problems would be faced by creating a slide-presentation mode. I don't regret the sprint at all but it certainly didn't result in completed work. I had to go over all of that development and turn the prototype into well designed code. This naturally took quite a bit longer but resulted in much more coherent designs and a stronger overall structure. This is done now and my LCA2016 presentation is now in my mainline of edlib development and works well.

There were a number of structural changes that I made while revising all this functionality, I'll just address a few of them here.

LCA-2016 presentation is done

I've been busy of the last couple of months. A number of family and personal things meant I have less time for edlib, but I had a lot to do for edlib too. I really wanted to use edlib to give my presentation at linux.conf.au 2016 in beautiful Geelong.

edlib - render-lines, what a good idea.

I had a brain-wave last week. I thought it would take me about a day to implement but it took 3 to get it working almost reasonably and the rest of the week to polish it and create some proper use-cases to exercise it properly. But that is done now and I am quite happy with the result. It related to the rendering side of edlib.