Interested in Org but not enough to monitor the mailing list? In this blog I digest developments, and each month regurgitate what I consider the highlights, along with anything else Org-related I consider nifty.
Posts are published in .html, .org, .org.html, .txt, and .pdf forms — because why not 😛
May 2022 2022-05-31
Finding time as of late has been more difficult than I anticipated, and on top of that, just as I was thinking of writing last month’s post, I got distracted by an exciting patchset that has been in the works for over a year finally getting sorted out and landing. So, I hope that some of the fun developments in this post will make up the absense of the last one 🙂.
Since it’s been longer than I thought since the last standard post, we’ve got a fair few commits to catch up on — about 200. Most of these are miscellaneous minor improvements and bugfixes, but a few notable changes have arrived too.
Folding
The fabulous new folding engine (org-fold-core) should noticeably improve Org’s performance with large files. It contains a number of key optimisations to improve speed, namely:
- Deferring fontification of folded regions
- Using text properties (\(\mathcal{O}(n \log n)\)) instead of overlays (\(\mathcal{O}(n^2)\)) for folded regions
- A collection of aggressive optimisations available under
org-fold-core--optimise-for-huge-buffers - Convert text properties to overlays for isearch (which currently only supports overlays)
How noticeable is the overall performance impact? Well, I poked Ihor and he was kind enough to whip up some benchmarks.
org-shifttab showing file contents, as file size increases, with and without org-fold.
org-shifttab showing the entire file, as file size increases, with and without org-fold.Well this looks very promising1! Let’s see how much of an improvement this is overall.
| File size (Mb) | Headings (thousands) | Bugfix (no org-fold) | Main (with org-fold) | Improvement |
|---|---|---|---|---|
| 18 | 36 | 115.31 | 0.89 | 99% |
| 8.8 | 24 | 19.03 | 0.48 | 97% |
| 4.4 | 5 | 3.79 | 0.13 | 97% |
| 2.2 | 2 | 1.29 | 0.08 | 94% |
| 1.1 | 1 | 0.50 | 0.045 | 91% |
To be clear, even the smallest file in this data — a 1.1 Mb Org file with around a thousand headings, is fairly large. So, it’s unlikely you’ll notice much of a difference with small–medium files, but if you a few large+ files this should be a fantastic improvement. Once again, thanks Ihor!
The change to text properties instead of overlays breaks a number of third party
packages like evil-search and consult’s consult-line.
If you are involved in any packages affected by this, you’ll either want to
consider supporting invisible text, or look at isearch-filter-predicate and
isearch-mode-end-hook, which org-fold now uses.
If you’re an end-user, perhaps politely make an issue on the repo for a project
if no issue currently exists, and either:
- Stay off Org’s bleeding edge till the package ecosystem has adapted to this change
- Help the packages you use adapt to this change
- Set
org-fold-core-styletooverlaysto restore the old behaviour
Engraved source code blocks in LaTeX
All too often exporting code to LaTeX has been a disappointment, with lovely
syntax highlighting from Emacs major modes replaced with a markedly inferior
attempt by pygments (setting org-latex-listings to minted) in a colour scheme I
don’t really like.
A bit over a year ago, a project called engrave-faces started with the aim of making Emacs’ font-lock more exportable, like a generalised htmlize.el. This has recently been used to provide a new option for inline and block source code exports in LaTeX.
To use this, simply install the package and set org-latex-src-block-backend (a
rename of org-latex-listings to better reflect its usage) to engraved.
While this is sufficient to get started, this new backend also allows for some
new options. The theme used for engraving a source block can be set globally
with the new variable org-latex-engraved-theme, or per-file with the
#+latex_engraved_theme keyword. It takes either the name of a theme, or the
symbol t as a stand-in for the current theme.
The theme can also be set on a per-block level using the LaTeX attribute :engraved-theme.
Here’s what using these new capabilities looks like in practice.
Org mode
#+title: Engraving source blocks #+begin_src emacs-lisp (message "look ma, some %s" 'code) #+end_src #+begin_src shell echo "This is shell code" #+end_src
This may well be the best syntax-highlighting solution available for PDFs/LaTeX currently available, but I am a tad biased 😛.
TexInfo export improvements
Jonas Bernoulli has been using a custom TexInfo backend for Magit’s documentation for a while now, and over the past few months he’s worked the features he was missing into Org’s built-in TexInfo exporter.
Upstreaming like this always takes a fair bit of effort, so thank you Jonas for going through with this!
Toggle noweb prefix handling
Previously, whenever a noweb reference appeared on a non-empty line, a multi-line replacement would duplicate the content before the noweb reference.
Clearly, this is not always desirable, and this behaviour can now be turned of by setting the new header argument :noweb-prefix no.
Org mode
#+begin_src emacs-lisp :noweb yes :noweb-prefix no (setq example-data "<<example>>") #+end_src Will now expand to #+begin_src emacs-lisp (setq example-data "some multi-line content") #+end_src Instead of #+begin_src emacs-lisp (setq example-data "some (setq example-data "multiline (setq example-data "content") #+end_src
Package highlight: org-modern
I think we’ve all seen plenty of org-mode prettification packages before, so what makes Minad’s org-modern special? It’s actually doing something similar to Ihor’s org-fold improvements, switching out slower overlay-based approaches for text properties. I can confirm that switching out org-superstar-mode for org-modern has made a substantial improvement in my experience, halving the first-load time of my config.org to around 20 seconds. If you’re a fan of Org prettification and haven’t taken a look at this package, I highly recommend giving it a shot.
Other improvements
- Clean up some magic numbers in org-attach Marco Wahl
- Allow any command form in
org-attach-commands(including keyboard macros) Marco Wahl - Allow dest in
org-list-send-itemto be a buffer position Sacha Chua - Improve CSL-JSON date handling in oc-basic David Lukes
- Add TOML and desktop language aliases TEC
- Speed up cached bibliography retrieval in oc-basic Ihor Radchenko
- Allow setting PlantUML jar arguments Ihor Radchenko
- Allow for customisation of property separators with
org-property-separatorsTyler Grinn - New ox-latex maintainer, Daniel Fleischer
- More unit tests Kyle Keyer, Nick Dokos
- Documentation improvements Kyle Meyer, Juan Manuel Macias, Bastien, Karl Fogel, Cody Harris
Bugfixes
- An Emacs <28 bug in org-persist Ihor Radchenko
- Author extraction in oc-basic Nicolas Goaziou
- Fix behaviour of
org-copy-visiblewith adjacent tex andbuffer-invisibility-specKyle Meyer - Parsing of inline footnotes with parentheses Nicolas Goaziou
- Honor
default-directoryin ob-gnuplot Ihor Radchenko - Heading fontification bug Anders Johansson
- Template expansion where one key is a substring of another Andrew Arensburger
Special Announcement 2022-04-01
It has become apparent to the Org Mode developers that Org is suffering from a severe lack of enterprise adoption. To rectify this, we will be leveraging our collective decades working on holistic human-markup interaction tools to rapidly pivot to what we believe to be the true markup format of the future — Confluence Wiki Markup.
With this paradigm shift, you can look forward to a much more intuitive syntax, empowering you to create next-generation agile documents.
To assist you in this transition, we’ll give you a brief overview of the changes you can expect. Text formatting is almost unaffected, with a few sensible changes made to the surrounding characters.
February 2022 2022-02-30
Timothy (TEC) here. This month we have a guest post from a different part of the Org ecosystem, to highlight one of the most promising efforts to provide a good experience outside Emacs.
“But I use Emacs, I don’t care” you may say. In that case, I’d like to point out that wider spread and better Org support enriches the Org ecosystem as a whole. It makes the format more approachable, and useful for other people. This is good for everybody.
Without any further ado, here’s the guest post kindly written by Kristijan. Enjoy!
Like every beginner Vim user, at some point I ran into a usual editor war post: Vim vs Emacs. At that time, I didn’t have an idea what “Emacs” was.
January 2022 2022-01-31
There are less “obvious user-facing changes” this month, but a lot of good work has been happening 🙂.
The cache continues to make strides
Nine months ago, Ihor started working on the org-element cache. The cache has existed for a while now, but it used to be too buggy to be useful.
This is no longer the case! We’re finally confident enough to flip
org-element-use-cache’s default from nil to t. I mentioned some of the
improvements in , however Ihor has continued to make strides with the
cache.
Getting the cache right is tough though, and so while the situation is much improved there are still lingering bugs which need to be chased down. If you only take one thing away from the post, I’d like it to be this:
December 2021 2021-12-31
Just over eight months ago, I kicked off This Month in Org with an emphatic announcement of the blog’s in the form of a Welcome post. If you haven’t guessed, this is the first “blog post” I’ve ever written. In that welcome post, I gave my motivation for starting the blog — essentially to bridge a perceived gap in information sources between a subscription to the Org project mailing list, and nothing.
That is why I thought this blog should exist, but until now I have neglected to mention what I want to accomplish with it. By starting TMiO I hoped to:
- Engage more people with the improvements being made to Org[1].
- Highlight some of the great work being done by Org[1] contributors.
- Read more…
November 2021 2021-11-30
With a 9.5 release highlight post last month, and the month before skipped, it’s now three months since the last regular instalment of TMIO. Let’s get back up to date on some of the latest happenings with Org.
Org as markup
Looking at the wider ecosystem, it certainly appears that there is a growing appetite for Org markup outside org-mode. More projects like Hugo and Logseq seem to be interested in supporting Org markup, and there has been a recent growth in editor extensions like Neovim’s orgmode.nvim (started in March this year) and Sublime Text’s OrgExtended (started in June this year).
October 2021 2021-10-31
Turns out that life became busy enough that instead of delaying, last month’s post had to be cancelled. We’re now back to business as usual though 🙂.
There have been some cool recent developments in Org over the past two months, but you’ll have to learn about those is next month’s edition as Org 9.5 has been released 🎉. So, let’s go over some of the changes I’m most excited about, in no particular order. To get a more complete picture of the latest changes, see ORG-NEWS.
The new citation engine
August 2021 2021-08-38
Dear readers, unfortunately I’ve been unusually busy as of late, and will continue to be for a bit over a month. As such, it’s entirely likely that the next post or two may
be latemake use of creative dates too.
In terms of Org development, we’ve had a fairly slow month — mostly tweaks and fixes, concentrated on the new org-cite functionality. However, there have been big developments with Roam as of late.
July 2021 2021-07-31
Last month I not-at-all-subtly hinted that a certain long-awaited feature was arriving imminently. At this point, I think it’s a good idea to set the tone for the rest of this post.
Citations
June 2021 2021-06-34
The previous two months have been pretty good for Org development — with many bug fixes and feature improvements. This month has been substantially slower than the last[1], but that’s not to say not much is happening: in fact, there are some rather nifty contributions lined up for the not-too-distant future and a certain long-awaited feature branch[2] is getting very close to merging 😀. You’ll just have to stick around to hear about those in a future edition of TMIO 😉.
Customise the reference command used in LaTeX
May 2021 2021-05-31
Async Babel sessions have landed
Jack Kamm has made a fantastic contribution to Org: ob-comint.el now supports asynchronous output from source blocks with full support for sessions 🎉.
If you haven’t used sessions before, you’re in for a treat! By simply assigning a named session to a source code block, e.g. :session foo, the same process will be reused for any other code blocks that specify the foo session. To do this for a certain language across the entire document, you can set the header argument property, i.e.
Along with the asynchronous process communication in ob-comint.el, we have an implementation for Python, and we should see support for R and Ruby in the not-too-distant future 😍.
April 2021 2021-04-30
A discussion on contributor support
Concerns were raised[1] about some contributors’ patches languishing, and it not being made clear how long it might take to get a response from someone.
In response to this, a the new role of Contributor Steward has been created to: help ensure contributors get a timely response, help out with preliminary patch feedback, and keep updates.orgmode.org up to date.
Org now has three Contributor Stewards to ease the process for patch submitters and core maintainers:
- Timothy / TEC
- Tim Cross
- John Corless
Welcome 2021-04-26
Introduction
Org is an absolutely marvellous project. However, a quick glance at https://orgmode.org can lead one to think “so… it’s an Emacs version of Markdown? What’s the big deal?”. While it’s easy to understand how someone might think that at first, that impression misses two crucial points:
- While for simple constructs (bold, italic, headlines, etc.) the syntax is very much analogous[1], Org scales to much more powerful forms that are a headache to replicate in Markdown
- Org mode was developed in Emacs, for Emacs. The integrations for Org put every other plaintext markup editing experience to shame. This is a bold statement, and I stand by it.
Footnotes:
Note the difference in scale, org-fold makes the most difference in the graph where the times are an order of magnitude more.