(By [[https://www.howardism.org/][Howard Abrams]], 2015 Nov 24)
Basic idea of this file is to be both a *guide* and a *practice pad*
for leveling up on Emacs’ keyboard macro system.
To use, just [[https://github.com/howardabrams/pdx-emacs-hackers/raw/master/workshops/keyboard-macros.org][download this file]] to your system, and edit it in Emacs.
Then just follow along... If you are not familiar with org-mode files,
here are the basics for this workshop:
- Hit a tab on the header expands or collapses it
- Narrow to show just one section with: C-x n s
- Return by widening normally with: C-x n w
- Hyperlinks can be clicked on to view the Emacs Manual
Pretty much treat the rest of this document as a text file.
** Basic Usage
Let’s begin with something simple to try out this project.
In the /block area/ below, hit: C-c ' (yes, the apostrophe).
When done with the tasks, hit: C-x C-s to close and return here.
#+BEGIN_SRC org
- This foobar should be Interesting
- Nulla foobar Posuere
- Nullam foobar Tempus
- Etiam foobar laoreet quam sed Arcu
- Donec foobar hendrerit tempor Tellus
- Mauris foobar mollis tincidunt Felis
- etiam laoreet quam sed arcu
- nullam rutrum
- sed diam
- lorem ipsum dolor sit amet, consectetuer adipiscing elit
- mauris ac felis vel velit tristique imperdiet
- fusce suscipit, wisi nec facilisis facilisis, est dui fermentum leo, quis tempor ligula erat quis odio
- donec hendrerit tempor tellus
#+END_SRC
Notice that the frame is split with this document still visible?
Good, let’s play around. Let’s capitalize the first letter of each line.
Position point at the beginning of the first line, and then:
- F3
- M-c
- C-n
- C-a
- F4
Press F4 repeatedly to capitalize the first word on each line.
Don’t capitalize /every/ line... leave a few alone for a moment.
Now you have the basics.
Let’s make a new macro to capitalize the /last/ word on each line.
Pop back up to the top of the list, and:
- F3
- C-e
- M-b
- M-c
- C-n
- F4
Once again, leave a few lines alone.
Just one more macro, as I want to show you something interesting.
Jump back to the beginning, and:
- F3
- C-a
- M-f
- Space
- foobar (or type any single word you find amusing)
- C-n
- F4
Type: C-x C-k C-k
Notice it behaves the same as F4, but you can keep typing C-k to
execute your macro.
Start typing C-n and/or C-p and notice the minibuffer. It alternates
between the following:
- M-c C-n C-a
- C-e M-b M-c C-n
- C-a M-f SPC f 2*o b a r C-n
Typing C-k at any point will execute that macro. You’ve now see the
Emacs Macro Ring, and can manipulate it. You are now at Level 2.
Type C-c ' (apostrophe) to close that side window.
** Naming Macros
While that =foobar= macro may be quite handy, looping around the
keyboard macro ring to select it may be inefficient. Name it!
C-x C-k n
Give it a name of foobar, and now, you can: M-x foobar
to have the macro run.
Perhaps it may be more useful to bind that macro to some un-used
key, like C-c k or F2 ... type:
C-x C-k b
And then type your binding, e.g. C-c k
You know that C-c followed by a single letter is supposed to be free
of usage and available for your nefarious reasons, right?
Read the [[info:emacs#Save%20Keyboard%20Macro][the Emacs Manual]] for details. If you actually tried this
example, then you have earned enough experience points to be level 3.
** What do you mean On Every Line?
Keyboard macros are slick, and while we /could/ prefix the F4 with
the numbering prefix to run it multiple times at once, e.g. C-9 F4,
a typical use case is to run it on every line in a region.
Position the cursor in the block, and type: C-c ‘
#+BEGIN_SRC org
I will not take out the trash
I should not stop swearing
I may not stop to smell the roses
I do not want to go shopping
I could not care less
#+END_SRC
With the point on the first line, type:
- F3
- C-a
- M-f
- M-f
- Space
- not (type the word, =not=)
- F4
Now select the remaining 4 lines any way you like, and type:
- C-x C-k r
Read [[info:emacs#Basic%20Keyboard%20Macro][the Emacs Manual]] for complete details, and welcome to level 4.
** Numbering
Numbering things seems to be important to people. Let’s see if we
can easily number the worst song ever inflicted on humanity.
Once again, position the point inside the block, and type: C-c '
#+BEGIN_SRC org
1 Drummers Drumming
2 Pipers Piping
3 Lords a Leaping
4 Ladies Dancing
5 Maids a Milking
6 Swans a Swimming
7 Geese a Laying
8 Golden Rings
9 Calling Birds
10 French Hens
11 Turtle Doves
12 Partridge in a Pear Tree
#+END_SRC
Follow along at home. First, position the point at the end (on the
line with the partridge), and:
- C-x C-k C-c
- 1
- Return (to set a counter to 1)
- F3
- C-a
- F3
- Spacebar
- C-n
- F4
Yes, hitting the F3 key twice (once to start the macro, and again to
insert the value of the counter) is a bit odd. You can also use the
binding: C-x C-k C-i
Each time you hit F4, you increment the counter that is inserted.
By the way, if you didn’t do the whole =C-x C-k C-c= to set the
counter to 1, the first value would be 0. Read [[info:emacs#Keyboard%20Macro%20Counter][the manual]] for
details.
** Fixing Macros
Now that you are a Level 5 Elf Keyboard Macro-er, and you can now
sling macros everywhere, you may notice that sometimes, in the
middle of a long macro, you flub it.
If you hit F4 too soon (it happens), hit: C-u C-u F3 to pick up
where you left off and enter the rest of the macro. Hit F4 when you
are really finished.
If you need to fix a keyboard macro with more finesse, type:
C-x C-k C-e
And behold the glory. Tis a simple language that should be pretty
obvious to a skilled Macro-er as yourself. Type C-h m once in that
buffer to display details of how to edit the macro. When you are
finished editing, type: C-c C-c
I’ll let you play around with your own example for this one.
Read [[info:emacs#Edit%20Keyboard%20Macro][the manual]] for complete details.
** Variations on a Theme
To gain the Level 7 Keyboard Macro-er title, one should know how to
customize a macro /while/ running it. The following section of HTML
code needs some textual changes. We want to add one of the following
phrases to the /end/ of every paragraph that has a =class= of =change=:
* Because I said so. Got it?
* Because I'm the boss. Got it?
* You heard me. Got it?
* Just do it. Got it?
To begin, first move to the following block (hint: C-c M-f) and hit
TAB to collapse the block (you gotta see all the instructions,
right?) Next, issue a C-c ' on this block to show it in a new
window. If you have trouble with your HTML mode, change the =html=
to =text=.
#+BEGIN_SRC html
<!DOCTYPE html>
<html>
<body>
<p>
Proin neque massa, cursus ut, gravida ut, lobortis eget, lacus.
Praesent augue. Sed diam. Nunc eleifend leo vitae magna. Nunc
rutrum turpis sed pede.
</p>
<p class="change">
Nullam rutrum. Nunc rutrum turpis sed pede.
</p>
<p>
Phasellus at dui in ligula mollis ultricies. Curabitur lacinia
pulvinar nibh. Donec pretium posuere tellus. Praesent
fermentum tempor tellus. Proin quam nisl, tincidunt et, mattis
eget, convallis nec, purus.
</p>
<p class="change">
Fusce sagittis, libero non molestie mollis, magna orci ultrices
dolor, at vulputate neque nulla lacinia eros. Sed diam. Nam
vestibulum accumsan nisl.
</p>
<p>
Aliquam feugiat tellus ut neque. Nam vestibulum accumsan
nisl. Praesent fermentum tempor tellus.
</p>
<p>
Vivamus id enim. Suspendisse potenti. Curabitur lacinia
pulvinar nibh. Mauris ac felis vel velit tristique imperdiet.
</p>
<p class="change">
Donec vitae dolor. Mauris ac felis vel velit tristique
imperdiet. Nunc aliquet, augue nec adipiscing interdum, lacus
tellus malesuada massa, quis varius mi purus non odio. Proin
quam nisl, tincidunt et, mattis eget, convallis nec, purus. Nam
euismod tellus id erat.
</p>
<p>
Nullam rutrum.
</p>
</body>
</html>
#+END_SRC
Type the following:
- F3
- C-s
- Type: class="change"
- C-e
- C-s
- Type: </p>
- C-p
- C-e
- C-x q
- Type: Got it?
- C-n
- F4
In this particular case, we actually didn't do anything special, so
move to the beginning of the buffer, and hit F4, and the cursor will
go to the end of every paragraph that needs changing, and stop with
a prompt:
Proceed with macro? (Y, N, RET, C-l, C-r)
Type C-r and begin typing one of our phrases, and when you are done,
type: C-M-c
The prompt will be re-displayed, so finish the macro with 'Y'.
Check out [[info:emacs#Keyboard%20Macro%20Query][the manual]] for details on this =C-x q= business.
This isn't a bad approximation, see [[https://www.google.com/search?q=y+-+y%5E3+%2F+6+%2B+y%5E5+%2F+120+-+y%5E7+%2F+5040+%2B+y%5E9+%2F+362880][Google’s answer]] for comparison.
Responding to [[http://irreal.org/blog/?p=6924][yesterday's post]], Sacha asks if I could post the code for
=jcs-insert-url= for others to use. I thought I'd already done that but
apparently not. That's probably because except for the part identical to
=jcs-get-link=, which I /did/ [[http://irreal.org/blog/?p=2895][write about]], it's pretty trivial. In any event,
here it is:
#+BEGIN_SRC emacs-lisp
(defun jcs-insert-url ()
"Insert URL of current browser page into Emacs buffer."
(interactive)
(insert (jcs-retrieve-url)))
#+END_SRC
The =jcs-retrieve-url= function does all the work, of course, and is
just the code that I abstracted out of =jcs-get-link= to actually
retrieve the URL from Safari:
#+BEGIN_SRC emacs-lisp
(defun jcs-retrieve-url ()
"Retrieve the URL of the current Safari page as a string."
(org-trim (shell-command-to-string
"osascript -e 'tell application \"Safari\" to return URL of document 1'")))
#+END_SRC
One obvious problem with all this is that it works only for macOS. Not to
despair, though, because in the comments to the original post, [[http://irreal.org/blog/?p=6924#comment-3732979999][Brad Collins
suggests a solution]] that uses [[https://github.com/xuchunyang/grab-x-link][grab-x-link]] to do the same thing for FireFox and
Chrome on other systems. Be sure to read Brad's comment because there is---or at
least was---an issue with the MELPA version.
Finally, Sacha took the part about looking for ways to make your workflow easier
seriously and came up with a bit of Elisp to [[http://sachachua.com/blog/2018/01/org-mode-inserting-a-function-definition/][insert a function definition at the
point]], regardless of where it's defined. That's very handy and I immediately
stole her code and used it to insert the two functions above. My old method was
to switch to =init.el=, find the function, copy it to the kill ring, switch back
to the original buffer, add the source block fences, and insert the code between
them. Sacha's code did all of that for me and I didn't even have to leave my
current buffer. That's splendid. If you find yourself having to add function
definitions to your text, be sure to read Sacha's post. It will save you a lot
After writing about Florian Adamsky's post on [[http://irreal.org/blog/?p=7040][acronyms in AUCTeX]], I snooped
around on his site and came across a [[https://florian.adamsky.it/2016/03/31/emacs-calc-for-programmers-and-cs.html][nice post]] on [[https://www.gnu.org/software/emacs/manual/html_node/calc/index.html][Emacs Calc]] from a programmer's
and computer scientist's point of view. As regular readers know, I've been
working to increase my calc-fu lately so I read the post with interest.
Adamsky demonstrates some of the Calc functions that are useful to programmers
and computer scientists. This includes such things as entering and displaying
numbers in various radixes and performing the standard logical operations on
(the usually binary representation of) numbers. He even shows how to add a new
“units” representation to Calc---in this case bits/bytes/bits per second.
Calc is a large subsystem and famously hard to master but worth the effort. It's
been described as a “poor man's Mathematica.” It's not nearly as powerful as
Mathematica, of course, but it's surprising how many things it can do. If you're
a programmer/computer scientist and an Emacs user you should spend a little time
investigating Calc. It really can make your life easier. An easy way to get
started is to read Adamsky's post. It covers only a small slice of Calc but will
The other day, I saw [[https://www.reddit.com/r/emacs/comments/89bxe0/org_mode_longterm_time_tracking_for_freelancers/][this query]] on the reddit Emacs subreddit. I already have
solutions for this type of problem but I'm always interested in the how people
use Org mode to record and report data so I followed the link that primitiveinds
provided for [[https://alexpeits.github.io/programming/2017/02/12/org-timesheets.html][his solution]] to generating time reports.
Even if, like me, you already have your time tracking and reporting needs under
control, primitiveinds' solution is worth looking at for its own sake. It works
by looking for CLOCK entries in an Org buffer and accumulating the relevant
information in the CLOCK line as well data about the associated task. That might
seem like it would require routine but tedious text manipulation but
primitiveinds leverages the org-element functionality to easily handle the task.
He starts by calling =org-element-parse-buffer= to generate a tree
representation of the Org buffer. Then he uses =org-element-map= to examine each
CLOCK element (and only CLOCK elements) to extract the necessary
information. It's a great technique that can easily be adapted for other parsing
of Org data. The code that primitiveinds presents is easy to follow and he
provides a nice explanation of what it's doing.
If you need to programmatically examine Org data for further processing, you
should take a look at primitiveinds' post. It's definitely worth a read.
If you're like me, you automatically think of the Org mode table editor (or
Orgtbl minor mode) when you think of tables in Emacs. It's hard to beat that
functionality and Orgtbl mode makes it available everywhere in Emacs, even if
you're not in an Org buffer. Sometimes, though, you'd like to have special
formatting for some or all of the table. That's where =delim-col= comes in.
=Delim-col= is /built-in/ Emacs functionality that allows you to do things like
adjust what string separates the columns, add a beginning or ending string to
each item, add an ending string for each row, and adjust the padding in the
table. It can be really handy for copying and pasting and then reformatting
tables from an external source.
I didn't know about =delim-col= until I read about it [[https://emacsnotes.wordpress.com/2018/09/24/delim-col-a-handy-tool-for-creating-pretty-tables-and-converting-those-to-different-table-formats/][over at Emacs Notes]], where
you'll find a good explanation of the facility and what it can do. The Emacs
Notes post also offers at bit of Elisp to make choosing the strings and
delimiters a bit easier. By default you have to set them using a series of
=setq= statements if you want something different from the built-in choices. The
Emacs Notes codes arranges for you to be prompted for the values.
You probably won't need the =delim-col= functionality very often but when you do
it's much easier than using something like a keyboard macro. Take a look at the
Way back in 2014, I [[http://irreal.org/blog/?p=2575][posted]] about Eric Neilsen's excellent [[http://ehneilsen.net/notebook/orgExamples/org-examples.html][Emacs org-mode
examples and cookbook]]. I recently came across a reference to it and was reminded
what a great resource it is. It's easy to browse through and just read one or
two entries when you have time. In skimming through it, I learned---or perhaps
relearned---how to [[http://ehneilsen.net/notebook/orgExamples/org-examples.html#sec-10][insert in-line calculations in a document]].
As I wrote in the original post, Neilsen is a researcher and his cookbook is
oriented at using Org mode to produce documents of various types. Still, that
covers a lot of territory and there are many good examples of powerful Org mode
use cases in it. The Document has moved or, really, taken up a second
residence. It was originally hosted at [[http://fnal.gov/][Fermilab]], where Neilsen works, and it's
still there but it's also available at his own site. The two documents are
identical so it doesn't matter if you use the new link or the original one
pointing to FNAL.
If you're an Org user, especially if you use Org to produce documents,
you should take a look at Neilsen's cookbook and bookmark it for future
*TLDR*: I started using Emacs about 3 years ago. I couldn't be more grateful to
have seen the light, and to have been rescued from the darkness of Windoze,
Goggle and/or friends. After enlightenment, I've taken upon myself the task of
customising an environment to write my PhD thesis with Org Mode.*
*** Why
Post created in response to the [[https://www.reddit.com/r/emacs/comments/9ynsvc/write_a_thesis_using_emacs_and_orgmode/][current thread]] in /r/emacs/ on thesis writing
with Org Mode.\\ I see most people's reason to avoid Org mode for scientific
writing is the fact that supervisors or co-authors use Mic. Word. I'll try to
argue that that's not enough reason to accept subpar tools.
*** What I'll talk about
I'll mention a bit of my motivations, and then I'll discuss how to make use of
(mostly) built in Org functionality such as tagging, export, [[https://orgmode.org/manual/In_002dbuffer-settings.html][setupfiles]] and
includes, reference management, keyboard shortcuts and advanced searching; all
with the purpose of building a useful thesis writing environment. Readers should
have a minimum knowledge of Org mode, the Org export system and LaTeX.
*** My requirements
Here in the Netherlands, most PhD thesis consist of an introduction, 3 to 4
research chapters (as submitted for publication), a summary, bibliography and
appendices. What this means for me is that my writing environment has to
/necessarily/ satisfy the following *minimum requirements*:
- Inclusion of (parts of) external files
- Keeping track of references
- Include and reference figures
- Version control documents
- Support for sharing with my supervisor in whatever format he wants
Failure to comply with any of these means the editor is unfit for
purpose^{#fn.1”>1}. Unfortunately, this set of requirements are not seamlessly
satisfied by likes of Mic. Word or G. Docs. I reckon they can probably be
configured to satisfy them, but why bother.
Additionally, a PhD thesis writing environment should also provide the following
features:
- Extended searching facilities for both text and references
- Simple syntax for tables and equations
- Support within a proper text editor
- Shortcuts to reach my files and build the thesis
To the best of my knowledge, /only/ Emacs with Org Mode + ox-latex provide all
of these out of the box.
*** Moulding Org Mode for thesis writing
Most of my inspiration comes from reading Kitchin's blogs and code, and reading
the Org Mode documentation, mailing list and Emacs Stack Exchange. Here' I'll go
one by one through all of the requirements listed above, and how to deal with
them.
**** Prelude: File structure
I have a main /thesis.org/ document, with latex heading declarations and a
commented setup file. I also have /research.org/ files, in different
directories, with their own latex heading declarations and commented setup
files.
The first lines of /thesis.org/ look like the following:
'(("Open in Emacs" . org-ref-open-pdf-at-point-in-emacs))))
#+end_src
**** Include and reference figures
For each research project I keep a =./media= directory, where all my figures
live. You can include figures in Org mode by using the following syntax:
#+begin_src org
#+NAME: figurename
#+CAPTION: This is a figure caption
[[path_to_figure][link_description]]
#+end_src
Currently there is a bug in the ELPA version of Org mode, such that relative
paths to figures in =#+INCLUDE= 'd files aren't adapted with respect to the
including file, so the latex export cannot find them. I've [[https://code.orgmode.org/bzg/org-mode/commit/d81a1d088c74e605c99e90a2835c55df5144f43e][submitted a fix]]
which should land in the next release of Org.
**** Version control documents
[[https://magit.vc/][Magit]]. I thought about having the research chapters as git submodules in a
thesis git project directory, but I currently don't. This would allow me to
always have the thesis code in a saved state, even if I further work on my
research chapters to answer to reviewers questions.
**** Support for sharing with my supervisor
Unfortunately, my supervisor likes to write comments in Mic. Word. I give in
that sharing your writing with colleagues is a fundamental part of
research.\\ Fortunately, [[https://github.com/jkitchin/scimax/blob/master/ox-word.el][ox-word]] export via Pandoc & LaTeX is capable of
creating nice looking, structured Word files which I send to my supervisor. I
then manually work through each comment step by step, though I'm looking for a
way to improve this part of my workflow. I think the Emacs community is missing
a minor mode to track Word document changes from within Org Mode. There are some
ideas laying around on how to implement it [[https://lists.gnu.org/archive/html/emacs-orgmode/2015-06/msg00246.html][hidden deep in the mailing list]], or
in [[https://emacs.stackexchange.com/questions/34923/merging-changes-to-from-docx-files-into-org-files][this Emacs Exchange thread]].
I may update this post with more information later.
**** Extended search facilities
By extended search facilities I mean the ability to quickly search for
information in references, and to keep notes linked to the literature. For
searching I make use of [[https://github.com/jkitchin/org-ref/issues/597][org-ref + pdfgrep]], as shown in my org-ref setup. For
notes linked to documents I've recently started to use [[https://github.com/weirdNox/org-noter][Org-noter.]]
**** Simple syntax for tables and equations
Org tables are a pleasure to work with. The following: