Malinka, an Emacs C/C++ project configuration package

Introducing malinka, a C/C++ project configuration package for Emacs. Malinka can be used to configure projects in C or C++ and provide correnc input to other packages like rtags and flycheck so that a modern code navigation and editing experience can be achieved.

Working with C/C++ code in Emacs can sometimes become a really frustrating undertaking. Using ctags, gtags or Cedet does not always provide the best results that one would desire from a modern editor when navigating C/C++ code.

In an earlier post I had written of a way to force Semantic to parse all files under a particular source tree so that it could know about all the tags of a project. While it worked most of the times, it is a crude way to deal with the problem. Not to mention it’s terrible lisp code. Since then I dealt a little bit more with elisp and emacs configuration and came to the conclusion that there are better ways of dealing with the problem.

Introducing Malinka

Enter Malinka, a C/C++ project configuration package for Emacs. It’s a kind of a meta-package combining the power of a few other packages to create a modern Emacs development experience for C/C++ programmers.

The main functionality that malinka provides is a way to configure C/C++ projects and through that configuration empower rtags and optionally other packages like flycheck with correct input at the correct time so that a modern C/C++ code editing and especially navigation experience is achieved.

Prerequisites

In order to be able to use Malinka you need to have a few basic things setup in your .emacs.d. As long as you have dealt with the first prerequisite you don’t have to worry about the rest. It’s just a matter of M-x el-get-install RET malinka RET if you are using el-get or M-x package-instal RET malinka RET if you are using melpa. Those commands would pull and build all other bullet points but I am mentioning them here for documentation purposes

  • el-get or melpa: el-get is a modern way to manage your emacs packages. The easiest way to install malinka is by using el-get to pull and build all dependencies. Alternatively you can use package-instal having the melpa package repositories configured in your package archives.
  • rtags: rtags is a dependency that el-get would get and build for you manually. It’s a package based on a server and client model. The rtags daemon is called rdm and uses clang to parse and maintain all tags of a malinka project and inotify to reparse any files that have been edited. The rtags client is called rc and is the method by which we communicate with the daemon. If desired you can start the daemon manually by invoking something like
    rdm > /dev/null 2>&1 &
    

    But there is no need to worry about that since as long as the rtags package is installed malinka will assert that rdm is running and if not it will attempt to start it.

  • projectile: projectile is a generic project creation and management library for Emacs. It contains a lot of very useful functionality to detect and manage projects mainly using VCS. In order to not reinvent the wheel some of projectile’s functionality is used by malinka.
  • flycheck: (optional) Flycheck is an optional dependency, but one that can integrate very nicely. Using flycheck and clang’s syntax checker malinka can provide correct input data to flycheck depending on the currently visited buffer. As a result the syntax checker can show all warnings being aware of the different compilation parameters of each buffer.

Setup

After you have installed malinka using el-get or any other method, you need to set it up so that it becomes aware of your C/C++ projects. The setup step is basically about letting clang know the compile commands of your project.

The suggested method from rtags itself is to symlink the rtags wrapper script to your compiler. It is indeed the most hassle-free method but with malinka we are going to be trying a different approach. The malinka approach is far from perfect and as can be seen in the future work section there are some plans to improve it.

Simple project setup

The way to let it know about a particular project is by using (malinka-define-project). As a very minimal example picture the setup of Refu C library, an old project of mine.

(require 'malinka)

(malinka-define-project
 :name "Refulib"
 :cpp-defines '("REFU_COMPILING" "REFU_LINUX_VERSION" "REFU_TEST"
                "RF_MODULE_DF_XML" "RF_MODULE_DS_ARRAY" "RF_MODULE_DS_BARRAY"
                "RF_MODULE_DS_LIST" "RF_MODULE_IO" "RF_MODULE_IO_TEXTFILE"
                "RF_MODULE_REGEX" "RF_MODULE_STRINGS" "RF_MODULE_SYSTEM"
                "RF_MODULE_THREAD" "RF_MODULE_THREADX" "RF_MODULE_TIME_DATE"
                "RF_MODULE_TIME_TIMER" "RF_OPTION_DEFAULT_ARGUMENTS"
                "RF_OPTION_FGETS_READ_BYTESN=512" "RF_OPTION_FILE_ADDBOM"
                "RF_OPTION_LIST_CAPACITY_M=2"
                "RF_OPTION_LOCALSTACKMEMORY_SIZE=1048576"
                "RF_OPTION_REGEX_NODES_ALLOCMULTIPLIER=4"
                "RF_OPTION_REGEX_STACKSIZE=250"
                "RF_OPTION_STRINGX_CAPACITY_M=2"
                "RF_OPTION_THREADX_MSGQUEUE=10" "RF_OPTION_VERBOSE_ERRORS"
                "_FILE_OFFSET_BITS=64" "_GNU_SOURCE" "_LARGEFILE64_SOURCE"
                "RF_MODULE_LIST_EXTRA" "RF_MODULE_LIST")
:compiler-flags '("-Wall" "-g")
:include-dirs '("/home/lefteris/w/Refulib/include")
:root-directory "~/w/Refulib")

(malinka-define-project) can accept more arguments than seen in this example so we should look at all of them one by one. (For a full and up to date list always do check the source code too!):

  • name: The name of the project. Should be the same name as the root directory of the project.
  • cpp-defines: The macros definitions to be passed to the C preprocessor.
  • include-dirs: The include directories of the project where the compiler should search for headers.
  • compiler-flags: Other flags to pass to the compiler.
  • root-directory: The full path to the root directory of the project. If not specified then projectile is used to try and find the root judging by the current’s buffer location and the malinka project name.
  • compiler-executable: The full path to the compiler executable. Defaults to /usr/bin/gcc.

The difficulty in this approach is that we need to manually copy all of the compiler options in the malinka project definition. In the future work section one can see suggested improvements to this.

Setup of multiple versions of a project

Some teams have their workflow organized in a way so that different versions of the code is kept under different directories. The reason for such a setup is only if the product you are working on is complex and comprised of multiple repositories. Then it makes sense to have 2 different versions of all repos checked out so that one can work on both at the same time easily. For example:

+
|
+ version 1.20 +
|              |
|              project_foo +
|              |           |
|              |           sources
|              |           ...
|              lib_foo     +
|              |           |
|              |           sources
|              ...      
|
+ version 1.60 +
|              |
|              project_foo +
|              |           |
|              |           sources
|              |           ...
|              lib_foo     +
|              |           |
|              |           sources
|              ...      
|

In such a case the setup that you would keep for either project_foo or lib_foo would not specify a root directory but utilize the same-name-check parameter of (malinka-define-project)

(malinka-define-project
 :name "project_foo"
 :cpp-defines '("SOMETHING_IS_DEFINEd")
 :compiler-flags '("-Wall" "-g")
 :include-dirs '("/path/to/include")
 :same-name-check nil)

Having the same-name-check parameter set to nil along with the absence of a root-directory allows for such a multiple versions of the same project setup to work. We will see how in the next section.

Usage

The first time you want to use malinka in a project you have to configure that project for your system. The way to achieve this is by calling M-x malinka-project-configure from any buffer belonging to the project you need to configure. Once this is done a json file containing the clang compilation database for this project will be created. The name of the file is compile_commands.json. I suggest you add it to the ignores of your version control system.

Note that you do not really need to be in a buffer belonging to the project in order to call M-x malinka-project-configure. It will prompt you with a choice of project names that it knows about. If malinka notices that the current buffer belongs to a known project it will be provided as the default choice. So it’s generally good practise to be in a buffer belonging to the project you want to configure.

In the case of a project with multiple versions as shown here, there is an extra selection that the user is prompted for when invoking M-x malinka-project-configure after having selected the project name. Malinka will prompt you for the root directory of the project so that you can choose which version/checkout you need configured.

After that is done and after rtags have indexed all of the project (it can take some time for large codebases) you can use all of the rtags commands in order to jump around and navigate in the C/C++ source code of your project.

Using rtags with malinka

For all the possible commands and in order to be up to date I suggest directly checking the rtags documentation and their source code to see what is possible. As a simple example and to give you a taste of the power of rtags take a look at the default keybindings of their API.

(defun rtags-enable-standard-keybindings (&optional map prefix)
  (interactive)
  (unless map
    (setq map c-mode-base-map))
  (unless prefix
    (setq prefix "\C-xr"))
  (ignore-errors
    (define-key map (concat prefix ".") (function rtags-find-symbol-at-point))
    (define-key map (concat prefix ",") (function rtags-find-references-at-point))
    (define-key map (concat prefix "v") (function rtags-find-virtuals-at-point))
    (define-key map (concat prefix "V") (function rtags-print-enum-value-at-point))
    (define-key map (concat prefix "/") (function rtags-find-all-references-at-point))
    (define-key map (concat prefix "Y") (function rtags-cycle-overlays-on-screen))
    (define-key map (concat prefix ">") (function rtags-find-symbol))
    (define-key map (concat prefix "<") (function rtags-find-references))
    (define-key map (concat prefix "[") (function rtags-location-stack-back))
    (define-key map (concat prefix "]") (function rtags-location-stack-forward))
    (define-key map (concat prefix "D") (function rtags-diagnostics))
    (define-key map (concat prefix "G") (function rtags-guess-function-at-point))
    (define-key map (concat prefix "p") (function rtags-set-current-project))
    (define-key map (concat prefix "P") (function rtags-print-dependencies))
    (define-key map (concat prefix "e") (function rtags-reparse-file))
    (define-key map (concat prefix "E") (function rtags-preprocess-file))
    (define-key map (concat prefix "R") (function rtags-rename-symbol))
    (define-key map (concat prefix "U") (function rtags-print-cursorinfo))
    (define-key map (concat prefix "O") (function rtags-goto-offset))
    (define-key map (concat prefix ";") (function rtags-find-file))
    (define-key map (concat prefix "F") (function rtags-fixit))
    (define-key map (concat prefix "x") (function rtags-fix-fixit-at-point))
    (define-key map (concat prefix "B") (function rtags-show-rtags-buffer))
    (define-key map (concat prefix "I") (function rtags-imenu))
    (define-key map (concat prefix "T") (function rtags-taglist))))

I will let it as an exercise to the user to experiment with most of those. But the main strength of malinka is that it empowers rtags with the correct input data so that (rtags-find-symbol-at-point) and (rtags-find-references-at-point) work properly.

Go to any source file of your project and no matter how complicated the definition of a tag is, no matter how far away in the include chain it lies, your cursor will always correctly jump to it.

After that if you want to jump back to the previous tags and follow the tag jumping backwards you would call (rtags-location-stack-back). And vice versa if you need to go forward again you would call (rtags-location-stack-forward).

I suggest binding all those to convenient keys since you will be using them quite often 🙂

Malinka API

At the moment these are the functions one can use to interact with a project that has been registered with malinka:

  • M-x malinka-project-configure: As seen here this is the command to configure a project that has been defined with (malinka-define-project). Configuring means creating the clang compilation database and registering it with the rtags daemon.
  • M-x malinka-project-add-file: Allows adding a file to the project after configuration and indexes it with the rtags daemon.
  • M-x malinka-project-add-include-dir: Allows adding an include directory to the project’s compile commands and updates the rtags daemon.
  • M-x malinka-project-add-cpp-define: Allows adding a macro definition to the project’s compile commands and updates the rtags daemon.

Future Work / Conclusions

Malinka is a first approach of mine to create an Emacs package whose sole purpose is to manage C/C++ projects using the most modern related packages available to Emacs as of this moment. There is definitely room for improvement.

The main improvement that can be implemented is some form of Makefile or build command parsing so that cpp-defines, include directories and generally all compiler parameters are automatically acquired by malinka and passed directly to rtags. With such a feature, any compiler parameters given at (malinka-define-project) would simply be appended to what has already been parsed.

Finally I am no emacs lisp expert and I am always looking to learn. As a result any contributions to malinka are more than welcome, be it in the form of issue-reporting or pull requests/patches.

That’s all! Until next time.

Running emacs as a daemon with systemd

This is a quick post explaining how to use systemd to automatically load emacs at the startup of your operating system and then use emacs client to connect to it

This is a quick post explaining how to use systemd to automatically load emacs at the startup of your operating system and then use emacs client to connect to it. This requires emacs version greater or equal to 23 and also a linux distro that is using systemd. In my system I am running Arch Linux with emacs 24.3 and at the time of this post I was using systemd version 212.

UPDATE 27/06/2013:This post explained how to have Emacs start as a general systemd service ran by root. Go to the end of the blogpost for an even better way to configure emacs as a user systemd service!

Step 1. Create the systemd service

First of all we need to create the systemd service file as is shown in the emacs wiki. Place the following into a file named /etc/systemd/system/emacs@.service

[Unit]
Description=Emacs: the extensible, self-documenting text editor

[Service]
Type=forking
ExecStart=/usr/bin/emacs --daemon
ExecStop=/usr/bin/emacsclient --eval "(progn (setq kill-emacs-hook 'nil) (kill-emacs))"
Restart=always
User=%i
WorkingDirectory=%h

[Install]
WantedBy=multi-user.target

In case you are wondering how systemd works you can visit the man page or the Arch Linux wiki page on the subject. The above should start your emacs in daemon mode at startup and restart it even if it’s killed. It will start by the user who enables/starts this service file and use his init file for startup.

To stop the emacs unit we use the emacsclient to send some lisp code for evaluation. This code simply exits the process and stops the daemon.

(progn (setq kill-emacs-hook 'nil) (kill-emacs))

Step 2. Enable the service

With systemd you can among other things, enable, disable, start, stop or query a status of a service. We achieve all these by the following commands for emacs as a service.

~$ sudo systemctl enable emacs@lefteris.service
~$ sudo systemctl disable emacs@lefteris.service
~$ sudo systemctl start emacs@lefteris.service
~$ sudo systemctl stop emacs@lefteris.service

~$ systemctl status emacs@lefteris.service

Please note that you should replace lefteris with your own user name. Also .service can be ommited. It simply serves to determine the type of systemd unit we are referring to. Service is the default. Other units can be mountpoints, devices and sockets.

  • enable: The service will start from statup next time you boot the system
  • disable: Undo an enable
  • start: Starts the unit using the execution start command
  • start: Stops the unit using the execution stop command
  • status: Returns a status report about the unit

I would suggest you enable the emacs systemd unit now using the above command. Then start it. You can also check the status of the unit and get output similar to the one below.

systemctl status emacs.service

Step 3. Connect to the server using emacsclient

Now it should be pretty straight forward to connect to the running emacs daemon using emacsclient. For example running emacsclient -c should connect you to the server.

emacsclient: connect: Connection refused
emacsclient: No socket or alternate editor.  Please use:

        --socket-name
        --server-file      (or environment variable EMACS_SERVER_FILE)
        --alternate-editor (or environment variable ALTERNATE_EDITOR)

Wow! What’s this? But the server is indeed running. You can quickly check it with a ps aux | grep emacs . And to add to that the owner of the process is our user and not the root user. So what is the problem?

We should also check if the server is indeed listening and on which socket by using the following command:

~$ sudo netstat -xaupen | grep emacs

unix  2      [ ACC ]     STREAM     LISTENING     88697    5924/emacs           /tmp/emacs1000/server

Hmm … the output looks all right but there is something weird about the socket. Usually sockets opened by user programs should be under /tmp/USERNAME/... . That explains why emacsclient can’t connect. So all we have to do is alter the command a bit:

emacsclient -c --socket-name /tmp/emacs1000/server

And voila! This connects us to the correct emacs server instance.

I can’t say I am sure if this can be considered a bug of emacsclient, a bug of emacs in daemon mode or neither. In any case using the --socket-name argument solves the problem and allows us to connect to emacs at any point in time having it started as a systemd unit at startup. If you have any comments or suggestions, especially concerning the socket argument please don’t hesitate to write below.

Until next time!

UPDATE 27/06/2013 – Configuring Emacs as a user systemd service

After working for some time with Emacs as a systemd service and learning more about systemd I realized that there is a better (and easier) way to configure emacs as a systemd service specified only for the user you need. Accomplishing this is really easy. Look at the following service file.

[Unit]
Description=Emacs: the extensible, self-documenting text editor

[Service]
Type=forking
ExecStart=/usr/bin/emacs --daemon
ExecStop=/usr/bin/emacsclient --eval "(progn (setq kill-emacs-hook 'nil) (kill-emacs))"
Restart=always

# Remove the limit in startup timeout, since emacs
# cloning and building all packages can take time
TimeoutStartSec=0

[Install]
WantedBy=default.target

The service file is almost the same as the one in the original post except for the fact that User and Working directory do not need to be specified. Moreover I have added TimeoutStartSec=0 to avoid systemd killing emacs if for some reason it takes too long to start and send the sd_notify message. This can happen if for example you are using el-get to clone and build all your Emacs packages in a clean environment.

Finally notice that multi-user target has been replaced by the default.target. If you are using some other –user systemd scripts you can configure that section properly. If you don’t know what a target is then just leave it as the default.target. In –user service files there is no multi-user.target so this is a field that you will need to change.

Now all that you need to do is copy this service file as emacs.service to ~/.config/systemd/user/emacs.service. After that you can work with it just as you did in the original blog post but with the main difference of having to add the --user argument after systemctl. For example:

systemctl --user enable emacs
systemctl --user start emacs
systemctl --user stop emacs
systemctl --user disable emacs