A Minimal Development Environment: Part Two

In part 1 of this series, I outlined the reasons for attempting to build a minimal development environment and set a goal for this project. I also summarized my experiences with trying to set up such an environment using a variety of small-size Linux distros. In this segment, I share the results of building out the environment on Debian Linux with the NeoVim editor as the primary development tool.

There is so much detailed information about exactly how to install various tools, which Vim plugins worked and which didn’t, and where the “gotchas” are, that I decided it was too much for a blog post. What follows is a summary of the end results without too much description, except where an explanation is warranted.

Constraints

I limited myself to a 512 MB instance so that I would feel confident there was ample margin for running necessary software in a more normal, yet still small system such as a Raspberry Pi 3B+, which has 1 GB of RAM. I also wanted to load it up with as many languages and Vim plugins as I could, to prove the case that there was ample room both in memory and in persistent storage for all these tools. This is not because I expect anyone to work with 30 or 40 different languages on a routine basis, but rather to make the case that any professional developer ought to be able to configure a suitable environment within these memory and storage constraints.

Lessons Learned

• It’s feasible to put together a usable and productive development environment in a small-footprint OS instance (as small as 512 MB RAM) and using a minimum of different tools (NeoVim editor with plugins, plus a few command-line tools). Of 35 programming, scripting, and markup languages tested, 34 worked at least to some extent, and the remaining one would not build given the tight memory constraint I set. That result indicates it would be very feasible to configure a lightweight development environment that supported the handful of languages a real person might actually use. No one I’ve ever heard of has a workaday development setup for 35 languages.
• A mouseless configuration is comfortable, based on Debian Linux, the OpenBox window manager for X, and NeoVim. When I say “comfortable,” I mean it feels natural, as opposed to using an IDE that had some keyboard shortcuts added to it in hindsight.
• For most languages, you sacrifice some conveniences of IDEs but you can still get work done; might be okay if you need to work with multiple languages and want a simple environment overall. Also might be okay if you prefer working on the command line anyway, and you find all the “help” provided by IDEs to be less helpful than intended.
• For some languages, the developer workflow in the minimal environment is actually better than that with a full-featured IDE; notably Scala, Ruby, and COBOL; .NET developer experience is comparable to using VSCode; better than using VisualStudio in my opinion. Working with Markdown or with markup languages and supersets like Haml is comparable to using Sublime Text 3 or Atom.
• For the most part, the Vim plugins available are not well documented or supported. Some of them are very good, but on the whole you cannot count on them to work or even to install, and when something goes wrong you must expect to troubleshoot it yourself. At least half the plugins I attempted to use did not work. In many cases you can get reasonable syntax highlighting and indenting, and in some cases reasonable completion. Beyond that, be grateful for anything that works.

A Nice Workflow

When trying out Scala, I found a very nice developer workflow was possible with sbt. I suspect a lot of people who work with Scala every day already do this. When I installed sbt I realized it hadn’t occurred to me to set up this sort of workflow when using an IDE unless the tooling was baked into the IDE. I had become lazy, in that sense. This simpler tool stack made everything inescapably visible.

With an sbt console watching the Scala project directory in one window, and an editor open in another window, you can move smoothly between code changes and the results of the changes. The video below shows this on the minimal VM. NeoVim is open in the upper window, and sbt console is running in the lower window. The ~run command in sbt console causes sbt to run a build every time it detects a change to Scala source files in the project. For me, it’s easier to do this using a simple text editor and a command line window than it is when working within an IDE.

For the experiment I didn’t take the time to define a full-blown project setup with tests and so forth, but it would be easy to extrapolate this workflow to a red-green-refactor cycle. You may have seen and worked with similar setups using autotest for Ruby or Infinitest for Java, or even a command-line script executing in a loop on a timer. The lightweight environment can support this sort of thing quite easily.

Almost, but not quite, all unicorns and rainbows: The sbt console doesn’t always shut down cleanly. It sometimes leaves an orphan Java process in memory. In this small environment, you find out the very next time you try to load a program because you’re already out of memory. You might have to kill the sbt Java process before you can do whatever you wanted to do next. Easy enough to script that, but it’s one more thing to write to get the environment squared away.

Arduino Development

For this experiment, I didn’t try Arduino development. However, based on a fairly recent (2017) article by Chris Rittersdorf, “Arduino Programming in Vim”, it sounds feasible.

I’ve mentioned that Vim plugins aren’t always reliable and aren’t necessarily maintained over the long term. Chris discovered the same thing when he set out to configure Vim for Arduino development. His article warns against certain Vim plugins you might find in an Internet search.

Ultimately he settled on three plugins that support code snippets and syntax highlighting for Arduino code. He also found a tool called Arduino Makefile that builds code and uploads it to an Arduino device. Read his article if you are interested in giving this setup a try.

Special Cases – Vim Not Recommended

Vim fans are fond of saying their favorite plugin “turns Vim into a full IDE,” and some of them go to great lengths to duplicate or mimic the functionality of specialized development tools through Vim configuration and plugins. There are no cases where that assertion is true, and it isn’t the intent of building a minimalist environment in the first place.

It’s unrealistic to expect this lightweight configuration to behave like a commercially-built and -supported IDE. If anything, the more plugins you add, the more fragile it becomes.

The whole idea behind this is simplicity. Once you go beyond the point of diminishing returns with plugins, you’ve subverted the goal of simplicity.

There are more than a few special situations that call for specialized development tools, and a generalized and minimal Vim-based development setup will not support them adequately for professional work. Here are a few that I recognized in the course of this exercise.

R — recommend RStudio
A serious user of R is probably a Quant rather than a polyglot application developer, and would not be interested in a multi-language setup like this. The best Vim plugin for R is Rvim-R, which doesn’t work as advertised and even if it did, would not provide a developer experience even remotely comparable to RStudio.

Java/Groovy — recommend IntelliJ IDEA or Spring Tool Suite
With appropriate Vim plugins installed, the developer experience is not bad but still not as smooth as with a full IDE. For occasional use, or for a person very familiar with Java who doesn’t need much hand-holding by an IDE, this environment would be fine. (Note: Other JVM-based languages are okay, like Clojure, Kotlin, and Scala.)

Lisp/Scheme/Racket — recommend Emacs/Slime for Lisp/Scheme, and DrRacket for Racket
Editing these languages is a pain with this tool stack. The command-line repls work well, but the developer experience with Vim is very poor. Plugins intended to “help” with parentheses make things worse. If you want to have a Lisp-like language available on this environment, I recommend Scheme with no Vim plugins added; it seemed to be the best of the three. I used mit-scheme.

Android development — recommend Android Studio
Vim aficionados say you can do development for the Android platform by using the Eclim plugin. One of the ways Eclim can work is to communicate with a headless instance of Eclipse, to tap into Eclipse features. Eclipse with an add-on called ADT used to be the standard way to develop Android apps. Apart from introducing overhead by having two development tools running in tandem, using Eclim with Vim creates a dependency on tooling that is being sunsetted. It also creates a fairly complicated stack comprising components not intended to be piled on top of each other in quite that way. In the end, if you can get this setup to work at all, the developer experience would be worse than Eclipse with ADT, and Eclipse with ADT is worse than Android Studio. Why go there?

iOS development — recommend XCode on OSX
People have tried and failed in myriad ways to configure Vim for iOS development. Those who are dead-set on using Vim should consider going the other way around: Use the Xvim plugin for XCode, instead of trying to add everything they need from XCode into Vim.

What About Legacy Code?

My comments about the differences between an IDE and an editor have mainly concerned new development, or have assumed we’re working with a clean code base. When working with an existing code base, there’s often a need for extensive refactoring. Even if we practice incremental refactoring, significant design debt may exist in the code base for a long time.

Good IDEs make it easier and less error-prone to refactor than editors. This is one of the most useful features of an IDE. When working with a legacy code base, I turn to an IDE for that reason even if I would otherwise prefer a text editor for the particular language.

Environment Summary

The minimal development environment worked well configured as follows.

General notes

• With total memory of 512 MB, there was no time during this experiment when the system felt sluggish, and no memory-related errors except while installing Idris, and initially when starting the Scala sbt.
• Loaded to the hilt with tools and libraries, the virtual hard drive used 15 GB of space. That includes the operating system. Everything was in a single partition.
• I did not install version managers such as rvm, rbenv, and nvm. The rationale was to use Debian’s alternatives system to manage multiple versions of software. I wanted to avoid loading redundant tools for version management, in keeping with the minimalist approach.

Virtual machine

• VMware guest
• 64-bit
• 512 MB memory
• 20 GB virtual disk

Operating system

• Debian 9
• developer packages as required by the various languages

Window manager

• OpenBox

Persistence

• Sqlite3 – relational DB
• PostgreSQL – relational DB
• MongoDB – document DB
• BerkeleyDB – indexed files

HTTP servers

• Nginx
• Lighttpd
• various baked into language distributions (Npm, Python, Java, etc.)

Testing tools

• SoapUI

Editor

• NeoVim with:
  • pathogen – package manager
  • vim-plug – package manager
  • deoplete – multi-language completion
  • neomake – integrated make tool
  • rainbow_parentheses – parentheses matcher
  • NerdTree – integrated file browser
  • Spacegray color scheme
  • and numerous language-specific plugins

Programming/scripting/markup languages & NeoVim plugins
[0] Fail
[1] Works on environment, but no Vim help
[2] Vim plugin partially works (or I couldn’t figure it out)
[3] Vim plugin seems to work
[4] Vim plugin works well
[5] Developer work flow is good (in my opinion)

• [2] Awk
• [2] Bash
• [3] C/C++
• [5] Clojure
• [5] COBOL
• [2] CoffeeScript – integrated make worked, split window mode did not
• [5] C#
• [3] Elixir
• [3] Erlang
• [5] F#
• [4] Go
• [3] Groovy
• [3] Haml
• [4] Haskell – used ghc, not Vim plugin
• [3] HTML
• [0] Idris – out of memory on install
• [2] J – docs are poor; you have to fiddle with it
• [3] Java
• [3] JavaScript/Node
• [3] JSON
• [3] Kotlin
• [3] LaTeX
• [1] Lisp – problems with Vim plugins
• [3] Lua
• [2] Markdown – problems with editing plugin; preview plugin OK
• [3] Matlab
• [1] OCaml – problems with Vim plugins
• [3] Perl
• [3] Python3
• [1] R – problems with Vim plugins
• [1] Racket – problems with Vim plugins
• [5] Ruby
• [3] Sass
• [5] Scala
• [1] Scheme – problems with Vim plugins
• [3] SCSS
• [2] TypeScript – several plugins highly touted; very little delivered
• [3] XML
• [3] VimScript
• [5] VisualBasic
• [3] XML

Conclusion

From this exercise I conclude it’s feasible to create a lightweight environment that’s completely suitable for professional software development that would comfortably fit in a small VM, a Docker container, or a Raspberry Pi. There are some kinds of development for which that environment would be unsuitable, but for general application development in most languages it would be fine.

I’m curious to build a development machine based on cheap hardware and a few of these development tools and see how that goes. I don’t need 35 languages, after all; I just wanted to stuff this thing as full as possible to prove or disprove the hypothesis that it would work. The next experiment could be to put a system together and use it for real work for a while, and then report on the pros and cons. That would be a narrower and deeper trial of the idea, with a selected set of development tools fully configured for real-world use.