A Minimal Development Environment: Part One

Over the years, software development tools have grown in complexity and in system resource consumption. Most developers today rely on integrated development environments (IDEs) that require a fairly hefty desktop or laptop computer to run, like Microsoft VisualStudio, JetBrains IntelliJ IDEA and the family of language-specific IDEs based on it, Eclipse and numerous IDEs based on it, and so forth.

I found myself wondering one day if we really needed all that “stuff” and if maybe the world of IDEs is experiencing the same kind of “feature bloat” that has made office software like Microsoft Office and Corel Office so “heavy” and hard to use. What would a minimal, yet functional development environment look like?

David Hussman came up with a useful and simple idea, called Dude’s Law. It goes like this:

V = W/H

It means, Value equals the Why over the How.

As IDEs began to combine useful tools in a seamless way, the H became smaller. The V became larger. That is, people found more value in using an IDE than they obtained by working with editors, debuggers, and other tools separately.

Have IDEs passed the point of diminishing returns in adding features? Is the H becoming larger, thereby reducing the V of using them? What would a minimal, yet functional development environment look like? An environment that moves us back toward a lower H and a higher V…but not too far back?

Before getting into that, let’s review the pros and cons of IDEs.

IDEs: The Benefits

IDEs are helpful because they tie together a set of tools that developers commonly use in concert. Central to these are editors, and editors are complemented by other tools like test frameworks, debuggers, profilers, and integrated pass-through to build tools, command line windows, and other system tools.

There are add-in components to provide autocompletion and color-coding for various programming languages, previews of text markup, basic refactorings, and other useful features. You can even run a server from within an IDE to help you test and debug your applications without leaving the environment.

Probably the single most significant benefit of an IDE (at least in my opinion) is that they “understand” the structure of your project and the relationships among the files it comprises. They know where your project root is located, which directories contain sources, which contain resources, which contain tests, and how to run various builds and packaging tools. All these details vary by programming language, and good IDEs “know” how to work with projects in the language they were designed to support. Text editors, however configurable they may be, lack this kind of support.

IDEs: The Trade-Offs

The trade-offs for integrating numerous functions include:

• memory footprint
• configuration complexity
• instability or sluggishness
• compatibility issues
• too much stuff

Memory footprint

On developercommunity.visualstudio.com as recently as 4th quarter 2017, some users of VisualStudio were reporting memory usage over 2 GB, as high as 2.7 GB in one case. Some of this was attributed to reported bugs that Microsoft has fixed or is fixing, and some is a consequence of longer-term issues with the product. My interpretation is that the product has so much functionality baked into it that it requires substantial memory.

VSCode is a lighter-weight alternative to VisualStudio that’s catching on quickly, even within Microsoft. But it, too, has had reports of high memory usage and high CPU. You can find reports on github.com/Microsoft/vscode/issues of memory usage as high as 3 GB, dating from as recently as 4th quarter 2017.

Several popular IDEs and families of IDEs are based on Java. Users frequently report memory-related problems that boil down to JVM memory settings, and not the IDEs as such. Key products in this category are the JetBrains suite of IDEs, Eclipse-based IDEs from various vendors, NetBeans, and Android Studio. These IDEs require significant memory because they contain so many features and functions.

The intellij-support.jetbrains.com site has many questions from users of JetBrains IDEs related to high memory usage. One user reported memory usage in WebStorm of 1 GB, which froze her development box. That issue turned out to be related to the Markdown Support plugin, as well as the -Xmx setting to control JVM memory allocation. Those causes point to the other three general issues with larger IDEs, mentioned above. In fairness to JetBrains, most of the other issues reported in late 2017 appear to be related to the Markdown Support plugin, as well.

Eclipse-based IDEs also garner a large number of user questions related to memory. On www.eclipse.org/forums, many users report running out of memory. These issues are almost always due to JVM memory settings, especially the -Xmx value. Oracle’s JDeveloper and IBM’s Rational Application Developer (RAD) have similar issues, and a fair amount of space on user forums and blogs is devoted to memory-related tuning tips to make those tools run well.

Lighter-weight “smart” editors offer an alternative to thick IDEs that ought to enable us to work within a smaller memory footprint, but this is not always the case. Atom, based on Electron.js, is notorious for high memory usage. It can take up a GB of memory to edit a single file. The issues stem from the underlying framework, Electron.js.

Sublime Text is another popular alternative to full-blown IDEs. It doesn’t appear to have significant memory problems, although there was a memory leak in a January 2018 build. Even so, it isn’t exactly small, and people like to add packages to it to customize the tool to their own needs. Anyone can write a package, just as anyone can write an Eclipse plug-in or a VSCode extension, and you never know if the add-on will behave well.

Configuration complexity

Each “heavy” IDE pays attention to some OS-level settings and ignores others. Often, you have to configure something at the OS level and then configure it again from inside the IDE. Running a build from a command line may or may not give the same result as running a build from within the IDE. The process of getting both sets of configurations right can be time-consuming and tedious, and it’s all too easy to overlook something.

Instability or sluggishness

What I mean by “instability” in this context is closely related to configuration complexity. IDEs sometimes “forget” configuration settings that you’ve made previously. The cause is often mysterious and not worth the time and effort to track down, as we’re trying to get real work done as opposed to tinkering with our tools. Most people make the configuration setting again each time this happens, so they can continue working.

When you run a server from within an IDE, one or both those tools may experience instability. The IDE may lock up periodically, requiring you to kill it. If the IDE doesn’t lock up, it may become confused about the started/stopped state of the server, or the server itself may hang in a way the IDE doesn’t detect or handle well. The socket connection to the server might be closed for some reason, and you receive no notification. You may have to restart the server and/or the IDE several times per day when using it in this mode.

A server that is supposed to auto-deploy modified components may not do so, and you receive no warnings. You may spend several minutes or even a full hour looking for the reason the application doesn’t seem to “see” your modification before you realize what is happening.

It’s worth mentioning that if you’re working on a machine with ample memory, say 32 GB, you probably won’t see these problems as often as you might with a more constrained system. But if you’re working on a “normal” machine, it’s quite likely you will see stability issues.

When I say “sluggishness” I’m thinking of the times when the IDE pauses to perform some under-the-covers maintenance activity, such as “building indexes.” I find this sometimes interrupts my train of thought, as I typically work in very small TDD cycles. Plain old editors don’t exhibit this behavior because they don’t have this degree of “magic” built in.

Other odd behaviors occur from time to time, often appearing in a new release of an IDE and being fixed eventually. One example: A recent article describes an oddity of VisualStudio 2017 whereby source control syncing eats up 75% of the CPU when the IDE is idle. It offers two solutions (or workarounds), one of which is to disable source control syncing and using the command line to interact with your source control system. I don’t know about you, but that’s normally the way I work anyway. Using the IDE for editing and the command line for everything else neatly side-steps any (and maybe all) IDE oddities.

IDE support for version control systems always seemed flakey to me, and often seems to work in a counterintuitive way; not only VisualStudio, but also Eclipse and the JetBrains IDEs. So, is the IDE helping or hindering, even when it isn’t causing problems like this? Is it really necessary to bundle every conceivable function directly into the IDE?

Compatibility issues

Every dependency your tools have is a potential compatibility issue. The more features and functions an IDE supports, the more dependencies it has. Every feature is supported by a library, either baked in or provided as an add-on component. From time to time, you’ll find the IDE will not work with the latest and greatest release of a key library that you need for your application. Eventually, the vendor will add support, but in the interim you have to work around the incompatibility somehow. The time you spend doing so is time lost to productive work.

Too much stuff

Okay, maybe this doesn’t really matter so much to most people. After all, it’s common to find laptops with 32 GB of RAM and 2 Terabytes of persistent storage, so who cares if you have lots and lots of tools installed? But to me it seems like a whole lot of clutter. And all those IDEs have their own conventions and defaults and configuration nuances. Even the JetBrains family of IDEs aren’t entirely consistent in their keyboard mappings.

These are the IDEs and editors currently installed on my laptop:

• Android Studio
• AppCode
• Arduino Studio
• Atom
• C-Lion
• DataGrip
• DrRacket
• Eclipse
• Eiffel Studio
• GoLand
• Nano
• NeoVim
• NetBeans
• IntelliJ IDEA
• Open COBOL IDE
• PhpStorm
• PyCharm
• Qt
• Rider
• RStudio
• Ruby Mine
• Spring Tool Suite
• Sublime Text 3
• VSCode
• WebStorm
• XCode

It’s just too much stuff.

In Search of Simplicity

While I appreciate the convenience of grouping tools together that are used together, I don’t see a big difference between working with a set of panels within an IDE’s window and working with several separate windows. I rarely use a debugger, as I’m one of “those TDD people.” For the same reason, I don’t benefit much from being able to click on a test failure message to jump to the offending source line, as the error is almost always the line of code I just modified, and my editor window is already positioned there anyway.

Autocompletion is a very useful feature of IDEs, but it’s also supported in many “smart” editors. Besides that, autocompletion pop-ups often feel like a distraction rather than a help. My fingers can type the rest of a statement in less time than it takes to get the autocompletion pop-up out of the way so I can see the code I’m working on. Autocompletion is more useful for people who can’t type, I think.

Intellisense is very useful when you aren’t sure what code is possible at a given point in an expression. When using an object-oriented language, it’s great to be able to see which methods are available in context. But once I’ve become familiar with a language, intellisense becomes progressively less valuable.

One of the simpler editor features, syntax highlighting, actually offers more value (to me) than the more-sophisticated features. The color-coding tells me at a glance when I’ve forgotten one of those silly but important things like a closing parenthesis, quote, or curly brace. Many editors support syntax highlighting, and you don’t absolutely need a heavy IDE to get it.

Another basic feature available on configurable editors as well as in IDEs is intelligent indentation. Not the most complicated or “advanced” feature, but one of the most convenient.

In general, developers prefer using keyboard shortcuts whenever they can and avoid excessive use of the mouse. With some IDEs, I find the most efficient way is to combine the two. For example, with JetBrains IntelliJ IDEA, I like to disable editor tabs and use distraction-free mode. Navigating within a file in the editor, starting builds, running individual microtests, and switching to other open file buffers are all faster using the keyboard.

But navigating among the various different panels and views is faster using the mouse. It’s a trade-off. The cost of reaching for the mouse is lower than the cost of the multiple, multi-key “shortcuts” for that kind of navigation. (Maybe those key sequences should be called “longcuts”.) An editor that was designed for keyboard-only use would be slightly better, in my opinion.

One feature of IDEs that I don’t see at a mature level in smart editors is support for refactoring. VSCode strikes me as midway between a smart editor and an IDE, and as of recently you can add Roslyn refactoring support via Omnisharp (but I have not tried it myself). I know of packages for Sublime Text to support refactoring in JavaScript and Python. Atom offers packages that supposedly support refactoring in JavaScript, CoffeeScript, and PHP. I don’t know how well those work, but I don’t have high expectations of them. Even the best-functioning refactoring support in sophisticated IDEs often doesn’t do what you really want, and you have to adjust the code manually. Project-wide rename is probably the most useful built-in refactoring in most tools.

None of this rises to the level of refactoring support that we find in VisualStudio/Resharper, IntelliJ IDEA and relatives, or Eclipse. People have independently built plug-ins for the editors they use to support the languages they use. There’s a nice emacs-refactor project on Github that supports a few basic refactorings in several languages for Emacs. You can also invest some sweat equity into building crude but functional refactoring tools for customizable editors such as Vim; but: V = W / H?

Less is More

What’s the minimum setup you’d need to do serious software development?

A bare-bones Linux distro will have C/C++ support, make, and nano (a minimal editor). With those tools you can recompile the kernel, build packages from source, and edit configuration files. Normally, once you’ve bootstrapped the system beyond a basic level of functionality, you’d install additional development tools.

A typical consumer-oriented Linux distro will have more development tools than those. It’s likely to have Python, Ruby, and Java installed to support some of the packages that are bundled with the distro. It may or may not have a Java SDK, but it probably has at least a JRE. Different distros come with various text editors that are easier to work with than nano, too, such as vi, Gedit, Leafpad, or Geany.

So, in principle you have a usable development environment the moment you finish installing Linux, and you can work with well under 1 GB total RAM.

A Little Less Less, Please

But that might be taking things a little too far. Sure, you’ve avoided the bloat, complexity, instability, and compatibility issues of a fat IDE, but at what cost? What’s the value of H in the formula, V = W/H?LI began with the assumption that a configurable text editor, as opposed to a feature-rich IDE, could provide enough functionality to be practical and usable for most developers. A handful of candidates exist in that category, most notably Vim and Emacs, and more recently VSCode.

Atom is nice to use but has issues because of Electron.js. Sublime Text is very usable and I’ve used it extensively for “real” development work; its package system allows for adding on useful support for syntax highlighting, snippets, and more.

There are also several pretty decent lightweight text editors that don’t pretend to be IDE-killers but that are sufficient for a developer who is already familiar with the programming language they’re using, like Geany, Leafpad, and Notepad++.

I’ve been satisfied using Sublime Text for shell scripts, various text formats and markup languages, and even for serious development in Ruby, Python, and JavaScript. I think Atom, Geany, and others would be fine, too. But when you need to work on a large Java or C# project, you start to hit a wall with tools like those. There are just too many “moving parts,” like project metadata, packages, and classpaths. Too much of “less” makes for a miserable day. Sometimes you need a little less less and a little more more.

There are those who insist they can’t get along without all the bells and whistles of a full-featured IDE. At the opposite end of the sprectrum are minimal editors like nano and vi. You already use them to edit configuration files when you set up new Unix/Linux instances. You can edit program source files with them, too. In a pinch, you could do all your development work that way; ultimately, source code is text, after all. The question is neither (a) “How much feature bloat can we stuff into our favorite IDE?” nor (b) “How close to the bone can we work to prove we’re ninjas?” but rather, “What tools make sense for our needs in our context?”

Vim and Emacs are very mature and people have created plugins to support just about any type of development work. VSCode is maturing rapidly and has the potential to be another member of that club. I know people who work at Microsoft and used VisualStudio for 20+ years who have switched to VSCode and have no intention of opening VisualStudio ever again. These editors actually do have the potential to serve as replacements for fat IDEs, unless you really need some of the more sophisticated features of IDEs in your work.

It’s interesting that the JetBrains IDEs allow you to disable editor tabs, operate in “distraction-free mode,” and switch between open buffers via keystrokes; just as Vim and Emacs do natively. Are editors trying to become IDEs, or are IDEs trying to become editors? Does it matter?

A Lightweight Yet Functional Environment

What do we need to bring the pendulum to the center, between bloaty IDEs and bare-nekkid text editors? Not much, I think:

• Small Linux distro
• Lightweight window manager
• Configurable editor
• Compilers, etc.

To prove (or disprove) the idea, I tried out several lightweight, small-footprint Linux distros using VMware Fusion on a MacBook Air. I decided to go with Neovim, a modern implementation of Vim, as the editor. That’s not a comment on Emacs or VSCode; just want to minimize the number of variables.

Goals

I had a couple of goals for this experiment:

• Configure a usable and practical development environment compact enough to run on a Raspberry Pi (or similar) with 1 GB of memory.
• Replace the constellation of language-specific and platform-specific IDEs with a single development tool that could support all mainstream programming languages, for some definition of all, some definition of mainstream, and some definition of support.

Why all the caveats? Well, not every developer cares about support for “all” languages and “all” platforms. That goal is probably overkill for most people. But if it can be done, it would demonstrate that a minimalistic development environment is indeed possible for a subset of languages and platforms, and that could be quite useful.

The bit about “some definition of support” alludes to the fact that if we switch from an IDE to a text editor, we’re going to give up at least some functionality around intellisense, assisted refactoring, and integrated debugging and profiling. Syntax highlighting, some degree of autocompletion, and language-aware automatic indentation are definitely possible with a good text editor. Those of us who rigorously use test-driven development rarely need a debugger, and with the trend toward building applications out of very small services the real challenge in debugging occurs when we’re knitting the threads together.

Why do I call it an “experiment?” The purpose of an experiment is to learn something by trying to disprove a hypothesis. My hypothesis is that it’s possible to configure a very lightweight development environment that provides all the tooling necessary for “real-world” development work, as an alternative to working with multiple IDEs, each of which eats up huge quantities of RAM and other system resources. I aim to disprove that.

The “control” for the experiment is the wonderful world of fat IDEs that we already live in. I have more than 20 IDEs installed on the machine where I’m typing this. Is that really necessary or helpful?

Approach

Initially, my approach was to start with a “tiny” Linux distro and build up from there by adding packages needed for software development. That promised to be easier than starting from source, and still offered the possibility of a compact OS. I tried several of these before changing the approach:

• TinyCore CorePlus
• TinyCore Core
• Vector Linux Light
• Puppy Linux Xenialpup
• Arch Linux
• Damn Small Linux

Each one proved unsuitable to the purpose for one reason or another. Granted, you could get most of them to support some of the work I wanted this environment to support, but not all. I made extensive notes on all the difficulties I experienced in trying to configure these distros, but all that detail is irrelevant to the experiment at this point.

Many of the “tiny” Linux distros don’t even install properly. In general they aren’t well packaged, well documented, or well maintained. Arch Linux was the best of that bunch. It has particularly good documentation, an active user community, and good support. Unfortunately, because of poor mirror management the installation always hung at some point during the base package group load. I never got past that first step in configuring the instance. I got frustrated with that and moved on to other candidates.

The opposite approach is to start with a “desktop” distro and remove unnecessary packages to trim it down. That strategy proved harder than it sounds. I was still looking for a small footprint, so I didn’t go with Ubuntu, Mint, or Absolute. Those distros come with so many programs pre-installed that it would be a challenge to remove packages without breaking something important. That’s understandable, as their purpose is to support a general computer user out of the box.

Summary of what didn’t work

I tried these small-footprint Linux distros:

• Lubuntu
• LXLE
• Raspbian Desktop

The closest I got to a working environment from those options was Lubuntu. The problem there was support for C/C++ and CppUnit with automake. If you know you’ll never work with those tools, the Lubuntu configuration could be good enough. There are other ways to build C/C++ applications; and not everyone needs to work on C/C++ code. I was being picky. I suspect the automake issues with Lubuntu are temporary and will go away in a future release. But this was not the time for me to work through OS issues. I spent several hours on it, and called it a day.

Debian was the last “desktop” distro I intended to try before moving to a strategy of building from Gentoo or just from source. On the site I noticed there is an ARM build, which means this could work on a Raspberry Pi if it proves out in the VMware test environment. Raspbian is a Debian variant. It would be fun to set up a really cheap but full-featured development environment with a small footprint in the physical world as well as the virtual.

Coming in Part 2: Configuring Debian and Vim

In the next instalment, I’ll step through the process of installing and configuring a Debian instance with a lightweight window manager and Neovim, a modern implementation of Vim. We’ll add support for several programming languages and see if we can disprove the hypthesis that we can set up a lightweight yet robust and usable development environment in which a single editor supports all our development languages.