The Software Scrutiny Journal

I was laying in my hammock on one sunny day and began to think about text files. If you’re someone that knows me, you are probably well aware that I am a hardcore text file user and advocator. Anyways, I became bored and Googled “Hacker News text files” to feed myself with some confirmation bias. While reading the comments I realized that I either practice what is preached, or have refined it.

Fox example, one strategy is the format of todos:

2020-05-25T13:28:09-05:00
- [X] Check Basecamp
- [X] Cut lawn
- [ ] Finish revising latest blog

Another strategy is how my calendar evolved.

Another could be how my files are structured. Vimwiki uses the “diary” subdirectory and has files named yyyy-mm-dd.markdown. I also have directories of companies that I’ve consulted for and some for my real estate.

However, there was one common theme among some replies that I had not yet considered: Move all of your data into one, enormous file.

I currently use vimwiki, but have since used less and less features over time. Initially, I followed the vimwiki mardown convention, and even exported the files to an html format for web publishing. Now, everything is just plain text which uses my homegrown formatting conventions. The only features of vimwiki that I use nowadays are:

• the syntax highlighting (for urls and such)
• the keybinding to open the vimwiki file (<leader>ww)

Ditching the vimwiki plugin seems to be pretty do-able at this point.

Navigating through one enourmous file does not scare me. Being a whiz at Vim, I can pinpoint my location in a second or two. One thing that I was concerned about was performance and/or memory use (which I now see as comical).

To verify that this migration would be okay, I moved everything into one file. This was accomplished with using cat and filepath wildcards. Since I am using git, this is easy to revert if things don’t pan out.

This file ended up being about 30,000 lines long. When running ls -lah, I found that the file totals 1.4M. Not bad at all.

Then I opened the file and ran pmap for the neovim process:

$ pmap <pid_for_nvim>
...
total         99388K

I also wanted to compare this with two tabs of Firefox.

$ pmap <ff> <ff1> <ff2> <ff3> <ff4>
total          2379420K

I found it interesting that Firefox uses 5 processes total for two tabs. Anyways, it is obvious that this is an apples to oranges comparison. Firefox by itself is basically its own operating system. My primary concern was that my computer would behave with reading and writing to this massive file. Fortunately the vim process consumes 23x less memory than a firefox instance of two tabs. It’s nice that I don’t have to ever worry about the size of my file.

As for productivity levels, I would say that I am probably about the same. If anything, there may have been some marginal improvements.

• Combining all diary files allows for easier weekly review.
• Instead of fuzzy file searching via the_silver_searcher, I can now use Vim searching. This feels quicker (and is one less keystroke).
• I can move the most accessed data to the top of the file, so that I can easily get to it. Since each topic was in its own file, I had to separately navigate to each.
• And of course, there is a feeling of bliss having everything tucked into a single file. No need to manage directories and subdirectories of data.