Elixir collections

Elixir is a function programming language that I have been using a lot in recent months to build all kinds of applications. Understanding of built-in collection types is essential to use any language effectively and Elixir is no different.

This posts summarizes all collection type along with pros/cons/gotchas for each one of them.

Collection Example When
Tuples {:ok, "All good"} Returning data from a function
Lists [1, "two", :three] For a collection of items
Keyword lists [one: 1, two: 2] Passing options to a function
Maps %{one: 1, two: 2} Flexible key/value store
Structs %User{name: "John", age: 32} Typed/fixed key/value store

Tuples

  • {:ok, foo} “tagged” tuple since begins with an atom like :ok or :error like {:error, 543, "some error"}

Examples:

[Read More]
elixir 

Subtle Errors in C++ Programs

I recently stumbled upon a subtle bug in a benchmark code which again reminds me to never use C++ again, if I can.

Here’s a buggy snippet from this code (simplified):

// BUGGY
ostringstream os;
int i = 1;
os << "foo-" << i << ".dat";
const char *filename = os.str().c_str();
int fd = open(filename, O_RDONLY);

You may expect above code to try open a file named foo-1.dat but that’s not what is happening here.

In this snippet, os.str() create a temporary string object which is destroyed immediately after call to c_str() method. So, filename ends up pointing to freed memory which can of course contain arbitrary content (till you reach a NULL).

[Read More]

Setting Up Backup Snapshots on Linux

For some time I’ve been looking for a backup solution for Linux that can periodically take snapshots of data, allowing me to go back in history of any file just like git. I finally found restic which fits these requirements. Here is how I set it up to take snapshots of particular directories, say every 15 minutes.

Installing restic

Though restic is available in repositories of almost all Linux distros, I recommend downloading the latest release directly from GitHub to avoid dealing with potentially outdated version. Helpfully, you can stay current with restic release with restic self-update.

[Read More]

Google Drive on Linux

There is no official Google drive client for Linux. I tried many different clients found all over GitHub but none of them worked reliably for me except rclone. I also tried third-party proprietary clients like Insync but allowing read-write access to all your Google drive files to a closed source blob is too much to swallow.

Once caveat with rclone is that it does not natively support bi-directional sync (github issue) but someone developed a python script rclonesync-V2 which is a wrapper around rclone which does the job. With these two pieces of software we can get close-to-official Google drive client experience.

[Read More]

Faster compilation with distcc

Often, you have more than one system at your disposal but no clear way of distributing your compilation workloads over to all or some of them. They might be running different OSes which makes it look even more difficult. In my case, I have one laptop (2 cores) and a desktop (4 cores) connected with a WiFi network. The laptop runs Linux (Fedora 13 64-bit) while the desktop runs Windows 7 (64-bit). I wanted to somehow offload Linux kernel compilation over to my powerful desktop and keep my laptop cool :)

[Read More]

Compressed RAM disk for Windows, The Virtual Way!

Recently, I developed Linux kernel driver which creates generic RAM based compressed block devices (called zram). Being RAM disks, they do not provide persistent storage but there are many use cases where persistence is not required: /tmp, various caches under /var, swap disks etc. These cases can benefit greatly from high speed RAM disks along with savings which compression brings!

However, all this seems to be completely Linux centric. But with virtualization, zram can be used for Windows too! The trick is a expose zram as a ‘raw disk’ to Windows running inside a Virtual Machine (VM). I will be using VirtualBox as example but exposing raw disks should be supported by other Virtualization solutions like VMware, KVM too.

[Read More]

Comprehensive graphical Git diff viewer

Comprehensive graphical Git diff viewer

Since a long time, I was looking for a graphical git diff viewer which could show original and modified file side-by-side and highlight the changes. There are few solutions but none of them is sufficient:

  • A tool included with git called ‘git-difftool’ is partially helpful – it can show changes graphically but diff for each file is shown one-by-one. This is very irritating. In fact, unusable even with just 10-15 files.
  • Another alternative is the meld diff viewer which is “git aware”. The problem here is that it can show diff for uncommitted changes only which is very limiting. What if you want to see what changes between Linux kernel, say 2.6.33-rc1 and 2.6.33-rc2? or changes between last two commits? meld cannot do it, AFAIK.
  • Finally, with kompare, you can do something like: ‘git diff master | kompare -o -’. This method however, does not show original and new files side-by-side. It is simply prettier diff highlighting.

None of above methods are sufficient. So, I wrote the following script which solves our problem: show complete contents of original and new files side-by-side and highlight the differences.

[Read More]
linux  git 

Linux kernel workflow with Git

You worked on some part of Linux kernel. It works great. Now, how to generate the patch series and send it out for review? For this, I always used to generate diffs, create a set of draft mails (one for each patch) in KMail or Thunderbird, and send all these mails one-by-one. This workflow quickly became a big headache. Then I learned Git (and some related tools) to do all this from command line and wow! what a relief!

[Read More]
linux  git 

ccache to speed-up Linux kernel compile

In case you are unfamiliar with ccache, its a “compiler cache”. Compiling is primarily CPU intensive task. So, ccache caches compiled objects - so next time we compile same code, it reuses these objects thereby significantly speeding-up compilation.

I need to recompile Linux kernel usually several times a day, with different permutations of config settings. This almost forces a ‘make clean’ or ‘make mrproper’ which deletes all compiled objects in build tree and then we have to rebuild everything all over again. This takes enormous amount of time. ccache comes to rescue! I’m surprised why I didn’t use it earlier.

[Read More]
linux 

SLOB memory allocator

Linux kernel has few SLAB allocator variants included: SLAB, SLUB and SLOB. Of these, SLOB is especially meant to be used on embedded devices – it tries to be more memory space efficient than other SLAB variants.

Yesterday, I had a detailed look at SLOB allocator for possible use in compcache poject and found it unacceptable for the purpose. I did it in response to feedback on xvmalloc allocator – as part of compcache patches posted of inclusion in mainline Linux kernel: http://lkml.org/lkml/2009/3/17/116

[Read More]
linux