Wednesday, July 22, 2015

Building Hardware optimized ATLAS on Ubuntu 14.04

The Automatically Tuned Linear Algebra Software (ATLAS) is used by many scientific software packages such as Mathworks Matlab, Julia, Kaldi (speech recognition), and many more. ATLAS can be optimized to the current hardware with a few simple steps, giving much higher performance than the default available packages.

This guide helps you create and install an optimized Debian package on Ubuntu 14.04, but it might work similarly on other GNU Linux distributions. One problem with Ubuntu 14.04 on Intel processors is that it uses a new kernel module called intel_pstate for managing the CPU frequency. This driver will lower the CPU frequency for the idle CPUs (throttling) even if the profile is set to "performance". Unfortunately ATLAS will not build if the CPU frequency is not fixed to the maximum allowed. The workaround is to disable this module in the boot loader (GRUB) temporarily for the build. Here are the steps to follow:

1. Configure the boot loader to disable the intel_pstate module:
• edit /etc/default/grub and add the intel_pstate=disable option to the GRUB_CMDLINE_LINUX_DEFAULT variable. In my case the line before and after the change looked like this:
before:
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash nomdmonddf nomdmonisw"
after:
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash nomdmonddf nomdmonisw intel_pstate=disable"
• make the change active by running:
sudo grub-mkconfig -o /boot/grub/grub.cfg
• reboot
2. run the following commands:
sudo apt-get source atlassudo apt-get build-dep atlassudo apt-get install devscripts gfortran f2c libcnf-devcd atlas-*/sudo fakeroot debian/rules custom
3. the last step (that can take a long time to finish) will build ATLAS and create a Debian package file one level up. Install the package with:
sudo dpkg -i ../libatlas3-base_*.deb
4. restore the /etc/default/grub file to the original, run the sudo grub-mkconfig -o /boot/grub/grub.cfg command and reboot.
This procedure was tested with ATLAS 3.10.1. Hope this works for you as well. Let me know how it goes in the comments.

NOTES:

1. the building process took between 6 and 7 hours to complete on an Intel(R) Core(TM) i7-4930K CPU @ 3.40GHz.
2. Although there were 12 such cores, setting the environmental variable DEB_BUILD_OPTIONS=parallel=12 has no effect on the building process or speed
3. there are 6 deb files generated by the building process:
libatlas3-base_3.10.1-4+custom13_amd64.deblibatlas3gf-base_3.10.1-4+custom13_all.deblibatlas-base-dev_3.10.1-4+custom13_amd64.deblibatlas-dev_3.10.1-4+custom13_all.deblibatlas-doc_3.10.1-4+custom13_all.deblibatlas-test_3.10.1-4+custom13_amd64.deb
4. when installing the libatlas-doc package I got the following errors:
Processing triggers for doc-base (0.10.5) …Processing 3 added doc-base files…Error in /usr/share/doc-base/atlas-install', line 14: all Format' sections are invalid.Error in /usr/share/doc-base/atlas-devel', line 10: all Format' sections are invalid.Error in /usr/share/doc-base/atlas-contrib', line 14: all Format' sections are invalid.Note: install-docs --verbose --check file_name' may give more details about the above errors.`