Multiarch

What it is, what it does, why it matters

Multiarch is a proper general solution to the problem of installing
libraries of more than one architecture at a time on a system, i.e a
generalisation of lib32/lib64 which deals with one common example of
the problem. 

The genesis, and much of the incentive, came from ia32-libs, which
became the biggest package in Debian in 2005.

This was always intended as a temporary solution until something
proper came along. That took more than 6 years!

Fundamentally it is a very simple thing:
Put libraries into architecture-specific paths
  /usr/lib/x86_64-linux-gnu
  /usr/lib/arm-linux-gnueabi
  /usr/lib/i386-linux-gnu

The really important way to think about this is that binaries have one
canonical path, which allow the same binary to be installed in a
native or non-native context. It also makes cross-compiling enormously
simpler - the paths don't change between build- and runtime. 

It doesn't matter if you are installing foreign binaries to run on the
native hardware (i386/amd64), to run on qemu, or to be an emulated
foreign OS (Linux on BSD, solaris on linux). 

Things multiarch does:
Cheap emulated environments - emulate only the parts you need to
Cross-compilation is no longer special - you get it for free!
Support for cross-grading a system from one architecture to another
(arm→ armel, i386→ amd64, armel→ armhf)
Better support for binary-only software

Things multiarch doesn't do:
Install more than one arch of binaries/tools in /bin
Anything to do with ABI-compatible capabilities (SSE/NEON/ALTIVEC/MMX)

Things multiarch allows:
Partial architectures
Cross-dependencies (e.g for cross-compilers)
Cross-built architectures


Things we learned along the way about making large infrastructural
change:
Use written specs to record shared understanding
Split your work into bite-sized deliverables
Make it clear how people can help: http://wiki.debian.org/Multiarch/Implementation



Timeline:
2004 BOF at Debconf 4 
2005 Talk at Debconf 5
2006 FOSDEM multiarch meeting 
2008.06 Dpkg multiarch patches uploaded 
2009.05 apt and dpkg maintainers agree on a package management spec at UDS in Barcelona
2010.08 Tuple proposal for ABI names drafted
2011.02 dpkg multiarch implementation (sponsored by Linaro) lands in Ubuntu. Tuple spec proposed to LSB
2011.03 new directory names scrapped, but finalized (and normalized) GNU triplets adopted
2011.04: Ubuntu 11.04 released with 83 libraries multiarched, +14 in a ppa: enough to cross-install flash plugin
Feb 2012: All of ubuntu main, many base packages in Ubuntu and Debian Wheezy 

Current state:

133 source, 261 binary packages in precise 
110 out of 112 (source), 175 out of 176 libs
192 source, 389 binary packages in wheezy out of 628 (source) 1120 (binary) libs
grep-apt-avail
425 source out of 2398 source libs, precise 
479 source out of 7273 source packages, precise
360 source out of 2162 source libs, wheezy
401 source out of 7906 source packages, wheezy

There is no flag-day for multiarch, packages are converted and
uploaded one by one until all the ones anyone cares about are done.

(done with: 
grep-aptavail -FMulti-Arch: -s Source --regex "allowed\|same\|foreign" | sort | uniq | wc -l
grep-aptavail -Fsection:: -s Source  --regex ".*" | sort | uniq | wc -l
grep-aptavail -Fsection:: -s Source  libs | sort | uniq | wc -l


How it works
GNU triplets are used for the arch-qualifiers 
This leaves some historical cruft: (i386/486/586/686 special cased)

Multi-arch-ready packages are given an extra field
Multiarch: same, foreign or allowed

same means it can be co-installed and can only satisfy deps within the
arch
foreign means can not be co-installed can satisfy deps for any arch
allowed means can be either. packages depending on it specify which case

use dpkg-architecture -qDEB HOST MULTIARCH to query the name of the
subdirectory for libraries

and equivalent lsb mechanism is needed for non-debian distros. 

Changes needed in distro:
Lots of packages (all libraries, most -dev packages)
dpkg and apt
Compilers need system path to include multiarch library and header paths
Other tools that need to know about new library paths:
libc (loader)
make (foo: -lbar syntax)
pkg-config
pmake
debhelper
lintian
libffi
cmake
openjdk (lib-jna)
dpkg-cross


How it's used:
dpkg --add-architecture i386
or
echo "foreign-architecture i386" >> /etc/dpkg/dpkg.cfg.d/multiarch 
apt-get update
apt-get install libattr1-dev:i386

deb-src lines get an arch field:
deb [arch=amd64,i386] http://archive.ubuntu.com/ubuntu precise main universe
deb [arch=armel] http://ports.ubuntu.com/ precise main
deb-src http://ports.ubuntu.com/ precise main

dpkg --get-selections gives:
libattr1               install
libattr1:armel         install
libattr1-dev           install
libattr1-dev:armel     install

APT::Architecture Arch to use when fetching and parsing package lists
APT::Architectures All suported arches (native + foreign)


Cross building requires cross-dependencies for build-deps. Generally
required packages are either libs or tools. Some can be both/either.
We could annotate every dep, but in fact the multiarch info generally
provides the info we need. Libs are M-A: same, tools are M-A: foreign
The dependency is a feature of the depending package. 
The M-A info is a feature of the depended-upon package.
We only have to annotate the cases that are opposite what is expected.

We also have to deal with inlcude files which differ between arches.
by putting them in /usr/include/<triplet>/

And dev packages need converting in order to be able to install both
native and foreign versions. (not strictly required, but convenient).
That means moving out all binaries. 

Cross-deps: Running apt-get -aarmel build-dep acl
Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
  autoconf automake autotools-dev bsdmainutils debhelper gcc-4.6-base:armel
  gettext gettext-base groff-base html2text intltool-debian libattr1:armel
  libattr1-dev:armel libc6:armel libc6-dev:armel libcroco3 libgcc1:armel
  libgettextpo0 libpipeline1 libtool libunistring0 libxml2
  linux-libc-dev:armel m4 man-db po-debconf
  
Implementation details

Lots of info at:
http://wiki.debian.org/Multiarch
Spec: https://wiki.ubuntu.com/MultiarchSpec
Implemntation advice: http://wiki.debian.org/Multiarch/Implementation
