The Apt-Get in debian systems use their own downloading system to download packages from the internet. It supports file resuming and pipelining. However, it doesn’t support downloading using multiple threads, and it won’t allow you to external applications to download.

However, the http downloader is a separate application(/usr/lib/apt/methods/http) from apt-get, and the apt-get spawns the executable to download the files. Those two programs communicate via stdin and stdout. I have written a new downloading application which can communicate with the apt-get. So, I just copied the executable to /usr/lib/apt/methods/http. The new application spawns another program(In my case a shell script) with two arguments; the url and the local file.

So, after all my shell script calls a python script. My program doesn’t call the python script directly because when the python scripts gives some output to stdout/stderr the apt-get program crashes. So, my shell script redirects the all output to /dev/null. Then, the python script will get the url, do a HEAD request, get the file size. If the file size is less than 100K, it’ll execute wget, if not it’ll execute axel. So, the files will get downloaded and apt-get will install the software.

The apt-get update command fetches many small files from the same server. Since the original downloader uses pipelining, it’s much faster. I have to modify my code to do something about pipelining.

Files :

scripts.tar.gz

apt.tar.gz

This entry was posted on Wednesday, February 14th, 2007 at 1:25 am and is filed under Hacks, Linux, Open Source, Programming, Python. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.