Planet Exherbo

Ciaran McCreesh: Paludis 0.44.2 Released

Sun, 07 Mar 2010 13:18:21 +0000

Paludis 0.44.2 has been released:

Various documentation updates.
When disambiguating a spec like foo::bar, where cat-1/pkg::foo and cat-2/pkg::bar exist, we now recognise that you must want cat-2.
A bug causing NoConfig clients to fail to find profiles when used with a supplemental repository that does not provide its own profiles description has been fixed.

Filed under: paludis releases Tagged: paludis

Bryan Østergaard: Exherbo at Open Source Days

Thu, 04 Mar 2010 00:29:58 +0000

The danish open source conference, Open Source Days, will run this friday and saturday. Exherbo will be present both days with quite a few developers in attendance.

I'm sure there will be a (little) hacking on Exherbo and related projects like Genesis but it's also a very good, if somewhat rare, opportunity to meet some of the leading Exherbo developers and talk to them about the current status of Exherbo and what's in the future.

Personally I'm hoping to have some good discussions about the kind of problems people currently face when they use Linux in various business settings and ways that those problems might be solved. I'm also hoping that you can learn something from the way the Exherbo project is managed and get an idea how the project manages to move at such a fast pace.

And finally I'm looking forward to meeting lots of people and having a good time :)

Bryan Østergaard: On Genesis commits

Fri, 26 Feb 2010 15:42:11 +0000

I've been asked a few times by different people whether they should push simple patches to genesis. The answer is a resounding 'Yes, please go ahead' but it seems like stating my policy towards other peoples contributions might not go entirely amiss.

My current policy (which will stay until genesis starts stabilising a lot more) is:

Any contribution is welcome so you just need to find somebody with push access (the usual Exherbo devs will do)
Large patches / feature contributions are also extremely welcome but you might want to contact me first and make sure I won't undo you hard work.

I'm not in any case going to complain about contributions (large or small) so don't hold back. And remember that I have a pressing need for people to play around with genesis scripts and tell me what you need from genesis. git://git.exherbo.org/genesis-scripts.git was created yesterday for this purpose.

And for those of you who don't have direct push access you can just cue your patches using the hacchi patch bot in #exherbo on irc://chat.freenode.net and I'm sure one of the friendly exherbo developers will help you :)

Bryan Østergaard: Genesis just got internal events

Thu, 25 Feb 2010 14:41:11 +0000

One of the big problems with Genesis was that you'd get its coldplugging events in that the events wasn't necessarily delivered in a sane order. I briefly considered different ways of controlling the order of the events but quickly came to the conclusion that was madness.

The solution I decided upon instead is generating an internal event after coldplugging is finished and send that event to all event modules. This way we can simply trigger on the 'genesis-started' event and start mounting filesystems and whatever else is needed for bootup.

So how can you help? What I need right now from you is just playing around with some homegrown scripts, trying to catch some events and telling me about all the things that are absolutely impossible to do without. Testing Genesis is quite simple and can easily be done without interfering with the rest of your system.

How to test Genesis:
1. Clone the git://git.exherbo.org/genesis.git repository
2. Run ./autogen.sh in the genesis repository
3. Run ./configure
4. Run make

5. Run sudo make install (by default it installs to /usr/local and won't conflict with anything)

6. Write a /usr/local/etc/genesis/config file

My current config file looks like:


[genesis]
# Currently supported log destinations are file and console
logging = console
logfile = /var/log/genesis.log

[modules]
command = yes
netlink-uevent = yes
netlink-route = yes

[netlink-uevent]
coldplug = yes
coldplug_mounts_sysfs = no
log_matched_events = yes
log_unmatched_events = yes

Ignore all the options except the coldplug_* options. The other options might be necessary due to the rather haphazard way configuration is implemented currently but don't expect them to work right now. The coldplug option enables/disables coldplug events and the coldplyg_mounts_sysfs option controls whether the coldplugging part tries to mount and umount /sys. You want to keep this option disabled unless you're doing actual boot testing.

Stick one or more bash scripts in /usr/local/etc/genesis/netlink-uevent/ or /usr/local/etc/genesis/netlink-route/ and try running: sudo /usr/local/sbin/genesis

The test scripts I'm using currently looks like (from /usr/local/etc/genesis/netlink-uevent/foo.sh):

SUBSCRIPTIONS_add=vcs10

add()
{
echo "netlink-uevent::add" >> /var/tmp/genesis.log
}

You can add any number of event subscriptions you like in a single script or spread subscriptions over several scripts. Each subscription is defined as SUBSCRIPTION_function=trigger

Trigger is simply a regex that matches the metadata from events in an internally serialized string format. Function denotes the function in the script called when matching that event. So all the above script does is triggering the add() function on all events matching the string 'vcs10'. Since I have coldplugging enabled I'm guaranteed to see that event.

And that's all there is to it.. One of the things high on my TODO list is to replace the regex matching with a udev like language where I can match on subsystem, MAC address or whatever other metadata events generate. I'd very much appreciate comments on what you think is needed in a simple language like that.

Bryan Østergaard: Remember Genesis? The oft mentioned new init system for Exherbo?

Wed, 24 Feb 2010 12:46:53 +0000

If not you might want to read this old exherbo-dev post to refresh your mind.

Genesis started out as me being rather frustrated with current init systems and their huge failures. My basic idea was that all the different init systems fail badly at solving any real problems for system administrators and that it was about time fixing that situation.

So as I started really thinking about the problems that init systems should solve I decided that writing a new init system from scratch was the approach..

A short while later I discussed most of my ideas in a talk at FOSS Aalborg and had a fair bit of positive feedback. It's worth noting that I haven't started writing any code at this point and consequently my problems hadn't started yet.

When I did start writing code I quickly got even more ideas for Genesis and it became an even more ambitious project. And then it changed radically.. again and again.

Due to all the changes and lack of direction I didn't want to publish the code and it was often compared to Duke Nukem Forever (for good reasons I might add). But now that I finally seem to be reasonably sure about the direction I've finally released the code for everybody to peruse and contribute to.

The current status of the code is quite messy and it's not yet usable as an init system at all. You can glean the basic ideas of it however and there's a TODO list that you're most welcome to take a swing at. A few people have already contributed some minor patches for the build system for which I am grateful. I'd get around to fixing those things myself but for now I'd rather focus more on Genesis design/architecture and make sure that development progresses quickly.

As for progress I'm trying to make sure I do at least one Genesis commit every day and tracking that on http://calendaraboutnothing.com/~kloeri. Some days might only see very small commits but I'm shooting for larger, daily updates.

My primary goals right now is getting Genesis to a state where it can boot a system as well as making the code more hacker friendly so more people can contribute.

As for booting I expect to set up a separate git repository for genesis scripts as soon as Genesis is ready for that. It's my hope that most scripts will be contributed by other Exherbo contributors and that I can keep my focus on Genesis itself.

Ali Polatel: Batch tagging of audio files from the command line

Wed, 27 Jan 2010 08:00:00 +0000

As many of you know MusicDNS is an acoustic fingerprinting service and a software development kit provided by MusicIP. The fingerprinting client library that looks up and identifies audio files based on existing fingerprints is called libofa. MusicBrainz has a great audio tagger called Picard which can tag audio files by querying this MusicDNS service.

There is, however, a simple problem. Picard is a GUI and thus doesn’t allow batch tagging of audio files from command line.

Hence I decided to write my own tool for generating acoustic fingerprints and for querying MusicDNS service. I’ve chosen to use libsndfile to do the decoding as libofa expects raw audio data. libsndfile is a C library for reading and writing files containing sampled sound through one standard library interface. It’s pretty easy to use and its API hides most of the low-level details from the programmer.

The tool is named afprint, released under GPLv2. Following the UNIX philosophy it just does one thing, calculation of acoustic fingerprint and duration of the given audio file.

Usage is simple:

alip@harikalardiyari> afprint -h
afprint-0.1.0-7b17577 audio fingerprinting tool
Usage: afprint [-hVv0] <infile>

Options:
    -h, --help      Display usage and exit
    -V, --version   Display version and exit
    -v, --verbose   Be verbose
    -0, --print0    Delimit path and fingerprint by null character instead of space
If <infile> is '-' afprint reads from standard input.
alip@harikalardiyari> afprint -v sample.ogg
[dump_print.294] Format: OGG (OGG Container format)
[dump_print.295] Frames: 2188368
[dump_print.296] Channels: 1
[dump_print.297] Samplerate: 44100Hz
[dump_print.298] Duration: 49735ms
[dump_print.302] essential frames: 5953500 > frames: 2188368, adjusting
sample.ogg 49735 ARaJDAgL...

afprint decodes the audio data using libsndfile and feeds it to libofa. It also calculates the duration of the audio file and prints them in format: FILENAME DURATION FINGERPRINT

Reading from standard input is tricky because pipes aren’t seekable thus it’s not possible to calculate the duration of the audio file. For this reason, when the audio data is fed via standard input, when <infile> is -, afprint saves this data into a temporary file and reads from it. This makes it possible to calculate acoustic fingerprints of Mp3 files, which libsndfile doesn’t support, easily.

alip@harikalardiyari> mpg123 -q --au - 01_san_francisco.mp3|afprint -v -
[wav.c:388] warning: Cannot rewind AU file. File-format isn't fully conform now.
[wav.c:388] warning: Cannot rewind AU file. File-format isn't fully conform now.
[dump_print.294] Format: AU (Sun/NeXT)
[dump_print.295] Frames: 8000111
[dump_print.296] Channels: 2
[dump_print.297] Samplerate: 44100Hz
[dump_print.298] Duration: 181820ms
/dev/stdin.au 181820 AQMZN...

Note the --au option passed to mpg123 as --wav doesn’t work.

So far so good, now we need a tool to query the MusicDNS server to find out the PUID of the audio file and query MusicBrainz to get the audio tags.

I’ve written a simple Perl script to do the job. The script, which has the name puidlookup, reads audio fingerprints from standard input and queries the MusicDNS server. Optionally it can query MusicBrainz as well to receive the tags.

Here are the requirements:

Usage is simple, just pipe afprint’s output to puidlookup.

alip@harikalardiyari> puidlookup -h
Usage: puidlookup [-hVv0]
    -h, --help          Display usage and exit
    -V, --version       Display version and exit
    -v, --verbose       Be verbose
    -0, --null          Expect input is null delimited
    -m, --musicbrainz   Look up PUIDs from MusicBrainz
                        (requires WebService-MusicBrainz)
    -l, --limit         Limit results to the given number
puidlookup reads filename, duration and audio fingerprint from standard input

The --null option responds to afprint’s --print0 option. These options are useful if filenames have spaces or other weird characters in it.

By default it only queries MusicDNS:

alip@harikalardiyari> afprint 04sheep.ogg | puidlookup
ARTIST='Pink Floyd'
TITLE='Sheep'
PUID=930806c1-e1e0-588a-b7de-2dacb1b8b11e

The --musicbrainz option can be used to query MusicBrainz:

alip@harikalardiyari> afprint 04sheep.ogg | puidlookup --musicbrainz
PUID=930806c1-e1e0-588a-b7de-2dacb1b8b11e
TRACKID=431a85dd-e22b-4626-91c9-c0abb8058d3f
ARTISTID=83d91898-7763-47d7-b03b-b92132375c47
ARTIST='Pink Floyd'
TITLE='Sheep'
TRACK=4
ALBUM='Animals'

The output is quoted so it’s safe to pass to eval, making it easy to integrate with shell scripts.

Last step is writing a tagger script to tag audio files. I’ve written a shell script called ofatag which uses envtag. It recognizes Mp3 files using the file command and decodes using mpg123, other formats are directly fed to afprint.

Now, to tag your files using MusicBrainz web services just do
ofatag /path/to/music/*.mp3 /path/to/music/*.ogg
etc.

I haven’t released a version yet because it’s all pretty new and needs testing. So please test it and report back! Any comments, thoughts, patches are appreciated.

Ali Polatel: sydbox-0.6.3

Sat, 23 Jan 2010 08:00:00 +0000

sydbox-0.6.3 is released. ( tarball, sign, sha1sum )

Resolve path of non-abstract UNIX sockets
Intercept dup family calls and fcntl calls to see if a socket descriptor we care about has been duplicated

Ali Polatel: ptrace on BSD

Sat, 16 Jan 2010 08:00:00 +0000

ptrace is a system call which is used for process tracing and debugging. This system call is available on many operating systems. However each operating system has different versions.

I want to explain about my efforts to port sydbox to FreeBSD. The ptrace implementation of FreeBSD is similar to Linux’. The request PT_SYSCALL is available to stop the traced process at every system call and exit similar to PTRACE_SYSCALL of Linux. In addition to that FreeBSD has the requests PT_TO_SCE and PT_TO_SCX which stops the traced process only at the beginning of system call entry or exit. This is a feature I really miss on Linux.

There is, however, a big difference, I’m inclined to call it a bug, about ptrace on FreeBSD. When a traced process is stopped at the entry of a system call, there’s no way to prevent the execution of this system call. On Linux this is done by changing the system call number to either something invalid like 0xbadca11 or something harmless like getpid.

Here is an example:

    /* denying system calls using ptrace on Linux
     */

    #include <assert.h>
    #include <fcntl.h>
    #include <signal.h>
    #include <stdio.h>
    #include <stdlib.h>
    #include <unistd.h>
    #include <sys/reg.h>
    #include <sys/types.h>
    #include <sys/wait.h>
    #include <sys/ptrace.h>
    #include <linux/ptrace.h>

    #if defined(__x86__)
    #define ORIG_ACCUM    (4 * ORIG_EAX)
    #elif defined(__x86_64__)
    #define ORIG_ACCUM    (8 * ORIG_RAX)
    #else
    #error unsupported architecture
    #endif

    int main(void)
    {
            int status;
            pid_t pid;

            if ((pid = fork()) < 0) {
                    perror("fork");
                    abort();
            }
            else if (pid == 0) {
                    ptrace(PTRACE_TRACEME, 0, NULL, NULL);
                    kill(getpid(), SIGSTOP);
                    open("foo.bar", O_WRONLY | O_CREAT);
                    _exit(0);
            }

            if (waitpid(pid, &status, 0) < 0) {
                    perror("waitpid");
                    abort();
            }

            assert(WIFSTOPPED(status));
            assert(WSTOPSIG(status) == SIGSTOP);

            if (ptrace(PTRACE_SYSCALL, pid, NULL, NULL) < 0) {
                    perror("ptrace(PTRACE_SYSCALL, ...)");
                    ptrace(PTRACE_KILL, pid, NULL, NULL);
                    abort();
            }

            if (waitpid(pid, &status, 0) < 0) {
                    perror("waitpid");
                    ptrace(PTRACE_KILL, pid, NULL, NULL);
                    abort();
            }

            assert(WIFSTOPPED(status));
            assert(WSTOPSIG(status) == SIGTRAP);

            /* Change the system call to something invalid, so it will be denied.
             */
            if (ptrace(PTRACE_POKEUSER, pid, ORIG_ACCUM, 0xbadca11) < 0) {
                    perror("ptrace(PTRACE_POKEUSER, ...)");
                    ptrace(PTRACE_KILL, pid, NULL, NULL);
                    abort();
            }

            /* Let the process continue */
            ptrace(PTRACE_CONT, pid, NULL, NULL);

            waitpid(pid, &status, 0);
            assert(WIFEXITED(status));
            exit(WEXITSTATUS(status));
    }

Now although the traced process calls open("foo.bar", O_WRONLY | O_CREAT) the file foo.bar won’t be created because the tracer process denies the system call.

Here is the same example for FreeBSD:

    /* denying system calls using ptrace on FreeBSD
     */

    #include <assert.h>
    #include <fcntl.h>
    #include <signal.h>
    #include <stdio.h>
    #include <stdlib.h>
    #include <unistd.h>
    #include <sys/types.h>
    #include <sys/wait.h>
    #include <sys/ptrace.h>
    #include <machine/reg.h>

    int main(void)
    {
            int status;
            pid_t pid;
            struct reg r;

            if ((pid = fork()) < 0) {
                    perror("fork");
                    abort();
            }
            else if (pid == 0) {
                    ptrace(PT_TRACE_ME, 0, NULL, 0);
                    kill(getpid(), SIGSTOP);
                    open("foo.bar", O_WRONLY | O_CREAT);
                    _exit(0);
            }

            if (waitpid(pid, &status, 0) < 0) {
                    perror("waitpid");
                    abort();
            }

            assert(WIFSTOPPED(status));
            assert(WSTOPSIG(status) == SIGSTOP);

            if (ptrace(PT_SYSCALL, pid, (caddr_t)1, 0) < 0) {
                    perror("ptrace(PT_SYSCALL, ...)");
                    ptrace(PT_KILL, pid, (caddr_t)1, 0);
                    abort();
            }

            if (waitpid(pid, &status, 0) < 0) {
                    perror("waitpid");
                    ptrace(PT_KILL, pid, (caddr_t)1, 0);
                    abort();
            }

            assert(WIFSTOPPED(status));
            assert(WSTOPSIG(status) == SIGTRAP);

            /* Change the system call to something invalid, so it will be denied.
             */
            if (ptrace(PT_GETREGS, pid, (caddr_t)&r, 0) < 0) {
                    perror("ptrace(PT_GETREGS, ...)");
                    ptrace(PT_KILL, pid, (caddr_t)1, 0);
                    abort();
            }

            r.r_eax = 0xbadca11;

            if (ptrace(PT_SETREGS, pid, (caddr_t)&r, 0) < 0) {
                    perror("ptrace(PT_SETREGS, ...)");
                    ptrace(PT_KILL, pid, (caddr_t)1, 0);
                    abort();
            }

            /* Let the process continue */
            ptrace(PT_CONTINUE, pid, (caddr_t)1, 0);

            exit(0);
    }

We expect the same to happen here, the file foo.bar shouldn’t be created. But it’s created. Replace the PT_GETREGS and PT_SETREGS calls with a PT_KILL to terminate process with signal SIGKILL. The file will still be created! So there’s no way to deny a system call using ptrace which makes it impossible to port sydbox to FreeBSD without patching the kernel.

None of the other BSD’s, neither NetBSD nor DragonFlyBSD nor OpenBSD, has the ptrace request PT_SYSCALL so I haven’t checked if the behaviour is the same on these systems.

Wulf C. Krueger: Recognition

Sun, 10 Jan 2010 17:19:49 +0000

What keeps me doing things in my life are primarily two factors: Money and recognition. Not necessarily in that order.

In my job, I’m being paid to do what I do but I couldn’t ever be satisfied with just that. What really thrills me is being recognised for the professional I am. Receiving an email from a customer that simply said “Thank you. You’re one of the few persons I can always rely on.” made my day. I don’t get that from receiving my pay-cheque.

In my private life, I’m mostly a father, a husband and, last but not least, a guy who loves to work on Linux. A machine that just works is a boring machine. Thus, I really love working on Exherbo.

Working on Exherbo allows me to do and try everything, make things work exactly the way I want them to, give back to the FOSS community – and being recognised for the professional I am. :-)

Recognition, thus, is very important for me. Now, Bryan “kloeri” Østergaard, has decided to remove Exherbo’s “Developers” page which lists all the core developers in favour of a simple list of all contributors ever.

This in itself is fine with me. What I really don’t like about it is the fact, that those of us who do most of the work on Exherbo will be buried somewhere in that rather huge list (after all, Exherbo currently has about 95 contributors).

I have Exherbo in my CV as well but am I supposed to send recruiters to a list of everyone and their dog and find me in there with no indication of my level of involvement?

As much of a trifle this may look, it annoys me and so I’m now using gitstats to create statistics and a list of authors myself. It’s not hosted on exherbo.org (linked from our “Resources” page, though) but on my own server:

http://www.mailstation.de/egitstats/

I’m going to add a few more graphs and stuff over time (like changes per package directory, category, etc.).

If you have any suggestions (preferredly upstream-able ones), please let me know.

Ciaran McCreesh: Paludis 0.44.0 Released

Sat, 09 Jan 2010 15:53:14 +0000

Paludis 0.44.0 has been released:

The ‘everything’ set is now called ‘installed-packages’. A new set named ‘installed-slots’ has been added, which is similar but includes slot restrictions matching installed slots.
kdebuild-1 support has been removed, following the Gentoo Council’s decision to remove all mention of it from the Package Manager Specification. Users with installed kdebuild-1 packages must remove them before upgrading.
Support for EAPI 4 (formerly known as EAPI 3) is present but not installed, since the specification has yet to be approved.
Support for the new EAPI 3 is present but not installed, since the specification has yet to be approved.
The [.key=value] syntax for user dep specs now works with sets, sequences and spec trees. If < is used instead of =, a less than comparison is used for numeric values, and for compound values, a match succeeds if any item of the key is equal to the specified pattern.
build_options: preserve_work can be used to avoid removing temporary working directories, and to force a non-destructive merge.
Profile updates (package and slot moves) are now enabled by default.
Workarounds for various interactivity abuses carried out by certain ebuilds have been added.
Various large code cleanups and build system cleanups.

Posted in paludis releases Tagged: paludis

Bryan Østergaard: Half the solution..

Thu, 07 Jan 2010 15:38:26 +0000

For a long time I've wanted to replace the Developers listing on Exherbos website with a list generated from git log showing all the authors.

A while ago this got much easier as Ingmar Vanhassel and others added .mailcap files to our repositories. This means that some of my commits that I've accidentally made as kloeri@localhost can be grouped with kloeri@exherbo.org commits.

The actual page generation is still missing however so I'm looking for a volunteer that want to tackle this task.

Steps involved should be something like:
1. Clone git://git.exherbo.org/www.git
2. Read the Makefile to get an idea how our website is maintained. Reading my old blog post on our website setup is also useful
3. Figure out how to get a list of authors from git log. I just want a plain list containing the real names of all the contributors but without email addresses, commit count or other stats like that
4. Sort the list so it's easier to read
5. Make sure your list can be parsed by Maruku (a Markdown processor) and make sure it's processed along all the static .mkd files

Limiting the author list to just cover the arbor repository is fine for now.

Finally, give me a git format-patch of your changes so I can push it and we can all enjoy the improved website. Of course, you're more than welcome to ask me for help as needed.

Ali Polatel: sydbox-0.4

Thu, 07 Jan 2010 08:00:00 +0000

sydbox-0.4 is released.

What’s new?

Make network sandboxing on by default.
When bind’s port argument is zero, look up the actual port from /proc/net/tcp{,6} after the subsequent listen call for network_restrict_connect.
GObject isn’t a dependency anymore.
Try hard to restore errno after ptrace errors.
Moved all check based unit tests to gtest. dev-libs/check isn’t a dependency anymore.

Download

tarball: sydbox-0.4.tar.bz2
sign: sydbox-0.4.tar.bz2.asc
sha1sum: 67ef223d21571d42a4df122206ce22c9221b142e

Ali Polatel: Network sandboxing and /proc

Wed, 06 Jan 2010 08:00:00 +0000

As many of you know sydbox can do network sandboxing but for some reasons we didn’t have it on by default on Exherbo.

For those who don’t know much about sydbox and network sandboxing let me explain it briefly. Network sandboxing has three modes:

allow: All network connections are allowed.
local: Only local network connections are allowed.
deny: No network connections are allowed.

In addition to that there’s a restrict_connect option which disallows connects to all addresses except addresses that one of the parents has bind()‘ed to.

There’s also a network white list which specifies the additional network addresses that are allowed in local and deny modes.

On Exherbo we use the mode local with restrict_connect option enabled.

One limitation of sydbox was it couldn’t white list bind() addresses whose port were zero. The reason is obvious. The only place we can look up the actual port is /proc/net/tcp, or /proc/net/tcp6 for ipv6, and we need to do this before the bind() call has completed. The problem arises here. The /proc/net/tcp entry is only created after the bind() call has succeeded.

The solution isn’t entirely trivial. We have to note the file descriptor argument of bind() along with the socket family and socket address and intercept the subsequent listen() call. Only then we can look up the port argument from /proc/net/tcp.

The sydbox master has a simple implementation to solve this problem. If the port argument of a bind() call is zero, we save the file descriptor and the corresponding socket family and address to a GHashTable. After that the subsequent listen() call is intercepted and if the file descriptor of the listen() call matches a file descriptor in the hash table, sydbox looks up the port from /proc/net/tcp, fills it in and white lists the address.

With sydbox-0.4, which I’ll release after some testing, network sandboxing will be on by default again for the Paludis profile.

Just to be on the secure side ;)

Ali Polatel: mpdcron-0.3

Mon, 04 Jan 2010 08:00:00 +0000

mpdcron-0.3 is released:

What’s new?

Added stats module to keep statistics of played songs in a sqlite database
Added notification module to send notifications via notify-send
Added scrobbler module to submit songs to Last.fm or Libre.fm
Added module support through GModule
Added initial manpage
Changed name to mpdcron

Download

Ali Polatel: Bringing Last.fm home with mpdcron (part 3)

Mon, 04 Jan 2010 08:00:00 +0000

I wrote a script to import Last.fm data to mpdcron’s statistics database with the name homescrape.

It’s written in ruby and requires nokogiri to parse HTML. Currently it can import play count and loved songs. By default it will import all your Last.fm tracks and if you don’t want that you can pass a date using the –since option. Optionally homescrape can make use of chronic to parse dates in a huge variety of date and time formats.

With this, the statistics module is complete feature-wise and I’ll release mpdcron-0.3 after some testing.