Frankie's World

Current tasks for DebianGis

2009-04-06T11:47:00.000+02:00

I'm still waiting some clarificatons by HDF team about their HDF5 roadmap with SONAMEs and libraries name. It seems they partially learned to manage libtool properly to follow C API changes, but still some oddities are present, such as using the same names for MPI and serial flavors. I will probably have to again diverge from upstream names, which is generally something I don't like, as in the case of GDAL. A summary document about HDF5 plans is due in a brief also.
When finished with HDF updates, I will concentrate on a couple of new beasts which still are lacking in DebianGis and are of so much interest, that are Ossim and the CNES Orfeo Toolbox. In the meantime I would hope that debian-science team will define a squeeze-related policy for MPI.

As always, I'm still worried of the low level of activities in the DebianGis team, which is a constant since years. Unfortunately, this is a situation I see quite common also in many other teams, but I still don't see a right way to improve the status of the team to avoid an almost one-man show. Also I would like to see again Qgis in main, but I'm still waiting for feed-backs by past/current contributors to my last qgis-dev list and off-lists polls.

OSM and N810

2008-10-14T23:41:00.002+02:00

I finally managed to use my Nokia N810 to do some GPS logging for OSM. The Nokia internal GPS is quite slow, so I used an external antenna and the well known Maemo Mapper tracking capability to log my first 60 kms in my area. I'm not too much happy with the result, because it seems the Nokia tablet is quite slow in tracking points: other trackers are someway configurable on those regards, but Mapper seems not taking in account any user preference about data sampling. Too bad, even if it is able to use OSM maps as default background, which is a great plus for a mapper. Definitively it needs a look to sources to understand what happens under the cover.

Vlogger with a named pipe

2008-01-17T12:52:00.000+01:00

In an old post I dealt with the nice mod_chroot module to secure your multisite server. I also adopted recently vlogger in order to manage better apache logs for virtual hosts, when you have dozens or hundreds of them. Unfortunately the suggested way to use it (a simple piped command) does not work nicely with mod_chroot, because the piped command has to run in the chroot jail.

Of course, it is not a good idea installing the whole perl intepreter and all required modules in a minimized jail, so I adopted an oldish classic trick (the old things are always the best) to solve the issue: using a named pipe. You basically need to do something like the following:

mkfifo -m 600 /var/run/apache2/logger && chown www-logs /var/run/apache2/logger

cat /var/run/apache2/logger | /usr/sbin/vlogger -u www-logs -g nogroup -s access.log /var/log/vlogger >/dev/null 2>&1 &

/etc/init.d/apache2 start

Of course your log directives will be something like:

LogFormat "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" vcombined
CustomLog "/var/run/apache2/logger" vcombined

You absolutely need to run the reader before the writer, else the init script would hang for ever. It is easy adding a simple init script to run vlogger before apache and it has also the big advantage of not requiring re-fork a perl intepreter at every damn log action. Piping rocks.

Beamers

2007-12-12T20:23:00.000+01:00

I ask myself again and again: why it is always a challenge using a laptop with a beamer? It is not rocket science, but I always find problems due to lack of sync with certain beamers and certain laptops combinations. If you are lucky, you can see an image, but probably some parts of the screen (and of your slides) are invisible. It happens also under Windows, but it happens too many times under Linux.
Lately, I'm finding difficult to use my Thinkpad X31 under that respect, even if I always managed to use it without great issues until some months ago. Does it need a macumba before use?

Lessons of the life...

2007-03-20T19:50:00.000+01:00

Waiting hours to complete GRASS importing for an Hyperion 242 bands data set, just to discover that all bands are zeroed as result. At least, I also find that the latest Envi 4.3 is not able to read those HDF files correctly. Free software is any way better...

To package or not to package...

2007-03-14T10:25:00.000+01:00

... this is the problem

I just casually saw Norbert post about the ion3 querelle. This a very unfortunate example of collapsing interactions among upstreams and maintainers. My general advice (but for suggesting to use wmii or dwm instead of ion3 :-)) is avoiding packaging of development branches, but this is a decision which sometimes is difficult to take: some programs seems in development for years and stable releases could result largerly unusable or limited. Surely our - and generally speaking any distribution - release cycle and maintainance are not adequate for many on-the-edge software out there.

When upstreams releases [a]periodical milestones, those could be packaged, but upstream will not support them: we have already our problems in supporting regular releases for security independently by upstream for mainstream programs, without adding pieces of casual crap around.

Packaging a casual snapshots is out of question IMHO, but for using it in sid/experimental, so I can understand the upstream opinion, because many users report problem to upstream instead of maintainers. I see definitively no silver bullet anyway, but for maintainers' capability of understanding what should be packaged or not and opening an unbreakable communication channel with upstreams to be up-to-date in respect with upstream roadmap.

Playing with mod_chroot...

2007-03-13T23:04:00.000+01:00

... or why PHP sucks so much?

It is quite common (at least for me) finding abused web applications on shared hosts around. The most typical case is finding some kind of IRC bot running as www-data and a few unauthorized files around used for phishing, cross-site scripting or what else. That motivated me to try playing with mod_chroot in order to minimize the possibility of webapps abuse from our friendly kiddies^Wcrackers. Mod_chroot is a nice tiny Apache module whose purpose is the confinment of webapps within a limited tree, where nothing but a few files and dirs are available to try exploits and abuse badly written code.

Unfortunately, I found that mod_chroot poses one major problem (among others, see its CAVEATS doc): the sucking mail() function is not working out of the box, because PHP folks - God knows why - decided to implement that stuff by calling a local /usr/sbin/sendmail program within a shell call. The most sane option is simply ignoring the issue and living happy with a disabled local mail() function. A nice solution in that case is using a PEAR Mail module, which is able to send mails via SMTP in much more elegant way. Unfortunately, there are quite a good number of morons out there, and that could be not a viable option if you are the system administrator of a bounce of shared hosts which are exposed to those morons, who absolutely need a working mail() function.

After a few googling around, I did find that many people had the same problem, but none found (or explained) a working and neat solution for a working mail() implementation, so I'm writing some notes about that. My implementation is based on a nice tiny program (mini_sendmail) which has the great advantage of not requiring a configuration file or a spool directory to work. It also works without suid bit on, which is a good thing as well. It is statically compiled by default, so it simplifies things a lot. In order to have mail() working you need also to install a (statically compiled) shell, e.g. bash-static as sh under the /bin directory of your chroot tree (a more tiny shell would be appropriate). I commented out the silly username autodetection code in the mini_sendmail.c source, because it creates some problem within an empty chroot tree (it failed with a "can't determine username" message with or without the /etc/passwd file available).

Now, the next step is using an entry like

sendmail_path = /usr/sbin/sendmail -t -fwww-data@your.domain -s127.0.0.1

in your php.ini file. You can also use another IP address if your SMTP server is not your localhost or you used some more exotic routing setup. Having a suitable /etc/hosts is also useful. A final trick is adding a var/lib/php4 or what ever directory is required to store PHP session files. No other files should be required in order to have all working. Of course, you have also to use inet sockets for any required connection (e.g. mysql) but this is typically the default in etch. I would also suggest to add only ad hoc statically-compiled binaries within the chroot tree when required, else you will need to store all required shared libs: in that case you will need to find binaries requirement by using ldd and chroot to find loading/running errors by means of an usual trial-and-error cycle.

Proftpd 1.3 out in experimental

2006-03-23T13:16:00.000+01:00

After being busy with DebianGis related things and a few other packages recently, finally I found the time (and will :-)) to complete and uploaded an experimental deeply re-packaged version of proftpd.

Now proftpd is quite near to the release of 1.3 and current 1.3.0rc5 is considered the final release candidate before that (due in a couple of weeks). So it is also time to test the new package and give feedbacks to me and upstream team. I removed the ancient multi-binary approach to promote the new dynamic shared objects to prime time. As always, there are quite a few feature and configuration changes in the new version, so people are warned about that.

I happily reduced the number of patches applied to the sources and refined scripts all around, but more work is required to manage better upgrades and also adding more contributed modules, as well. And some silly things surely have been overlooked. So, stay tuned...

Italian, really!

2006-01-14T19:07:00.001+01:00

Ok, ok, I like pizza, mandolino and do it better... I'm definitively condamned to live in the same country of Berlusconi, sigh!

Your Inner European is Italian!

Passionate and colorful.

You show the world what culture really is.

Who's Your Inner European?

My new toy landed

2005-12-10T20:45:00.000+01:00

After the sudden death of my beloved Compaq Presario 2715 (friends said that it finally did commit suicide after almost 5 years of tortures), I finally got a new Thinkpad T43P for my Debian and not-so-Debian activities at home. Transferring the full contents of the Compaq hard disk and changing a bit its configuration on fly for the new hardware (SATA controller and a few other things) went well. I really hate installing my own boxes from scratch and managed to use Ubuntu-Live to startup the box and fill in the disk with my sid tree and home.
So now my second Thinkpad is on the road...

Playing with a d-link modem

2005-11-30T10:05:00.000+01:00

I finally managed to hack a D-Link DSL300T series modem, which works using a Montavista embedded Linux distribution for AR7 (with a few proprietary modules and tools unfortunately). I have now a TTL-2-RS232 adapter to better control the box with a serial console, thanks to a friend of my LUG. I'd like to add at least openvpn to the box, to get a great low-cost vpn gateway.

On the embedded side, I just discovered that my recent handheld i-mate PDA2k is a Blue Angel compatible with GPE and Familiar. Just another toy to play with whenever I'll have time and another spare unit, which can be in a few (at least for the spare unit part).

BTW, I'm seriuosly starting to think 24 hours in a day are too few to do whatever I would.

We are secure. Or not?

2005-09-10T09:19:00.000+02:00

Recent post by aj points the no-end saga of (old-)stable security updates. Indeed, I think the situation is quite worst of what seems in LWN statistics. There are less known bugs not fixed for months. My own reports about proftpd remained parked for a couple of months on both stable and testing secteam lists, for instance. That's the bad news. The good news is that this is true for all distributions (and I will not talk about proprietary software, which could retain security bugs even for years after disclosure). That's not a Debian issue only, folks.

A few personal ideas about security updates:

There are of course priorities in security issues management, not all bugs are first class ones.
Security auditing is a slow process and not always upstreams are able to provide fast and decent fixes. A partial fix is not so better than a full one.
I don't know if our secteams use a tracking system off list, with task lists and assignments. It's definitively a good thing to do that, just in case, and maintainers should be able to submit patches and reports to it. One could think we should use BTS for that. I don't think BTS is the most useful tracking system for security, and in some cases things cannot be published before fixes (yes I know, non full disclosures are evil, don't hide problems, etc. but I think responsible reports are better).
A good number of issues are not tracked on BTS. I personally use rarely BTS for security, even for full disclosed issues, when they are not so known (no CANs, no official reports by upstreams, etc.). There are also issues known only by upstreams and me, sometimes. In those cases I only send e-mail to secteams.

UPDATE: indeed, I rembered wrong about proftpd, I found an ack by a testing secteam member about my report on 30th of June. So, they have been responsive, sorry joeyh.

Open comments are annoying due to spam/trolling

2005-09-03T08:54:00.000+02:00

Sorry, but I cannot agree with Kern's observation about blog comments... Unfortunately open blogs and wikis are subject to almost continuous and annoying spamming. I have a few wikis around and I had to close them to proper registered users, in order to avoid that problem. One cannot spend his lifetime to despam manually wikis, by revisioning or what else. Some programs do not allow easily despamming changelogs too.
I would add that a few of them allow easily bot registration also, so they do not protect the blog at all against smart spammers, when comments are activated.
That said, I'm yet quite annoyed by MLs flamewars, I would avoid to see blogs flamewars too and Planet flooding, even if we would live in a perfect world.

Silly bugs in proftpd yet around...

2005-09-02T21:04:00.000+02:00

A couple of users pointed me to a few silly and old bugs in proftpd package, never discovered before. Wow, I can really confirm now: more pairs of eyes look at a software, more bugs are found and fixed. Thanks Christoph and Ryo. This is the very special advantage of free software...

Race conditions accessing files

2005-08-30T23:56:00.000+02:00

After a brief thread with smb4k upstream, I noticed that it is not evident for many developers that accessing files (temporary or not) could expose to race conditions exploits in some situations, if the program runs in privileged mode or not. That's quite worrying, because Debian GNU/Linux has tons of little tools which are not so often inspected for security auditing. So folks, be so nice to read the following recommendations and consider that accessing files could be potentially a risky business:

do not use fixed or easily guessable pathnames for files in /tmp or any other world -accessible directory.
do not use nothing different from mkstemp() or tmpfile() to open temporary files and consider that mkstemp() does not work properly on NFS (due to O_EXCL use)
double check if you are using correct ownership and permission
unlink temporary files just after opening and use the open handle after
check if you are accessing a symlink or a pipe and do not accept anything different from a plain file
do not accept any path which has a symlink in a subpath
do not overwrite an existent file and if your temporary files need explicit deletion do not forget to do it on exit
checks and opening must be done atomically
tempnam() and its sister calls are evil
creating a temporary directory and files there is better for atomicity against race conditions
doing IPC without a named file is surely safer

Proftpd 1.3.0 is going ready

2005-08-30T13:47:00.000+02:00

New version of proftpd will use DSO, that caused a complete rewriting of the packages in sid.
That's a good news, because it will allow a simplified approach, without the need of using many different flavors of binaries for every authentication layer. Also, I'm moving to dpatch and removing all dbs patching rests. The bad news is of course that the new package will delay and stay in experimental too for a while eventually. Anyway I have not intention to release until 1.3.0 will become official for TJ and the Proftpd team. In the meantime I'm cleaning the 1.2.10 release by removing a whole series of imperfections and little bugs.

The proftpd saga

2005-08-30T11:51:00.000+02:00

I'm receiving almost no-end off-BTS reports about sarge proftpd package (1.2.10-15). It is amazing to see how few people care to consult the Debian Bugs Tracking System to know possible issues and problems with Debian packages. The old package in stable has a few gotchas (segfaults and cpu hogging) due to mod_delay module, which stabilized only recently, about one month or so after sarge release.

My suggestion is using 1.2.10-20 release on any production server, if you would not experiment DoSes and CPU consumption under heavy load. I packaged a stable backport with needed patches and uploaded to my own repo sitory on people. You can also add an apt resource like:

deb http://people.debian.org/~frankie/debian/sarge/ ./

I hope a proposed update with those changes enter a next point release of sarge. Incidentally, -20 solves also a couple of security issues pointed recently by Secunia and full disclosed since then. They will be object of a secure team update (thanks Michael Stone), due in a few.

Frankie's World blog started

2005-08-30T11:11:00.000+02:00

Ok, time to start blogging Debian activities, just to inform a few people about flaws and status of my packages and whatever...
Blogging is also the stardard way to complain about Debian issues too, as we recently discovered :-)