Adminlinux.org (......)

How to install Flash Plug-in on Fedora 11 64 bit Firefox

2009-06-13T06:15:00.000-07:00

I found this easy but tricky :)

Download the flash plug- in

wget http://download.macromedia.com/pub/labs/flashplayer10/libflashplayer-10.0.22.87.linux-x86_64.so.tar.gz

Copy the plugin to the mozilla plug-ins directory

cp /home/usr/libflashplayer.so /usr/lib64/mozilla/plugins/

That's all !!

How to autofs NFS automount

2009-06-03T03:49:00.000-07:00

Automount will mount a filesystem on demand, just by issuing an ls or a cd to a directory

An example to mount myserver:/oracle/test on /export/oracle in the local machine :

1) Create the mount point, and define the file to use by autofs

a) vi / etc / auto.master

/ export /etc/ auto.export

****We are defining the mount point, and the file that contains the mount information

2) We must now create the /etc/ auto.export file

a) vi / etc / auto.export

oracle soft, int, rw myserver: /oracle/ test

3) We must change the value of BROWSE_MODE in / etc / sysconfig / autofs to let mount the filesystem with a cd or ls.

a) vi / etc / sysconfig / autofs

# BROWSE_MODE - maps are browsable by default.
#
BROWSE_MODE = "yes"

4) Start autofs

a) /etc/init.d/ autofs start
b) chkonfig autofs on

No you can mount myserver:/oracle/test on /export/oracle by jus cd in to the directory.

How To disable pam Checking against Dictionary.

2009-06-01T05:54:00.000-07:00

Creating Strong passwords

The best password in the world does little good if you cannot remember it; use acronyms or other mnemonic devices to aid in memorizing passwords.
Here is how to create a simple way to create a good password :

Over eight characters
Combines letters, numbers and symbols
Easy for you to remember, but difficult for others to guess

Make all (or some) of the following substitutions

For the word "are", use "r"
For the word "be", use "b"
For the word "for", use 4
For the word "nothing", use 0 (zero)
For the word "to", use 2
For the word "you" use "u"
For the word "we", use "v"
For any numbers, type the number
If permitted, use "&" for and, or "@" for at.

Example.

"To be or not to be, that is the question" becomes : 2bOn2BtItQ

Or

"lost is the best Series from 2009" becomes : LsdbSft2&9

A friend of mine asked how to disable password checking against dictionary words in Red Hat using pam here is little how to

Open the following file with your favorite editor /etc/pam.d/system-auth eg # vi /etc/pam.d/system-auth

1) Comment out the line referencing to pam_cracklib

# password optional pam_cracklib.so try_first_pass retry=3

2) Remove "use_authtok" from the line following line.

# password sufficient /lib/security/$ISA/pam_unix.so nullok use_authtok md5 shadow

This will disable Dictionary password checking when creating or changing passwords.

Telnet login RHEL no password

2009-04-21T01:34:00.000-07:00

No passord for telnet User with the following procedure :

a) created a user "manolo"

#useradd manolo
b) added a password

# passwd manolo
c) opened /etc/shadow and deleted the password manually for the user.

# vi /etc/shadow

output

manolo:$1$kAOxC9VT$rgJjMuzFz229KTjsKEBxP/:14354:0::7:::

d) deleted the password manually:

manolo::14354:0::7:::

e) then telnet using manolo to my host.

# telnet -l manolo 10.33.8.105

Managed to log in with out prompting for a password .

Please be Advised that I recommend ssh with shared keys or telnet through an ssh tunnel

SSH no password rsa-key RHEL 5

2009-04-20T13:21:00.000-07:00

SSH no password rsa-key RHEL

1) On the machine you are trying to access open a terminal and (Skip these step if you never generated a key before and or you want a fresh start)

a) As root backup .ssh directory for the user by moving it to ssh.backup

# mv /home/user/.ssh/ ssh.backup

2) On the machine that we will generate the RSA-Key open a terminal and (Skip these step if you never generated a key before and or you want a fresh start)

a) As root backup .ssh directory for the user by moving it to ssh.backup

# mv /home/user/.ssh/ ssh.backup

3)On the machine that we will generate the RSA-Key open a terminal and

a) Create the RSA key as the user you want to access the remote machine

# ssh-keygen

b) Press enter to accept defaults

Generating public/private rsa key pair.
Enter file in which to save the key (/home/manuken/.ssh/id_rsa):
Created directory '/home/manuken/.ssh'.

c)Press enter leave the passphrase blank

Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/manuken/.ssh/id_rsa.
Your public key has been saved in /home/manuken/.ssh/id_rsa.pub.
The key fingerprint is:
8a:8f:d3:9e:db:d5:93:b7:c4:fd:d8:0a:21:e1:47:91 manuken@manuken-laptop.lan

4) Now copy the key to the remote machine .

# ssh-copy-id -i $HOME/.ssh/id_rsa.pub user@remotehost

5) ssh in to the machine you should be able to access without a password.

SSH no password rsa-key RHEL 4.6

2009-04-20T13:14:00.000-07:00

Hi there creating rsa keys to log in with out a password on RHEL 4.6 follow this how to is a bit different from RHEL 5 release,

1) On the machine you are trying to access open a terminal and (Skip these step if you never generated a key before and or you want a fresh start)

a) As root backup .ssh directory for the user by moving it to ssh.backup

# mv /home/user/.ssh/ ssh.backup

2) On the machine that we will generate the RSA-Key open a terminal and(Skip these step if you never generated a key before and or you want a fresh start)

a) As root backup .ssh directory for the user by moving it to ssh.backup

# mv /home/user/.ssh/ ssh.backup

3)On the machine that we will generate the RSA-Key open a terminal and

a) Create the RSA key as the user you want to access the remote machine

# ssh-keygen -t rsa

b) Press enter to accept defaults

Generating public/private rsa key pair.
Enter file in which to save the key (/home/manuken/.ssh/id_rsa):
Created directory '/home/manuken/.ssh'.

c)Press enter leave the passphrase blank

Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/manuken/.ssh/id_rsa.
Your public key has been saved in /home/manuken/.ssh/id_rsa.pub.
The key fingerprint is:
8a:8f:d3:9e:db:d5:93:b7:c4:fd:d8:0a:21:e1:47:91 manuken@manuken-laptop.lan

4) Now copy the key to the remote machine .

a)Issue the comman below in order to cat the id_rsa.pu

# cat $HOME/.ssh/id_rsa.pub

b) The output will be similar to the one below copy this to the remote machine

ssh-rsa
AAAAB3NzaC1yc2EAAAABIwAAAIEAq98nHs9R+yldDsJkSwW84
GMG3jGoXVBwl4XLHU8prW3sMewgmSAr3pU5aWMewtdEuQnwt6/BzhR7lvbpRxC85NfljIZJyo
78uw/ZkzJZF3CdVqGts9KIHvWzX/LLE2dp3M4o7jQB8ASBp9jP0Je/cEnS86C0t0bUSpz1UYGZrgk= root@remote

c) Copy the output ( copy the output with your mouse) and ssh in to the remote machine.

# ssh user@remote

d) On the remote machine create the /.ssh dir:

# mkdir /home/user/.ssh

e) Change the directory to /home/user/.ssh and create the file authorized_keys

# cd /home/user/.ssh

# nano authorized_keys

d) Paste the output from cat $HOME/.ssh/id_rsa.pub in to the authorized_keys

5) ssh in to the machine you should be able to access without a password.

How to Rebuild Your rpm Database

2009-03-12T03:47:00.000-07:00

Sometimes seems that the Pubkeys on your rpm database doesn't rebuild properly. Try the following to ensure proper update of the Pubkeys :

1. cd /var/lib
2. tar -czvf /root/rpmdb-old.tar.gz rpm
3. rm -f /var/lib/rpm/__db*
4. cd /var/lib/rpm
5. mv Pubkeys Pubkeys.old
6. rpm -vv --rebuilddb

How to Create a centralized log in server with php-syslog-ng and mysql

2009-02-10T02:32:00.000-08:00

How to Create a centralized log in server

Configure syslog-ng to log to mysql database using fifo template /etc/syslog-ng/sylog-ng.conf:

destination d_mysql {
pipe("/tmp/mysql.pipe"
template("INSERT INTO logs (host, facility, priority, level, tag, date,
time, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL','$TAG',
'$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes));
};

log { source(net); destination(d_mysql); };

Comment out the following line

#source src { unix-dgram("/dev/log"); internal(); };

Uncomment out the following lines

source src { unix-dgram("/etc/log/log"); internal(); };
source net { udp(); };

Create the fifo pipe for syslog-ng to export out logs

mkfifo /tmp/mysql.pipe

Create syslog database

CREATE DATABASE syslog
USE syslog

CREATE TABLE logs (
host varchar(32) default NULL,
facility varchar(10) default NULL,
priority varchar(10) default NULL,
level varchar(10) default NULL,
tag varchar(10) default NULL,
date date default NULL,
time time default NULL,
program varchar(15) default NULL,
msg text,
seq int(10) unsigned NOT NULL auto_increment,
PRIMARY KEY (seq),
KEY host (host),
KEY seq (seq),
KEY program (program),
KEY time (time),
KEY date (date),
KEY priority (priority),
KEY facility (facility)
) TYPE=MyISAM;

Create the fifo pipe for syslog-ng to export out logs

mkfifo /tmp/mysql.pipe

Restart syslog-ng process

Stop syslog-ng

/etc/init.d/syslog-ng stop

Start syslog-ng

/etc/ini.d/syslog-ng start

Pipe Insert scripts

# Created by Matthias Buch
#

In the syslog-ng.conf we use:
destination d_oracle {
pipe("/dev/ora.pipe"
template("INSERT INTO logs (LL_HOST, LL_facility, LL_priority, LL_level, LL_tag,
LL_DATE, LL_program, LL_msg) VALUES ( '$HOST', '$FACILILITY', '$PRIORITY',
'$LEVEL', '$TAG',
to_date('$YEAR.$MONTH.$DAY $HOUR:$MIN:$SEC', 'yyyy.mm.dd hh24:mi:ss'),
'$PROGRAM', substr('$MSG',1,511));\n COMMIT;\n") template-escape(yes));

This script is used to pipe syslog-ng to mysql

#
# Created by Tadghe Patrick Danu
#
#!/bin/bash

if [ -e /tmp/mysql.pipe ]; then
while [ -e /tmp/mysql.pipe ]
do
mysql -u theuserid --password=thepassword syslogdb < /tmp/mysql.pipe done else mkfifo /tmp/mysql.pipe fi

CronJob

#Rotate logs once a month

0 0 1 * *  /var/www/php-syslog-ng/scripts/logrotate.php

@reboot           /muysql #the name is right  is th script that pipes the local logs to mysql.

Linux Technician Visual Effects Training Facility London job offer

2009-01-30T04:06:00.000-08:00

Hi there

A position has arised in my work place, I thought it could be interesting for someone .

The role involves assisting with the running and maintenance of the studios as well as aiding the development of the network infrastructure.

Essential Requirements:

- Excellent knowledge of the Linux operating system (particularly RH distributions).
- Proficiency in installing and upgrading workstations and servers.
- Ability to learn quickly and problem solve is essential.
- Excellent communication and team working skills.
- Accuracy and attention to detail.
- Capacity to manage a busy workload with high productivity.

Responsibilities:

- LAMP server administration including IPTables.
- Basic BASH scripting
- Support studio IT issues (Linux and Basic Windows)
- Research open source solutions on virtualized servers.

Useful Knowledge/Interest:

- VFX software such as Maya, Renderman, Shake.
- Network Management tools.

Location:
Escape Studios, Shepherds West. W14 0DA

Escape Studios is a hub of 3D animation, visual effects and games expertise. We provide the very best services to support the high-end digital

content creation industry. From aspiring students to industry leading companies we aim to deliver the same level of quality and customer care.

If you're interested in a career in computer graphics then we can provide you with all the resources to make it happen. For our industry partners we

provide the tools, training and staff, all by utilising the wealth of knowledge and experience we have in-house. The result is a facility where

students, professionals and companies can all benefit in equal measure.

Salary: £22K

Please send CV's with covering letter to linuxtech@escapestudios.com

Please feel free to email if you have any questions.

How to create .tga Targa files with mplayer, for SHAKE editing

2009-01-22T08:49:00.000-08:00

On Centos 5.2 / 4.6

Yum install mplayer

This one liner can be used to create the .tga files
mplayer filename.mpg -vf scale=1920:1080 -vf format=bgr24 -vo tga -nosound

On Ubuntu
ap-get install mplayer

This one liner can be used to create the .tga files
mplayer t2-039.mpg -vf scale=1920:1080 -vf format=bgr24 -vo tga -nosound

How to Install Shake, Nvidia driver and Centos 5.2

2009-01-20T05:47:00.000-08:00

Install Centos 5.2

Boot from your first CentOS 5.2 DVD. Press at the boot prompt.

It can take a long time to test the installation media so we skip this test.

The welcome screen of the CentOS installer appears. Click on Next.

Choose your language .

Select your keyboard layout

If you are installing CentOS 5.2 on a fresh system, answer Yes to the question Would you like to initialize this drive, erasing ALL DATA?

Now we must select a partitioning scheme for our installation. For simplicity's sake I select Remove linux partitions on selected drives and create default layout. This will result in a small /boot and a large / partition as well as a swap partition. Of course, you're free to partition your hard drive however you like it. Then I hit Next.

Answer the following question (Are you sure you want to do this?) with Yes.

On to the network settings. The default setting here is to configure the network interfaces with DHCP, but we are installing a server, so static IP addresses are not a bad idea... Click on the Edit button at the top right.

Choose your time zone

Give root a password

The installer checks the dependencies of the selected packages:

Click on Next to start the installation:

The hard drive is being formatted

Finally, the installation is complete, and you can remove your CD or DVD from the computer and reboot it

Installing Nvidia drivers

INSTRUCTIONS: HOW-TO INSTALL NVIDIA DRIVERS IN CENTOS 5.2

Browse to nvidia.com and downloaded the linux drivers (x86_64 in my case):

http://www.nvidia.com/object/linux_display_amd64_173.14.09.html (or whatever current one is)

2. Download the file: NVIDIA-Linux-x86_64-173.14.09-pkg2.run (or whatever current one is)

3. Get out of X. At a shell, type: init 3

You cannot be in X while doing this. Running 'init 3' will kick you out of X, back to a console prompt.

4. Login as root

5. Move the downloaded file to ~ (/root)

mv /home/whateveryournameis/Desktop/NVIDIA* /root

6. You have to install the kernel source code with yum, before running the nvidia utility:

Just run: yum install kernel-devel

7. Unpack the archive, and start the installer:

Just run: sh NVIDIA-Linux-x86_64-173.14.09-pkg2.run

This will unpack the archive, and automatically run the 'nvidia-installer' for you.

Just follow the prompts.

8. After driver is installed, you need to update your /etc/X11/xorg.conf file to use the new driver.

Just run: nvidia-xconfig

It will modify your xorg.conf file for you automatically.

9. Reboot

10. Enjoy :)

Installing shake

shake-linux-v4.10.0606a.tar.gz

Go to applications then sytem tools and open the terminal.

all the following commands are to be executed in the terminal or just copy and paste:

tar -zxvf shake-linux-v4.10.0606a.tar.gz .

pwd

Copy the output of the “pwd” command with your mouse then:

nano /etc/profile

scroll down until the end of the document and write:

PATH=$PATH:/paste/from/your/mouse/

press control-x and “yes” to save the settings

3. Open a terminal again and write: shake you should be able to run shake.

Basic Samba Troubleshooting Commands

2009-01-06T02:20:00.000-08:00

Samba Troubleshooting

Basic “survival” commands:

check samba configuration file :

testparm -S

check samba status :

smbstatus -L

check samba services :

smbclient -L localhost

connect to a service :

smbclient \\\\localhost\\share

How to Rebuild a corrupt Ldap Database

2008-12-23T02:24:00.000-08:00

Rebuilding the Ldap Database is very simple, and yes you will have to rebuild the database sooner or later.

Here is how.

#Stop the services:
/etc/init.d/ldap stop

#rename folder
mv /var/lib/ldap /var/lib/ldap.backup.date

#create folder
mkdir /var/lib/ldap

#add database info
slapadd –l /backups/ldap_backup.ldif

#Set Owner
chown –R ldap:ldap /var/lib/ldap

#Set Perms
chmod –R 755 /var/lib/ldap

#Start Services
/etc/init.d/ldap start

Ldap should OK now.

Red Hat Linux Commercial

2008-12-06T11:35:00.000-08:00

How to SELinux: could not open policy file <= /etc/selinux/targeted/policy/policy.23: Halting now.

2008-12-05T03:56:00.000-08:00

Two days ago I had a problem when disabling SElinux, when i rebooted I had this Kernel panic!


Booting `Fedora (2.6.25-14.fc9.i686)´

root (hd0,0)
Filesystem type is ext2fs, partition type 0x83
kernel /boot/vmlinuz-2.6.25-14.fc9.i686 ro root=UUID=64d8152d-508c-46ff-b8cd-9c2f4dc52510 rhgb quiet
[Linux-bzImage, setup=0x37d5d000, size=0x1fab60]
initrd /boot/initr-2.5.25-14.fc9.i686.img
[Linux-initrd @ 0x37d5d000, 0x292b6f bytes]


Decompressing Linux... done.
Booting the kernel.
Red Hat nash version 2.0.52 starting
SELinux: could not open policy file <= /etc/selinux/targeted/policy/policy.23: No such file or directory Unable to load SELinux policy (No such file of directory). Halting now.

The Solution was to boot the server, select an option from the boot screen press "e" add

Code:

selinux=0

Code i.e :

kernel /boot/vmlinuz-2.6.25-14.fc9.i686 ro root=UUID=64d8152d-508c-46ff-b8cd-9c2f4dc52510 rhgb quiet selinux=0

Then Press "b"

Increase MySQL connection limit

2008-11-24T06:29:00.000-08:00

The default is 100, but sometimes that’s not enough!

For MySQL 3.x:
# nano /etc/my.cnf
set-variable = max_connections = 250

For MySQL 4.x and 5.x:
# nano /etc/my.cnf
max_connections = 250

Restart MySQL once you’ve made the changes and verify with:
echo “show variables like ‘max_connections’;” | mysql

Lazy Linux: 11 secrets for lazy cluster admins

2008-11-07T00:39:00.000-08:00

Lazy Linux: 11 secrets for lazy cluster admins

This info was found on IBM excellent tips

Visit the inner sanctum of lazy Linux admins and discover how to reduce effort, regardless of number of nodes

Level: Intermediate

Vallard Benincosa, Certified Technical Sales Specialist, IBM
Egan Ford (egan@us.ibm.com), Linux Cluster Executive IT Specialist, IBM

22 Oct 2008

Cluster means different things to different people. In the context of this article, cluster is best defined as scale-out -- scale-out clusters generally have a lot of the same type of components like Web farms, render farms, and high performance computing (HPC) systems. Administrators will tell you that with scale-out clusters any change, no matter how small, must be repeated up to hundreds of thousands of times; the laziest of admins have mastered techniques of scale-out management so that regardless of the number of nodes, the effort is the same. In this article, the authors peer into the minds of the laziest Linux® admins on Earth and divulge their secrets.

Since their first appearance in 1998 in the list of the Top 500 fastest computers in the world, Linux clusters have risen from an obscure science experiment to the position of today's dominant force in supercomputing technology. In fact, the number of Linux clusters in the Top 500 list has grown from 1 system in 1998 (1 cluster, 1 Linux OS system) to four-fifths of the list in 2008 (400 clusters, 458 Linux OS systems).

Managing Linux clusters requires a unique set of skills that are not usually found among the single-system or small-networked-systems IT administrators -- it requires an in-depth knowledge of networking, operating systems, and pretty much all subsystems in the architecture.

But this is not all: It requires a different attitude. It requires laziness above all else. It requires the admin to do what Scrooge McDuck told his nephews in Duckburg: "Work smarter, not harder."

In this article we discuss some of the best secrets of the laziest Linux cluster admins. While they can hardly be called secrets, for some reason people either do not understand or they underestimate the power of these ideas. To clear this matter up, we'll present the secrets here with an explanation of their importance.

More ways to laze

The secrets are

Don't reinvent the wheel.
Use open source.
Automate everything.
Design for scale - plan to be lazy from the start.
Design for hardware manageability.
Use good cluster management software - don't dig wells with teaspoons.
Use open source monitoring solutions.
Control your users with a queuing system.
Verify you get what you pay for - benchmark it!
Manage cluster communication.
Look for ways to become lazier.

1. Don't reinvent the wheel

The lazy Linux cluster admin is not in the business of wheel-making; he focuses on building upon the work of others. There is no sense in wasting time building an application when a free supported solution already exists.

One of the rarest things in the world is an original idea or an original problem -- especially in the Linux cluster world. You will rarely come across something that has not been struggled over and solved back in 2004. This shouldn't make you feel unimportant or unoriginal; rather you should feel confident that there is really no problem that can not be solved (speaking technically, not politically or socially). So accept the fact that most problems and their solutions have been recognized, diagnosed, and solved.

To waste less time, the efficient admin spends more time:

Researching solutions that exist and adapting them to his own needs. Isaac Newton quoted Bernard of Chartres when he noted that in his work he stood "on the shoulders of giants." Think what would have been lost if he did not first try to understand Euclid's Elements first and then build off of others.
Contributing or customizing open source projects rather than reinventing something that has already been done - knowing full well that if he writes something, it will more than likely fizzle out when he leaves to a new job because no one else will understand it.

We don't mean to stifle your creativity -- quite the opposite. Using work already made by others allows you to build the next layer that will make your environment far superior and efficient than those in other organizations.

2. Use open source

The most successful Linux cluster admins that we have worked with have a vast knowledge of current open source projects. They are the frequent contributors to mailing lists and their names are the ones associated with current projects when you search on the Internet. They usually troll http://sourceforge.net and http://freshmeat.net for new projects that interest them.

Open source tools have properties that allow them to live longer than the person championing them, especially the popular ones. There is a very good reason that tools like Ganglia, Nagios, and TORQUE are still in use even though they have been around a long time. They are good -- they save administration in software costs and licensing schemes.

One other aspect of the laziest cluster admins is that they are quite passionate about open source and use it in their own personal pursuits. This may be their own Web servers at home or applications that they run on their own Linux notebook. From Pidgin to Firefox, you'll find that the laziest Linux admins run Linux in some other aspect of their lives distinct from the clusters they manage at work.

3. Automate everything

Using scripts on the command line and other quick writes are a big part of the Linux admin's tool chest. Scripting (as long as it doesn't reinvent anything) provides two useful results:

The first and most obvious is that it saves typing and provides a repeatable pattern.
The second is that it is self documenting and can be referred back to later.

We quite commonly see skilled admins with directories for scripts they have written on their machines. These scripts do everything from checking the firmware versions on the nodes to mapping GUIDs in an InfiniBand cluster.

One example where scripting is quite appropriate is that of generating an operating system image, whether it be stateless or stateful. If an admin has a "golden image" that needs to be propagated to each compute node on the system, he should know what is in it. Having a script that creates that image is the best documentation available because it explains exactly what is done and it is repeatable. Without scripts to build images, image bloat occurs and that eats up more space and slows the system down.

Too often we run across organizations with a golden image that they have been nurturing since 2000. The biggest reason: They don't know how to rebuild it. The second and probably the best reason: Because their application has been tested and "certified" on this image. Certified is one of those terms you run across that is as nebulous as the definition of cloud computing (which by the way is not a patented nor trademarked term).

The secret of why you should want to automate things is this: It takes more brain power to get out of work than it does to actually do work. The lazy Linux cluster admin does not accept work that turns his brain into fluff. If you have to ssh into every machine in the cluster and run a command, you are not being lazy enough. All commands to nodes should be done in one fell swoop using parallel commands or procedures. If your hardware vendor does not have Linux tools to automate BIOS updates or subsystem flashing, you should factor that into your acquisition cost.

Tips 8 and 10 in our last article on "Lazy Linux: 10 essential tricks for admins" documented several command-line scripting techniques we use often. There are many other ways to do it, some of which may be more efficient, but those tips just give you an idea of what can be done.

4. Design for scale and plan to be lazy from the start

Secret Number 3 (automate, automate, automate) is a great goal, but it is just a step on the path to complete idleness. For the laziest of admins, complete idleness can only be achieved with autonomous scale-out management. The first step in this quest is a system immune to operations that do not scale.

Very large scale-out clusters are plagued with bottlenecks. That is, most scale-out admins use TFTP to network boot or install large sets of machines. As any experienced scale-out-oriented admin can tell you, TFTP is unreliable and does not scale. Without proper remote hardware control, a massive TFTP failure could require the lazy admin to actually get out of his chair (bed) and walk (bum a ride) to the data center (not home) to reset each and every machine (busywork)! Even with proper remote hardware control, the lazy admin will have to stop playing WoW long enough to periodically issue the commands (again, busywork) to reset the nodes at a smaller scale to get the system up.

With a little upfront planning management, bottlenecks (like the following) can be avoided.

Bottleneck #1: Provisioning services

DHCP, TFTP, HTTP, NFS, and DNS are the most common services used to provision clusters. Each of them has a threshold -- TFTP is the worst one when it comes to scaling. Fortunately all of them can be easily replicated to assist with scale.

Tip: Isolating DHCP and TFTP to a different NIC will dramatically increase scalability. For instance, we have measured TFTP scaling at 40:1 if sharing the NIC with other provisioning services; 80:1 if not sharing services or stateless booting.

Bottleneck #2: The network

The network it is often the most overlooked part of any design. We are referring to the GigE network used for management, not specialized high performance networks used for application traffic. Although in many cases there is only one network that must be shared for data and management; this can compound any scaling issues.

Be careful when designing hierarchal networks that you do not oversubscribe too much. If an 80:1 node-to-service-node ratio is required, make sure that ratio is maintained or exceeded throughout the fabric.

Bottleneck #3: Don't bite off more than you can chew

When we design large scale-out clusters, we take a cluster-of-clusters approach. Each sub-cluster or scalable-unit (SU) is a building block that scales within itself for all cluster operations, (for example, install, network boot, BIOS flashing, monitoring, etc.). Each SU has a one or more (depending on the size of the SU) service nodes to provide the services necessary to control, monitor, and provision all the nodes in the SU. To further aid in scalable management, each SU has its own broadcast domain (SU-to-SU and SU-to-World communications are routed -- check for bottlenecks).

The central management node and service nodes have a private physical or virtual network so that aggregation of information from the service nodes and data pushed to the service nodes do not compete with other cluster traffic. We refer to this network, the management node, and the service nodes as the hierarchal management cloud or HMC. Its setup and operation is solely in the domain of the admins.

This cluster-of-clusters approach will allow the lazy admin to design systems that can scale beyond any budget and allow the same admin central control without fear that massive operations will fail.

5. Design for hardware manageability

We are surprised at the number of admins who don't think in terms of "lights out" when designing their clusters. Efficient administrators operate lights-out clusters, meaning their clusters sit in dark rooms away from humans and ideally it would be weeks or months before they actually need to see the physical machines they work on daily. In some cases, they'll never see the machines because they are managing them from the other side of the world. Of course, the laziest don't even know where the data center is -- it is just a set of host names or IP addresses.

Data centers are loud and sometimes cold (and who knows, maybe dangerous); the lazy admin should avoid them at all costs. Who knows the as-yet undiscovered health hazards of being in rooms with lots of machines. As power/cooling/staffing costs rise, the trends are towards moving data centers to places that are less expensive to operated. With this in mind, having absolute remote control should be regarded as essential to managing a Linux cluster now and for the foreseeable future.

Hardware vendors have largely given into customer desires for standards to remotely manage systems. IPMI 2.0 has become the current standard for most managed Linux clusters managed. IPMI offers a way to remotely power-cycle a machine, as well as have remote console viewable to see the machine boot from BIOS. At one customer site, we were able to troubleshoot a machine that was 60 miles away from us in the comfort of the customer's office. (The customer was one of those lazy Linux admins whose office was only lit by the neon signs on his wall. His office-turned-bachelor-pad was also equipped with two fridges laden with energy drinks and sugar-coated snacks. Needless to say, we didn't want to leave the office.)

IPMI is powerful -- we could change BIOS settings, reboot the nodes, and watch them boot up and see the screen dump without ever seeing the machine - it should be set up on all clusters. You should always demand at the least:

Remote powering of the machines and
Remote console or better viewing of the machines to see boot up problems that may arise.

With IPMI, we see little need in the Linux cluster space for other boxes that merely provide us a glorified interface to run IPMI other than perhaps a management node. Instead, we recommend standard open source tools like ipmitool that come packaged with most Linux distributions already. We find that our laziest Linux cluster admins will live and die by the command line.

What is still open to debate is the dependability of IPMI for remote console. We recognize that there are times when a real out-of-band terminal server can be of value. Terminal servers such as the Cyclades ACS48 remain a reasonable investment and provide out-of-band access and reliability that IPMI does not quite deliver.

In addition, IPMI 1.5 was not the most reliable IOHO (and it was an industry-wide issue). IPMI 2.0 does a much better job and many vendors add fancy Web pages around it to make it seem like it's out-of-band enough. There are arguments to include and not include terminal servers and just use IPMI native on the machine. Most of our customer's trains of thought chug along like this: Every lazy Linux admin knows that he spends a great deal of time troubleshooting 5 percent of the nodes when 95 percent of them are doing just fine. In a case like this, perhaps it is better to just buy 5 percent more nodes instead of infrastructure and have spares. This then means the budget is spent more on compute power than on infrastructure.

The argument that counters this one is that if one terminal server can save a plane trip across the country to troubleshoot a node, then the expense is worth it. We let the lazy Linux admin make the call -- after all, it's him that has to get in the plane. We've seen strong opinions on both sides.

6. Good cluster management software keeps you from digging wells with teaspoons

Cluster tools have come a long way since we first started installing Linux clusters in 1999. Back then, there were not many cluster management tools available and as such, most admins created home-grown suites of tools that deployed, monitored, and managed their clusters.

The laziest admins have either adopted open source tools or made the tools they developed back in 1999 available to the community. Seldom does anyone have an environment so unique that open source tools can not fill this gap. Most often, those who champion their own tools usually are alone and when they leave an organization, their tools disappear. However, we do recognize that there are many sites where customized tools work just fine.

If you are not satisfied with your home-grown tools or are looking for something better, consider looking at several open source tools. Among the most prevalent for managing clusters are OSCAR (System Imager), ROCKS, Perceus, and or our personal favorite xCAT 2. All of which are open source.

Perhaps the most popular open source cluster deployment/control solution today is ROCKS. ROCKS was created and maintained by UCSD and they have done a good job of making clusters user friendly. Our only gripe is its lack of flexibility, primarily at the OS level. ROCKS is based on Red Hat distributions which is fine for many people, but not those who use SUSE or wish to use images they have created based on RH 6.2 distributions. In addition, ROCKS is not a cloning solution which we find many IT organizations using.

Perceus is another such solution that differs from ROCKS in that it is a stateless installation. For the purposes of this article, we define stateless computing as running your operating system in memory instead of keeping it on disk. A disk is not required but can be used for local scratch or for swap.

What we like about xCAT other than our vested interest (full disclosure; we contribute code and actively develop xCAT) is that it has more flexibility, scales more, and has more power than any of the other tools. (And it has the most handsome and intelligent contributors.) The fastest supercomputer on earth, the LANL RoadRunner system (the Cell/B.E.™/Opteron™ hybrid that is the first one-petaflops system and first one-petaflops Linux cluster) is managed by xCAT.

xCAT allows imaging, kickstart, autoyast, iscsi, and stateless for nearly every enterprise Linux distribution available. In addition, it has command-line tools that abstract IPMI commands from remote power to console setup and uses them in the same framework. xCAT has been actively developed since October 31, 1999 and was open sourced by IBM in 2007.

But to be fair, we'll mention the drawbacks to xCAT as well:

It can be difficult to set up; with so much flexibility there are a lot of configurable parameters. We have set up a wiki to make this easier and there is lots of support via mailing lists and IRC channels.
xCAT 2 is a complete rewrite of the venerable xCAT 1.3 and as such, there are features that need to be ironed out.
Like many open source tools, the documentation could be improved.

There are many other cluster management solutions and we could easily have the Emacs vs. vi debate on the topic. But we'll just end with this: Give xCAT a try if you are looking for ideas or something better.

7. Use open source monitoring solutions

Last year at SC'07 we met with several top labs in the US to discuss the difficult problem of monitoring a Linux cluster. That lead to a few calls in early 2008 to discuss this problem. Monitoring is difficult for several reasons:

As the cluster gets larger, the data collected is larger and performance can be felt on the machine receiving monitored information. For example, if there are 2,000 machines and each one is sending metrics to an infrastructure node, the node can be brought to its knees and leave you wondering if it's responding.
Collecting data with agents on compute nodes can suck memory and CPU cycles from the user's jobs. Many shops want no agents on the nodes because it lowers performance of applications. Striking the balance requires trade-offs: Is it worth the CPU cycles taken to get the data?
We have not seen a scaleable tool that ties user jobs with machine performance to profile jobs in a way that is usable and visually appealing.
There are no tools that do it all exactly as you would like it to be done. This is what is perhaps most striking about this space: The tools used are incomplete and miss doing at least one something everyone would want. Most admins use a combination of one or two management products. When we see products that promise to monitor your cluster that surpass all other solutions we are skeptical: Every cluster is different and no one size fits all. Customized monitoring seems to be the only way out of it.

So given the complexity, here's how some of the laziest administrators we know are solving the problem.

The most common solution we've noticed at large cluster shops (including top universities and government labs) was to use Nagios for alerting and Ganglia for monitoring. Between these two very customizable tools an admin can get great insight into the multitudes of things happening on the cluster. Ganglia has proven to scale extremely well.

But there are other points of view as well. At USC, Garrick Staples wrote pbstop as a plug-in to TORQUE to visually see what each job is doing and where it is running. He says this is all the monitoring that he needs and doesn't use anything else.

The most popular open source monitoring tools we have seen used by the scale-out cluster community are:

Nagios.
Ganglia.
Cacti.
Zenoss.
CluMon.

We can say that many of these tools in turn make grand use of RRDtool in their implementation. CluMon also uses Performance Co-Pilot (PCP) underneath which is also quite popular. xCAT will have support for Ganglia and PCP in a future release.

To recap, the lazy Linux cluster admin knows

Monitoring doesn't have a one-size-fits-all solution. Monitoring means different things to different people. Some people are interested only in machine alerting, others in performance metrics, others in job profiling, and others just want to know if certain services or applications are running.
Customization will be different at different sites and on different clusters within the same sites.
The trade-offs between what is being monitored, what it is used for, and what resources it requires needs to be weighed carefully.

8. Control/manage users with a queuing scheduler

The lazy admin knows that users are the root of all problems. Preventing users from having root powers or permissions therefore is extremely important. You can even go further than that: You should do everything you can to keep users off your machine.

Queuing systems provide this functionality: A user submits a job and the queuing system decides which nodes it will run on. Unless users are running a job, they should stay off of the machine.

Today's popular queuing systems include some pay for products such as: LSF and PBS Pro. These products are used by many commercial customers, as well as government labs and universities. For many systems, plain open source solutions like TORQUE and SLURM work just fine.

We do a trick with TORQUE coupled with the Maui scheduler to keep our users off the cluster unless they are running a job. In Linux, this is done by first setting the /etc/security/access.conf file so that only root can login and no one else is allowed. For example, on each node if you run the command:

echo "-:ALL EXCEPT root:ALL" >>/etc/security/access.conf

then only root will be able to log into this machine. Next, you create a TORQUE prologue script that allows the user to login that runs something like this:

perl -pi -e "s/:ALL$/ $USER:ALL/" /etc/security/access.conf

(Hint: The $USER variable is the second variable that is passed to the prologue script by TORQUE when the script runs.) Since the root user runs the prologue script, the user will be allowed to log into the cluster. When the job is completed, an epilogue script is run that removes the user from /etc/security/access.conf so that he can no longer log into the node perl -pi -e "s/ $USER\b//g" /etc/security/access.conf. This prevents users from clobbering each other.

We have seen "performance" problems on clusters that have nothing to do with the machine; the real problem is that multiple users run on the same machine and the jobs they run each require 100 percent of the CPU.

It's no secret that user management is imperative. But what is often overlooked in simple troubleshooting is that users themselves are creating the problem. We strongly suggest that users be kept off the system unless they enter in through a controlled environment like a resource scheduler. In addition, we urge that the cluster network itself (the gigabit management or user network) be separate from the rest of the corporate or campus WAN with only certain user nodes providing front-in access.

9. Benchmark! Find performance problems before they find you

The last thing you want to deal with is a torch-bearing mob threatening to burn down your village because performance is weak and results are incorrect. So keep in mind that all too often, hardware diagnostics are the only litmus tests when determining cluster worthiness, but hardware diagnostics may paint an incomplete picture.

Hardware diagnostics are usually pass/fail with a vendor-defined threshold -- your threshold may be higher or lower. If a hardware diagnostic test fails, then you do have a problem; however no failures does not mean there are no problems.

Here are some roblems we've encountered that had a measurable impact on performance on with systems that passed vendor diagnostics:

Double-bit memory errors.
Single-bit memory errors.
SRAM parity errors.
PCI bus errors.
Numerical errors.
Cache errors.
Disk errors.
Performance inconsistencies.

Often there are problems that have nothing to do with hardware, but with software instead. Applications, libraries, compilers, firmware, and any part of the operating system can be the source many problems undetected by hardware diagnostics. Hardware diagnostics often do not run in the same runtime environment as the applications and do not stress the subsystems the same way as applications -- then problems created by software will be missed.

Clearly you need to run some type of relevant workload with your operating environment to verify that your cluster actually can do good work. This can be accomplished by running a few industry-accepted benchmarks. The purpose of benchmarking is not to get the best results but to get consistent, repeatable, accurate results that are also the best results.

How do you know if the results are the best results? A cluster can be broken down into the following major subsystems:

Memory.
CPU.
Disk.
Network.

Your hardware vendor should have benchmark data stating the expected Memory, CPU (FLOPS), Disk, and Network performance.

How do you know if the results are consistent?

Statistics. Each benchmark is run one or more times per node (or set of nodes for multi-node tests) and then the best representative of each node (or set of nodes) is grouped together and analyzed as a single population. The results are not as interesting as the shape of the distribution of the results. Empirical evidence for all the benchmarks in this article suggests that they should all form a normal distribution. A normal distribution is the classic bell curve that appears so frequently in statistics. It is the sum of smaller, independent (may be unobservable), identically-distributed variables or random events.

Benchmarks also have many small independent (may be unobservable) identically-distributed variables that may affect performance, such as:

Small competing processes.
Context switching.
Hardware interrupts.
Software interrupts.
Memory management.
Process/thread scheduling.
Cosmic rays.

These variable may be unavoidable, but they are a part of the source of a normal distribution.

Benchmarks may also have non-identically-distributed observable variables that may affect performance:

Large competing processes.
Memory configuration.
BIOS version and settings.
Processor speed.
Operating system.
Kernel type (like NUMA vs SMP vs UNI) and version.
Bad memory (such as excessive ECCs).
Chipset revisions.
Hyperthreading or SMT.
Non-uniform competing processes (such as httpd running on some nodes but not others).
Shared library versions.

These variables are avoidable. Avoidable inconsistencies may lead to multimodal or non-normal distributions and may have a measurable impact on application performance.

With a goal of consistent, repeatable, accurate results, it is best to start with as few variables as possible. Start with single node benchmarks like STREAM. If all machines have similar STREAM results, then memory can be ruled out as a factor with other benchmark anomalies. Next, work your way up to processor and disk benchmarks, then two-node (parallel) benchmarks, then multi-node (parallel) benchmarks. After each more complicated benchmark, run a check for consistent, repeatable, accurate results before continuing.

In the outline following is a path we recommend (the benchmarks report the performance of the components in bold).

Single-node (serial) benchmarks:
1. STREAM (memory MBps)
2. NPB Serial (uni-processor FLOPS and memory)
3. NPB OpenMP (multi-processor FLOPS and memory)
4. HPL MPI Shared Memory (multi-processor FLOPS and memory)
5. IOzone (disk MBps, memory, and processor)
Parallel benchmarks (for MPI systems only):
1. Ping-Pong (interconnect microsec and MBps)
2. NAS Parallel (multi-node FLOPS, memory, and interconnect)
3. HPL MPI Distributed Memory (multi-node FLOPS, memory, and interconnect)

Wait, that sounds like a lot of work!

It is, but it is also necessary if you plan to be lazy later on. Fortunately we have the tools and documentation to make this easy. A few days upfront planning can save weeks of frustration later on. We'll talk about these tools and graphs in a future article; they will also publish as part of a future xCAT RPM that will greatly increase productivity.

10. Manage cluster admin communication

Setting up MediaWiki
To set it up, first designate a server to host the Wiki. We usually use the management server if none other is available:
ssh mgmt

Now make sure an http server, a mysql server, and php5 are installed. If you were using Red Hat 5.1 or its derivatives you would type:
yum install http mysql php5 mysql-server

Next, configure mysql:

service mysqld start
mysql_install_db
mysqladmin -u root password 'mypasswd'

Now install media wiki:

cd /tmp/
wget http://download.wikimedia.org/mediawiki/1.13/mediawiki-1.13.0.tar.gz
tar zxvf mediawiki-1.13.0.tar.gz
mv mediawiki-1.13.0 /var/www/html/wiki
chmod a+x /var/www/html/wiki/config

Once you make it this far, navigate your Web browser to http://localhost/wiki. You can just follow the menus to install the rest of it.

Once you finish, it will instruct you to do this:

cd /var/www/html/wiki
mv config/LocalSettings.php .

Once you collect information about a system, the information should be stored some place useful so that the rest of the cluster staff can access it easily. We'd like to welcome you to the year 2000 -- documents in Word or Excel are neither cool nor is that the way to do this efficiently. The most productive practice for this we have seen is to set up an internal wiki. This is because the lazy admin gets tired of answering the same questions over and over again. Instead of having to look it up or run some command to give an answer, he simply says: "Check the wiki." And his task is done.

Every site should maintain a wiki that contains all the information about the cluster that is generally asked for like:

A log of changes to system including when administrative action was performed on the cluster.
Inventory of cluster: Firmware versions, models, serial numbers, number of nodes in cluster, processor type, memory.
Links to open support tickets with the vendor and where to get updates.
Documentation for common tasks using the management software.
Information on how OS images are created.

In short: The wiki should have enough information so that if someone asks a question about the cluster the admin need only say "Check the Wiki." In turn, any time you give someone an answer to something that is not in the wiki, you should tell them they need to pay it forward and right down that knowledge you gave them in the wiki. In addition, it is good for the churn in personal that is the reality of the wild IT world.

Why's a wiki better than other forms of documentation?

Wiki's are editable at a central place and access control can be given to people who need it. We've seen documents that reside in a share repository, but to view them, one must first navigate to this repository, then save to their hard drive and then open. Ineffective. One link is faster and less frustrating to get to.
Writing information in a Word document or spreadsheet is static and is not nearly as easy to edit, resave, and send out revisions. Not to mention, we see too many iterations of the same document around and people don't realize if they have the latest version or not. Especially when more than one person is editing it. In effect: Living documents live longer on wikis.

Setting up a wiki is extremely easy. We use MediaWiki. It's free, easy to get, and easy to install and configure. (See sidebar.)

Wiki syntax is much easier than HTML and there are many useful links on the Web that show how to use it. There are also good extensions that you can get for highlighting code syntax in perl or bash if that is what you use.

We find little resistance in organizations when we propose a wiki and hope it makes you lazier by installing one.

11. Look for ways to be lazier

We frequently see people in the cluster business doing things the way they do it because that's the way they've always done it. We think this is a good way for a Linux cluster shop to lose talent and get the least work out of their cluster. Change is the name of the game and new ideas come along frequently.

Naturally, we don't expect anyone to be able to investigate every idea that comes their way, but being familiar with newer trends is something that sets apart the good admins from the mediocre ones. Having said that, in this fast moving space, no one can possibly know something about everything and very few know everything about something. But good cluster admins know some things pretty well, have tested even more things, and ask questions about things they haven't heard about.

So if someone starts talking about something you haven't heard, the lazy Linux admin will ask a question because he's too lazy to go off afterwards and use his favorite search engine to find information about it. The scariest Linux cluster admins are the types that never ask questions. The lazy Linux cluster admin is not afraid of saying he doesn't know something. He is confident in his skill that if he doesn't know it, then someone else in the room doesn't either.

Today, there are great things happening to the world of managing Linux clusters. The most interesting that we have seen are:

More of a shift to stateless computing. Makes it easier to administer images and easier to keep nodes in synch.
Looking at clusters from the data center perspective. This means taking into account power and cooling and the payoff in terms of three or more years as opposed to just acquisition cost.
Liquid cooling efficiencies as opposed to air cooling. Did you know that liquid cooling is 90 percent more efficient than air cooling?
Adaptive management. This is where the queuing system is able to provision nodes on demand. This is true cloud computing and we have demonstrated this with Moab and xCAT. This is the final step of the path to complete idleness.

Summary

If this article has done its job, then you should now have ideas for how you can do less work and get better control of your existing Linux cluster environment and plan for your next one. We are confident that the ideas and practices we have set forth in this article contribute to better cluster utilization, a more professional science around it, and leaner and more efficient cluster administrative staff.

Having less people and less to problems means less meetings, less work, and more time for WoW, herding goats, sleeping, or doing whatever your lazy pursuits may be.

Resources

Learn

Tips 8 and 10 from the article "Lazy Linux: 10 essential tricks for admins" (developerWorks, July 2008) documents several command-line scripting techniques we like to use often.
See more Linux tips on developerWorks.
Some of the resources we mentioned in this article:
- Top500 Supercomputer sites list has been listing the world's fastest computers since 1993.
- I believe we mentioned the Supercomputing 2007 conference. (There's at least one of these gems each year.)
In the developerWorks Linux zone, find more resources for Linux developers (including developers who are new to Linux), and scan our most popular articles and tutorials.
See all Linux tips and Linux tutorials on developerWorks.
Stay current with developerWorks technical events and Webcasts.

Get products and technologies

Here's where to get some of the resources we mentioned in this article:
- Rocks is an open-source Linux cluster distribution that enables end users to easily build computational clusters, grid endpoints, and visualization tiled-display walls. The group has one goal: Make clusters easy.
- OSCAR (Open Source Cluster Application Resources) lets users, regardless of their experience level with a *NIX environment, install a Beowulf-type HPC cluster.
- Perceus is the next generation of enterprise and cluster provisioning toolkit created by the developers of Warewulf.
- xCAT (eXtreme Cluster Administration Toolkit) is a scalable distributed computing management and provisioning tool that provides a unified interface for hardware control, discovery, and OS diskful/diskfree deployment.
- Nagios is a host and service monitor designed to inform you of network problems before your clients, users, or managers do -- it was designed for Linux but works fine under most *NIX variants.
- Ganglia is a scalable distributed monitoring system for HPC clusters and grids that is based on a hierarchical design targeted at federations of clusters.
- pbstop is an ncurses monitoring and admin tool distributed inside perl-PBS (Portable Batch System) and is used by the admins of the largest clusters in the world.
- TORQUE is an open source resource manager providing control over batch jobs and distributed compute nodes, a community effort based on the original *PBS project.
- Cacti is a complete network graphing solution designed to harness the power of RRDTool's data storage and graphing functionality; it includes a fast poller, advanced graph templating, multiple data acquisition methods, and user management features out of the box.
- Zenoss delivers commercial open source IT management solutions.
- CluMon is a cluster monitoring system developed at the National Center for Supercomputing Applications to keep track of its Linux super-clusters; it is a tunable system that can be made to work for almost any set of Linux machines.
- RRDtool is an open source, high performance data logging and graphing system for time series data.
- Performance Co-Pilot provides a framework and services to support system-level performance monitoring and performance management; SGI provided an open source version in 2000.
- SLURM, the "highly scalable resource manager" (and a popular worm-based soft drink on the cartoon Futurama), is an open-source resource manager designed for Linux clusters of all sizes that provides three key functions -- it allocates exclusive and/or non-exclusive access to resources; it provides a framework for starting, executing, and monitoring work on a set of allocated nodes; and it arbitrates contention for resources by managing a queue of pending work.
- Moab Workload Manager is a policy-based job scheduler and event engine that enables utility-based computing for clusters.
Order the SEK for Linux, a two-DVD set containing the latest IBM trial software for Linux from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
With IBM trial software, available for download directly from developerWorks, build your next development project on Linux.

Discuss

Get involved in the developerWorks community through blogs, forums, podcasts, and spaces.

About the authors


		Vallard Benincosa is a lazy Linux Certified IT professional working for the IBM Linux Clusters team. He lives in Portland, OR, with his wife and two kids.


		Egan Ford started building Web and HPC Linux clusters in 1999 and was the chief architect for IBM's first large HPC cluster (Los Lobos at the University of New Mexico). Since then Egan has led in the design and implementation of some of IBM's largest systems including AIST, LANL Roadrunner, and the US National Science Foundation Teragrid (teragrid.org). Egan is the creator of IBM's first cluster management solution (xCAT) and co-author of two IBM RedBooks on Linux HPC.

Enable Commercial DVD Playback in Ubuntu

2008-10-30T16:20:00.000-07:00

Like Adobe Flash and proprietary media formats, playback of encrypted DVDs is something that Ubuntu can’t distribute in the default desktop. Adding support for playing your commercial DVD videos in Ubuntu isn’t a difficult task.

There’s a script included in Ubuntu that can install software for decrypting DVDs, run it by pasting the line below into your terminal:
sudo /usr/share/doc/libdvdread3/install-css.sh

If all went well, that script should have downloaded and installed the libdvdcss2 package for you.

Whenever you insert a DVD, the Totem Movie Player should launch and begin playing the disk. Unfortunately, Totem makes a really poor DVD player as it does not support DVD menus. It’s possible to install a different back-end for Totem to play DVD menus, but I just like to use the VLC media player instead. I’ve found that VLC does a fantastic job with DVDs.

Install VLC from the package vlc (click the link to install), or by running the command below in your terminal:
sudo apt-get install vlc

Open VLC from Applications->Sound & Video->VLC media player. To play a DVD, click File->Open Disk.

There’s a lot of options in this dialog, just make sure the disk type is set to DVD (menus), click OK, and the movie should begin to play.

Mount ISO files in Linux

2008-10-29T16:44:00.000-07:00

It is convenient to mount an ISO file directly instead of burning it to a CD first. This recipe describes the command used to mount an ISO image on a Linux system.

To mount the ISO image file.iso to the mount point /mnt/test use this command:

mount -o loop -t iso9660 file.iso /mnt/test

How to Install Shake on Ubuntu 8.0.4

2008-08-21T03:10:00.000-07:00

The problem is a missing x11 library that is not included with hardy, but is found in recent suse and fedora distros - and so we'll take the library from an extracted fedora *rpm file, libX11-1.0.3-8.fc7.i386.rpm

download the file libX11-1.0.3-8.fc7.i386.rpm
install the 'alien' rpm conversion utility
extract the rpm file
locate libX11.so.6.2.0 and copy to the lib directory used by shake
rename your moved copy of 'libX11.so.6.2.0' to 'libX11.so.6'
- if you are running shake from the original directory that was created from the software's extraction, then the lib directory is found in the shake directory
- (~./shake*/lib/)
if you are using shake 4.1, then you should be able to run shake

Instructions for Installing RPM Files Using Alien

Installing Alien

You can install alien itself from the Ubuntu Universe repository by adding the repository to your list of sources and doing:
$sudo apt-get update $sudo apt-get install alien

Installing the .rpm file

To install the .rpm file, you first need to convert it to a .deb file which can be installed on Ubuntu.
I assume that you downloaded the package to your Desktop (~/Desktop is the directory)
You can convert the .rpm to a .deb by using the following commands.
$cd ~/Desktop
-This will change the directory to your desktop, where you have the .rpm file.

$sudo alien -k name-of-rpm-file.rpm
- This will convert the .rpm to a .deb.
- The “-k” will keep the version number. Otherwise alien adds a “1″ to the version number.
- Tip: Use Smart Tab Completion to avoid mistyping the file names

$sudo dpkg -i name-of-deb-file.deb
- This will install the .deb package

Try reading the alien manpage for more details on how to convert other kinds of packages and the options available.

Google talk onyour Iphone

2008-07-21T14:46:00.000-07:00

iPhone 3G gets 50 per cent O2 Wi-Fi boost

2008-07-19T13:59:00.000-07:00

Over 3,000 more Wi-Fi hotspots will be accessible to O2 iPhone users from July 11 following a deal with BT Openzone

"The agreement with BT Openzone, along with O2's existing arrangements with The Cloud, will give all iPhone users 'unlimited' access to over 9,000 public Wi-Fi hotspots"

O2 iPhone customers will be have free access to 50 per cent more public Wi-Fi hotspots from 11 July, launch day for the iPhone 3G.

O2 has signed a deal with BT Openzone to give iPhone customers access to more than 3,000 BT Openzone Wi-Fi hotspots across the UK as part of their standard iPhone data package.

The agreement with BT Openzone, along with O2's existing arrangements for access to The Cloud's Wi-Fi network, will give all iPhone users 'unlimited' access to over 9,000 public Wi-Fi hotspots.

The deal covers existing iPhone users as well as iPhone 3G buyers, with either device automatically switching to the fastest network available when in range of The Cloud or BT Openzone hotspots. Users can download content or browse the web from their iPhone as much as they want at no extra cost - subject to O2's excessive use policy.

The BT Openzone covers a variety of UK locations including 13 BT wireless city centres, coffee shops, railway stations, airport lounges, motorway service stations and marinas.

Is my life about to change?

2008-07-17T08:15:00.000-07:00

Does destiny exist?.

Hi there,

I have been interested in Linux for a while now, installing, configuring, crying etc...
Last week when I was job Hunting, I found the perfect position to apply for "Linux technician"
Red hat, Apache Mysql etc but the most important the willing to learn ..... well thats me me me I thought.

I sent them my CV and after a couple of days they called me !!!!! I was so excited.

The interview.

I woke up and I wasn't nervous usually is really stressing to attend an interview but not this time.

The place was great Pictures of Animated caracters hanging in the wall, you could breathe art and harmony in there, the interviewer was great not intimidating at all he just wanted to interview me, with out any psichological torture.

Well the interview was 90% OK now the destiny bit .......

When I was walking inside the underground I saw some posters on the wall saying "Big Decissions are beign made around you"......
I thought that was hilarious but wait there is more "how can we make that happen..." very interesting after attending an interview.

Today I woke up and in my way to the bus, I saw a van with a message containing the name of !!the company I applyed for ....... th van was from a diferent company!!

Good interview, Good signs will I get my dream Job, are this coincidence or my life is about to change.........

Essential Math for Games Programmers

2008-07-11T15:22:00.000-07:00

Current Description

As the visual quality of games has improved, more attention has been given to other aspects of a game to increase the feeling of reality during gameplay and distinguish it from its competitors. One of the most rapidly growing fields is physical simulation, as shown by discussion boards, the latest games, and a recent burst of articles and papers. Creating such a simulation may appear to be a daunting task, but given the right background it is not too difficult, and can add a great deal of realism to animation systems, and interactions between avatars and the world.

This tutorial deepens the approach of the previous years' Essential Math for Games Programmers, by focusing in on this topic of physical simulation. It, like the previous tutorial, provides a toolbox of techniques for programmers interested in creating physics engines, with references and links for those looking for more information. The focus of the course is to study the simulation pipeline and show how problems along the way can be solved and optimized using standard 3D mathematical concepts and engineering know-how.

Topics include rigid body dynamics, verlet and implicit integration methods, constraint systems, collision detection, collision response, rag doll and IK systems. Potential special topics include dynamic destruction, fluid dynamics and using GPUs to improve simulation performance. Sample code libraries and examples are provided.

Current Materials

Slides

Fully updated versions of the slides and sample code used to present the tutorial at GDC 2008 are as follows:

Problem Overview (Squirrel Eiserloh)
Dynamics 101 (Marq Singer)
Numeric Integration Methods (Marq Singer)
Numerical Robustness (for Geometric Computations) (Christer Ericson)
Reframing the Problem (Squirrel Eiserloh)
Collisions using separating-axis tests (Christer Ericson)
Collision Detection (Gino van den Bergen)
Modelling and Solving Constraints (Erin Catto)
Physics Parallelization (Erwin Coumans)
GPU Physics (Marq Singer)

See below for further materials from past years that may be useful. The code samples in particular contain code for simple dynamics and collision.

Past Materials

Fully updated versions of the slides and sample code used to present the tutorial at GDC 2007 are as follows:

Problem Overview (Squirrel Eiserloh)
Dynamics 101 (Jim Van Verth)
Numeric Integration Methods (Jim Van Verth)
Numerical Robustness (for Geometric Computations) (Christer Ericson)
Reframing the Problem (Squirrel Eiserloh)
Collisions using separating-axis tests (Christer Ericson)
Collision Detection (Gino van den Bergen)
Modelling and Solving Constraints & Contact Manifolds (Erin Catto)
Soft Body (Marq Singer)
Destruction (Marq Singer)
Squishing and Sound (Marq Singer)

Fully updated versions of the slides and sample code used to present the tutorial at GDC 2006 are as follows:

Introduction (Jim Van Verth)
Dynamics 101 (Jim Van Verth)
Numerical Robustness (for Geometric Computations) (Christer Ericson)
Motion and Collision - It's All Relative (Squirrel Eiserloh)
Continuous Collision Detection (Gino van den Bergen)
Fast and Simple Physics using Sequential Impulses (Erin Catto)
Bending, Breaking and Squishing Stuff (Marq Singer)

For those slides in PowerPoint Show format, the original PowerPoint materials are also available upon request.

In past years this tutorial was more general, providing a toolbox of techniques for programmers interested in improving their 3D background. The focus of the course was to follow the rendering and dynamics pipelines and show how problems along the way can be solved and optimized using 3D mathematical concepts. The following PowerPoint Show files and PDFs have been collated from all six years of the tutorial and organized by subject. They provide background material for those who are interested in the basics of 3D math and programming. The original PowerPoint materials are also available upon request.

Core Mathematics

Vectors and Points, Lines and Planes (pps)
Linear Transformations and Matrices (pps)
Affine Transformations (pps)
Fixed Point and Floating Point (pps)

Rendering

Introduction to the 3D Pipeline and Scene Graphs (pps)
Viewing, Projecting and Picking (pps)
Rasterization, Lighting and Texturing (pps)

Animation

Curves and Interpolating Position (pps)
Quaternions and Interpolating Orientation (pps)

Simulation

Rigid Body Dynamics (pps)
Numeric Integration (pps)
Constraints (pps)
Inverse Kinematics (pps)
Collision Detection (pps)
Collision Response (pps)
Deriving the Angular Response Equation (pdf)

Demo Code

This collection of demo code is from the GDC 2005 presentation, and covers many of the topics listed above. The following archive has all the files you need to compile and run the demo code on Windows. You will need at least Microsoft Visual Studio 6. They should transfer to Microsoft Visual .NET with no trouble.
Windows 2000/XP Source

In the past we have supported the Mac, but at the moment none of us have a working machine much less one with MacOS X. The following files are provided for historical purposes only, and are no longer supported. You will need Codewarrior 6, at least MacOS 8, and the OpenGL SDK.
MacOS 8/9 Source

The copyright on these files is held by Red Storm Entertainment and Ubisoft. You can use them for personal purposes, but cannot redistribute them in source or binary form. No warranty is implied; use at your own risk.

For more polished code and demos with a less restictive license allowing them to be used in game development, and that can be compiled with non-Microsoft compilers, MacOS X, and Linux, please purchase a copy of our book.

Mathematics for Information Technology

2008-07-11T15:03:00.000-07:00

For more info Visit http://www.maths.gla.ac.uk/research/groups/it/

Research in Mathematics for Information Technology

Mathematics for IT is a 'cross-cultural' group, having connections with various mathematical areas including Computer Science. Our work has mainly involved: (1) error-correcting codes that change according to the degree of interference through which the message stream passes; (2) the mathematics of computer graphics; (3) polynomials over finite fields which have important applications in aspects of information technology such as communications and secrecy; (4) the use of fractals in representing nature, and in image compression and reconstruction, (5) neural networks, (6) mathematics (especially cellular automata) and music

Contacts

Professor S D Cohen : Applications of Finite Fields
Dr S G Hoggar : Fractal Compression

Recent Theses

"Jigsaws and Faster Fractal Pictures " by Lindsey Menzies
"Weight Enumerators and Weight Distribution of KM Codes" by Keith Pickavance
"Prediction of Earthquakes " by Euan Fraser
"Applications of Dynamical Systems to Music Composition " by Kenneth McAlpine
"M-sequences related to the multifocal electroretinogram: identification of primitive polynomials to avoid cross-contamination in mutlifocal electroretinogram responses " by Jill Ireland
"LVQ and Kohonen nets as human, for comparing ASM generated faces " by Hataikin Porncharoensin