Work smoothly

From OpenMosixWiki

Jump to: navigation, search

'openMosix can migrate most Linux processes with no program changes required.'

Applications using shared memory continue to run as under standard Linux, but currently will not migrate. Also, applications using special hardware or using shared mmap to read/write a file will not be relocated to another server.

Threads other than Java green thread JVMs cannot be migrated by Linux, so openMosix cannot migrate programs that use them. More info on Java and Mosix

This is the place to share your success story or to offer a work-a-round for an application with known issues.

'Below is a list of applications that migrate or work smoothly with openMosix.'

Contents

[edit]

Numerical Applications

  • 'Matlab 5' (numerical computations) migrates just fine. (Matlab 6 requires special setup to migrate. Some ML6 programs won't migrate effectively) (More on Matlab)

Note: "migrates" does not mean "does useful work." No version of matlab will do any useful work when migrated, period.

  • 'Octave ' is an opensource work-a-like to Matlab and is migratable. (verified on v 2.1.50)
  • 'R' (statistical analysis) [1]

' R version 1.7.1 under OM on kernel 2.4.18 seems to migrate fine without needing SETJMP changes. ' ''R 1.5' and 'R 1.6' needs a patch but then works smoothly.

Change the SETJMP macro in src/include/Defn.h from
# define SETJMP(x) sigsetjmp(x,1)
to
# define SETJMP(x) sigsetjmp(x,0)

' Although this change helped R to migrate for R 1.5 and 1.6, it did break Ctrl-C handling.

  • 'Maple� 8' computational functions for symbolic and numeric mathematics. [2]
[edit]

Biology Applications

  • 'MODELLER' [3] works fine without any modifications
[edit]

Audio/Visual Applications

  • 'MJPEG Tools' Because it uses both i/o intense and cpu intense pipes, it works very well on small cluster. Encoding and decoding are bound on the home node (very fast connections between nodes will let these migrate)but the filters are migrated to the nodes increasing performace of very high quality file encoding.
  • 'bladeenc' easily rips your mp3 collection with:

cdparanoia -B

for n in `ls *.wav`; do bladeenc -quit -quiet $n -256 -copy -crc & done;

  • Sean has a description of his CD ripping cluster: [4]
  • 'FLAC' is a lossless audio encoder. [5]
  • 'Grip' is a gtk-based cd-player and cd-ripper. You can rip on one node with cdparanoia and encode on the rest of the nodes with 'oggenc' [6]
  • 'LAME' Ain't an MP3 Encoder [7]

' Note: Lame and other applications often get compiled for a specific platform, as in march=pentium4 because they initially only run on 1 node. If later other nodes get attached with a different processor, such as a Pentium3, this can cause crashes, however not related to openMosix

  • 'mpg321(mpg123)' migrates well when playing and encoding wave files.
  • 'LiVES' Linux (i) Video Editing Studio using mplayer and mencoder. Migrates very nicely due to its use of several sub-programs to do the actual audio and video editing.
  • 'Xnest' The Xnest server in Xfree86 migrates cleanly and without the migshm patch.
  • 'TightVNC' (VNC client/server) migrates well without the migshm patch.
[edit]

Visual Applications

  • 'kstars' (Desktop Planetarium for KDE) has been reported to work smoothly with openMosix. [8]
  • 'Croquet' (3D immersive technology) migrates successfully with openMosix. [9]


[edit]

Multiprocessing Tools

  • 'FSU-Pthread library' Ghilardi Gian Paolo (aka rejected) did some research on migrating applications that use these threads, he made an interesting report on threadson_om/benchmark.htm

' For ''MPICH' (Portable MPI), the "ch_p4" device is the most general and supports SMP nodes, MPMD programs, and heterogeneous collections of systems. http://www-unix.mcs.anl.gov/mpi/mpich/ '* The "ch_p4mpd" device supports only homogenous clusters of uniprocessors, but provides for far faster and more scalable startup. '* The "ch_shmem" device is inappropriate for openMosix since it is for a single shared-memory system.

'''''MPICH-GM' is for Myrinet switch-connected clusters.

[edit]

Codes and Encryption

  • 'John the Ripper' Unix password cracker. edit john.conf List.External:Parallel then run with -external:parallel [10]
  • 'CISILIA' a cluster aware password cracking utility [11]
[edit]

Cooperative Computing

  • 'Dnetc' is distributed.net distributed computing. You need the 'x86/ELF/pthreads' client (uses pthreads for systems without shared memory, e.g. MOSIX) [12]. For starting the client: (Note: The 0 forces single-threading.)

./dnetc -numcpu 0

  • 'SETI@home' the setiathome client migrates great on openMosix. [13].

' Download the client for i686 machines from [[14]]. ' You can see the status of the openMosix team at [15]. ' Also, please be aware of an ''exploit' for setiathome; it's been pointed out in a post on Slashdot.org: 03/04/06/156217&modethread&tid=172.

  • 'Operation Project X' 
                Operation Project X is a internet based distributed computing project to crack the keys
                used to protect the X-Box in order to prevent any installations of other Operating
                Systems. If the key would be successfully be cracked its possible to install Linux
                without any modifications on a xbox.

                Project infos:

* http://www.theneoproject.com/opx/main.asp (dead site) * http://sourceforge.net/projects/opx 

                The pxclient migrates very well on OpenMosix.

                Personally it is suggested to compile the pxclient to fit with the used libraries.
[edit]

Weather

  • 'WRF' is Weather Research and Forecasting (WRF) Model [16]

' Running WRF directly migrated to another node and using oMFS worked fine for me

e.g. from node 2
runon 1 /mfs/1/usr/local/wrf/wrf.exe

' WRF request huge memory regions for mapping the input-dataset from files into RAM. openMosix maybe does want to migrate this process automatically because of the huge-memory request. 

Using oMFS with DFSA enabled and the "runon" command is recommended.
[edit]

Chess

  • Dr. Robert Hyatt's 'Crafty' (chess player) works under certain circumstances. [17]

' [18]: openMosix setup instructions.

[edit]

Computing Utilities

  • 'Make' migrates quite well. With 4 nodes I observed a 2 times speedup in compilation of the Linux kernel.
  • 'Postfix' A high-performance mail transport agent. [19]
  • 'GridEngine' Distributed resource management which complements openMosix nicely to throttle jobs.
  • Intel Fortran compiler versions more recent than 7.0.79
  • 'NAGWare Fortran 95 compiler Release 4.2(464)' and 'NAG fortran 77 libraries (Mark 20)' produce programs that migrate perfectly (flap@germinal.toulouse.inra.fr)]
  • 'ZLIB' Of course zlib decompression with tar and gunzip migrates very well on openmosix.
  • 'GAIM CVS 0.60' Migrates very well and fast to make room on the home node
  • 'Distributed Folding dot Net' Processes migrate just fine as well.

' ''NOTE:' The August '03 release of the distributedfoldingdot_net client actually does not seem to migrate well. From what I can tell, migration activity causes the program to segfault pretty consistantly.

  • 'SpamAssassin' The top opensource anti-spam tool detection deamon spamd forks it self to handle multiple requests and each request migrates independent of the others.
  • 'Fluent' Serial and parallel processes (tested versions 6.0.12, 6.1.22) migrate fine, on Fedora Core 1 and similar disttributions you may need to set the environment variable 
   LDASSUMEKERNEL=2.2.5
otherwise you could observe crashes with messages 
   EOF on stdout. The fluent process could not be started.
  • 'Ansys' Version 8.1 has it's own cluster function but setup ist not as easy as just to use openmosix cluster - long running processes migrate very well (tested with Redhat 9)
Retrieved from "http://howto.krisbuytaert.be/openMosixWiki/index.php/Work_smoothly"
Personal tools