ARGOS / FlyBase Next Generation -- Installation

Quick start: Use web form install.cgi for instructions and configuration.
      Name                        Last modified       Size  Description

[DIR] Parent Directory 16-Jul-2011 21:34 - [TXT] Readme-current.txt 05-Mar-2004 16:29 10k [   ] argoconf.tar 06-Aug-2004 16:14 70k [TXT] argos-crontab.txt 12-Feb-2004 17:57 1k [TXT] argos-ftp-notes.txt 01-Apr-2005 14:35 7k [TXT] argos-system-management.txt 03-Mar-2004 02:48 9k [   ] argos-update-ant.xml 02-Sep-2004 18:29 6k [TXT] install 20-Sep-2003 22:43 1k Argos installation scr> [TXT] install.conf 05-Aug-2005 15:11 4k Argos installation scr> [TXT] installargos.pl 05-Aug-2005 15:06 44k Argos installation scr> [TXT] packages.conf 01-Apr-2005 13:57 10k [TXT] reconfigure.conf 27-Mar-2004 11:31 2k [TXT] rsync-notes.txt 03-Dec-2003 17:27 6k [TXT] rsync.exclude 14-Feb-2004 23:49 1k [TXT] run-apache 10-Sep-2004 00:01 14k [TXT] run-postgres 28-Mar-2004 14:27 7k [TXT] run-tomcat 24-Mar-2005 17:43 15k [TXT] update 27-Mar-2004 13:04 1k


ARGOS SERVER INSTALLATION NOTES

alpha 0.7 release;    Don Gilbert;    March 2004

This ARGOS system is designed for automatic installation
and updates of FlyBase and other genome information servers.
This includes scripts, configurations, data, and Unix binaries
for all needed programs except Perl, Java and rsync.  Rsync is
used as the primary distribution program.

ARGOS currently forms the genome database/web server infrastructure
for the public genome databases at Indiana University including
FlyBase.net, euGenes.org, and the nascent Daphnia water flea database.
Basic development of this system continues, as part of the GMOD group
project for generic organism genome databases, but it is now usable for
replication of genome databases, with flexibility to add new such
databases.

Uses of automated database server distribution include local cluster
load distribution, world-wide mirror sites for rapid and redundant
access, developing bio-grid uses, institution/company servers for local
projects and data mining. An automatable mirroring system differs from
the method of providing software and data downloads by FTP in that
packages of data and software in this system are kept up-to-date without
human intervention.  Similar package management systems such as RPM,
PacMan, the Grid replication tools, and others share common functions,
but have proved not quite suitable yet for bio-database distribution.

Server packages are currently hosted at rsync://flybase.net/argos/ Web
notes are at http://eugenes.org/argos/ or http://flybase.net/argos/
Installation of a full FlyBase or other server will require several
gigabytes of disk space.
  
Some of these hosts have this server running
  http://flybase.bio.indiana.edu:8081/   -- solaris
  http://iubio.bio.indiana.edu:8081/     -- solaris
  http://chervil.bio.indiana.edu:8081/   -- linux
  http://marmot.bio.indiana.edu:8081/    -- macosx

See also install/argos-system-management.txt and background reading in
the docs/ folder.


REQUIREMENTS
============
 Unix systems supported for binary packages:
   Sun Solaris v8 & v9
   Intel Linux (redhat v8,9 tested)
   Apple MacOSX v10.2   
   
 These packages need to be and commonly are preinstalled 
 except for Solaris v8
  Perl v5.6 or later  - http://www.perl.com/
  Java v1.3 or later  - http://java.sun.com/
  rsync v2.5 or later - http://rsync.samba.org/
  

PREPARATION
===========

This system can be run from any Unix account with disk space
and ability to use Unix ports.  For a public server that is
automatically updated, we recommend

  1.  Create a normal unix account just for this service (e.g. argosadm)
  locating it on a disk with room for data (see in install.cgi a list of disk
  use).  The ARGOS_ROOT path will be set to this account folder or one inside it.
  Replication will manage files inside ARGOS_ROOT.
  
  2. Optionally create a second, underpriviledged account (like nobody)
  for the web server processes.  This will require you run the apache
  server as root.  It is not needed for lightly used servers, but recommended
  for heavy use.

     
INSTALLATION
============

Start with install.cgi web form
------------------------------------
http://flybase.net/argos/install/install.cgi

This is the recommended starting point.  The web
form handles Steps 0 and 2 below for producing the
local configuration for your system, and lists other
necessary steps (as below). 

Summary of steps
----------------

1. Download  install script : 
    http://flybase.net/argos/install/installargos.pl

2. Bootstrap installation :
   From command line, run this, changing -root= to your preferred location
    'perl installargos.pl -root=/bio/argos/ -install -v' 

   This creates install/ and a few other folders at the root path and fetches
   default configuration files.
    
3. Configure : 
   Use the install.cgi web form to create install.conf.local with paths
   and port numbers for your system.   This configuration file for your
   system needs to be placed in the install/ folder created in (2).

4.  Install a full data and software set :  
    'cd /bio/argos; install/install  -install -v' 
   OR 'install/install -install -v >& log.install &'  to run in background.
   
   This can take a while.  Check installation files and log.install 
   for errors when this finishes.

5.  Start the web server:
    'install/run-apache start' 

    The Argos web server should then be available at
    http://localhost:8081/ with installed packages at the ports that
    you configured it to use.  Read the logs/ folder for the *.err
    error logs to explain if this isn't working.
  
6.  Read the argos-system-management.txt document.  As appropriate,
    add cron tasks to this argos account for updating and server restarting.
   

Use command-line installargos.pl script
-------------------------------------------

This Perl script handles the bulk of data installation and updates.
installargos.pl [ options ] -info|install|list|update|reconfigure 

The -info and -list commands make no changes to disk.
All commands are benign if used with -n[orun] option.
Some other installargos options:

  installargos.pl -info  == show packages information
  
  installargos.pl -list -pack flybase  == list all files in package, 
    and required packages, from distribution servers.

  installargos.pl -update == update installed packages on frequent basis
  
  installargos.pl -no -install == view steps without doing them
  
  installargos.pl -root=/usr/local/argos == use new root path
   


HISTORY
============
winter 2004: Argos now is public server for FlyBase-NG, euGenes
fall 2003  : updates for FlyBase public server including slave servers for
      BLAST load balancing service; GBrowse (fb) and others
summer 2003: added genome services: euGenes.org, daphnia-wFleaBase, centaurbase (test)
may 2003   : renamed from flybaseng to argos 
march 2003 : changed preliminary biodb to flybase-ng



WORKED EXAMPLE
==============
Installing all of flybase-ng may be time consuming - it still needs
an easier installation script.  Don't install as root -- but as
a regular user, such as yourself.

The basic step is to copy the installargos.pl perl script to a local folder
and run are below in my test.

# ---------
# STEP 1 -- DOWNLOAD
#   http://flybase.net/flybase-ng/install/ installargos.pl
#   to a local folder where you want to install.  
# ---------

dghome2% perl installargos.pl
Usage:
 installargos.pl [ options ] -info|install|list|update|reconfigure 

** 1. Run installargos.pl -install [-root=/bio/argos]  == bootstrap install 
** 2. Edit /bio/argos/install/install.conf.local to set configuration.
   3. Run /bio/argos/install/install -install   == copy server packages
 Packages selected from packages.conf will by copied from servers.
 Start web servers with /bio/argos/install/run-apache
 Web notes are at http://flybase.net/flybase-ng/

 -- details omitted --
 
 configuration= PACKAGES=argos-install, ROOT=/bio/argos
 
 packages= argos-install

# ---------
# STEP 2 -- run with parameters
# IF YOU WANT TO USE default path /bio/biodb, skip the -root=...  option
# USE THE '-n' option to see what it will do without doing it.
# ---------

dghome2% perl installargos.pl -root=/Users/gilbertd/temp/argos -install -v

config file missing: ./packages.conf
config file missing: ./install.conf
config file missing: ./install.conf.local
config file missing: ./reconfigure.conf

# INSTALL server directories
mkdir -p /Users/gilbertd/temp/argos/install
# OS=apple-powerpc-darwin

# INSTALL package=argos-install
# INSTALL rsync://flybase.net/argos/install/
rsync -a --exclude-from=/Users/gilbertd/temp/argos/install/rsync.exclude.local rsync://flybase.net/argos/install/ /Users/gilbertd/temp/argos/install/
cp -p /Users/gilbertd/temp/argos/install/install.conf /Users/gilbertd/temp/argos/install/install.conf.local
perl -pi -e s,^ROOT=\S+,ROOT=/Users/gilbertd/temp/argos,; /Users/gilbertd/temp/argos/install/install.conf.local
mkdir -p /Users/gilbertd/temp/argos/common/
ln -s ./systems/apple-powerpc-darwin /Users/gilbertd/temp/argos/common/system-local

# RECONFIGURE packages

dghome2% ls
common/          install/         installargos.pl* 

# ---------
# STEP 3
# EDIT BY HAND install/install.conf.local
# OR see for web form configuring (in progress)
#   http://flybase.net/flybase-ng/install/ install.cgi
# ---------

# CHANGE:

# path to installation
ROOT=/bio/biodb
# packages to install
PACKAGES="biodb-root flybase"
FLYBASE_CONF=#FLYBASE_CONF

# TO THESE (or other desired):

# path to installation
ROOT=/Users/gilbertd/argos1
# packages to install
PACKAGES="biodb-root flybase"
FLYBASE_CONF=


# ---------
# STEP 4 -- run after bootstrap as install/install
#  -- check configuration values match above
# ---------

dghome2% install/install
Usage:
 install/installargos.pl [ options ] -info|install|list|update|reconfigure 

   1. Run install/installargos.pl -install   == bootstrap install 
   2. Edit /bio/biodb/install/install.conf.local to set configuration.
   3. Run /bio/biodb/install/install -install   == copy server packages
...
 configuration= APACHE_PORT=8081, DAPHNIA_CONF=#DAPHNIA_CONF,
 EUGENES_CONF=#EUGENES_CONF, EUGENES_PORT=7072, FLYBASE_CONF=,
 FLYBASE_PORT=7082, GMODWEB_CONF=#GMODWEB_CONF, PACKAGES=biodb-root
 flybase, PGPORT=7302, ROOT=/Users/gilbertd/argos1, TOMCAT_FLYBASE=7882,
 TOMCAT_PORT=7880

---------

# ACTUALLY DO THE installation for flybase...
dghome2%  install/install -install -v

# if you want list of all files copied, use -rsopt=-v

# SINCE it takes a while,

dghome2%  install/install -install -v >& log.install &


# ---------
# STEP 5 -- check installation, run web server:
# ---------

install/run-apache 

install/run-apache start

# check result in logs/ folder

---------





Argos: A Replicable Genome infOrmation System version 1.0 ; August 2005
Copy this from /install Web home at eugenes.org/argos/ .. iubio.bio.indiana.edu/argos/ Comments to 'argos AT eugenes.org'