Roadtech Administrative Toolkit index.

General

The toolkit comprise a set programs written mainly in Korn shell script, to aid in administering and using UNIX based systems with Roadtech software. Roadrunner and related software. Bundled with a small set of binaries.

Host System

The scripts are generally fairly tolerant of their install location, and the file system layout. To run you need symbolic links from one of the directories in the search path, to the key scripts in the directory to which they were installed. They will mostly sort the rest out from there.

Resources

CPU and memory requirements are generally fairly light, for Roadrunner allow around 5MB RAM per user, plus about 5% of data set size. Usual performance limit is file I/O. Using a disk controller with non volatile write cache, or solid state storage, yields better performance per £ spent, than spending on CPU performance.

Picking a processor with fewer but faster cores, will often yield better responsiveness, per £ spent.

Virtual machine

Yes they can be run on a virtual machine. That is how we QA/QC test for OS compatibility. Virtual machines using partition backed storage, yield better file I/O than ones using file backed storage.

Main issue with a VM is removable media for backing up. Most of the development effort for hyper-visors has been put in to making guests portable, and independent of host hardware.

iSCSI tape

Only dependent of network connectivity, universally support by hyper-visors.

Host Optical drive

All hyper-visors that I have used can export an optical drive to a guest for reading. Not all will export for writing.

PCIe controller pass through

Subject to support from both the host hardware and the hyper-visor used, it may be possible to pass a PCIe controller to the guest. Where this provides SATA, SCSI, USB, FireWire, Thunderbolt, interfaces. The guest gets full use of attached devices.

USB pass through of storage devices

This gets very messy. It depends on both the hyper visor and the type of device. A USB storage device appears to the host as a chain of things.

Life is simplest where you can have a media change event, without also having USB remove and attach events. Think of an optical drive, or an RDX enclosure. In this case you attach the enclosure to the host before starting the VM, the guest then handles the media changes.

Hot swapping a USB flash key while a guest is running is the most problematic. If I have two USB flash keys of the same make and model, they are generally indistinguishable until the host has accessed the UUIDs, of the file systems contained in the flash memory.

Security

It is recommended that the host system is installed and configured in line with industry best practice as set out in the guides published by the NSA and CIS.

Recommended file system layout for AIX

All supported AIX systems, use IBM's LVM for partitioning. It is recommended that additional logical volumes are created for storing scripts, and application data. Scripts will work with jfs, or jfs2 file systems.

If using a disk to disk stage in your backup procedure, there is a performance advantage if the source and destination are on physical volumes (volume groups).

Recommended file system layout for Linux

It is recommended that LVM is used to manage the storage space for multiple file systems. This follows practice on AIX systems, and allows space to be assigned as the system develops. The following is consistent with CIS guidelines.

For Linux systems it is further recommended that the layout fits in with the Filesystem Hierarchy Standard. Current version was published back in 2004, at the time it was published the layouts of the most popular distributions were markedly different. Which was confusing and created extra work for the people porting packages between distributions.

RedHat finished migrating with Fedora 17 29/05/2012, which scrapped /bin, /sbin, /lib, and /lib64. It moved their contents to the corresponding sub directories in /usr. it also allowed for /usr to be mounted read only, except during package updating.

Mount PointFS flagsComment
/ Consider SSD.
/boot 
/homenodevIf not a network share, consider SSD.
/ehomenodevLocal directory for EDI user accounts. Consider SSD.
/lhomenodevLocal accounts if /home is a network share. Consider SSD.
/tmpnodev,nosuid,noexecMay be a RAM disk. Contents may be wiped on a reboot
/srv 
/srv/RoadTech RoadTech scripts and data. Consider SSD.
/var 
/var/cache/RoadTech/RTstaging Work space for backup and other disk hungry tasks.
/var/log Aim for enough space for 4 weeks retention.
/var/log/audit Aim for enough space for 4 weeks retention.
/var/tmpnodev,nosuid,noexec

Presuming that the system has the above layout of file systems the following directories, could be suitable choice for installing the scripts, and associated applications.

DirectoryUsed forVariable
/srv/RoadTech/dataRoadrunner live data set.RR_Dir
/srv/RoadTech/testareaRoadrunner test data set.N/A
/srv/RoadTech/RT_AdminRoadtech scripts.RT_Admin
/srv/RoadTech/RT_Admin/binPrograms or utilities shipped with the toolkit.N/A
/srv/RoadTech/RT_Admin/cgi-binWeb Server extensionsIL_CGI
/srv/RoadTech/ImageLibraryRoot for Website to serve images from. IL_Dir
/srv/RoadTech/ImageLibrary/pod_cacheCache of re-coded image files.
/srv/RoadTech/ImageLibrary/pod_imagesOriginal document files.

SeLinux

Only applicable to Linux. Scripts will all run on a system with SeLinux installed and enforcing the Targeted policy.

Login accounts

Depending on what functionality you are installing on a server you will need a number of accounts creating. The scripts use variables for these accounts so you can set them to pretty much anything you like. doing so may however confuse our support department.

RT_Owner

Variable set to account code of user who "owns" the scripts. Historically this is also the account that owns the COBOL run time, and the Roadrunner programs and data.

RR_Owner

Account that owns the COBOL run time, Roadrunner programs and data. See RT_Owner.

IL_Owner

Variable set to account code of users who "owns" the image files.

Account Groups

RT_group

Set to group with permission to use scripts. Historically this is also the group that can invoke the COBOL run time.

IL_Group

Group with rights to view images. This is the group that the web server runs as. For Linux this is usually the group "apache".

RT_EdiGroup

Default value is datalink.

RR_ResourceGroup

Used in an AIX cluster. Value defaults to "roadtech".

Shared home

Scripts should work on a group of systems, that share /home, between them. By means of a cluster file system, NFS, or autofs.

RoadRunner cluster

Theoretically it is possible to run RoadRunner off a cluster, where you have several compute nodes sharing the data via a cluster file system. As of 2010 I do not know any one using this arrangement. It is far more efficient to run on one node with adequate processing power and a fast storage subsystem.

User credentials

Where it is available scripts use the tool getent to access security information, whether from local files, or a shared directory via LDAP. Rather that accessing /etc/passwd directly.

This works well in conjunction with the shared home support noted above.

Installation and configuration notes.

Is there more than 1 version?

Yes there have been a number of releases, these can be identified by the code displayed at the top left of the menu screens. The code is based on the revision date of the script rr85.sh.

These notes are for the late 2016 rewrite, unless you are QA testing you want these notes instead.

Do I have the right version for my system?

This depends on the version of both the operating system and roadrunner.

New version should support the following OS, to full or near full functionality.

The following should be supported with varying levels of functionality.

The version you are running can be distinguished by the date code displayed on the top left of the opening screen. If code ends "exp" it is a development version and should not be used on a production system.

Customers should in general be running a recent version.

Environment

Since the 2003 release the scripts have been written to use the library file setenv.lib, to configure options, and set environment variables to indicate the presence and locations of various hardware devices, and software packages.

If you are writing any scripts to use with RoadRunner see the notes from 2002-3.

Backups.

To be useful the contents of a backup must be consistent. This is particularly important for database type applications such as RoadRunner. The backup scripts make checks for this, they will fail and record the failure if any files within the sets being backed up change or are deleted during the backup.

All scripts that may change or remove files, must check for the existence of a backup flag, either global, per application, or specific to a single DATASET within an application. If the flag file exists then the script must exit gracefully without making changes.

All of the standard scripts will abort gracefully if a flag is found on any of these checks. No files shall, under any circumstances, be altered within the file tree of any DATASET, unless the corresponding flags for the DATAPATH, have been checked within the preceding second.

Dynamic menus

The menu options will change depending on :-

Example

An obvious example of dynamic menus is backup.

Fresh install

I have not tested this for a while.

Make sure that the system is set up in a secure and sane manner. Making changes once, you have applications installed and in use, is tricky and disruptive.

Have you considered.

Presuming that you are happy with your system proceed as follows.

Install

ISO

Mount CD.

Post install

Systemd

On a OS using systemd /var/run is a symbolic link to a tmpfs file system /run. Install provides a template systemd unit file and a script RTinit to set up the required entries on boot of the OS.

This applies to

Upgrading from a previous version

There is a fundamental issue, the upgrade is performed using the current programs. It the new version contains enhancements that require different configuration then, an old update routine may not know about them.

Wherever possible the update routine of the current program set is updated first. To know about forthcoming changes. For this reason you should always use the built in update system, and avoid skipping a large number of intermediate versions.

Where there is a major structural change, it is very strongly recommended that you update to the last incremental update of the previous set (19/8/2016) before attempting an upgrade to the new major version. Major changes came in in 1999, 2003, 2010, and this set for 2017. Take a look at History.html or Changes.html.

Updates between version are normally small and the routines are written to allow live updating. If however you are skipping a lot of versions, I would suggest that the update is done after a backup, and with users off the system.

Secondly the suite includes check routines to validate the configuration.

Preparation

Note current version, the version you are going to.

Note current configuration.

Load Update

As the Road Tech administrative user, run RTadmin. Select the utilities menu. The select the Administration menu. Select the Update Menu

Select the update method that goes with the provided update media.

Wait 3 minutes before proceeding to the next step.

Check configuration options

As the Administrative user. Go to the System utilities menu Select the option to "Show configuration"

If anything looks wrong, please provide a screen dump of the configuration, and what version you were upgrading from an to.

Exit the menus and fix via the system options file if possible.

Repeat if you changed anything.

Check Roadrunner configuration

As the Road Tech administrative user, run RTadmin. Select the utilities menu. The select the Dataset menu.

Select the option to "Show DATASET Settings">

If anything looks wrong, please provide a screen dump of the configuration, and what version you were upgrading from and to.

Exit the menus and fix via the roadrunner options file if possible.

Repeat if you changed anything.

If you have multiple datasets repeat for each set.

Validate configuration

As the Administrative user. Go to the System utilities menu Go to the Check menu.

Select the option to "Check links and ownerships"

This function checks things in phases. Some steps depend on what options have been installed.

StageAction
Obsolete filesRemove
symbolic links
rt.varIf missing create, if out of date create new template as rt.var.new, exit for user to reconcile.
RTadminPerm
rr.varIf roadrunner is installed check rr.var, if out of date or missing create new template as rr.var.new, exit for user to reconcile.
RR permissionsIf Roadrunner is installed set file permissions.
dataset.varPer data set variables. If missing create, if out of date create new template as dataset.var.new, exit for user to reconcile.
il.varOnly if rt.var indicates the ImageLibrary is installed.

Usage

The scripts are either command line based, or display a menu. There are some that can be either.

Exact options and behavior depend on who is running the utilities.

Command line

Command line scripts run from a prompt and may take optional arguments. They are mostly intended to run with out further user input.

Menu based

All of the scripts and menus take a similar format.

18/02/2003      AIX Ver 4 rel 2
Default Print Queue := postscript

        1  Media menu.
        2  Print services.
        3  Mail menu.
        6  hostman's programme menu.
        Q  Quit.


                [ ]

The top two lines contain a general data block giving a date code indicating the script version, the operating system and version, and the default print destination.

They are displayed by the library function menu_header

Additional header info may be displayed, that is relevant in the context of the menu.

For example the Roadrunner menus use the function menu_header_r to display information relevant to roadrunner.

RTedi uses the function menu_header_e to display relevant information at the top of the screen.

RTadmin or rr85

Main RoadTech Administration script, it is in some ways overworked :-)

Links are created in the program directory for the platform usually /usr/bin, under both names pointing to where this administrative program was installed.

The old name is reminder that these programs stared life back in 1990 purely to administer roadrunner on AIX.

When invoked on its own it displays a context sensitive menu. the exact options, depend on the type of system, the user who is running it, and what optional modules are installed on the system.

rr or RR

This is the script to start roadrunner. Links are created in the program directory for the platform usually /usr/bin, under both names pointing to where the administrative programs were installed.

Invoked on its own it will start roadrunner using the last DATASET used.

It may alternatively be invoked with an optional Parameter indicating the DATAPATH to use.
rr /usr/share/rr85

One other variant exists, if the variable definition
MENU=TRUE
appears in the file $rrdir/rr.var a menu of the available datasets will be built using the information from the file $rrdir/.datapaths.

RTedi

Road Tech EDI menu and scripts. RTedi