Appendix A - Descriptions of the SRB Client API
Appendix B - Descriptions of the DataCutter Client API
1) SRB server - The Storage Resource Broker (SRB) is a middleware that provides distributed clients with uniform access to diverse storage resources in a heterogeneous computing Environment. Storage systems handled by the current release of the SRB include the UNIX file system, archival storage systems such as UNITREE and HPSS, and database Large Objects managed by various DBMS including DB2, Oracle and Illustra.
2) Client library - Client applications are provided with a set of API for sending requests and receiving response to/from the SRB servers.
3) MCAT server - The meta data of the SRB system is managed by the MCAT server which is based on relational database technology. The MCAT server stores meta data associated with data sets, users and resources managed by the SRB. Detailed descriptions on the design of MCAT are given in http://www.npaci.edu/DICE/Software/SRB/mcat.html. Meta data includes information for access control, and data structures required for implementing "collection" (directory) abstraction. Since the MCAT is implemented with relational database technology, it can be extended beyond the capability of traditional file system, such as implementing a more complex access control system, proxy operations, and information discovery based on system level and application level meta data.
4) A set of UNIX like utilities (e.g., ls, cp, chmod, etc) for manipulating and querying collections and datasets in the SRB space.
5) A JAVA based SRB client GUI - The srbBrowser is a JAVA based SRB client GUI which can be used to perform a variety of client level operations including replication, copy and paste, registration of datasets and metadata manipulation and query, etc.
6) A logical view of data - The MCAT presents clients with a logical view of data sets stored in the SRB. Similar to the file name in the file system paradigm, each data set stored in SRB has a logical name, which may be used as a handle for data operation. Unlike the file system where the physical location of a file is implicitly implied in its path name through its mount point, the physical location of a data set in the SRB environment is logically mapped to the data sets. Therefore, the actual data of data sets belonging to the same collection may physically reside in different storage systems. A client does not need to remember the physical mapping of a data set. It is stored as the meta data associated with the data set.
7) The global logical name space Similar to file system paradigm, data sets can be arranged in a directory-like structure, which we call collection. Collections provide a logical grouping mechanism where each collection may contain a group of physically distributed data sets and/or sub-collections.
8) The container abstraction Due to the relative high overhead of creating/opening files in archival storage systems such as HPSS, they are not suitable for storing large amount of small files typically found in digital library systems. The container concept was specifically created to handle this type limitation. Through the use of containers, many small files can be aggregated in the cache system before storage in the archival storage system. A caching system has been designed to handle container I/O. The current version of the SRB server has been designed to handle multiple archival and cache storage resources.
9) "Server driven" parallel I/O - The "Server driven" parallel I/O scheme has been implemented throughout the SRB infrastructure. It is the preferred data transfer mechanism for data import, export, copy and replication. The parallel I/O uses the more efficient "mover" APIs and protocol to access HPSS resources.
10) A Mass Storage System (MSS) implementation within SRB - The system can be configured to have a pool of disk cache resources in the frontend and a STK tape silo library in the backend. Data files stored in this system are automatically staged to the disk cache whenever accessed and migrate to the tape library when more free cache space is needed.
11) Support four authentication schemes - The SRB can now support four authentication schemes - GSI, SEA (developed at SDSC), plain text password and encrypted password. GSI (Grid Security Infrastructure) is a security infrastructure based on X.509 certificates developed by the Globus group (http://www.npaci.edu/DICE/security/index.html). SEA is a authentication and encryption scheme developed at SDSC based on RSA key encryption scheme. (http://www.npaci.edu/DICE/Software/SEA/index.html)
12) Ticket - The Ticket abstraction has been implemented to facilitate the sharing of data among users. The Ticket implementation works as follows:
i) The owner of a data object or collection may grant read only access to a user or a group of users through issuance of tickets on the data object or collection.
ii) A ticket is a 10 character alphanumeric string and can be passed from the ticket issuer to the ticket users. A ticket user can then use this ticket to open and read data objects.
iii) A ticket can be issued to either MCAT registered or unregistered users. Unregistered users who wish to use a ticket to access a data object must use a special call to connect to the SRB sever. The normal user authentication will then be bypassed but the resulting connection has only limited privileges.
13) Logical resource implementation - the RESOURCE meta data has been modified to allow grouping of two or more physical resources into a resource group or logical resource. Logical resources can be used in more than one way. A logical resource can be used to group together a pool of disk cache and appears as one large disk cache, or it can be used to automatically replicate data in each of the physical resource belonging to the logical resource.
13) Data replication The SRB supports data replication. The following gives some details of the implementation:
i) Automatic replication of new data - When creating a data set, a client can specify a logical resource rather than a physical resource as the resource input. With the COPIES=ALL flag set, a copy of the data will be created in each of the physical resource belonging to the logical resource.
ii) Replication of existing data - A user can use the Sreplicate command or the srbObjReplicate() API to replicate existing data to the specified resource.
14) Replica synchronization - Copies of replica can modified individually and replica of an object can be out of sync. Now, the most recently modified copy will be marked as "dirty". A new command - "Ssyncd" and API - srbSyncData() has been created to synchronize all copies (replica) of an SRB object with the dirty copy. A "%" character to the left of the file name of a "Sls -l" output indicates the given copy is "dirty".
15) Proxy operation - Some operations can be more efficiently done by the server without much involvement by the client. An example is the copy operation where it is more efficient for the server to do all the read and write operations on behalf of the client than passing the data read to the client and then for the client passing it back to the server for the write operation. A framework for handling proxy operations has been set up to facilitate the building of new proxy functions. These proxy functions can be in the form of either direct codes in the SRB server or individual executables which are accessible by users the the Spcommand utility or the srbExecCommand() API
16) Bulk Ingestion - A bulk ingestion mode has been implemented. This feature enables, a user to create a container and register the requisite metadata in parallel and in bulk. The system has been tested at SDSC to ingest into SRB medium-sized data files in large directory structures at more than 400 files per second. The feature is available through the command called Sbload.
17) Auditing - Auditing provides a facility to audit the usage of data sets.
18) srbMon - a SRB server monitoring daemon. The srbMon is a SRB monitoring daemon which remotely monitors and logs all SRB servers belonging to a federation, and attempts to restart any servers that were found to be down.
19) UNIX Transparency - This involves using the PRELOAD feature of the Solaris and Linux platform to trap UNIX I/O system calls and replaced them with SRB I/O calls so that regular UNIX commands and applications can be used to access and manipulate data stored in SRB without recompiling. Please read README.transparency for more info on this subject.
20) Support file size larger than 2GB - The SRB software will now support file size larger than 2 GB for the Solaris, Linix, Aix and C90 platforms.
21) Prespawning server processes - To improve interactivity, the SRB servers can now be configured to prespawn a specific number of server processes.
22) Platforms supported The SRB has been implemented on AIX, Sun Solaris(32 and 64 bits address), Linux 32 and 64 bits address), SunOs, DEC OSF, SGI, MacOS X and the Cray C90. The MCAT catalog now runs on both the Oracle and DB2 DBMS.
1.2 An Overview of the SRB architecture
The Storage Resource Broker (SRB) is a middleware that provides distributed clients with uniform access to diverse storage resources in a heterogeneous computing Environment.
Figure 1 gives a simplified view of the SRB architecture. The model consists of three components: the meta data catalog (MCAT) service, SRB servers and SRB clients, connected to each other via network.
The MCAT stores meta data associated with data sets, users and resources managed by the SRB. The MCAT server handles requests from the SRB servers. These requests include information queries as well as instructions for meta data creation and update.
Client applications are provided with a set of API for sending requests and receiving response to/from the SRB servers. The SRB server is responsible for carrying out tasks to satisfy the client requests. These tasks include interacting with the MCAT service, and performing I/O on behalf of the clients. A client uses the same common API to access every storage systems managed by the SRB. The complex tasks of interacting with various types of storage system and OS/hardware architecture, are handled by the SRB server.
1.3 Implementation details
1.3.1 The SRB process model
Figure 2 depicts the SRB process model. The design of the SRB server is based on the traditional network connected client/server model. It is composed of two separate servers, SRB Master and SRB Server. The SRB Master is the main daemon listening continuously on a well-known port for connection requests from clients. Once a connection from a client is established and authenticated, it forks and execs a copy of the SRB Server, which we call SRB agent, to service the connection. From that point onward, the client and the SRB agent communicate using a different port and the SRB Master goes back to listening for more connections. A client can use the same SRB agent to service multiple requests.
Client applications communicate with the SRB agent using a set of API via TCP/IP sockets. The client library sends requests using pre-defined request stubs to the SRB agent, and receives and parses replies from the SRB agent. The model is distributed in the sense that clients and servers may be running on different hosts.
1.3.2 Federated SRB servers
A group of distributed SRB servers coordinating with each other to service client requests, can be configured to form a federation. The advantages of a federated SRB are as follows:
o For technical and policy reasons, various storage systems must run on different hosts.
o Improved performance.
o Improved data reliability and availability Replica of data may be stored in different storage systems and on different hosts.
Figure 3 depicts the working of a federated SRB, which consists of two SRB masters, running on hosts A and B. The SRB server on A is MCAT enabled meaning that it can talk to the MCAT server. In this example, a client "open data set" request may result in the following steps:
1) After completing the connection sequence with the SRB master on host A, the client sends a request to the SRB agent on host A to open a data set for reading.
2) The SRB agent makes a MCAT call passing the clients user ID and the data set name to the MCAT server to check if the client has the proper access permission to the data set. If it checks out, the MCAT call returns a data structure that contains the physical location where the data set is stored. The physical location data structure includes the host name, the storage system type (e.g., UNIX, HPSS, DB2 Large Object, etc) and the path name (e.g. a unix file path).
3) The SRB agent on host A realizes the requested data set is on host B and carries a remote open on behalf on the client passing along the storage system type and the path name to the SRB agent on host B.
4) The SRB agent on host B uses the storage system type to invoke the appropriate low level storage driver to invoke to handle the open call, passing along the path name to be opened. Upon completion of the open call, the SRB agent on host B returns the opened file descriptor or error code to the SRB agent on host A.
5) If the open call is successful, the SRB agent on host A stores the returned file descriptor and other information such as host name, etc in a internal data structure and pass back the pointer to this data structure to the client. The client can then use this pointer in subsequent read calls; if the open call is not successful, an error code is returned instead.
1.3.3 - The SRB agent design details
As described above, the SRB is designed based on the traditional client/server model. Client applications are provided with a set of simple API to communicate with the SRB servers. The main daemon, the SRB master, is responsible for the simple tasks of listening for incoming connections, and spawning a SRB agent to handle each connection once the client is authenticated. The SRB agents are responsible for receiving and servicing all subsequent client requests.
Figure 4 gives a simplified view of the SRB agent design.
At the top is the "dispatcher" module, which listens for incoming client requests and dispatches the requests to the proper request handlers. The "dispatcher" is also responsible for returning the results to clients.
Clients are provided with two sets of API, the high-level and low-level API. The high-level API handles data sets that use the MCAT server for meta data management, while the low-level API handles data sets that do not use the MCAT. When using the high-level API to create a data set, the data set is automatically registered in MCAT and the MCAT keeps all relevant information associated with the data set until the data set is deleted. On the other hand, data sets created using the low-level API are not registered in MCAT. When accessing a data set using the low-level API, a client needs to provide all required information such as the physical location (host address and full path name) and the storage system type of the data set. Detailed descriptions of the client API are given in Appendix A.
The high-level request handler of the SRB agent handles all high-level client requests, and the low-level request handler handles low-level client requests.
Through the high-level API, a client can access the following type of services provided by the SRB agents:
1) Create and remove data sets.
2) Perform I/O operations on data sets.
3) Manipulate and query meta data associated with the data sets and collections.
4) Manipulate and query meta data associated with other objects such as users, user-groups and resources managed by the SRB.
All these operations require the high-level request handler to interact with the MCAT service to manipulate and query the meta data stored in MCAT. In addition, for requests that perform operations on data set (1) and 2)), low-level requests will be generated and dispatched to the low-level request handler for further processing.
A set of library calls is used by the high-level request handler to interact with the MCAT service. These calls allows the SRB agent to register, unregister and modify meta data associated with data sets, users and storage resources, and to query the meta data. Basically, these routines translate easy-to-understood input parameters into complex SQL queries and send them to the MCAT server. Upon receiving the query results from the MCAT server, these routines parse the query results into simple forms before returning them to the caller.
The type of MCAT call made by the SRB agent depends on the type of client call the SRB agent is trying to handle. For example, to handle a "create data set" call, a query is sent to the MCAT server on the requested storage resource. If the client is allowed to create data set in this storage resource, it returns the physical location (host name, directory path) and storage system type of the requested storage system to the caller. Using the returned directory path, a unique physical path name (e.g., UNIX or HPSS path name) is generated. A low-level "create" call is generated and dispatched, passing on the host name, path name and storage system type parameters to the low-level request handler. If the low-level create call is successful, a "register data set" call is made to the MCAT server, passing on the data set name, the collection in which to put the data set, the storage resource, the physical path name and the user name. The MCAT registration routine does a few sanity checks (such as the permission to create data sets in the specified collection, duplicate data set name, etc). If the registration is successful, a set of meta data associated with the newly created data set is added to the MCAT catalog. Finally, the SRB agent returns a handle to the client. The client can then use this handle for subsequent read/write calls. If the data set register call is unsuccessful, a low-level unlink call is dispatched and an error is returned to the client.
The low-level request handler handles low-level client requests dispatched directly from the request dispatch, or high-level client requests passed indirectly through the high-level request handler module. This module performs the basic I/O operations on behalf of the client on the various storage systems managed by the SRB agent. The common parameters passed to this module are the resource location (host name), the storage system type (UNIX, HPSS, DB2, etc) and the physical path name (UNIX path name). If the requested storage system is not local, it dispatches a remote call to the appropriate SRB agent. Otherwise, it calls its low-level drivers to carry out the I/O operation.
Two types of drivers, the file-type and the DB-type, are supported. The file-type drivers provide UNIX-like file I/O interfaces to the file system of the storage systems, i.e., UNIX, UniTree and HPSS. Other non-file system type drivers, such as FTP and HTTP, are also grouped under this category because the similarities in the I/O interface. I/O operations supported by the file-type drivers are create, open, close, unlink, read, write, sync, seek, stat, chmod, mkdir, opendir, closedir and readdir.
The DB-type drivers provide I/O interface to DB large objects stored in DBMS, i.e., DB2, Oracle and Illustra. . I/O operations supported by the DB-type drivers are create, open, close, unlink, read, write and seek.
1.3.4 - The SRB container design
Due to the relative high overhead of creating/opening files in archival storage systems such as HPSS, they are not suitable for storing large number of small files typically found in digital library systems. The container concept was specifically designed to circumvent this type of limitation. The idea is through the use of containers, many small files can be aggregated in the cache system before storage in the archival storage system. The metadata for translating an inContainer object to file path, offset and size are kept in the MCAT and the SRB I/O drivers have been adapted to handle inContainer objects.
A caching system has been designed to handle container I/O. All container writes are done to the cache copy first. When a client is done filling up the container, a syncContainer call can be made to write the entire container copy to the archival storage system. For reading an object that has been stored in a container (inContainer object), if the container is not already on cache, the SRB server will stage the entire container to the cache first. Reading is done only on the cache copy. The metadata for translating an inContainer object to file path, offset and size are kept in the MCAT and the SRB I/O drivers have been adapted to handle inContainer objects.
The current version of the SRB server has been designed to handle multiple archival and cache storage resources. Multiple storage resources for containers was implemented through the use of logical resource. At the time of container creation, a logical resource is associated with the container being created through the srbContainerCreate() API.
Logical resource is an abstraction that contains one or more physical resources. Physical resource is a SRB abstraction that describe the underlying storage resources that can be accessed by the current SRB server configuration. In MCAT, a set of attributes is associated with a physical resource. Among other things, these attributes identify the location of the resource (host address and the path of the SRB vault), the type of resource (UNIX file system, HPSS file system and DB large objects), and the resource class (cache or permanent, primary or normal), etc.
The logical resource used to create a container should contain both classes of physical resources: "permanent/archival" (e.g., HPSS) and "cache" (e.g., UNIX disk file system) resources. A "cache" copy can be purged while a "permanent" copy is not purgable. Multiple resources for each resource class are allowed in which case, one and only one must be designated as "primary" for each resource class.
All container I/Os are done only to the copy on the "cache" type resource. If a "cache" copy does not exist (purged), the SRB server will attempt to stage a copy from the primary permanent resource to the primary cache resource first. If the primary permanent copy is not available (system or device down), other permanent copies will be used. Similarly, if the primary cache is not available, other cache resources belonging to the same logical resource will be used. It should be noted that the staging is done for the whole container rather than individual container segments. Once a "cache" copy is found or staged, read/write I/O can be carried out on objects contained in the container. If the cache copy has been modified, the srbSyncContainer() call or the Ssyncont command can be used to synchronize the cache copy to the archival resource.
The following software has been developed to support containers:
API - Five new APIs, srbContainerCreate(), srbRmContainer(), srbGetContainerInfo(), srbSyncContainer() and srbReplContainer() have been created to create, remove, query container info, synchronize and replicate(or stage) copies of a container.
Command line utilities - Five command line utilities: Smkcont, Srmcont, Ssyncont, Slscont and Sreplcont to create, remove, sync, list meta-data and replicate (stage) copies of a container. The Slscont command allows the listing of all containers owned or accessible by a user as well as listing all inContainer objects stored in a given container.
2.0 The SRB client API
2.1 Overview
Clients are provided with a set of API for sending requests and receiving response to/from the SRB servers.
A majority of the API performs some kind of operations on data sets. Based on the type of data sets they operate on, these API may be classified into two groups, the high-level and low-level API. The high-level API handles data sets that use the MCAT server for meta data management, while the low-level API handles data sets that do not use the MCAT. When using the high-level API to create a data set, the data set is automatically registered in MCAT and the MCAT keeps all relevant information associated with the data set until the data set is deleted. On the other hand, data sets created using the low-level API are not registered in MCAT.
The low-level API can be subdivided further into file-type and DB-type API because of the different types of operation they support.
The rest of the API handles client/server connections, interacts with MCAT for information query and meta data creation and update.
2.2 Description of API
A brief description of the SRB client API is given below. The API are grouped into six categories according to functionality and the type of data set they operate on. A more detailed description of the API can be found in Appendix A.
1) Client/server connection API This set of API handles opening and closing connection to a SRB server.
clConnect - Initiate a connection to a SRB server.
tiUserConnect Initiate a connection to a SRB server by a TICKET USER. Normal authentication will be bypassed for a TICKET USER.
clFinish - Close the current connection and free the srbConn data structure.
clErrorMessage - Return the ErrorMessage of a connection. This is the error message returned by the SRB server from the current client call.
2) High-Level API:
srbObjOpen - Open a SRB data set.
srbObjOpenWithTicket - Open a SRB data set using a ticket.
srbObjCreate - Create a SRB data set.
srbObjClose - Close an opened data set.
srbObjUnlink - Unlink a data set.
srbObjRead - Read a block of data from a data set into buffer.
srbObjWrite - Write content of buffer to a data set.
srbObjSeek - Change the current read or write location on an data set.
srbObjSync - Sync an opened SRB object.
srbObjStat - Stat an SRB object.
srbObjStat64 - Stat an SRB object and put results in a stat64 struct.
srbObjGetdents - The SRB equivalent of the UNIX getdents call.
srbObjGetdents64 - The SRB equivalent of the UNIX getdents64 call.
srbObjProxyOpr - Proxy Operation request.
srbExecCommand - Remote execution of command.
srbObjReplicate - Make a copy of a data set.
srbObjMove - Move a copy of a data set to a new location.
srbCreateCollect - Create a SRB collection.
srbListCollect - List a SRB collection.
srbModifyCollect - Modify a SRB collection.
srbIssueTicket - Issue a ticket.
srbRemoveTicket - Cancel a ticket.
srbContainerCreate - Create a container.
srbRmContainer - Remove a container.
srbSyncContainer - Synchronize copies of a container.
srbReplContainer - Replicate/stage a container.
srbObjGet - Export a file from SRB to local file system using parallel I/O.
srbObjPut - Import a file from local file system to SRB using parallel I/O.
srbSyncData - Synchronize all copies (replica) of an SRB object with the most recently modified version
3) Low-Level File-type API:
srbFileOpen - Opens a file.
srbFileCreate - Create a file.
srbFileUnlink - Unlink a file.
srbFileClose - Close a file.
srbFileRead - Read a block of data from a file into buffer.
srbFileWrite - Write content of a buffer to a file.
srbFileSeek - Change the current read or write location on a file.
srbFileSync - Sync a file.
srbFileStat Get the status of a file.
srbFileFstat Get the status of a file descriptor.
srbFileMkdir - Create a new directory.
srbFileChmod - Change the mode of a file or directory.
srbFileRmdir - Remove a directory.
srbOpendir - Opens a directory
srbClosedir - Close an opened directory
srbReaddir - Read a directory entry
srbSetStorAttri - Set Attributes for a Storage System; establish connection.
Only used by FTP file type,
4) Lower-level DB-type API:
srbDbLobjOpen - Open a DB Large Object.
srbDbLobjCreate Create a DB Large Object.
srbDbLobjClose - Close a DB Large Object.
srbDbLobjRead - Read a block of data from a DB large object into buffer.
srbDbLobjWrite - Write content of a buffer to a DB large object.
srbDbLobjSeek - Change the current read or write location of a DB large object.
srbDbLobjUnlink - Unlink a DB large object.
5) Lower-level DB table object API:
srbDbTableOpen - Open a DB Table Object.
srbDbTableCreate Create a DB Table Object.
srbDbTableClose - Close a DB Table Object.
srbDbTableRead - Read a block of data from a DB Table object into buffer.
srbDbTableWrite - Write content of a buffer to a DB Table object.
srbDbTableSeek - Change the current read or write location of a DB Table object.
srbDbTableUnlink - Unlink a DB Table object.
6) API that interact with MCAT This set of API interacts with MCAT for information query and meta data creation and update.
srbGetDatasetInfo - Get some default Info on a SRB data set.
srbGetDataDirInfo - Get arbitrary meta data info on users, data sets or resources by querying the MCAT catalog.
srbRegisterDataset - Register a SRB data object.
srbUnregisterDataset - Unregister a SRB data object.
srbSetAuditTrail - Setting and Unsetting Audit Trail.
srbModifyDataset - Modify a SRB data set
srbChkMdasAuth - Authenticate a userName/passwd.
srbChkMdasSysAuth - Authenticate a userName/passwd for sys admin access.
srbRegisterUserGrp - Register a user group
srbRegisterUser - Register a user.
srbModifyUser - Modify a user info.
srbGetPrivUsers - Read the privileged users list.
srbGetMoreRows - Get more rows of result from a srbGetDatasetInfo, srbGetDataDirInfo, srbListCollect or srbGetPrivUsers call.
freeSqlResult - Free any memory associated with a SQL result struct including the SQL result struct itself.
clearSqlResult - Clear the memory associated with a SQL result struct, but the SQL result struct is not freed.
printSqlResult - Print the content of a SQL result struct.
srbGetContainerInfo - Query metadata associated with a container.
srbRegisterLocation - Register/create a new location.
srbIngestToken - Ingest/create a new Token of type ResourceType, DataType,UserType, Domain, or Action.
srbRegisterResource - Register/create a new resource (physical or compound).
srbRegisterLogicalResource - Register/create a new logical resource.
srbRegisterReplicateResourceInfo - Add another physical resource to a logical or compound resource.
srbDeleteValue - Delete/remove a value of type location, user, or resource (other types are implemented but untested).
srbSetupSessionPublicKey - Get the MCAT-enabled server's public key in preparation for transferring encryptioned information.
srbSetupSession - set up a session (for encryption) with the MCAT-enabled server.
6) Miscellaneous SRB API:
srb_perror - Emits an SRB error message string corresponding to the input error code.
srbVaultInfo - Get Info on the SRB storage vault.
srbFreeVaultInfo - Free memory taken by the vaultQueElement link list.
srbPrintVaultInfo - Print out the VaultInfo link list.
srbHostConfig - Get the storage vault information on a SRB host.
srbPrintHostInfo - Print the HostInfo link list.
srbFreeHostInfo - Free memory taken by the clHostElement link list.
3.0 Setting up Client Environment
3.1 Building and installing the SRB client library
This section describes the steps for building and installing the SRB client library libSrbClient.a.
It should be noted that if the SEA authentication scheme is to be used and if the SEA library (libsea.a) does not already exist on your build platform, the SEA software which can be obtained at the URL http://www.npaci.edu/DICE/Software/SEA/index.html, must be built first.
1) Edit the mk/mk.config file. All configurable parameters for building the SRB server and the client library are defined in this file. The parameters are self-explanatory through the comments given in this file. Some of the more important parameters are discussed below:
installDir - The absolute path of the SRB install directory.
PORTNAME - The OS platform of this SRB port. Currently, the SRB software runs on 8 platforms. i.e., valid PORTNAME's are : PORTNAME_solaris, PORTNAME_sunos, PORTNAME_aix, PORTNAME_linux, PORTNAME_osx, PORTNAME_alpha, PORTNAME_c90 and PORTNAME_sgi.
SRB_LARGEFILE64 - defines whether the 64 bit file size is supported by the underlining driver of this SRB server. Current, 64 bit file size is supported by the PORTNAME_solaris, PORTNAME_aix, PORTNAME_linux and PORTNAME_c90 platforms..
MDAS_AUTH - defines whether the MDAS authorization scheme will be supported for authentication. If used, the user/passwd pair registered with the MDAS catalog will be used to authenticate a user. Comment it out if your SRB server should not support MDAS authorization. It is ON (not commented out) by default.
SEA_AUTH - defines whether SEA authorization scheme will be supported. The software can be configured to support both SEA_AUTH and MDAS_AUTH. This is OFF (commented out) by default.
LIB_SEA - Is needed only if SEA_AUTH is defined. LIB_SEA specifies where the SEA client library is located. This library can be built from the SEA source branch in this package. This is commented out by default.
GSI_AUTH - defines whether GSI authorization scheme will be supported. The software can be configured to support GSI_AUTH, SEA_AUTH and MDAS_AUTH. This is OFF (commented out) by default.
LIB_GSI_AUTH - Is needed only if GSI_AUTH is defined. GSI_AUTH specifies where the GSI client library is located. This is commented out by default.
ADR_PROXY - defines whether the ADR DataCutter proxy operation will be supported.
JAVA_GUI and javaDir - JAVA_GUI defines whether the srbBrowser should be built. javaDir specifies the directory where the JDK software is installed. (e.g. /usr/local/apps/Java). In addition, if JAVA_GUI and javaDir are set, you must setenv CLASSPATH to "." and the path where the swing.jar is installed on your machine. e.g. setenv CLASSPATH /local/generic/lib/java/swing/swing.jar:.
Note: This software must be built using JDK 1.1.6 or 1.1.7 and swing 1.0.2.
2) "cd" to the the main SRB directory and type in "gmake clean" and then "gmake" to make the SRB software.
3) Type in "gmake install" to install the software in the $(installDir) directory. This procedure installs the following module in the $(installDir) directory:
bin/libSrbClient.a - The client library.
3.2 Setting up the client user environment
Before a user can connect to the SRB server, the following user environment setup must be carried out:
1) The ~/.srb/..MdasEnv file - A template of the file can be found in utilities/envFiles/.MdasEnv of the distribution package. This file contains four parameters:
a) mdasCollectionHome - This is equivalent to the user's home directory (collect) and is created by the MCAT administrator at the time of the user registration.
b) mdasDomainHome - This is the domain name associated with the user. This domain name is assigned to the user at the time of the user registration.
c) srbUser - This is the user name of the user. This name is assigned to the user at the time of the user registration.
d) srbHost - This is the default host address of the SRB server when a SRB client initiates a SRB connection.
e) AUTH_SCHEME - This parameter defines the authentication scheme to use. Valid input values are:
'PASSWD_AUTH' - Use the plain text password Mdas Authentication. If this option is chosen, an additional file containing the password the the .MdasAuth is required.
'ENCRYPT1' - Same as PASSWD_AUTH except the passwords are encrypted using a self contained scheme.
'SEA_AUTH' - Use the SEA authentication scheme.
'SEA_ENCRYPT' - Use the SEA authentication scheme plus encryption.
'GSI_AUTH' - Use the GSI authentication scheme. If this option is chosen, an additional parameter SERVER_DN given below is required.
'GSI_SECURE_COMM' - Use the GSI authentication scheme and use the GSI I/O library for all socket communication between client and server. If this option is chosen, an additional parameter SERVER_DN given below is required.
If this parameter is not defined, the Mdas Authentication scheme will be used.
2) Setting up user authentication - The SRB software can be built to support the GSI, SEA and password authentication schemes. The AUTH_SCHEME parameter in the .MdasEnv file is used to define the authentication scheme to use. Each authentication scheme requires a different setup by the user.
a) SEA authentication setup A client needs to register him/herself with the SEA server by running the following command:
seaauth reg
This is a one time exercise which in effect, triggers the registration of the user and obtaining a password encrypted private key which will be placed in a file named ~/.SEAuuuuu@ddddd (where uuuuu is the user ID and ddddd is the user domain name). Then type in the following command:
seaauth auto
to generate an unencrypted private key in the /tmp directory which can be used directly by the SEA library routines to authenticate the user.
b) Password authentication setup - This is a plain text password setup where the password is generated at the time of user registration with the MCAT catalog. Upon receiving this password from the MCAT administrator, a user should place it in a file named ~/.srb/.MdasAuth which will be used directly by the SRB library to authenticate the user.
c) GSI authentication setup - Please read README.gsi and follow the HTTP links for GSI client environment setup suggested by in the document. Once the certification/key has been obtained and converted to PEM format for GSI use, look into the cert pem file with a text editor and locate the user "Distinguish Name" string. The string representing the "Distinguish Name" should look like the following:
subject=/C=US/O=NPACI/OU=SDSC/UID=srb/CN=Storage Resource Broker/Email=srb@sdsc.edu
Copy this string and send it to the SRB administrator for registration.
NOTE: Currently, the SDSC SRB servers are configured to handle both Mdas and SEA authentication.
3.3 Locations of the SRB client library and header files
The SRB client library is installed in $(installDir)/bin/libSrbClient.a.
A client program should include the header file "srbClient.h" and the compilation header file search directive should include the following directories:
$(buildDir)/src/include
$(buildDir)/src/catalog/include
4.0 Client Programming
4.1 Connecting and disconnecting to the SRB Server
The client uses the clConnect(char* srbHost, char* srbPort, char* srbAuth) call to connect to the SRB server. srbHost specifies the host where the SRB server is running. If the input value in NULL, it will use the env variable "srbHost" if it is defined. If it is not defined, the ~/.srb/.MdasEnv file will be checked next. If not, it will use the hostname of the client machine.
The srbPort input defines the port number (in character string) to connect to the SRB server. If the input value in NULL, it will use the env variable "srbPort" if it is defined. If not, it defaults to "5558".
The srbAuth input is used to define the password for MDAS or SEA authentication. For SEA authentication, this is the password used by the SEA library to decrypt the encrypted private key stored in the file ~/.SEAuuuuu@ddddd (where uuuuu is the user ID and ddddd is the user domain name). This input is not needed if an unencrypted private key is available in the /tmp directory (generated using the 'seaauth auto' command). To provide additional flexibility, a client may also use the env variable "srbAuth" to specify the password. A client may also supply the password in the ~.srb/.MdasAuth file. This file should only be readable by the file owner. If a client uses more than one method to specify the password, the value given in the clConnect () call will take precedent, then the env variable "srbAuth", and lastly, the ~.srb/.MdasAuth file.
After a connection to the SRB server has been opened, the client can then use the connection to issue requests to the SRB server.
Upon completion, the client should issue a clFinish(conn) call to
close the connection to the SRB server.
4.2 Error handling
The SRB uses two methods to return error conditions to the client. The first is the commonly used method of returning a value with the client API. In general, for integer type return values, a negative returned value implies error. For string type returns, a NULL returned value implies error. The error code for integer type returns are given in include/srb_error.h . Typically, these are error codes that are internally generated by the SRB and MCAT. Given the integer error code, an error message string can be generated using the srb_perror() API or the Serror utility.
The SRB server can also return a more informative error conditions in the form of a text string, but the client must use the clErrorMessage(conn) call to retrieve the message. Typically, it contains a short message describing the error condition and often times, the error codes from the underlying storage system (e.g., HPSS, UniTree, etc) is also included.
4.3 Client programming examples:
Examples of SRB client can be found in the test/examples/src directory of the distribution.
The following are examples of using the High-Level Object API:
testsrb.c - Example of using a mixed bag of High Level Object API - srbSetAuditTrail, srbObjOpen, srbObjRead, srbObjClose, srbListCollect, srbGetMoreRows, srbGetDatasetInfo, etc.
srbcat.c - Cat a High Level Object - srbObjOpen, srbObjRead, srbObjClose.
testcoll.c - Example of using High Level Object API to manipulate collections - srbListCollect, srbGetMoreRows, srbCreateCollect, srbModifyCollect, etc.
testcre.c - Example of creating High Level Object - srbObjCreate, srbObjWrite, srbObjClose, etc.
testunlink.c - Example of removing High Level Object - srbObjUnlink.
testrepl.c - Example of making replicate of an object - srbObjReplicate.
testmove.c - Example of moving a High Level Object from one physical location to another - srbObjMove.
testproxy.c - Example of a proxy copy operation - srbObjOpen, srbObjCreate, srbObjProxyOpr.
issuetick.c - Example of issuing a ticket - srbIssueTicket.
rmtick.c - Example of removing a ticket - srbRemoveTicket.
usetick.c - Example of using a ticket to read a dataset - srbObjOpenWithTicket and srbObjRead.
testtick.c - Example of a using a mixed bag of ticket type API - srbIssueTicket, tiUserConnect, clConnect, srbObjOpenWithTicket, srbRemoveTicket, srbObjRead, srbObjClose.
testddir.c - Example of using the srbGetDataDirInfo call to query MCAT information - srbGetDataDirInfo.
testuser.c - Example of using sys admin API to query and manipulate MCAT user information - srbChkMdasSysAuth, srbRegisterUser, srbRegisterUserGrp and srbRegisterUser.
The following are examples of using the Low-Level File-type API:
testexf.c - Example of using a mixed bag of Extended File API - srbFileMkdir, srbFileRmdir, srbFileChmod, srbFileUnlink, srbFileCreate, srbFileWrite, srbFileClose, srbFileStat, etc.
exfls.c - Example of using srbOpendir, srbReaddir and srbFileStat.
exfrm.c - Remove an extended file - srbFileUnlink.
testftp.c - Example of using the File-type API for the FTP file type.
testhttp.c - Example of using the File-type API for the HTTP file type.
The following are examples of using the Low-Level DB-type Large Object API:
testLobj.c - Example of using the DB-type Large Object calls - srbDbLobjCreate, srbDbLobjUnlink, srbDbLobjOpen and srbDbLobjClose.
Other examples can be found in the test/testsuite and utilities/src directories.
The following gives a few specific examples of using the SRB client API:
1) An example for using clConnect(), clFinish() and clErrorMessage():
#include "srbClient.h"
/* Use the srbAuth and srbHost values defined in ~/.srb/.MdasEnv */
#define SRB_AUTH NULL
#define HOST_ADDR NULL
main(int argc, char **argv)
{
srbConn *conn;
int dirDesc;
int storSysType;
.
.
.
/* Connect to the SRB server */
conn = clConnect (HOST_ADDR, NULL ,SRB_AUTH);
/* check to see if the connection was successful */
if (clStatus(conn) != CLI_CONNECTION_OK) {
fprintf(stderr,"Connection to SRB server failed.\n");
fprintf(stderr,"%s",clErrorMessage(conn));
exit_nicely(conn);
}
/* Open a file-type directory */
dirDesc = srbOpendir (conn, storSysType, HOST_ADDR, ".");
if (dirDesc < 0) {
fprintf(stderr,"Unable to srbOpendir . \n");
fprintf(stderr,"%s",clErrorMessage(conn));
exit_nicely(conn);
}
/* Disconnect */
clFinish(conn);
2) An example for creating a high-level data object and writing to it.
This example assumed a connection has already been made using example 1). The object is created in a storage resource named "unix-sdsc". This resource must have already been registered in MCAT.
#define DATATYPE "ascii text"
#define RESOURCE "unix-sdsc"
#define COLLECTION "/srbtest"
.
.
int in_fd, out_fd;
int nbytes, tmp;
char buf[BUFSIZE];
.
.
/* Create a data object with objID = argv[1] */
out_fd = srbObjCreate (conn, MDAS_CATALOG, argv[1],
DATATYPE, RESOURCE ,COLLECTION, NULL, 0);
if (out_fd < 0) { /* error */
fprintf(stderr, "can't create obj \"%s\", status = %d\n", argv[1],
out_fd);
fprintf(stderr,"%s",clErrorMessage(conn));
exit_nicely(conn);
}
/* Open a local file with filename = inFileName. */
in_fd = open (inFileName, O_RDONLY, 0);
if (in_fd < 0) { /* error */
fprintf(stderr, "can't open file\"%s\"\n",
inFileName);
exit_nicely(conn);
}
/* Read from the local file and write to the just created data object */
while ((nbytes = read(in_fd, buf, BUFSIZE)) > 0) {
/* Write to the data object */
tmp = srbObjWrite(conn, out_fd, buf, nbytes);
if (tmp < nbytes) {
fprintf(stderr, "Error: Read %d bytes, Wrote %d bytes.\n ",
nbytes, tmp);
exit_nicely(conn);
}
}
srbObjClose (conn, out_fd);
close (in_fd);
3) An example for creating and listing collections.
This example creates a new collection in a parent collection, and then the parent collection is listed.
#Include "srbClient.h"
#define PARRENT_COLLECTION "/srbtest"
#define NEW_COLLECTION "foo"
#define ROWS_WANTED 2
char * parentCollection, newCollection;
int status;
mdasC_sql_result_struct *collResult;
.
.
parentCollection = PARRENT_COLLECTION;
newCollection = NEW_COLLECTION;
/* Create a new collection */
if ((status = srbCreateCollect (conn, MDAS_CATALOG, parentCollection,
newCollection)) < 0) {
fprintf(stderr, "can't srbCreateCollect, status = %d\n", status);
exit_nicely(conn);
}
/* List the parent collection */
collResult = malloc (sizeof (mdasC_sql_result_struct));
if ((status = srbListCollect (conn, 0, COLLECTION, "R",
collResult, ROWS_WANTED)) < 0) {
fprintf(stderr, "can't srbListCollect, status = %d\n", status);
exit_nicely(conn);
}
printSqlResult (collResult); /* print the result */
clearSqlResult (collResult); /* clear the content of collResult */
/* Any more result ? */
while (collResult->continuation_index >= 0) {
/* Yes, we have more. Get the next ROWS_WANTED rows */
if ((status = srbGetMoreRows(conn, 0,
collResult->continuation_index, collResult, ROWS_WANTED)) < 0)
break;
/* Dump the result */
printSqlResult (collResult);
clearSqlResult (collResult);
}
4) An example for a proxy copy operation.
This example performs a proxy copy operation. The source data object is opened and the destination data object is created first. Then a proxy copy operation is initiated.
#include "srbClient.h"
#define DATATYPE "ascii text"
#define RESOURCE "unix-sdsc"
#define COLLECTION "/srbtest"
.
.
int i, nbytes, tmp, in_fd, out_fd;
char in_fd_str[SML_BUF_SZ], out_fd_str[SML_BUF_SZ];
int status, nbytes;
.
.
.
/* open the source object with objID = argv[1] */
in_fd = srbObjOpen (conn, argv[1], O_RDONLY, COLLECTION);
if (in_fd < 0) { /* error */
fprintf(stderr, "can't open obj\"%s\"\n", argv[1]);
exit_nicely(conn);
}
/* Create the destination object with objID = argv[2] */
out_fd = srbObjCreate (conn, MDAS_CATALOG, argv[2],
DATATYPE, RESOURCE, COLLECTION, NULL, 0);
if (out_fd < 0) { /* error */
fprintf(stderr, "can't create obj \"%s\", status = %d\n", argv[2],
out_fd);
fprintf(stderr,"%s",clErrorMessage(conn));
exit_nicely(conn);
}
/* Convert fd to string */
sprintf (in_fd_str, "%d", in_fd);
sprintf (out_fd_str, "%d", out_fd);
/* Do a proxy copy operation */
nbytes = srbObjProxyOpr (conn, OPR_COPY, in_fd_str, out_fd_str);
printf ("\n\nBytes written: %d\n", nbytes);
srbObjClose (conn, out_fd);
srbObjClose (conn, in_fd);
5) An example for issuing tickets and using tickets to open and read data objects.
A ticket can be issued to either MCAT registered or unregistered users. Unregistered users who wish to use a ticket to access a data object must use a special call to connect to the SRB sever. The normal user authentication will then be bypassed but the resulting connection has only limited privileges.
In this example, the client issues a ticket on a data object to unregistered users meaning any user may use the ticket to open and read the data object. This example then makes a second connection to the SRB as an unregistered user ("ticketuser") and use the ticket to open and read the data object.
#include "srbClient.h"
#define BUFSIZE 4096
#define srbAuth NULL
#define HOST_ADDR NULL
#define COLLECTION "/srbtest"
#define BEGINTIME NULL
#define ENDTIME NULL
#define ACCESSCNT 1 /* Ticket is only good for 1 time access */
#define ROWS_WANTED 2
.
.
.
srbConn *conn, *conn1;
int nbytes, in_fd, out_fd;
char buf[BUFSIZE];
int status;
char *ticket;
char *ticketUser;
.
.
/* Set up the first connection to issue a ticket */
conn = clConnect (HOST_ADDR, NULL, srbAuth);
/* check to see that the backend connection was successfully made */
if ((status = clStatus(conn)) != CLI_CONNECTION_OK) {
fprintf(stderr,"Connection to SRB server failed.\n");
fprintf(stderr,"%s",clErrorMessage(conn));
exit_nicely(conn);
}
/* Issue a ticket on objID = argv[1] to any user, even unregistered */
ticketUser = ""; /* Any user */
if ((status = srbIssueTicket (conn, argv[1], COLLECTION, COLLFLAG,
BEGINTIME, ENDTIME, ACCESSCNT, ticketUser, &ticket)) < 0) {
fprintf(stderr, "can't srbIssueTicket obj\"%s\", %d\n",
argv[1], status);
exit_nicely(conn);
}
/* Print the ticket */
printf ("ticket = %s\n", ticket);
/* Make a second connection as a unregistered user ("ticketuser") */
conn1 = tiUserConnect (HOST_ADDR, NULL);
if ((status = clStatus(conn1)) != CLI_CONNECTION_OK) {
fprintf(stderr,"Connection to srbMaster failed.\n");
fprintf(stderr,"%s",clErrorMessage(conn));
srbRemoveTicket (conn, ticket);
exit_nicely(conn);
}
/* Use the 2nd connection and open the data object with the ticket */
in_fd = srbObjOpenWithTicket (conn1, argv[1], O_RDONLY, COLLECTION,
ticket);
if (in_fd < 0) { /* error */
fprintf(stderr, "can't open obj\"%s\"\n", argv[1]);
srbRemoveTicket (conn, ticket);
exit_nicely(conn);
}
/* Read the data object */
while ((nbytes = srbObjRead (conn1, in_fd, buf, BUFSIZE)) > 0) {
printf ("Content of the object: \n%s\n", buf);
}
srbObjClose (conn1, in_fd);
clFinish(conn1); /* Close the second connection */
/* remove the ticket */
status = srbRemoveTicket (conn, ticket);
if (status < 0) {
fprintf(stderr, "srbRemoveTicket error. status = %d\n", status);
fprintf(stderr,"%s",clErrorMessage(conn));
}
ClFinish(conn); /* Close the first connection */
6) An example for querying the MCAT catalog for information.
This example uses the srbGetDataDirInfo() call to query the MCAT catalog. This query selects the resource name, resource type, network address and path name, given the name of a data object.
#include "srbClient.h"
#define srbAuth NULL
#define HOST_ADDR NULL
#define COLLECTION "/srbtest"
#define ROWS_WANTED 2
.
.
.
mdasC_sql_result_struct *myresult;
char qval[MAX_DCS_NUM][MAX_TOKEN];
int selval[MAX_DCS_NUM];
int i;
int status;
/* Initialize the selval and qval array. */
for (i = 0; i < MAX_DCS_NUM; i++) {
selval[i] = 0;
sprintf(qval[i],"");
}
/* set up the query: Given the Data Object Name, select the path name,
* resource name, the network address and the resource type.
*/
sprintf(qval[DATA_NAME]," = '%s'",argv[1]);
selval[PATH_NAME] = 1;
selval[RSRC_NAME] = 1;
selval[RSRC_ADDR_NETPREFIX] = 1;
selval[RSRC_TYP_NAME] = 1;
myresult = malloc (sizeof (mdasC_sql_result_struct));
/* Send the query request */
if ((status = srbGetDataDirInfo(conn, 0, qval, selval, myresult,
ROWS_WANTED)) < 0) {
fprintf(stderr, "can't srbGetDataDirInfo. status = %d.\n",
status);
exit_nicely(conn);
}
/* Dump the result */
printSqlResult (myresult);
clearSqlResult (myresult); /* clear the content of myresult */
/* Print any additional rows */
while (myresult->continuation_index >= 0) {
if ((status = srbGetMoreRows(conn, 0,
myresult->continuation_index, myresult, ROWS_WANTED)) < 0)
break;
/* Dump the result */
printSqlResult (myresult);
clearSqlResult (myresult);
}
freeSqlResult (myresult);
7) An example for creating a container and put an object in the container.
This example uses the srbContainerCreate() call to create a container and the srbObjCreate() call to create an inContainer object.
#include "srbClient.h"
#define srbAuth NULL
#define HOST_ADDR NULL
#define COLLECTION "/srbtest"
#define DATATYPE "generic"
#define LOG_RESC "test-log-resc"
#define CONTAINER_NAME "testContainer"
#define CONTAINER_SZ 1000000
#define OBJECT_NAME "foo"
char objName[MAX_TOKEN];
int status, fd;
.
.
.
/* Create the container */
if ((status = srbContainerCreate (conn, MDAS_CATALOG, CONTAINER_NAME,
DATATYPE, LOG_RESC, CONTAINER_SZ)) < 0) {
fprintf(stderr, "can't srbContainerCreate. status = %d.\n",
status);
exit_nicely(conn);
}
/* Create the inContainer obj */
sprintf (objName, "%s&CONTAINER=%-s", OBJECT_NAME, CONTAINER_NAME);
out_fd = srbObjCreate (conn, MDAS_CATALOG, objName,
DATATYPE, "" ,COLLECTION, NULL, 0);
if (fd < 0) { /* error */
fprintf(stderr, "can't create obj \"%s\", status = %d\n", OBJECT_NAME,
fd);
fprintf(stderr,"%s",clErrorMessage(conn));
exit_nicely(conn);
}
.
.
.
5.0 Installing and configuring the SRB servers
This section describes the steps for installing and configuring the SRB servers. The SRB architecture supports multiple SRB servers running on various hosts. Metadata of data sets, resources and users managed by the SRB is stored in the MCAT catalog maintained in a relational database (DBMS). Therefore, a SRB installation may consist of multiple SRB servers running on various hosts, but only one SRB server is designated to interface with the MCAT database server. This server is built by defining the ORAMCAT or DB2MCAT keyword (see section 5.1 for definition) in the mk/mk.config file. This server is marked by the MDAS_ENABLED keyword in the data/hostConfig file.
Before any SRB server can be started, the MCAT catalog must be built and the MCAT server (DBMS) running. In addition, the user starting the SRB servers must have already been registered as a privileged user in MCAT. Currently at SDSC, we are running the MCAT server on a Oracle DBMS and the MDAS_ENABLED SRB server running on host gabor.sdsc.edu. Please email srb@sdsc.edu if a site wishes to use our MCAT server for its SRB metadata. If a site wants to setup and run its own MCAT catalog, the installation and configuration of the MCAT and the MCAT server are given in http://www.npaci.edu/DICE/Software/SRB/mcat.html.
If the SEA authentication scheme is to be used, and if the SEA library (libsea.a) does not already exist on your build platform, the SEA software which can be obtained at URL http://www.npaci.edu/DICE/Software/SEA/index.html, must be built first.
5.1 Building and installing the SRB server
The following describes the steps for building and installing the SRB server software:
1) Edit the mk/mk.config file. All configurable parameters for building the SRB server and the client library are defined in this file.
The SRB architecture supports multiple SRB servers running on various hosts. Each SRB server may be built with different options defined in the mk/mk.config file. For example, the SRB server on host A may include the driver for accessing HPPS and the SRB server on host B may include the driver for accessing the Lobj stored in DB2, etc.
The parameters are self-explanatory through the comments given in this file. Some of the more important parameters are discussed below:
a) Basic build configuration:
installDir - The absolute path of the SRB install directory.
PORTNAME - The OS platform of this SRB port. Currently, the SRB software runs on 8 platforms. Valid PORTNAMEs are: PORTNAME_solaris, PORTNAME_sunos, PORTNAME_linux, PORTNAME_aix, PORTNAME_alpha, PORTNAME_osx, PORTNAME_c90 and PORTNAME_sgi.
b) MCAT flags Enable any MCAT service that would be controlled by this SRB server.
ORAMCAT - defines that this SRB server being built is MDAS enabled and the MCAT is stored in Oracle DBMS. Normally, only one SRB server is MDAS enabled. Commented out by default.
DB2MCAT - defines that this SRB server being built is MDAS enabled and the MCAT is stored in Oracle DBMS. Normally, only one SRB server is MDAS enabled. Commented out by default.
NOTE: Both ORAMCAT and DB2MCAT cannot be defined at the same time.
c) Authentication flags
MDAS_AUTH - defines whether the MDAS authorization scheme will be supported for authentication. If used, the user/passwd pair registered with the MCAT catalog will be used to authenticate a user. Comment it out if the SRB server does not support MDAS authorization.
SEA_AUTH - defines whether SEA authorization scheme will be supported. The software can be configured to support both SEA_AUTH and MDAS_AUTH.
LIB_SEA - Is needed only if SEA_AUTH is defined. LIB_SEA specifies where the SEA client library is located. This library can be built from the SEA source branch in this package.
GSI_AUTH - defines whether GSI authorization scheme will be supported. The software can be configured to support GSI_AUTH, SEA_AUTH and MDAS_AUTH. This is OFF (commented out) by default.
LIB_GSI_AUTH - Is needed only if GSI_AUTH is defined. GSI_AUTH specifies where the GSI client library is located. This is commented out by default.
NOTE: Currently, the SDSC SRB servers are configured to handle all three authentication.
d) ADR_PROXY - defines whether the ADR DataCutter proxy operation will be supported.
e) JAVA_GUI and javaDir - JAVA_GUI defines whether the srbBrowser should be built. javaDir specifies the directory where the JDK software is installed. (e.g. /usr/local/apps/Java). In addition, if JAVA_GUI and javaDir are set, you must setenv CLASSPATH to "." and the path where the swing.jar is installed on your machine. e.g. setenv CLASSPATH /local/generic/lib/java/swing/swing.jar:.
Note: This software must be built using JDK 1.1.6 or 1.1.7 and swing 1.0.2.
f) Storage system flags Flags such as HPSS, UTREE, FTP, DB_DB2, DB_Illustra, etc define whether a particular storage system will be supported by this SRB server. UNIX, FTP and HTTP are enabled by default. Other flags related to storage system:
SRB_LARGEFILE64 - defines whether the 64 bit file size is supported by the underlining driver of this SRB server. Current, 64 bit file size is supported by the PORTNAME_solaris, PORTNAME_aix, PORTNAME_linux and PORTNAME_c90 platforms.:
If the HPSS flag is set, the NO_DCE flag specifies whether the DCEless client library developed by Mike Gleicher will be used. Please contact Mike at mkg@san.rr.com for informations regarding licensing this library. If this flag is not set, the regular DCE authentication is assumed. In addition, a number of HPSS library and header directory paths including HPSS_LIB_DIR, HPSS_HDR_DIR, etc need to be specified for the build.
2) "cd" to the main SRB directory and type in "gmake clean" and then "gmake" to make the SRB software. The Makefile contains various options to make and clean all or a subset of the build.
gmake --- build all.
gmake clean --- clean all.
gmake srb --- build only the SRB server and client.
gmake clean_srb --- clean only the SRB server and client.
gmake util --- build only the utilities (S commands).
gmake clean_util --- clean only the utilities.
gmake browser - build only the java srbBrowser GUI.
gmake clean_browser - clean only the java srbBrowser.
3) Type in "gmake install" to install the software in the $(installDir) directory. This procedure installs the following modules in the $(installDir) directory:
bin/runsrb - The script that starts the SRB
bin/srbMaster-2.0.0 - The frontend server.
bin/srbServer - The backend server (forked by the srbMaster-2.0.0 for each client connection).
bin/libSrbClient.a - The client library.
data/hostAuthConfig - The optional (needed only if HOST_BASED_AUTH in the mk.config file is set) host based authorization configuration file.
data/mcatHost - This file identifies the host on which the MCAT enabled SRB server and the authentication scheme to use to connect to this server.
data/hostConfig - This is the optional SRB host configuration file. It is only needed when when you want to add aliases to your local hostName.
data/hpssCosConfig - This is the optional HPSS Class of Services configuration file. It is only needed if HPSS in the mk.config file is set.
data/hpssNodceAuth - The file contains authentication info for non-dce HPSS. It is only needed if the HPSS and NO_DCE flags in the mk.config file are set.
data/MdasConfig - The MDAS configuration file.
data/metadata.fkrel - This file defines the foreign key relationship between the MDAS catalog tables and is used internally by the SRB for query generation. This file should not be changed between releases.
data/LobjConfig - The database configuration file for the DB Large Object driver. Basically, it contains the userID and password for accessing each database server.
Files installed in the data/ directory are templates for various configuration files. Details for configuring these files are given in the next section.
5.2 Configuring the SRB servers
5.2.1 Quick server setup
This section provides short notes on quick setup and startup of the SRB server. For more detail descriptions of setup, please read section 5.2.2
A) Setup for servers with UNIX driver only:
1) Configure the SRB user environment with the ~/.srb/.MdasEnv and ~/.srb/.MdasAuth files. Templates of these files can be found in the utilities/envFiles directory.
2) Configure the data/mcatHost file. If you are going to use SDSC MCAT, no change is needed.
3) To run the SRB server, type in:
cd bin
runsrb
This should start the SRB server. If it is not running,the data/srbLog file should give hints on why it is not running.
4) Check the data/srbLog file - The first line should contain something like:
LocalHostName: ghidorah, localhost, .....
followed by:
Local storage vault conf:
storSysType: 0, vaultPath: /projects/mdas/srb/SRBVaultTest
.
.
The LocalHostName list shows all the addresses of your local host recognized by the SRB server. It uses the gethostbyname() call to get the local addresses. There are situations where the local host address registered with MCAT may not show up with this call. Then, the "Local storage vault conf" will be empty. In this case, use the data/hostConfig file to include the MCAT registered host address to your local addresses.
B) Setup for servers with HPSS driver:
1) Build the SRB server with the HPSS flag set in the mk/mk.config file. In addition, the HPSS driver can use either the normally used DCE authentication or the NON-DCE implementation developed by Mike Gleicher. Please contact Mike at mkg@san.rr.com for informations regarding the licensing of the Non-DCE HPSS client/server software. If the NON-DCE authentication is used, the NO_DCE flag in mk/mk.config should be set before the build. If it is not set, DCE authentication is assumed.
2) Go through same procedures as A).
3) Configure the data/hpssCosConfig file.
4a) For NON-DCE authentication, configure the data/hpssNodceAuth file as described in the comment fields in the file. Basically, this file should contain the HPSS userID/password pair of the SRB user (the userID under which the SRB server runs).
NOTE: The NON-DCE authentication works only with the Non-DCE HPSS client/server implemented by Mike Gleicher.
4b) For DCE authentication, create a DCE keytab file and change the srbKeytabName parameter in bin/runsrb to point to this file path and the srbPrincipalName to the UserId associated with this keytab file.
C) Setup for servers with DB large object drivers:
1) Go through same procedures as A).
2) Configure the data/LobjConfig file.
5.2.2 Detailed server setup
1) Setting up the SRB user environment - Before a user can start the SRB server, the procedures for setting up user environment given in section 3.1 must be carried out. For GSI authentication, please read the section on " Steps to set up SRB Server certificates" in README.gsi.
2) Setting up the SRB configuration files - Modifications to the following configuration files are needed for each SRB server installation:
data/mcatHost - This file identifies the host on which the MCAT enabled server and the authentication scheme to use to connect to this server.
data/hostAuthConfig - The optional host based authorization configuration file. This file specifies who and from which IP address/domain the client connection may come from. The parameters are self-explanatory through the comments given in this file. It is needed only if HOST_BASED_AUTH is enabled in mk/mk.config
data/hostConfig - This is the optional SRB host configuration file. It is only needed when when you want to add aliases to your local hostName.
data/MdasConfig - The MDAS configuration file. Only the MDAS_ENABLED SRB server requires this file. The current MCAT catalog is kept in a Oracle database. This file contains basic informations required for SRB to interface with the MCAT catalog. Configurable parameters include SRB's Oracle userid, password and log file location.
data/LobjConfig - The configuration file for the DBMS Large Object drivers. Basically, it contains the userID and password required for authenticating the SRB server with DBMS where the large objects are stored. The parameters defined in this file are Db2User, Db2Passwd for DB2), IllusUser, IllusPasswd (for Illustra), OracleUser and OraclePasswd (for Oracle). Input to some of these parameters can be blank if the SRB server being started does not support a particular DBMS.
data/hpssCosConfig - This file configures the HPSS Class Of Service (COS). Basically, this file contains a table with two columns - "COS ID" and "Maximum file size in KBytes". It is used by the HPSS driver for choosing the appropriate COS based on the "dataSize" input parameter of the srbObjCreate() call.
bin/runsrb - This is the startup script for the SRB server. Modifications of some env variables defined in this file may be required. The parameters are self-explanatory through the comments given in this file.
data/hpssNodceAuth - This file specifies the userID/passwd pair for authenticating the SRB server with the NON-DCE HPSS system. This file is needed only when the NO_DCE in the mk.config file is set. Otherwise, for DCE HPSS authentication, create a DCE keytab file and change the srbKeytabName parameter in bin/runsrb to point to this file path.
5.3 Starting the SRB server
The servers in the SRB environment must be started in the following order:
1) The MCAT DBMS server.
2) The MDAS_ENABLED SRB server.
3) The rest of the SRB servers.
The script bin/runsrb is used to start the SRB server. To start the SRB server, type in:
. cd bin
. runsrb
5.4 Running the test suite
After installing the SRB server for the first time, the test suite given in the test/testsuite directory should be run. To run the test suite, modify inputs such as AUTH, DOMAIN, HOST and PORT as required in the "testsuite" script file in the test/testsuite directory and type in the following:
. cd test/testsuite
. gmake clean
. gmake
. testsuite
6.0 Building and Running the JAVA based srbBrowser
This section describes the steps for Building and Running the JAVA based srbBrowser. The srbBrowser is a JAVA based SRB client GUI. It can be used to perform a variety of client level operations including replication, copy and paste, registration of datasets and metadata manipulation and query, etc.
6.1 Building the JAVA based srbBrowser
Section 3.1 gives a brief description of building the srbBrowser software as part of the client library build. The following gives a more detailed descriptions of the build:
1) The SRB client library (e.g., gmake srb) and utilities (e.g., gmake util) must be built before building the srbBrowser.
2) setenv CLASSPATH to the path where the swing.jar is installed on your machine and ".". e.g.,
setenv CLASSPATH /local/generic/lib/java/swing/swing.jar:.
3) Edit the data/mk.config file and define the "JAVA_GUI" parameter and set the "javaDir" parameter to the directory where your JDK is installed.
4) "cd" to the main SRB directory and type in "gmake clean_browser" and then "gmake browser" or, "cd java" and type in "gmake clean" and then "gmake".
6.2 Running the JAVA based srbBrowser
To run the srbBrowser, setenv DISPLAY to your workstation, "cd java/bin" and type in "srbBrowser".
The following lists the SRB operations that can be carried out using the srbBrowser GUI:
1) The "Default" pull down menu allow a client to select the default storage resource and set the default comment. The default resource will be used for all subsequent dataset creation until it is changed. The default comment can be used for creating new comments for datasets.
2) The "Operations" menu shows the type of operations that can be performed depending what was selected. The following operations have been implemented:
"New" - make a new "Collection"
"Replicate" - Replicate a dataset or collection (recursively). Two choices are allowed: replicate to the default storage resource or choose a resource for replication.
"Copy" a dataset or collection - The following steps should be taken:
1) Select a dataset or a collection to be copied (source).
2) Select the "Copy" menu from the "Operation" pulldown menu.
3) Select the destination dataset or collection.
4) Select the "Paste" menu from the "Operation" pulldown menu.
"Delete" - Delete a dataset or a collection recursively.
"Import" - Import a file or directory from a local file system. The following steps should be taken:
1) Select the destination dataset or collection.
2) Select the "Import from Local FS" menu from the "Operation" pulldown menu.
3) A file chooser dialog box should appear. Browse through the local FS and selection a file or a directory to be imported from (source). Click on the "Open" button of the file chooser dialog box.
"Export" - Export a dataset or collection to a local file system. The following steps should be taken:
1) Select the source dataset or collection to be exported.
2) Select the "Export to Local FS" menu from the "Operation" pulldown menu.
3) A file chooser dialog box should appear. Browse through the local FS and selection a file or a directory to export to (destination). Click on the "Open" button of the file chooser dialog box.
"Register" and "Unregister" - Register and Unregister local files and directory. These functions may be recursive. Comparing to the "Import" and "Export" functions, registering a local file does not involve the the copying of the file physically. Rather, the file is registered with MCAT and hence becomes visible to the SRB world. For "Register" to work, your local file must be accessible by one of the SRB server in the federation. The following three steps must be taken to register a local file:
1) Set the default resource (through the "Default" menu) to the resource controlled by the SRB server that can access your local file.
2) Select a collection where you want this registered dataset or collection to reside.
3) Select the "Register Local FS" menu item from the "Operation" pull down menu and then selection the local file or directory you wish to register.
"Rename" - Rename a dataset.
"Display" - Display allows graphical (jpeg, gif, etc) and text (C, html, etc) files to be displayed in the srbBrowser window. This can also be achieved by double clicking the file displayed in the metadata window.
3) The "Metadata" menu allows the manipulation and querying of system level metadata. Manipulation of two attributes - "Access Control" and "Comments" has been implemented thus far.
The "Set Access Cntl" menu can be used to grant access permission of a dataset or a collection to individual users or group of users. Recursive operation is allowed for setting the access permission of a collection.
The "List Access Cntl" menu can be used to list the access permission of datasets and collections.
The "New Comment" menu can be used to attach a new "comment" to a dataset. A new comment can be created completely new or by adding to the default comment.
The "Modify Comment" menu can be used to modify an existing comment. Modification can be done by replacing the entire comment or appending to the current comment. The menu also allows the deletion of a comment.
The "Query Comment" menu can be used to query datasets based on the content of these comments. If a collection is selected for querying, the query will be done for all descendants of the collection. If a dataset is selected for querying, the content the comment for this dataset will be listed.
4) The "View" menu has only one sub-menu - "Refresh". The srbBrowser always try to update the tree structure and the metadata displayed in the window whenever changes were made by the srbBrowser. The "Refresh" function allowed the display to be refreshed after changes were made external to the srbBrowser.
7.0 Building and Running srbMon - a SRB server monitoring program
This section describes the steps for Building and Running the srbMon daemon. The srbMon daemon is a SRB monitoring daemon which remotely monitors and logs all SRB servers belonging to a federation, and attempts to restart any servers that were found to be down..
To build srbMon, "cd admin" and type in "gmake clean" and then "gmake".
To run srbMon, the following steps should be carried out:
1) srbMon should be run on a host which does not belong to the SRB federation to be monitored.
2) Configure the admin/data/srbHostFile file. This file should contain all the hosts in the federation to be monitored.
3) srbMon uses "rsh" to start any SRB servers that were found to be down. Therefore, SRB admin must make sure that remote shell type permissions are properly setup (i.e., .rhosts and /etc/hosts.equiv files, etc) on each host being monitored.
4) To run the daemon, "cd admin/bin" and type in "srbMon1_1_2". This daemon will sleep most of the time, waking up every 10 minutes to monitor the SRB servers.
5) The srbMon daemon generates a log file - admin/data/monLog, logging the status of each server and action performed.
Appendix A Descriptions of the SRB Client API
1) Client/Server connection API:
2) High-level API:
3) Low-level File-type API:
4) Low-level DB-type API:
5) Low-level DB Table API:
6) MCAT related API:
6) Miscellaneous SRB API.
Appendix B - Descriptions of the dataCutter Client API
NAME
srbConnect
SYNOPSIS
#include "srbClient.h"
srbConn* srbConnect(char *srbHost, char* srbPort, char* srbAuth, char *userName, char *domainName, char *authScheme, char *serverDn);
DESCRIPTION
Establish a connection to a srbServer through the srbMaster at the specified host and port. This API replaces the clConnect () API.
RETURN VALUES
Returns a srbConn* which is needed for all subsequent client calls. Error if the status field of the connection returned is CONNECTION_BAD.
ERRORS
CONNECTION_BAD in the status field of srbConn.
SEE ALSO
clConnect, tiUserConnect, clFinish and clErrorMessage.
NAME
clConnect
SYNOPSIS
#include "srbClient.h"
srbConn* clConnect(char* srbHost, char* srbPort, char* srbAuth);
DESCRIPTION
Establish a connection to a srbServer through the srbMaster at the specified host and port. This API is being replaced by the srbConnect () call.
RETURN VALUES
Returns a srbConn* which is needed for all subsequent client calls. Error if the status field of the connection returned is CONNECTION_BAD.
ERRORS
CONNECTION_BAD in the status field of srbConn.
SEE ALSO
srbConnect, tiUserConnect, clFinish and clErrorMessage.
NAME
tiUserConnect
SYNOPSIS
#include "srbClient.h"
srbConn* tiUserConnect(char *srbHost, char* srbPort);
DESCRIPTION
Establish a connection to a srbServer through the srbMaster at the specified host and port by a TICKET USER. Normal authentication will be bypassed for a TICKET USER.
RETURN VALUES
Returns a srbConn* which is needed for all subsequent client calls. Error if the status field of the connection returned is CONNECTION_BAD.
ERRORS
CONNECTION_BAD in the status field of srbConn.
SEE ALSO
srbConnect, clFinish and clErrorMessage.
NAME
clFinish
SYNOPSIS
#include "srbClient.h"
void clFinish (srbConn* conn);
DESCRIPTION
Close the current connection and free the srbConn data structure.
RETURN VALUES
None.
ERRORS
None.
SEE ALSO
srbConnect, tiUserConnect and clErrorMessage.
NAME
clErrorMessage
SYNOPSIS
#include "srbClient.h"
char* clErrorMessage(srbConn* conn);
DESCRIPTION
Return the ErrorMessage of a connection. This is the error message from the current client call returned by the SRB server. On encountering error on most client calls, in addition to returning an error code (usually a negative value), a text string describing the error is returned to the caller. The caller must use the clErrorMessage() call to retrieve the error message.
RETURN VALUES
Returns the pointer to the error message string. Returns a NULL if there is no error message.
ERRORS
SEE ALSO
srbConnect, tiUserConnect and clFinish.
NAME
srbObjOpen
SYNOPSIS
#include "srbClient.h"
int srbObjOpen(srbConn* conn, char *objID, int oflag, char *collectionName);
DESCRIPTION
Open a SRB data object for I/O operation.
RETURN VALUES
Returns the object descriptor. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbObjOpenWithTicket and srbObjClose.
NAME
srbObjOpenWithTicket
SYNOPSIS
#include "srbClient.h"
int srbObjOpenWithTicket (srbConn* conn, char *objID, int oflag,
char *collectionName, char *ticket);
DESCRIPTION
Open a SRB data object using a ticket.
RETURN VALUES
Returns the object descriptor. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbObjOpen and srbObjClose.
NAME
srbObjCreate
SYNOPSIS
#include "srbClient.h"
int srbObjCreate(srbConn* conn, int catType, char *objID,
char *dataTypeName, char *resourceName, char *collectionName,
char *pathName, int dataSize);
DESCRIPTION
Create a SRB data object.
RETURN VALUES
Returns the object descriptor. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbObjUnlink, srbContainerCreate.
NAME
srbObjClose
SYNOPSIS
#include "srbClient.h"
int srbObjClose (srbConn* conn, int desc);
DESCRIPTION
Close an opened data object.
RETURN VALUES
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbObjOpen and srbObjOpenWithTicket.
NAME
srbObjUnlink
SYNOPSIS
#include "srbClient.h"
int srbObjUnlink (srbConn* conn, char *objID, char *collectionName);
DESCRIPTION
Delete a SRB data object.
RETURN VALUES
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
NAME
srbObjRead
SYNOPSIS
#include "srbClient.h"
int srbObjRead(srbConn *conn, int desc, char *buf, int len);
DESCRIPTION
Read len bytes of the SRB data object into buf. The caller must have allocated enough space to hold the data read.
RETURN VALUES
Returns the length of bytes read. Returns a negative value upon failure.
ERRORS
SEE ALSO
NAME
srbObjSeek
SYNOPSIS
#include "srbClient.h"
int srbObjSeek(srbConn *conn, int desc, int offset, int whence);
DESCRIPTION
Change the current read or write location on an opened SRB data object.
SEEK_SET - pointer is set to the value of the Offset parameter.
SEEK_CUR - pointer is set to its current location plus the value of the Offset parameter.
SEEK_END - pointer is set to the size of the file plus the value of the Offset parameter.
RETURN VALUES
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbObjOpen, srbObjWrite and srbObjRead.
NAME
srbObjSync
SYNOPSIS
#include "srbClient.h"
int srbObjSync(srbConn* conn, int desc);
DESCRIPTION
Sync a SRB object.
RETURN VALUES
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbObjOpen, srbObjCreate, srbObjUnlink, srbObjClose, srbObjRead and srbObjWrite.
NAME
srbObjStat
SYNOPSIS
#include "srbClient.h"
int srbObjStat(srbConn* conn, int catType,
char *path, struct stat *statbuf);
DESCRIPTION
Get the status of an SRB object. The result is placed in statbuf.
RETURN VALUES
Stat result in statbuf.
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbObjStat64, srbObjOpen and srbObjCreate.
NAME
srbObjStat64
SYNOPSIS
#include "srbClient.h"
int srbObjStat64 (srbConn* conn, int catType,
char *path, struct stat64 *statbuf);
DESCRIPTION
Get the status of an SRB object. The result is placed in statbuf.
RETURN VALUES
Stat result in statbuf.
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbObjStat, srbObjOpen and srbObjCreate.
NAME
srbObjGetdents
SYNOPSIS
#include "srbClient.h"
int srbObjGetdents(srbConn* conn, int catType,
int fd, dirent_t *buf, size_t nbyte);
DESCRIPTION
The SRB equivalent of the UNIX getdents call. The function attempts to read "nbyte" bytes from the collection associated with the descriptor "fd" and to format them as file system independent directory entries in the buffer pointed to by buf. Since the collection entries are of variable lengths, in most cases the actual number of bytes returned will be less than "nbyte".
RETURN VALUES
Getdents result in buf.
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbObjOpen, srbObjGetdents64 and srbObjCreate.
NAME
srbObjGetdents64
SYNOPSIS
#include "srbClient.h"
int srbObjGetdents64(srbConn* conn, int catType,
int fd, dirent64_t *buf, size_t nbyte);
DESCRIPTION
The SRB equivalent of the UNIX getdents64 call which is the same as srbObjGetdents except the results are given in dirent64_t instead of dirent_t. The function attempts to read "nbyte" bytes from the collection associated with the descriptor "fd" and to format them as file system independent directory entries in the buffer pointed to by buf. Since the collection entries are of variable lengths, in most cases the actual number of bytes returned will be less than "nbyte".
RETURN VALUES
Getdents64 result in buf.
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbObjOpen, srbObjGetdents and srbObjCreate.
NAME
srbObjProxyOpr
SYNOPSIS
#include "srbClient.h"
int srbObjProxyOpr (srbConn *conn, ProxyOprId operation,
char *inputStr1, char *inputStr2);
DESCRIPTION
Perform a proxy Operation. Some operations can be more efficiently done by the SRB server without much involvement by the client. An example is the copy operation where it is more efficient for the server to do all the read and write operations on behalf of the client than passing the data read to the client and then for the client passing it back to the server for the write operation. A general framework for handling proxy operations has been set up to facilitate the building of new proxy functions. The input of the API consists of a ProxyOprId which identifies the type of proxy operation to be carried out and two input strings. The current implementation supports only one type of proxy operation, i.e., the "Copy" operation. Additional proxy operations can be implemented within this framework.
RETURN VALUES
Returns 0 or a positive value upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbObjReplicate, srbObjMove, srbObjOpen and srbObjCreate.
NAME
srbExecCommand
SYNOPSIS
#include "srbClient.h"
int srbExecCommand (srbConn *conn, char *command,
char *commandArgv, char *proxyAddr, int portalFlag);
DESCRIPTION
This is a special case of proxy operation. It performs the remote execution of arbitrary commands(executables) installed in a specific predefined directory on the remote host. Currently, this predefined directory is hard-coded as "commands" in the directory where the srbMaster and srbServer executables are installed. For example, if the srbMaster is installed in the /usr/local/srb/bin directory, the proxy commands should be installed in the /usr/local/srb/bin/commands directory.
The input argument "command" specifies the proxy command to be executed. Input arguments for the command is specified with the "commandArgv" argument. The argument "proxyAddr" specifies the host address where the proxy command is to be executed. If no address is specified, the default is the host where the client is connected to.
The input argument "portalFlag" specifies the mode of communication between the proxy command and the client. Valid values are PORTAL_OFF, PORTAL_ON and PORTAL_STD_IN_OUT. A value of PORTAL_OFF means there will be no communication between the proxy command and the client. PORTAL_ON means a socket will be created between the proxy command and the client. This call returns the descriptor of the socket which can be used by the client to communicate with the proxy command. On the server side, the socket descriptor is passed onto the proxy command through the environment variable defined by PORTAL_ENV. The proxy command can use this socket to communicate with the client. A value of PORTAL_STD_IN_OUT has a similar effect as PORTAL_ON except on the server side, this socket is now associated with the stdin, stdout and stderr of the proxy command. i.e., all stdout and stderr of the proxy command will be sent to the client through this socket and the client can use this socket to send messages to the stdin of the proxy command.
RETURN VALUES
Upon success, returns 0 if the "portalFlag" is PORTAL_OFF, and the portal socket descriptor if the "portalFlag" is PORTAL_ON or PORTAL_STD_IN_OUT. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbObjReplicate, srbObjMove, srbObjOpen and srbObjCreate.
NAME
srbObjReplicate
SYNOPSIS
#include "srbClient.h"
int srbObjReplicate (srbConn* conn, int catType, char *objID,
char *collectionName, char *newResourceName, char *newPathName);
DESCRIPTION
Replicate a SRB data object.
RETURN VALUES
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbSyncData, srbObjMove and srbObjProxyOpr.
NAME
srbObjMove
SYNOPSIS
#include "srbClient.h"
int srbObjMove (srbConn* conn, int catType, char *objID, char *collectionName,
char *newResourceName, char *newPathName);
DESCRIPTION
Move a copy of an SRB object to a new location.
RETURN VALUES
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbObjReplicate and srbObjProxyOpr.
NAME
srbObjWrite
SYNOPSIS
#include "srbClient.h"
int srbObjWrite(srbConn *conn, int desc, char *buf, int len);
DESCRIPTION
Write len bytes of buf into the Object fd
RETURN VALUES
Returns the number of bytes written. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbObjOpen, srbObjCreate and srbObjRead.
NAME
srbCreateCollect
SYNOPSIS
#include "srbClient.h"
int srbCreateCollect (srbConn* conn, int catType, char *parentCollect,
char *newCollect);
DESCRIPTION
Create a new collect.
RETURN VALUES
Returns 0 - success. Returns negative - failure.
ERRORS
SEE ALSO
srbModifyCollect and srbListCollect.
NAME
srbListCollect
SYNOPSIS
#include "srbClient.h"
int srbListCollect (srbConn* conn, int catType, char *collectionName,
char *flag, mdasC_sql_result_struct *listResult, int rowsWanted);
DESCRIPTION
List the content of a SRB collection.
RETURN VALUES
Returns 0 - success. Returns negative - failure.
ERRORS
SEE ALSO
srbObjCreate and srbModifyCollect.
NAME
srbModifyCollect
SYNOPSIS
#include "srbClient.h"
int srbModifyCollect (srbConn* conn, int catType, char *collectionName,
char *dataValue1, char *dataValue2, char *dataValue3, int retractionType);
DESCRIPTION
Modify a SRB collection.
RETURN VALUES
Returns 0 - success. Returns negative value - failure.
ERRORS
SEE ALSO
NAME
srbIssueTicket
SYNOPSIS
#include "srbClient.h"
int srbIssueTicket (srbConn* conn, char *objID, char *collectionName,
char *collectionFlag, char *beginTime, char *endTime, int accessCnt,
char *ticketUser, char **ticket);
DESCRIPTION
Issue a ticket for a SRB data object or collection.
user1@domain1&user2@domain2 ....
If it is NULL, => all users.
RETURN VALUES
Returns 0 - success. Returns negative - failure.
ERRORS
SEE ALSO
NAME
srbRemoveTicket
SYNOPSIS
#include "srbClient.h"
int srbRemoveTicket (srbConn* conn, char *ticket);
DESCRIPTION
Cancel a ticket.
RETURN VALUES
Returns 0 - success, negative - failure.
ERRORS
SEE ALSO
NAME
srbContainerCreate
SYNOPSIS
#include "srbClient.h"
int srbContainerCreate (srbConn* conn, int catType, char *containerName, char *containerType, char *resourceName, int containerSize);
DESCRIPTION
Create a container with the given container name in the given logical resource. The logical resource should contain two physical resources: a "permanent" (e.g., HPSS) and a "temporary" or "cache" (e.g., UNIX disk file system) resource. Once a container has been created, the srbObjCreate() call may be used to create objects in the container using the CONATINER keyword. The normal srbObjWrite() and srbObjRead() calls can be used to perform write/read operations to the container. Internally, all I/O are done only to the "cache" copy of the container. If the "cache" copy does not exist (e.g., purged), a replicate of the "permanent" copy will be made to the "cache" resource before any I/O operation can be made. The srbSyncContainer() call is used to synchronize the "cache" copy with the permanent copy. When a container is full (max container size exceeded), the SRB server automatically renames the full container by appending a unique integer (clock in seconds) to the container name and creates a new container with the original name. This way, the client does not have to worry about filling up containers.
RETURN VALUES
Returns 0 - success, negative - failure.
ERRORS
SEE ALSO
srbRmContainer, srbSyncContainer, srbReplContainer and srbGetContainerInfo.
NAME
srbRmContainer
SYNOPSIS
#include "srbClient.h"
int srbRmContainer (srbConn* conn, int catType, char *containerName);
DESCRIPTION
Remove an existing container. All inContainer objects stored in the given container must be removed first before making this call
RETURN VALUES
Returns 0 - success, negative - failure.
ERRORS
SEE ALSO
srbContainerCreate, srbSyncContainer, srbReplContainer and srbGetContainerInfo.
NAME
srbSyncContainer
SYNOPSIS
#include "srbClient.h"
int srbSyncContainer (srbConn* conn, int catType, char *containerName, int syncFlag);
DESCRIPTION
Synchronize the "permanent" copies of the container with the "cache" copy. When an inContainer object is created or opened for I/O, all I/O are done only to the "cache" copy. This call allows the synchronization of different copies. Valid values of the syncFlag are PURGE_FLAG and PRIMARY_FLAG. If PURGE_FLAG is set, the "cache" copy will be purged once the copies has been synchronized. If PRIMARY_FLAG is set, synchronization is done only to the primary archival resource. The default is to synchronize all archival resources.
RETURN VALUES
Returns 0 - success, negative - failure.
ERRORS
SEE ALSO
srbContainerCreate, srbRmContainer and srbGetContainerInfo.
NAME
srbReplContainer
SYNOPSIS
#include "srbClient.h"
int srbReplContainer (srbConn* conn, int catType, char *containerName, char *resourceName);
DESCRIPTION
Replicate or stage a copy of a container onto the specified physical resource. The specified physical resource must be a member of the logical resource associated with the container. The srbSyncContainer() call can be used to replicate copies to either the primary archival resource or to all resources, but not to a specific non-primary resource. This call can be used to synchronize a container to a specific non-primary archival resource or stage a container to a specific non-primary cache resource.
RETURN VALUES
Returns 0 - success, negative - failure.
ERRORS
SEE ALSO
srbContainerCreate, srbRmContainer, srbSyncContainer and srbGetContainerInfo.
NAME
srbObjGet
SYNOPSIS
#include "srbClient.h"
int srbObjGet (srbConn* conn, char *srcObjID, char *srcCollection, char *locFilePath);
DESCRIPTION
Download a dataset from SRB to the local file system using server-directed parallel I/O
RETURN VALUES
Returns: the number of bytes downloaded - success, negative - failure.
ERRORS
SEE ALSO
NAME
srbObjPut
SYNOPSIS
#include "srbClient.h"
int srbObjPut (srbConn* conn, char *destObjID, char *destCollection, char *destResLoc, char *dataType, char *destPath, char *locFilePath, srb_long_t size);
DESCRIPTION
Upload a dataset from the local file system to SRB using server-directed parallel I/O
RETURN VALUES
Returns: the number of bytes uploaded - success, negative - failure.
ERRORS
SEE ALSO
NAME
srbSyncData
SYNOPSIS
#include "srbClient.h"
int srbSyncData (srbConn* conn, int catType, char *objID, char *collectionName, char *resource);
DESCRIPTION
Synchronize all copies (replica) of an SRB object with the most recently modified version.
RETURN VALUES
Returns 0 - success, negative - failure.
ERRORS
SEE ALSO
NAME
srbGetContainerInfo
SYNOPSIS
#include "srbClient.h"
int srbGetContainerInfo (srbConn* conn, int catType, char *containerName, mdasC_sql_result_struct *myresult, int rowsWanted);
DESCRIPTION
Query the metadata associated with a container.
RETURN VALUES
Returns 0 - success, negative - failure.
ERRORS
SEE ALSO
srbContainerCreate, srbRmContainer and srbSyncContainer.
NAME
srbRegisterLocation
SYNOPSIS
#include "srbClient.h"
int srbRegisterLocation(srbConn* conn, char *locName, char *fullAddr, char *parentLoc, char *serverUser, char *serverUserDomain);
DESCRIPTION
Register location information
RETURN VALUES
Returns 0 - success, negative - failure.
ERRORS
SEE ALSO
srbIngestToken, srbRegisterResource, srbRegisterLogicalResource, srbRegisterReplicateResourceInfo, and srbDeleteValue.
NAME
srbIngestToken
SYNOPSIS
#include "srbClient.h"
int srbIngestToken(srbConn* conn, char *typeName, char *newValue, char *parentValue);
DESCRIPTION
Ingest Token
RETURN VALUES
Returns 0 - success, negative - failure.
ERRORS
SEE ALSO
srbRegisterLocation, srbRegisterResource, srbRegisterLogicalResource, srbRegisterReplicateResourceInfo, and srbDeleteValue.
NAME
srbRegisterResource
SYNOPSIS
#include "srbClient.h"
int srbRegisterResource(srbConn* conn, char *rescName, char *rescType, char *location, char *phyPath, char *class, int size);
DESCRIPTION
Register/create a new resource (physical or compound)
RETURN VALUES
Returns 0 - success, negative - failure.
ERRORS
SEE ALSO
srbRegisterLocation, srbIngestToken, srbRegisterLogicalResource, srbRegisterReplicateResourceInfo, and srbDeleteValue.
NAME
srbRegisterLogicalResource
SYNOPSIS
#include "srbClient.h"
int srbRegisterLogicalResource(srbConn* conn, char *rescName, char *rescType, char *phyResc, char *phyPath);
DESCRIPTION
Register/create a new logical resource
RETURN VALUES
Returns 0 - success, negative - failure.
ERRORS
SEE ALSO
srbRegisterLocation, srbIngestToken, srbRegisterResource, srbRegisterReplicateResourceInfo, and srbDeleteValue.
NAME
srbRegisterReplicateResourceInfo
SYNOPSIS
#include "srbClient.h"
int srbRegisterReplicateResourceInfo( srbConn* conn, char *physicalRescName, char *rescType, char *oldLogicalRescName, char *indefaultPath);
DESCRIPTION
Add another physical resource to a logical or compound resource
RETURN VALUES
Returns 0 - success, negative - failure.
ERRORS
SEE ALSO
srbRegisterLocation, srbIngestToken, srbRegisterResource, srbRegisterLogicalResource, and srbDeleteValue.
NAME
srbDeleteValue
SYNOPSIS
#include "srbClient.h"
int srbDeleteValue(srbConn* conn, int valueType, char *deleteValue);
DESCRIPTION
Delete/remove a value of type location, user, or resource (other types are implemented but untested).
RETURN VALUES
Returns 0 - success, negative - failure.
ERRORS
SEE ALSO
srbRegisterLocation, srbIngestToken, srbRegisterResource, srbRegisterLogicalResource, and srbRegisterReplicateResourceInfo.
NAME
srbSetupSessionPublicKey
SYNOPSIS
#include "srbClient.h"
int srbSetupSessionPublicKey (srbConn *conn, char *publicKey);
DESCRIPTION
Get the MCAT-enabled server's public key in preparation for transferring encryptioned information. Also see the sscSetupSessionPublicKey library routine.
RETURN VALUES
Returns positive - success or not-supported, negative - for some failures.
Returns a string representation of the publicKey in publicKey. If Secure Communications is not supported on the server side, an error message is returned in publicKey.
ERRORS
SEE ALSO
NAME
srbSetupSession
SYNOPSIS
#include "srbClient.h"
int srbSetupSession (srbConn *conn, char *sessionKey);
DESCRIPTION
Set up a session (for encryption) with the MCAT-enabled server. Also see the sscSetupSession library routine.
RETURN VALUES
Returns 0 - success, negative - failure.
ERRORS
SEE ALSO
NAME
srbFileOpen
SYNOPSIS
#include "srbClient.h"
int srbFileOpen(srbConn* conn, int storSysType, char *hostAddr,
char *filename, int flags, int mode);
DESCRIPTION
Opens a file-type file.
0 = Unix, 1 = UniTree, 2 = HPSS, 3 = FTP, 4 = HTTP.
RETURN VALUES
Returns the file descriptor for use in subsequent calls. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbFileCreate, srbFileClose, srbFileRead and srbFileWrite.
NAME
srbFileCreate
SYNOPSIS
#include "srbClient.h"
int srbFileCreate(srbConn* conn, int storSysType, char *hostAddr,
char *filename, int mode, int size);
DESCRIPTION
Create a File-type file.
RETURN VALUES
Returns the file descriptor for use in subsequent calls. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbFileOpen, srbFileUnlink, srbFileClose, srbFileRead and srbFileWrite.
NAME
srbFileUnlink
SYNOPSIS
#include "srbClient.h"
int srbFileUnlink(srbConn* conn, int storSysType, char *hostAddr,
char *filename);
DESCRIPTION
Unlink a File-type file.
0 = Unix, 1 = UniTree, 2 = HPSS, 3 = FTP (not supported for HTTP).
RETURN VALUES
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbFileOpen, srbFileCreate, srbFileClose, srbFileRead and srbFileWrite.
NAME
srbFileClose
SYNOPSIS
#include "srbClient.h"
int srbFileClose(srbConn* conn, int fd);
DESCRIPTION
Close a File-type file.
RETURN VALUES
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbFileOpen, srbFileCreate, srbFileUnlink, srbFileRead and srbFileWrite.
NAME
srbFileRead
SYNOPSIS
#include "srbClient.h"
int srbFileRead(srbConn *conn, int fd, char *buf, int len);
DESCRIPTION
Read len bytes of the File-type file into buf. The caller must have allocated enough space to hold the data read.
RETURN VALUES
Returns the number of bytes read. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbFileOpen, srbFileCreate, srbFileUnlink, srbFileClose and srbFileWrite.
NAME
srbFileWrite
SYNOPSIS
#include "srbClient.h"
int srbFileWrite(srbConn *conn, int fd, char *buf, int len);
DESCRIPTION
Write len bytes of buf into the File-type file fd.
RETURN VALUES
Returns the number of bytes written. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbFileOpen, srbFileCreate, srbFileUnlink, srbFileClose, srbFileRead.
NAME
srbFileSeek
SYNOPSIS
#include "srbClient.h"
int srbFileSeek(srbConn *conn, int fd, int offset, int whence);
DESCRIPTION
Change the current read or write location on a File-type file.
SEEK_SET - pointer is set to the value of the Offset parameter.
SEEK_CUR - pointer is set to its current location plus the value of the Offset parameter.
SEEK_END - Pointer is set to the size of the file plus the value of the Offset parameter.
RETURN VALUES
Returns the resulting pointer location, measured in bytes from the beginning of the file upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbFileOpen, srbFileCreate, srbFileUnlink, srbFileClose, srbFileRead and srbFileWrite.
NAME
srbFileSync
SYNOPSIS
#include "srbClient.h"
int srbFileSync(srbConn* conn, int fd);
DESCRIPTION
Sync a File-type file.
RETURN VALUES
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbFileOpen, srbFileCreate, srbFileUnlink, srbFileClose, srbFileRead and srbFileWrite.
NAME
srbFileStat
SYNOPSIS
#include "srbClient.h"
int srbFileStat(srbConn* conn, int storSysType, char *hostAddr,
char *filename, struct srbStat *statbuf);
DESCRIPTION
Get the status of a File-type file.
0 = Unix, 1 = UniTree, 2 = HPSS, 3 = FTP (not supported for HTTP).
RETURN VALUES
Stat result in statbuf.
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbFileOpen and srbFileCreate.
NAME
srbFileFstat
SYNOPSIS
#include "srbClient.h"
int srbFileFstat(srbConn* conn, int storSysType, char *hostAddr,
int fd, struct srbStat *statbuf);
DESCRIPTION
Get the status of a File-type file.
0 = Unix, 1 = UniTree, 2 = HPSS, 3 = FTP (not supported for HTTP).
RETURN VALUES
Stat result in statbuf.
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbFileOpen, srbFileStat and srbFileCreate.
NAME
srbFileMkdir
SYNOPSIS
#include "srbClient.h"
int srbFileMkdir(srbConn* conn, int storSysType, char *hostAddr,
char *filename, int mode);
DESCRIPTION
Create a new file-type directory.
0 = Unix, 1 = UniTree, 2 = HPSS, 3 = FTP (not supported for HTTP).
RETURN VALUES
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
NAME
srbFileChmod
SYNOPSIS
#include "srbClient.h"
int srbFileChmod(srbConn* conn, int storSysType, char *hostAddr,
char *filename, int mode);
DESCRIPTION
Change the mode of a file-type file or directory.
0 = Unix, 1 = UniTree, 2 = HPSS, 3 = FTP (not supported for HTTP).
RETURN VALUES
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
NAME
srbFileRmdir
SYNOPSIS
#include "srbClient.h"
int srbFileRmdir(srbConn* conn, int storSysType, char *hostAddr,
char *filename);
DESCRIPTION
Remove a file-type directory.
0 = Unix, 1 = UniTree, 2 = HPSS, 3 = FTP (not supported for HTTP).
RETURN VALUES
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
NAME
srbSetStorAttri
SYNOPSIS
#include "srbClient.h"
int srbSetStorAttri(srbConn* conn, int storSysType,
char *hostAddr, char *userAuth, char *dirPath);
DESCRIPTION
Set Attributes for a Storage System. Currently only defined for FTP files, connects to an FTP Server and/or sets the current directory.
3 = FTP. Currently only defined for FTP.
RETURN VALUES
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
NAME
srbOpendir
SYNOPSIS
#include "srbClient.h"
int srbOpendir(srbConn* conn, int storSysType, char *hostAddr, char *dirname);
DESCRIPTION
Open a file-type directory.
0 = Unix, 1 = UniTree, 2 = HPSS, 3 = FTP (not supported for HTTP).
RETURN VALUES
Returns the file descriptor for use in subsequent client calls. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbClosedir and srbReaddir.
NAME
srbClosedir
SYNOPSIS
#include "srbClient.h"
extern int srbClosedir(srbConn* conn, int fd);
DESCRIPTION
Close an opened file-type directory.
RETURN VALUES
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbOpendir and srbReaddir.
NAME
srbReaddir
SYNOPSIS
#include "srbClient.h"
struct srbDirent *srbReaddir(srbConn* conn, int dirDesc);
DESCRIPTION
Read a file-type directory entry.
RETURN VALUES
A pointer to struct srbDirent - The dirent read.
ERRORS
SEE ALSO
srbOpendir and srbClosedir.
NAME
srbDbLobjOpen
SYNOPSIS
#include "srbClient.h"
int srbDbLobjOpen(srbConn* conn, int storSysType,
char *resourceLoc, char *dataPath, int flags, int mode);
DESCRIPTION
Open a DB Large Object.
0 = DB2, 1 = Illustra, 5 = Oracle.
RETURN VALUES
Returns the file descriptor for use in later srbDbLobj* calls. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbDbLobjCreate, srbDbLobjClose, srbDbLobjRead, srbDbLobjWrite and srbDbLobjUnlink.
NAME
srbDbLobjCreate
SYNOPSIS
#include "srbClient.h"
int srbDbLobjCreate(srbConn* conn, int storSysType,
char * ResourceLoc, char *filename, int mode);
DESCRIPTION
Create a DB Large Object.
0 = DB2, 1 = Illustra, 5 = Oracle.
RETURN VALUES
Returns the file descriptor for use in later srbDbLobj* calls.
Returns a negative value upon failure.
ERRORS
SEE ALSO
srbDbLobjOpen, srbDbLobjClose, srbDbLobjRead, srbDbLobjWrite and srbDbLobjUnlink.
NAME
srbDbLobjClose
SYNOPSIS
#include "srbClient.h"
int srbDbLobjClose(srbConn* conn, int fd);
DESCRIPTION
Close an opened DB Large Object.
RETURN VALUES
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbDbLobjOpen, srbDbLobjCreate, srbDbLobjRead, srbDbLobjWrite and srbDbLobjUnlink.
NAME
srbDbLobjRead
SYNOPSIS
#include "srbClient.h"
int srbDbLobjRead(srbConn *conn, int fd, char *buf, int len);
DESCRIPTION
Read len bytes of the DB large object into buf. The caller must have allocated enough space to hold the data read.
RETURN VALUES
Returns the length of bytes read. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbDbLobjOpen, srbDbLobjCreate, srbDbLobjClose, srbDbLobjWrite and srbDbLobjUnlink.
NAME
srbDbLobjWrite
SYNOPSIS
#include "srbClient.h"
int srbDbLobjWrite(srbConn *conn, int fd, char *buf, int len);
DESCRIPTION
Write len bytes of buf into the dbLobj fd.
RETURN VALUES
Returns the number of bytes written. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbDbLobjOpen, srbDbLobjCreate, srbDbLobjClose, srbDbLobjRead and srbDbLobjUnlink.
NAME
srbDbLobjSeek
SYNOPSIS
#include "srbClient.h"
int srbDbLobjSeek(srbConn *conn, int fd, int offset, int whence);
DESCRIPTION
Change the current read or write location of dbLobj.
SEEK_SET - pointer is set to the value of the Offset parameter.
SEEK_CUR - pointer is set to its current location plus the value of the Offset parameter.
SEEK_END - pointer is set to the size of the file plus the value of the Offset parameter.
RETURN VALUES
Returns 0 upon success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbDbLobjOpen, srbDbLobjCreate, srbDbLobjClose, srbDbLobjRead, srbDbLobjWrite and srbDbLobjUnlink.
NAME
srbDbLobjUnlink
SYNOPSIS
#include "srbClient.h"
int srbDbLobjUnlink(srbConn* conn, int storSysType, char *resourceLoc,
char *dataPath);
DESCRIPTION
Unlink a DB large object.
0 = DB2, 1 = Illustra, 5 = Oracle.
RETURN VALUES
Returns 0 - success. Returns a negative value upon failure.
ERRORS
SEE ALSO
srbDbLobjOpen, srbDbLobjCreate, srbDbLobjClose, srbDbLobjRead and srbDbLobjWrite.
NAME
srbGetDatasetInfo
SYNOPSIS
#include "srbClient.h"
int srbGetDatasetInfo(srbConn* conn, int catType, char *objID,
char *collectionName, mdasC_sql_result_struct *myresult, int rowsWanted);
DESCRIPTION
Get some specific Info on a SRB data object. This call is normally used by the SRB server to check for access permission of data objects. If the permission checked out, it returns attributes such as "Resource Location", "Resource Type" and "Data Path Name" of the data object. These attributes are needed for subsequent access of the data by low-level drivers.
RETURN VALUES
Return 0 - success; myresult->continuation_index >= 0, ==> more results from the query. Use srbGetMoreRows() to retrieve more rows.
A negative value failure.
ERRORS
SEE ALSO
srbGetDataDirInfo and srbGetMoreRows.
NAME
srbGetDataDirInfo
SYNOPSIS
#include "srbClient.h"
int srbGetDataDirInfo(srbConn* conn, int catType, char qval[][MAX_TOKEN],
int *selval, mdasC_sql_result_struct *myresult, int rowsWanted);
DESCRIPTION
This is the general API for querying the MCAT catalog. The server uses the input qval[][] and selval[] array to generate and execute SQL queries and returns the query result in myresult. The selval[] array specifies a list of attributes to retrieve, and qval[][] specifies a lists of "=" predicates to search. Both selval[] and qval[][] must be arrays of size MAX_DCS_NUM and are indexed by values given in mdasC_db2_externs.h under the heading DCS-ATTRIBUTE-INDEX DEFINES.
For the selval[] array, setting an element of the array to 1 means that the attribute associated with this element is to be retrieved. e.g., selval[USER_NAME] = 1; means the "user_name" attribute is to be retrieved.
The qval[][] array specifies the "=" predicates to search. e.g., sprintf(qval[DATA_NAME],"'%s'", "unixFileObj1"); means that the search condition includes the term (data_name = "unixFileObj1").
An example of srbGetDataDirInfo:
mdasC_sql_result_struct myresult;
char qval[MAX_DCS_NUM][MAX_TOKEN];
int selval[MAX_DCS_NUM];
for (i = 0; i < MAX_DCS_NUM; i++) {
selval[i] = 0;
sprintf(qval[i],"");
}
sprintf(qval[DATA_NAME],"'%s'",argv[2]);
selval[PATH_NAME] = 1;
selval[RSRC_NAME] = 1;
.
.
if (srbGetDataDirInfo(conn, MDAS_CATALOG, qval, selval, &myresult) < 0) {
fprintf(stderr, "can't srbGetDataDirInfo \n");
exit_nicely(conn);
}
RETURN VALUES
Return 0 - success; myresult->continuation_index >= 0, ==> more results from the query. Use srbGetMoreRows() to retrieve more rows.
A negative value means failure.
ERRORS
SEE ALSO
srbGetMoreRows and srbGetDatasetInfo.
NAME
srbRegisterDataset
SYNOPSIS
#include "srbClient.h"
int srbRegisterDataset (srbConn* conn, int catType, char *objID, char *dataTypeName, char *resourceName, char *collectionName, char *pathName, srb_long_t dataSize);
DESCRIPTION
Register an SRB object.
RETURN VALUES
Returns 0 - success. A negative value - failure.
ERRORS
SEE ALSO
srbUnregisterDataset, srbModifyDataset and srbGetDatasetInfo.
NAME
srbUnregisterDataset
SYNOPSIS
#include "srbClient.h"
int srbUnregisterDataset (srbConn* conn, char *objID, char *collectionName);
DESCRIPTION
Unregister an SRB object.
RETURN VALUES
Returns 0 - success. A negative value - failure.
ERRORS
SEE ALSO
srbRegisterDataset, srbModifyDataset and srbGetDatasetInfo.
NAME
srbSetAuditTrail
SYNOPSIS
#include "srbClient.h"
int srbSetAuditTrail (srbConn* conn, int set_value);
DESCRIPTION
Set and Unset Audit Trail.
AUDIT_TRAIL_OFF - turn on audit trail.
AUDIT_TRAIL_ON - turn on audit trail.
GET_AUDIT_TRAIL_SETTING - return the current audit trail setting without modifying the setting.
RETURN VALUES
Returns the currently audit trail setting (after processing the latest change request).
ERRORS
SEE ALSO
NAME
srbModifyDataset
SYNOPSIS
#include "srbClient.h"
int srbModifyDataset (srbConn* conn, int catType, char *objID,
char *collectionName, char *resourceName, char *pathName,
char *dataValue1, char *dataValue2, int retractionType);
DESCRIPTION
Modify a SRB data object.
RETURN VALUES
Returns 0 - success. Returns negative - failure.
ERRORS
SEE ALSO
srbGetDatasetInfo and srbGetDataDirInfo.
NAME
srbChkMdasAuth
SYNOPSIS
#include "srbClient.h"
int srbChkMdasAuth (srbConn* conn, char *userName, char *srbAuth,
char *mdasDomain);
DESCRIPTION
Authenticate a userName/passwd for normal access. This is a call normally used by the SRB server to authenticate a user.
RETURN VALUES
Returns 0 success. The user is authenticated.
Returns negative - failure.
ERRORS
SEE ALSO
NAME
srbChkMdasSysAuth
SYNOPSIS
#include "srbClient.h"
int srbChkMdasSysAuth (srbConn* conn, char *userName, char *srbAuth,
char *mdasDomain);
DESCRIPTION
Authenticate a userName/passwd for sys admin access. This is a call normally used by the SRB server to authenticate a special user.
RETURN VALUES
Returns 0 success. The user is authenticated.
Returns negative - failure.
ERRORS
SEE ALSO
NAME
srbRegisterUserGrp
SYNOPSIS
#include "srbClient.h"
int srbRegisterUserGrp (srbConn* conn, int catType, char *userGrpName,
DESCRIPTION
Register a user group with MCAT.
"staff", "sdsc staff", "sdsc staff scientist", "sdsc senior staff scientist", "pto staff", "ucsd staff" "student", "sdsc student", "uva student", "project", "umd student", "public", "sysadmin", " deleted".
RETURN VALUES
Returns 0 - success. Returns negative - failure.
ERRORS
SEE ALSO
NAME
srbRegisterUser
SYNOPSIS
#include "srbClient.h"
int srbRegisterUser (srbConn* conn, int catType, char *userName,
DESCRIPTION
Register a user with MCAT.
"staff", "sdsc staff", "sdsc staff scientist", "sdsc senior staff scientist", "pto staff", "ucsd staff", "student", "sdsc student", "uva student", "project", "umd student", "public", "sysadmin", " deleted"
RETURN VALUES
Returns 0 - success. Returns negative - failure.
ERRORS
SEE ALSO
srbRegisterUserGrp and srbModifyUser.
NAME
srbModifyUser
SYNOPSIS
#include "srbClient.h"
int srbModifyUser (srbConn* conn, int catType, char *dataValue1,
char *dataValue2, int retractionType);
DESCRIPTION
Modify a users metadata.
RETURN VALUES
Returns 0 - success. Returns negative failure.
ERRORS
SEE ALSO
NAME
srbGetPrivUsers
SYNOPSIS
#include "srbClient.h"
int srbGetPrivUsers(srbConn *conn, int catalog,
mdasC_sql_result_struct *myresult, int rowsWanted);
DESCRIPTION
Read the privileged users list and put it in a user supplied char srbUserList[MAX_TOKEN][MAX_TOKEN].
RETURN VALUES
Return 0 - success; SrbUserList ->continuation_index >= 0, ==> more results from the query. Use srbGetMoreRows() to retrieve more rows.
A negative value failure.
ERRORS
SEE ALSO
NAME
srbGetMoreRows
SYNOPSIS
#include "srbClient.h"
Int srbGetMoreRows(srbConn *conn, int catalog,
int contDesc, mdasC_sql_result_struct *myresult, int rowsWanted);
DESCRIPTION
Get more rows of result from a srbGetDatasetInfo, srbGetDataDirInfo, srbListCollect or srbGetPrivUsers call, and put the results in a user supplied mdasC_sql_result_struct.
RETURN VALUES
Returns 0 - success. Returns negative failure.
ERRORS
SEE ALSO
srbGetDatasetInfo, srbGetDataDirInfo, srbListCollect and srbGetPrivUsers.
NAME
freeSqlResult
SYNOPSIS
#include "srbClient.h"
void freeSqlResult (mdasC_sql_result_struct *myresult);
DESCRIPTION
Free the content of myresult and then free myresult itself.
RETURN VALUES
None.
ERRORS
SEE ALSO
clearSqlResult and printSqlResult.
NAME
clearSqlResult
SYNOPSIS
#include "srbClient.h"
void clearSqlResult (mdasC_sql_result_struct *myresult);
DESCRIPTION
Clear the content of myresult. Myresult is not freed itself.
RETURN VALUES
None
ERRORS
SEE ALSO
freeSqlResult and printSqlResult.
NAME
printSqlResult
SYNOPSIS
#include "srbClient.h"
void printSqlResult (mdasC_sql_result_struct *myresult);
DESCRIPTION
Print the content of myresult.
RETURN VALUES
None.
ERRORS
SEE ALSO
freeSqlResult and clearSqlResult.
NAME
srb_perror
SYNOPSIS
#include "srbClient.h"
void srb_perror(int fd, int error_id, char *error_mnenomic, int flags);
DESCRIPTION
Print an SRB error message corresponding to the SRB error referred to by
RETURN VALUES
None.
ERRORS
SEE ALSO
NAME
srbVaultInfo
SYNOPSIS
#include "srbClient.h"
struct vaultQueElement *srbVaultInfo(srbConn* conn);
DESCRIPTION
Get Info on the SRB storage vault of a SRB server. The output is the head element of a link list of struct vaultQueElement. This call basically returns the content of the data/vaultConfig file.
RETURN VALUES
A pointer to the vaultQueElement struct. This is the head of a link list of struct vaultQueElement.
A NULL return means no result.
ERRORS
SEE ALSO
srbFreeVaultInfo and srbPrintVaultInfo.
NAME
srbFreeVaultInfo
SYNOPSIS
#include "srbClient.h"
void srbFreeVaultInfo(struct vaultQueElement *vaultQueHead);
DESCRIPTION
Free the memory taken by the vaultQueElement link list obtained from a vaultInfo call.
RETURN VALUES
None.
ERRORS
SEE ALSO
srbVaultInfo and srbPrintVaultInfo.
NAME
srbPrintVaultInfo
SYNOPSIS
#include "srbClient.h"
void srbPrintVaultInfo(struct vaultQueElement *vaultQueHead);
DESCRIPTION
Print the info represented by the vaultQueElement link list obtained from a vaultInfo call.
RETURN VALUES
None.
ERRORS
SEE ALSO
srbVaultInfo and srbFreeVaultInfo.
NAME
srbHostConfig
SYNOPSIS
#include "srbClient.h"
struct clHostElement *srbHostConfig(srbConn* conn);
DESCRIPTION
Get the host configuration of the SRB server. This call basically returns the content of the data/hostConfig file. The output is the head of a link list of clHostElement struct.
RETURN VALUES
A pointer to the clHostElement struct. This is the head of a link list of clHostElement struct.
A NULL return means no result.
ERRORS
SEE ALSO
srbPrintVaultInfo and srbFreeHostInfo.
NAME
srbPrintHostInfo
SYNOPSIS
#include "srbClient.h"
void srbPrintHostInfo(struct clHostElement *vaultQueHead);
DESCRIPTION
Print the info represented by the clHostElement link list obtained from a srbHostConfig call.
RETURN VALUES
None.
ERRORS
SEE ALSO
srbHostConfig and srbFreeHostInfo.
NAME
srbFreeHostInfo
SYNOPSIS
#include "srbClient.h"
void srbFreeHostInfo(struct clHostElement *hostQueHead);
DESCRIPTION
Free memory taken by the clHostElement link list obtained from a srbHostConfig call.
RETURN VALUES
Returns a 0 upon success, a negative value upon failure.
ERRORS
SEE ALSO
srbVaultInfo and srbPrintHostInfo.
NAME
sfoCreateIndex
SYNOPSIS
#include "srbClient.h"
int sfoCreateIndex (srbConn *conn, sfoClass class, int catType, char *hostName, char *inIndexName, char *outIndexName, char *resourceName)
DESCRIPTION
Creates an index for a set of data files stored in the SRB. Metadata about the data files to be indexed and the linear index files must have already been created in the collection "inIndexName" in the SRB. The resulting index files are stored in the collection "outIndexName" in the SRB. If the collection "outIndexName" does not exist, it will be created by the indexing service.
RETURN VALUES
Returns DC_OK upon success, a negative value upon failure.
ERRORS
SEE ALSO
sfoDeleteIndex and sfoSearchIndex.
Top of Client API Table - Appendix B
NAME
sfoDeleteIndex
SYNOPSIS
#include "srbClient.h"
int sfoDeleteIndex (srbConn *conn, sfoClass class, int catType, char *hostName, char *indexName)
DESCRIPTION
Deletes an index that has been created by the sfoCreateIndex() call.
RETURN VALUES
Returns DC_OK upon success, a negative value upon failure.
ERRORS
SEE ALSO
sfoCreateIndex and sfoSearchIndex.
Top of Client API Table - Appendix B
NAME
sfoSearchIndex
SYNOPSIS
#include "srbClient.h"
int sfoDeleteIndex (srbConn *conn, sfoClass class, char *hostName, char *indexName, void *query, indexSearchResult **myresult, int maxSegCount)
DESCRIPTION
Performs a search in the "indexName" to find the segments that intersect the range query given in the query parameter. The resulting segment metadata is written in a indexSearchResult type struct and the pointer to this struct is returned in the "myresult" parameter. The segment metadata is stored in the segments array of the "segmentInfo" struct of "myresult". The segments array is allocated by the indexing service of the server and the calling program is responsible for freeing the segment array. The "segmentCount" of "myresult" gives the number of entries in the segments array which may be less than "maxSegCount" input parameter. The returned "continueIndex" field of "myresult" will be greater than or equal to zero if there are more results to return; a -1 is returned if there is no more result. The DataCutter indexing service maintains an internal state for the query so that subsequent call of sfoGetMoreSearchResult() can return more results. If there are no more results to return, the indexing service deletes the internal query state and sets the "continueIndex" field of myresult to -1. "maxSegCount" limits the maximum number of segments to be returned from a single call to the sfoSearchIndex() function.
RETURN VALUES
Returns DC_OK upon success, a negative value upon failure.
ERRORS
SEE ALSO
sfoCreateIndex, sfoGetMoreSearchResult and sfoDeleteIndex.
Top of Client API Table - Appendix B
NAME
sfoGetMoreSearchResult
SYNOPSIS
#include "srbClient.h"
int sfoGetMoreSearchResult (srbConn *conn, sfoClass class, char *hostName, int continueIndex, indexSearchResult **myresult, int maxSegCount)
DESCRIPTION
Used to get more results after calling the sfoSearchIndex() function. The input parameter "continueIndex" should be greater than or equal to zero and set to the returned "continueIndex" value of a previous sfoSearchIndex() call to get more results. If it is set to -1, the DataCutter indexing service should close out the internal state of this query and return no more result. The definitions of the "myresult" and "maxSegCount" are the same as those of the sfoSearchIndex() function.It should be noted that in the current implementation, the dataCutter service maintains an internal state of the current query. Thus, only one query can be active at a time for a given client connection. Another call to sfoSearchIndex() will replace the state of the current query and any subsequent calls to sfoGetMoreSearchResult() will return results for the last query.
RETURN VALUES
Returns DC_OK upon success, a negative value upon failure.
ERRORS
SEE ALSO
sfoCreateIndex, sfoSearchIndex and sfoDeleteIndex.
Top of Client API Table - Appendix B
NAME
sfoApplyFilter
SYNOPSIS
#include "srbClient.h"
int sfoApplyFilter (srbConn *conn, sfoClass class, char *hostName, int filterID, char *filterArg, int numberOfInputSeg, segmentInfo *inputSeg, filterDataResult **myresult, int maxSegCount)
DESCRIPTION
Applies the filtering operation represented by the "filterID" input parameter to the segments given in the "inputSeg" input. The parameter "numberOfInputSeg" specifies the number of segments in the "inputSeg". Results of the filtering operation is written in a filterDataResult type struct and the pointer to this struct is returned in the "myresult" parameter. The returned "continueIndex" field of "myresult" will be greater than zero if there are more results to return; a -1 is returned if there is no more result. The DataCutter indexing service maintains an internal state for the operation so that subsequent call of sfoGetMoreFilterResult() can return more results. If there are no more results to return, the indexing service deletes the internal query state and sets the "continueIndex" field of myresult to -1. "maxSegCount" limits the maximum number of segments to be returned from a single call to the sfoApplyFilter() function.
RETURN VALUES
Returns DC_OK upon success, a negative value upon failure.
ERRORS
SEE ALSO
sfoSearchIndex, and sfoGetMoreFilterResult.
Top of Client API Table - Appendix B
NAME
sfoGetMoreFilterResult
SYNOPSIS
#include "srbClient.h"
int sfoGetMoreFilterResult (srbConn *conn, sfoClass class, char *hostName, int continueIndex, filterDataResult **myresult, int maxSegCount)
DESCRIPTION
Used to get more results after calling the sfoApplyFilter() function. The input parameter "continueIndex" should be greate than or equal to zero and set to the returned "continueIndex" value of a previous sfoApplyFilter() call to get more results. If it is set to -1, the DataCutter indexing service would close out the internal state of this operation and return no more result. The definitions of the "myresult" and "maxSegCount" are the same as those of the sfoApplyFilter() function.It should be noted that in the current implementation, the dataCutter service manintains an internal state of the current filtering operation. Thus, only one operation can be active at a time for a given client connection. Another call to sfoApplyFilter() will replace the state of the current operation and any subsequent calls to sfoGetMoreFilterResult() will return results for the last filtering operation.
RETURN VALUES
Returns DC_OK upon success, a negative value upon failure.
ERRORS
SEE ALSO
sfoGetMoreSearchResult and sfoSearchIndex.
rP>
Top of Client API Table - Appendix B
NAME
sfoFreeIndexResults
SYNOPSIS
#include "srbClient.h"
int sfoFreeIndexResults (indexSearchResult *myresult)
DESCRIPTION
Free all memory associated with a indexSearchResult type struct.
RETURN VALUES
Returns 0 upon success, a negative value upon failure.
ERRORS
SEE ALSO
sfoApplyFilter and sfoSearchIndex.
Top of Client API Table - Appendix B
NAME
sfoFreeFilterResults
SYNOPSIS
#include "srbClient.h"
int sfoFreeIndexResults (filterDataResult *myresult)
DESCRIPTION
Free all memory associated with a filterDataResult type struct.
RETURN VALUES
Returns 0 upon success, a negative value upon failure.
ERRORS
SEE ALSO
sfoApplyFilter and sfoGetMoreFilterResult.
Top of Client API Table - Appendix B
NAME
sfoFreeSegmentInfo
SYNOPSIS
#include "srbClient.h"
int sfoFreeSegmentInfo (segmentInfo *seg_i)
DESCRIPTION
Free all memory associated with a segmentInfo type struct.
RETURN VALUES
Returns 0 upon success, a negative value upon failure.
ERRORS
SEE ALSO
sfoApplyFilter and sfoSearchIndex.