Preface

DFdiscover Release 5.9.0

Getting Help

For software support, Please contact the DFdiscover team:

• via email, help@dfnetresearch.com.

• Visit our website, https://www.dfnetresearch.com.

Conventions

A number of conventions have been used throughout this document.

Any freestanding sections of code are generally shown like this:

 # this is example code
 code = code + overhead;

If a line starts with # or %, this character denotes the system prompt and is not typed by the user.

Text may also have several styles:

• Emphasized words are shown as follows: emphasized words.

• Filenames appear in the text like so: dummy.c.

• Code, constants, and literals in the text appear like so: main.c.

• Variable names appear in the text like so: nBytes.

• Text on user interface labels or menus is shown as: Printer name, while buttons in user interfaces are shown as Button .

• Menus and menu items are shown as: File > Exit.

Introduction

About This Guide

This guide is for programmers, and for those who aspire to be. It covers DFdiscover file formats and those tools that are executed at the shell level. As a result, some familiarity with the UNIX operating system and UNIX shell level programming is assumed.

If this is your first encounter with UNIX we strongly recommend that you look for a good UNIX programming book in your local bookstore. We would not hesitate to recommend The UNIX Programming Environment by Kernighan and Pike, and The Awk Programming Language by Aho, Kernighan and Weinberger. The former is an excellent introduction to writing UNIX shell scripts, while the latter describes awk, a simple C-like language which is ideal for manipulating data files. All of the tools described in these 2 books come as standard components of the UNIX operating system.

In the remaining chapters of this guide we will describe:

• DFdiscover Study Files A description of the location and format of all study files (including the configuration and data files) used in all DFdiscover studies

• Shell Level Programs Over a dozen shell level commands for doing such things as:

  • exporting data records from a study database

  • selecting individual data fields

  • reformatting exported data records

  • importing data records from other sources into a DFdiscover study database

  • sending and managing images and faxes

  • getting study configuration parameters

  These programs can be used in shell scripts to create your own programs. When combined with standard UNIX commands (like awk, sed, grep, etc.) they provide an extremely powerful and flexible way of creating study report programs.

• Utility Programs Descriptions of several infrequently used, yet useful, utility programs that can simplify some DFdiscover management tasks.

• Edit Checks A programming language for writing and executing edit checks that occur in real-time during data validation.

• Batch Edit Checks An extension of the edit checks language that permits execution of edit checks in batch, non-interactive mode.

• Writing Your Own Reports Create and install custom reports for a DFdiscover study.

• DFsas: DFdiscover to SAS® An environment for generating SAS® job and data files from a DFdiscover study database and study schema.

• DFsqlload: DFdiscover to Relational Database Tables A program that creates relational database tables from a DFdiscover study database and study schema.

DFdiscover Programming Limits

The following cribsheet is for DFdiscover programmers and summarizes all relevant DFdiscover database limits and formats. This cribsheet is to be used in conjunction with the information that follows in this chapter.

DFdiscover Study Files

This chapter describes the files maintained under each DFdiscover study directory. It starts with a brief overview of the top-level directories and then describes the files kept under each directory in detail. A similar chapter, System Administrator Guide, DFdiscover System Files, describes the files located under a DFdiscover installation directory.

DFdiscover study directories

The following directories are part of a standard DFdiscover study definition.

• bkgd This directory holds CRF background images created by DFsetup. Three files are created for each study plate. These include the plate background used by DFsetup (plt###), the data entry screen used by DFexplore (DFbkgd###), and a background used when printing CRFs containing database values from DFprintdb or DFexplore (DFbkgd###.png).

  NOTE: This directory must not be used to store any other files. Extraneous files will be deleted when CRF images are published from a development study to its production study, or when reverting a development study to the current state of its production study.

• data This directory holds the study data files, queries data file, and journal files. The following files are described in detail later in this chapter in The study data directory:

  • plt###.dat - per plate data files the study data files are created from the data recorded on the study CRFs

  • DFqc.dat - the query database the quality control data file contains all field level queries added to flag CRF problems and request data clarifications from investigators

  • DFreason.dat - reason for change records the reason for change data file contains all field level reasons for data changes clarifying, if required, why a field's data value was changed

  • DFin.dat - the new records database the new record queue contains all records that have been received and ICRed by the DFdiscover software but not yet validated

  • plt###.dat - missed records missed records serve as placeholders in data files indicating that requested data records will never be available

  • plt###.ndx - per plate index files per plate index files that speed database searching and hold record lock information

  • YYMM.jnl - monthly database journal files the journal files record all database transactions and provide an audit trail of additions and modifications

  • DFaudit.db - sqlite audit trail sqlite version of the journal files

• dfsas This directory contains any stored SAS® jobs that were created from DFexplore.

• dfschema This directory contains any stored DFschema files which are used to track changes to the study setup. These are used by DFaudittrace to generate records for DF_ATmods.

• drf This directory contains any .drf files (DFdiscover Retrieval Files) created by server-side tools, including reports and the program DFmkdrf.jnl. The common .drf files created by DFdiscover reports include:

  • DupKeys.drf Lists any duplicate primary keys found in the database by the last execution of DF_XXkeys.

  • VDillegal.drf Lists any illegal visit dates found in the database by the last execution of DF_XXkeys.

  • VDincon.drf Lists any inconsistent visit dates found in the database by the last execution of DF_XXkeys.

  • DFunexpected.drf Lists any unexpected data records found in the database by the last execution of DF_QCupdate.

• ecbin Any study specific scripts or programs that are run by edit check function dfexecute() and any Plate Arrival Trigger scripts defined in the DFsetup Plates View, must be stored in the study level ecbin directory. Executables can also be stored in the DFdiscover level ecbin directory for use in all studies. The study level ecbin directory has priority if the same program or script is stored in both locations.

• ecsrc This directory contains the edit check files: DFedits and any other edit check source files referenced in 'include' statements. Include files can also be stored in the DFdiscover level ecsrc directory. The study level ecsrc directory has priority if the same include file is stored in both locations.

• lib All study configuration files, created through both the DFadmin and DFsetup tools, are located in this directory. The following files are described in detail later in this chapter in The study lib directory:

  • DFcenters - sites database Study sites database includes information on all clinical sites participating in the study.

  • DFccycle_map - conditional cycle map Defines cycles that may be required, unexpected or optional, depending on specified database conditions (optional).

  • DFcvisit_map - conditional visit map Defines visits that may be required, unexpected or optional, depending on specified database conditions (optional).

  • DFcplate_map - conditional plate map Defines plates that may be required, unexpected or optional, depending on specified database conditions (optional).

  • DFcterm_map - conditional termination map Defines database conditions that signal termination of subject follow-up (optional).

  • DFCRFType_map - CRF type map Defines categories for different CRF background types (e.g. translations) (optional).

  • DFcrfbkgd_map - CRF background map Defines visits with CRF backgrounds used across multiple visits (optional).

  • DFedits.bin - published edit checks The "published" equivalent of the edit checks - for use by DFexplore clients only (optional).

  • DFfile_map - file map Specifies each unique CRF plate used in the study.

  • DFlogo.png - study logo Study logo displayed in data entry screens and reports.

  • DFlut_map - lookup table map Defines and associates lookup table names with directory names of lookup tables. Also includes definition for query lookup table, if it exists (optional).

  • DFmissing_map - missing value map Missing value codes (and labels) used in the study (optional).

  • DFpage_map - page map Used to specify descriptive labels to replace the visit/sequence and plate numbers that are used by default to identify problems on the Query Reports (optional).

  • DFqcsort - Query and CRF sort order Specifies a sort order for queries as written on Query Reports. The default is to sort by field number within plate number within visit/sequence number within subject ID (optional).

  • DFqcproblem_map - Query category code map This file contains all the system-defined and user-defined query category codes.

  • .DFreports.dat - reports history Personal file of DFdiscover report commands (and options) corresponding to reports which the user runs in the reports tool (optional).

  • DFschema - database schema Study database schema (or dictionary) describes all variables on all plates, as specified in the setup tool.

  • DFschema.stl - database schema styles Study database schema (or dictionary) describes all variable styles defined for the study in the setup tool.

  • DFsetup - study definition This is a JSON format file maintained and used by DFsetup to record all study setup information, including all plate, style and variable definitions.

  • DFsetup.db - sqlite setup database This is the sqlite database for setup and setup change history.

  • DFserver.cf - server configuration Configuration file for the study database server.

  • DFsubjectalias_map - subject alias map Used to specify descriptive labels to optionally replace the numeric subject ID used as an identifier throughout DFdiscover.

  • DFsubjectalias_map.log - subject alias change log Log all changes to the mapping of subject ID to subject alias over the course of a study.

  • DFtips - ICR tips This file contains the coordinates, field type and legal values for all variables defined on all plates. It is used by the ICR software to create the initial data record from new CRF pages that arrive as images.

  • DFvisit_map - visit map This file describes the scheduling of subject assessments during the trial and the CRF pages expected at each assessment.

  • DFwatermark - watermark This file describes watermarks used for printed output assigned by role.

  • QCcovers - Query cover pages This file contains formatting information to be included in a Query Report cover sheet. To include a cover sheet for a Query Report, QCcovers must first be defined.

  • QCmessages - Query Report messages This file contains messages to be included in Query Report cover sheets. More than one message may be included in a single Query Report, however, DF_QCreports expects the cover sheet information and all messages to fit on a single page.

  • QCtitles - Query Report titles This file describes how the report title and each title of the 3 sections of a DFdiscover Query Report are to be customized. DF_QCreports checks for the existence of this file and will use it if it exists, otherwise standard titles will be produced.

  • DFreportstyle - DFdiscover Report styling This file contains styling "instructions" for reports, excluding Legacy reports. Styling includes font choices and colors used in graphs and charts.

• lut This directory contains all study specific lookup tables. Lookup tables can also be stored in directory lut at the DFdiscover level. Study level files have priority if a lookup table with the same file name is stored in both locations. Lookup table file names must be associated with an edit check name in DFlut_map (found in the study /lib directory) before they can be used in edit checks.

• pages This directory holds all standard (SD) resolution (100 dpi) CRF image and supporting document files for all CRF pages received or uploaded during the study. Each CRF page received as an image or PDF is stored as a PNG file (Sun Rasterfile in pre-2014 DFdiscover releases), and requires about 25K bytes of disk space (i.e. 40 CRF pages per MB). Supporting documents can be audio, video, DICOM or PDF files and will be in their native format.

  DFdiscover creates subdirectories, below the study pages directory, in which to store the CRF image files. These subdirectories are named by the year and week (in yyww format) in which the CRFs arrived. For example, a study which started on January 1, 1995 would have subdirectories named: 9501, 9502, 9503, etc. through to the end of the trial.

  Within each of the yyww subdirectories, the individual CRF page files are named by the concatenation of the sequence number of the image (starting at 0001 each week) in which they arrived during that week, and their page number within the document. The file-naming format is ffffppp. For example, if the first document to arrive in the week of 9501 contained 3 pages, it would be represented by the following files beneath the study pages directory: 9501/0001001, 9501/0001002, and 9501/0001003. The ffff part of the file name is a sequential value constructed from 4 characters, each character taken from the alphabet:

  0 1 2 3 4 5 6 7 8 9 B C D F G H J K L M N P Q R S T V W Y Z

  This is essentially the digits 0 through 9 followed by the uppercase letters A through Z, with the exception of the letters A, E, I, O, U, and X. This part of the file name thus lies between 0001 and ZZZZ, which allows for a total of 30x30x30x30-1 = 809,999 images per week. Example file names within a week include:

  • 0001005 5^th page of 1^st document

  • 000B001 1^st page of 10^th document

  • 000Z011 11^th page of 29^th document

  • 0010002 2^nd page of 30^th document

  • 01C0004 4^th page of 1230^th document

• pages_hd This directory holds all higher (HD) resolution (300 dpi) CRF images and supporting document files received or uploaded during the study. DFdiscover creates subdirectories and stores the files in the same manner as the pages directory.

• reports It is recommended that all study specific reports, i.e. those programs which have been created specifically for a particular clinical trial, are stored in the study reports directory. This helps to standardize maintenance across studies.

  Documentation for study specific reports must also be stored in this directory in a file named .info, which uses the same format used to document the standard DFdiscover reports. The specific requirements of this file and its format are described in Writing Documentation for Study Specific Reports.

• reports/QC This directory is used to store the standard DFdiscover Query Reports, created by DF_QCreports. The files and directories maintained under this directory are briefly described below.

  ccc-yymmdd	The naming convention for Query Reports is a zero padded 3 digit site ID (if the site ID is greater than 999, it will be a 4 digit number), followed by a dash, and then the date on which the report was created. For example: 055-150815 would identify a Query Report created for site 55 on Aug 15, 2015. Thus you can tell from a Query Report name, both whom it was created for and when. Although this is helpful it does have one disadvantage, namely, it means that you cannot create more than one Query Report per day for an individual clinical site. If you do, the earlier one will be over-written. However, since Query Reports are sent to the clinical sites, and this isn't something you should do more than about once a week, and certainly not more than once a day, this approach to naming Query Reports works. Query Reports remain under QC until they are sent to the clinical sites. Thus any reports that you see at this level have not been transmitted to the study clinics.
  QC/sent	Query Reports are moved to this directory after they have been successfully sent to their respective clinical sites.
  QC/internal	DF_QCreports is also able to create named, internal Query Reports, which can include subjects from more than one site. These reports are written directly to this directory.
  QC_LOG	This is a plain ASCII file that lists any error messages reported during the last execution of DF_QCreports.
  QC_NEW	This is a plain ASCII file that lists the Query Reports created by the most recent execution of DF_QCreports.
  SENDFAX.log	This is a plain ASCII file in which DF_QCfax records the success or failure of its attempts to email or fax Query Reports to the clinical sites.
  SENDFAX.qup	This is a plain ASCII file that lists all Query Reports which have been queued up for transmission to the clinical sites.
  other files	Occasionally, DF_QCreports might be halted in mid-execution, by a power failure, or some other problem. In such cases it may not have time to remove the temporary files that it creates in the process of generating the Query Reports. These files generally contain a process ID#; for example you might see files that look like this 18273_N.refax, 18273_N.qalist. These temporary files can, and should, be removed.

• work The work directory contains temporary files, DFdiscover retrieval files and summary data files. It is important to be aware of the files that DFdiscover creates and maintains in this directory so that programmers do not inadvertently overwrite them.

  • Temporary Files The work directory is used for temporary files created by various reports and is the recommended location for temporary files created by study specific reports and for data files exported using DFexport.rpc. If DFexport.rpc is used to export study data files to this directory, or create temporary files here in the process of generating study specific reports, you need to be very careful in your selection of temporary file names, so that simultaneous execution of different programs does not result in temporary file name conflicts. A useful strategy is to include the process ID plus some part of the program name in all temporary file names, and also to check for and create a lock directory for any programs that should not be executed simultaneously by more than one user.

    Because the work directory is used for temporary file creation by various DFdiscover programs, you might find temporary files that were not removed because a program failed to complete due to a power failure or some other problem. Any file that is several days old, and has what looks like a process id number as part of it's name, is probably a temporary file left over when a program failed to complete successfully. Such files can be removed. This should be done periodically to clean up the study work directory.

  • Summary Files The work directory contains a number of summary data files created by DF_XXkeys and DF_QCupdate and used by other programs including DF_QCreports and DF_PTvisits. The following files are described in detail later in this chapter in The study work directory:

    • DFX_ccycle - cycle conditions met

    • DFX_cvisit - visit conditions met

    • DFX_cplate - plate conditions met

    • DFX_cterm - termination conditions met

    • DFX_keys - key fields for all required plates

    • DFX_schedule - subject scheduling and visit status

    • DFX_time1 - date and time from receipt to last modification

    • DFX_visit.dates - value in the database of all visit dates

Study File Permissions

The data, pages, and pages_hd directories must be owned by user datafax, with read,write and execute permissions for owner, and read and execute permissions for group.

The bkgd, dde, frame, lib, reports, and work directories must have read, write and execute permissions set for all users of the study group (typically this will be UNIX group studies).

These permissions are set by DFadmin on those directories which it creates when the study is first registered. The utility program DFstudyPerms is available for diagnosing and correcting study permission problems. A detailed list of study files and directories checked by this program can be found at System Administrator Guide, Maintaining study file system permissions

Format used to describe files

Each file documented in this section is described with the following attributes:

File attributes

DFdiscover Retrieval Files (DRF)

A DFdiscover Retrieval File (DRF) is an ASCII file in which each record identifies the subject ID, visit or sequence number and CRF plate number (in that order, | delimited) and optionally the image ID corresponding to a record in the study database.

DFdiscover retrieval files are created by DFexplore, DFdiscover reports, and the DFdiscover programs DFmkdrf.jnl and DFexport.rpc.

DFdiscover retrieval files created by DFexplore and DFdiscover reports and programs reside in the study drf directory. It is also recommended that DRFs created using DFexport.rpc also be saved to the study drf directory. All DRFs must end with the .drf extension. Exercise caution when selecting DRF names to avoid conflicts with those DRF file names generated by DFdiscover applications. Below is a description of the DRF file format.

Data Retrieval Files (DRF)

DFdiscover Retrieval File field descriptions

The study data directory

The study data directory holds all study data files, including the plate files that correspond to CRF plates, the query data file and the database transaction journals. These files are managed by the study database server and should not be accessed directly. Instead the records required should be exported from the study database to another file, as described in DFexport.rpc, and then read from the exported file.

Data records should never be added to these database files without doing so through the study database server. The program DFimport.rpc is available and is the recommended method of sending new or revised data records to the study database server.

Occasionally you may need to make extensive changes to an entire database file. The typical scenario is a change to one or more of the study CRFs to add or remove fields. In this case the plate definitions originally entered in the setup tool also need to be changed to match the new structure of the corresponding data files. This is a major operation. It requires that the study server be disabled while the database and setup definitions are being changed, and that the effected data files be re-indexed before the study database server is re-enabled.

Temporary data files

The following sections describe data and index files ending with .dat and .idx suffixes. Very rarely, you may notice files having the same name but with .tad or .xdi suffixes. These files are temporary files created and managed by the database server. They are present while the server performs sorting and garbage collection on a particular data file. As such they should be treated in the same manner as the other data and index files - basically, do not touch them. Typically, this is not a problem because the files are present for only a second or two.

Plate data files

plt###.dat - per plate data files

plt###.dat fields descriptions

plt###.dat - missed records

Missed record field descriptions

Query data files

DFqc.dat - the Query database

1|1|0000/0000000|
20|10|999|2100101|6|21|000000|0||1. Sample collection date
|11/10/05
|1|2|||user1 06/05/27 11:35:57|user1 06/05/27 11:35:57||1|

5|4|0000/0000000|20|3
97|2030|1415412|11|14|060424|31|1|A2. Whole Blood 1ml ID #
||1|2|||brian 06/03/23 1
2:57:20|brian 06/03/23 12:57:20|barry 06/05/11 14:10:04|1|

DFqc.dat field descriptions

Reason for change data files

DFreason.dat - reason for change records

1|3|000/0000000|254
|4|1|0123|9|phone|information provided by physician over t
he phone|nathan 03/01/10 10:23:23|nathan 03/01/10 10:23:23

Reason for data change record field descriptions

 Newly arrived data file

DFin.dat - the new records database

0|0|9145/0045001|099|4|1|0123||92/01/01|||

DFin.dat field descriptions

Journal Files

YYMM.jnl - monthly database journal files

980312|132
426|valid1|d|1|1|9807/0047008|254|5|24|99001|ABC|06/07/97|
171|097|172|096|2|055.1|121.5|2||1143|00|*|*|*|1|1|*|1|1|
98/03/12 13:24:26|98/03/12 13:24:26|

YYMM.jnl field descriptions

DFaudit.db sqlite audit trail

CREATE TABLE IF NOT EXISTS dfaudit (
  dftype    TEXT
  NOT NULL,  -- 1  type: N=new, C=changed field, D=deleted
  dfdate    INTEGER NOT NULL,  -- 2  date: yyyymmdd
  dftime    TEXT    NOT NULL,  -- 3  time: hhmiss
  dfuser    TEXT    NOT NULL,  -- 4  user
  dfsubject INTEGER,   -- 5  subject
  dfvisit   INTEGER,   -- 6  visit
  dfplate   INTEGER NOT NULL,  -- 7  plate
  dffieldid INTEGER,
-- 8  unique field id: data(=0), query(>0), reason(<0)
  dffieldchange INTEGER,   -- 9  changed
  field: data(0, unique id), query(field#), reason(field#)
  dfstatus
INTEGER,   -- 10 data, query and reason status
  dflevel   INTEGER,   -- 11 validation level
  dfmaxlevel
 INTEGER,   -- 12 maximum validation level reached
  dfcategory    TEXT,
 -- 13 missed record code, query category, reason code
  dfuse TEXT,
-- 14 missed record text, query usage, reason text
  dfoldval  TEXT,  -- 15 old value
  dfnewval  TEXT,  -- 16 new value
  dffieldnum    INTEGER,   -- 17 field number
  dffielddesc   TEXT,  -- 18 field description
  dfoldlbl  TEXT,  -- 19 old coded field label
  dfnewlbl  TEXT,  -- 20 new coded field label
  dfuid INTEGER)   -- 21 internal use for database restructure

CREATE IND
EX IF NOT EXISTS dfidx_date ON dfaudit (dfdate, dfsubject)
CREATE INDEX IF NOT EXISTS dfidx_user ON dfaudit (dfuser)
CREATE INDEX IF NOT EXISTS dfidx_fuid ON dfaudit (dfuid)
CR
EATE INDEX IF NOT EXISTS dfidx_fid  ON dfaudit (dffieldid)
CRE
ATE INDEX IF NOT EXISTS dfidx_fnum ON dfaudit (dffieldnum)
CREATE INDEX IF NOT
EXISTS dfidx_keys ON dfaudit (dfsubject, dfvisit, dfplate)
CREATE INDEX IF NOT EXISTS dfidx_vst  ON dfaudit (dfvisit)
CREATE INDEX IF NOT EXISTS dfidx_plt  ON dfaudit (dfplate)
CREATE IND
EX IF NOT EXISTS dfidx_sta  ON dfaudit (dfstatus, dflevel)
CREATE INDEX IF NOT EXISTS dfidx_lvl  ON dfaudit (dflevel)

Index files

plt###.ndx - per plate index files

Usual Name	plt###.ndx
Type	binary
Created By	DFserver.rpc
Used By	DFserver.rpc, DFshowIdx
Field Delimiter	NA
Record Delimiter	NA
Comment Delimiter	NA
Fields/Record	NA
Description	Every study data file has a corresponding index file that the study server uses to track the current status and location of each record in the data file. The index entry for a particular data record includes the value of the key fields id, plate, and visit/sequence number, the record status, the validation level, the offset of the beginning of the record into the data file, and the length of the data record. When searching for a data record by keys, it is much more efficient for the database server to search the index file for matching keys and then use the offset and length to extract the data record from the data file. Each time an existing data record is modified or a new record is added, a new entry is made at the end of the index file for the new, modified copy of that record, and the status of the old index entry (if there was one) is changed to indicate that it has been superseded by a new entry. Index and data files are sorted (on id and visit/sequence number) by the study database server, each time the server exits. Before sorting, an index file has M sorted entries at the top of the file, and N unsorted entries at the bottom of the file. When searching, the unsorted index entries are searched first in a linear fashion and then, if necessary, a binary search is performed on the sorted entries. The first 32 bytes of an index file are header information, consisting of four 4-byte numbers that identify attributes of the file as a whole, as described in plt###.ndx header bits, followed by 16 bytes of padding. Before the study database server exits, it checks this header information of each index file. If the number of unsorted entries and the number of pending deletes are both 0, then the file is already sorted and does not need further attention. The 32 bytes of header information are followed by the actual index entries. Each index entry is 32 bytes in size and is described in plt###.ndx record bits. Each index entry contains 8 bytes for the subject ID, 2 bytes for the visit number, 2 bytes for the plate number, 4 bytes for the offset into the data file, 2 bytes for the record length, a status byte and 13 bytes of padding. The status byte encodes three pieces of information: the record status (equivalent to the numeric value of the first byte of the data record), the record validation level (equivalent to the numeric value of the third byte of the data record), and the status of the index entry. This is encoded as illustrated below. The record status contains the same values as those allowed in the record status field of the data record, namely 0 through 7 (binary 111). Similarly, the validation level will take on the same values as those allowed in the validation level of the data record, again 0 through 7. The index status contains the value 2 if this is a new index entry, 1 if the index entry has been superseded by a newer one, and 0 otherwise. The size of all index files should always be a multiple of 32 bytes.

plt###.ndx header bits

plt###.ndx record bits

The study ecsrc directory

This directory contains the edit check source files.

DFedits - edit checks

edit SetInit()
{
if ( dfblank( init ) && !dfblank( init[,0,1] ) )
init = init[,0,1];
}
edit AgeOk()
{
number  age;
if ( !dfblank( p001v03 ) && !dfblank( p001v04 ) )
{
age = ( p001v03 - p001v04 ) / 365.25;

The study lib directory

This directory contains the study configuration files, those files that make this study unique from every other DFdiscover study.

DFcenters - sites database

|101 199|

|244 244|301 399|401 420|

011|Lisa Ondrejcek|DFnet|140 Lakeside Avenue, Suite 310,
Seattle, Washington, 98122|1-206-322-5932
country:USA;enroll:100|1-206-322-5931|Lisa Ondrejcek|
1-206-322-5931||1101 1149|1151 1199

Field definitions for DFcenters

DFccycle_map - conditional cycle map

IF|Visit List|Plate|Field|Value
AND|Visit List|Plate|Field|Value
[+-~]|List of conditional cycles

IF|0|1|22|6
+|2,5,8
-|3,6,9
~|4,7,10

IF|0|1|22|5
AND|0|9|13|>0
AND|0|9|36|!1
+|11

IF|1|3|9|~^A
-|11

DFcvisit_map - conditional visit map

IF|Visit List|Plate|Field|Value
AND|Visit List|Plate|Field|Value
[+-~]|List of conditional visits

IF|0|1|22|6
+|10-19
-|20-29
~|30

IF|0|1|22|5
AND|0|9|13|>0
AND|0|9|36|!1
+|40

IF|1|3|9|~HIV
-|40

DFcplate_map - conditional plate map

IF|Visit List|Plate|Field|Value
AND|Visit List|Plate|Field|Value
[+-~]Visit List|List of conditional plates

IF|0|1|22|6
+10,20|50,51
-10,20|40,41
~10,20|15

IF|0|1|22|5
AND|0|9|13|>0
AND|0|9|36|!1
+91-95|16

IF|1|3|9|yes
-91|16

DFcterm_map - conditional termination map

IF|Visit List|Plate|Field|Value
AND|Visit List|Plate|Field|Value
A or E

IF|0|1|22|6
A

IF|6|1|22|5
AND|6|9|13|>0
AND|6|9|36|!1
E

DFCRFType_map - CRF type map

PAPER|Print Version
CHINESE|Chinese
ENGLISH|English
SWEDISH|Swedish
FRENCH|French
SPANISH|Spanish

DFcrfbkgd_map - CRF background map

10|11,13-14,16-17,19-20|Quarterly visits
12|15,18|Annual visits
22|25,28|Annual lab work

DFedits.bin - published edit checks

DFfile_map - file map

001|Dosage of Study Drug (DOST-2)|2|2
002|Concomitant Medication (COM)|1|2
003|Written Consent (CONS-w)|1|2
005|Diagnosis (DSM-III-R)|1|2
006|Psychiatric History (PSH)|1|2
200|Death Report|1|1

DFlogo.png - study logo

DFlut_map - lookup table map

QC|DF_QClut
QCNOTE|DF_QCnotelut
QCREPLY|DF_QCreplylut
REASON|DF_Reasonlut
MEDS|meds.dict
COSTART|costart.dict
WHO|who.dict

DFmissing_map - missing value map

*|Not Available
.|Not Applicable
-|Temporarily Unavailable
.U|Unknown

DFpage_map - page map

PLATE   SEQNO

PAGE (PLATE-SEQ)
018|001|MED VISIT1
*|001|VISIT1 (%P-%S)
025|*|TERM (%P-%S)
*|*|(%P-%S)

DFqcsort - Query and CRF sort order

plate#|visit#|sort_order#

12|*|1

12|*|1  (1)
*|1|2   (2)
25|2|3  (3)
13|2|4  (4)
5|2|5   (5)
*|2|6   (6)
*|*|7   (7)

DFqcproblem_map - Query category code map

1|Missing|1|0
2|Illegal|1|0
3|Inconsistent|1|0
4|Illegible|1|0
5|Fax Noise|1|0
6|Other|1|0
21|Missing Page|0|0
22|Overdue Visit|0|0
23|EC Missing Page|0|0

1|Missing|1|0
2|Illegal|1|0
3|Inconsistent|1|0
4|Illegible|1|0
5|Fax Noise|1|0
6|Other|1|0
21|Missing Page|0|0
22|Overdue Visit|0|0
23|EC Missing Page|0|0
30|Clinical QC|1|1
50|Needs Review|0|1
37|Refuted|1|2

.DFreports.dat - reports history

DFschema - database schema

%z DFSTUDY_USER1 TITLE

%y TITLE Acme Pharma Next Gen Skin Rejuvenator

%S 254
%N 12
%B 1990
%Y 0 0
%Z 40

%P 1
%p Blood Pressure Screening Visits
%n 35
%t simple
%E 1
%R PrintCRF.shx

%I 1
%i 10
%V DFstatus1
%v DFSTATUS
%D DFdiscover Record Status
%T choice SimpleChoice
%A required
%W 1
%L 0~6
%C 0 lost
%C 1 final
%C 2 incomplete
%C 3 pending
%C 4 FINAL
%C 5 INCOMPLETE
%C 6 PENDING

%I 2
%i 11
%V DFvalid1
%v DFVALID
%D DFdiscover Validation Level

DFschema.stl - database schema styles

%S 254
%N 51

%I 1
%T int SimpleNumber
%A optional

%I 2
%T date SimpleDate 1940 0
%A optional
%W 8
%F dd/mm/yy

%I 3
%T string SimpleString
%A optional

%I 4
%T choice SimpleChoice
%A optional
%L $(choices)
%l 0
%c 0

Schema codes and their meaning

DFsetup - study definition

DFsetup.db - sqlite setup database

DFserver.cf - server configuration

STUDY_NUMBER=254
STUDY_NAME=DFdiscover Acceptance Test Study
STUDY_DIR=/opt/studies/val254
PAGE_DIR=$(STUDY_DIR)/pages
WORKING_DIR=$(STUDY_DIR)/work
PRINTER=hp4000
THRESHOLD=500000
STUDY_URL=http://www.ourstudy.com/index.html
LOCAL_CACHE=1
VERSION_STRICT=0
AUTO_LOGOUT=5|30
ADD_IDS=1||
VERIFY_IDS=0|

DFserver.cf configuration keywords

DFsubjectalias_map - subject alias map

150040|T1-B-40
150041|T1-B-41
150042|T1-B-42
150043|T1-B-43
150044|T1-B-44
150045|T1-B-45
200006|T2-A-6
200007|T2-A-7
200008|T2-A-8
200009|T2-A-9
200010|T2-A-10

DFsubjectalias_map.log - subject alias change log

200703|174748|userB|n|500003|T5-B-3|
200703|180202|userB|n|250009|T2-A-259|
200703|180219|userB|c|250009|T25-A-9|T2-A-259
200722|112345|userA|d|200004||T2-A-4
200722|112345|userA|d|200005||T2-A-5

Field definitions for DFsubjectalias_map.log

DFtips - ICR tips

# Study Number: 253
# Study Name: Demo Study 253
# Total Pages: 14
# Created: Wed Dec  1 12:33:29 2004
DELIMITER   |
PAGE    1
PLATE   1   # Blood Pressure Screening Visits
SEQ_CODE    1
PREOP   echo "$image $data" >> /tmp/test
DO_ICR  2
# FAX: plt001
FIELD   1 # ID001
TYPE    int
LEGAL   $(ids)
PART    106 84 2 50 25
PART    167 84 3 74 25
.
FIELD   2 # PINIT001
TYPE    string  skip
PART    380 84 3 74 25
.
FIELD   3 # AGE
TYPE    int
LEGAL   21-90
PART    79 182 2 50 25
.
FIELD   4 # SEX
TYPE    choice
LEGAL   0,1,2, blank
PART    0 0 0 0 0 # blank
PART    277 189 1 1 15 # male
PART    277 212 1 2 15 # female
.
FIELD   5 # RACE
TYPE    choice
LEGAL   0-65535
PART    0 0 0 0 0 # no choice
PART    464 188 1 1 15 # Caucasian
PART    464 213 1 2 15 # African American
PART    464 238 1 3 15 # Asian
PART    464 263 1 4 15 # Other
.
FIELD   6 # RACEOTH
TYPE    string  skip
PART    588 257 1 107 26
.
FIELD   7 # S1DATE
TYPE    date
FORMAT  dd/mm/yy
DATES   1940 0
LEGAL   01/01/01-today
PART    172 330 2 49 25
PART    232 330 2 50 25
PART    295 330 2 49 25

DFtips plate specific keywords

DFtips variable specific keywords

DFvisit_map - visit map

0|B|Baseli
ne|1|9 (mm/dd/yy)|0|0| 1 2 3 4 5 6 7 8||99|3 4 1 2 5 6 8 7
10|S|One Week Followup|9|9 (mm/dd/yy)|7|0| 9 10 14||||
20|S|Two Week Followup|9|9 (mm/dd/yy)|14|0| 9 10||||
30|T|Termination Visit|9|9 (mm/dd/yy)|21|0| 11 12||||

DFvisit_map field descriptions

Visit codes and their meaning

DFwatermark - watermark

draft|
Draft watermark|DRAFT|20|255,0,0|50|1|1|unrestricted,Sites

DFwatermark field descriptions

QCcovers - Query cover pages

<FOR center="#list">

<FOR centers="1~19,30~300">
<TEXT font="^P12 fB">
--------------------------------------------------------------
PLEASE DELIVER THIS REPORT TO THE FITNESS STUDY COORDINATOR
TO: <WHO>
Site <CENTER>: <WHERE>
<DATE>
</TEXT>
</FOR>
<FOR centers="20~29">
<TEXT font="^P12 fB">
-------------------------------------------------------------
DONNEZ CE RAPPORT AU COORDINATOR DE RECHERCHE POUR FITNESS,
S'IL VOUS PLAIT
TO: <WHO>
Site <CENTER>: <WHERE>
<DATE>
</TEXT>
</FOR>

QCmessages - Query Report messages

<FOR center="1~300">
<TEXT font ="^P12 fB">
--------------------------------------------------------------
PLEASE NOTE:
</TEXT>
<TEXT font="^P10 fCW">
The next FITNESS Study investigator's meeting will be held on
Study investigator's meeting will be held on October 9-10,
1999 in Orlando, Florida.
Location and times will follow at a later date.
</TEXT>
</FOR>
<FOR center="1,4,22~24,127">
<TEXT font="^P12 fB">
--------------------------------------------------------------
IMPORTANT!
</TEXT>
<TEXT font="^P10 fCW">
We have not received any corrections related to the
Quality Control reports we have sent to your site over the
last 2 months. Please make every attempt to resolve the
outstanding data queries as soon as possible. By protocol
and agreement with the FDA, all adverse events must be
reviewed by us within 3 days of each subject assessment.
If you are having problems of any kind, please let us know.
</TEXT>
</FOR>

QCtitles - Query Report titles

# TITLE AT THE TOP OF EACH PAGE OF AN EXTERNAL QUERY REPORT
<EXTERNAL font="^P12 fB">
FITNESS STUDY REPORT <NUM> for: <MON> <DAY>, <YEAR> at <TIME>
TO: <WHO>, <WHERE>
</EXTERNAL>
# TITLE AT THE TOP OF EACH PAGE OF AN INTERNAL QUERY REPORT
<INTERNAL font="^P12 fB">
INTERNAL QC REPORT: <NAME> page <PAGE> created:
<YEAR>-<MON>-<DAY>
</INTERNAL>
# SUB-TITLE ABOVE THE SUBJECT STATUS LIST
# (Part 1 of a standard external Query Report)
<STATUSLIST font="^P10 fB">
SUBJECT VISIT SCHEDULE:
Note: * identifies subjects with data queries.
</STATUSLIST>
# SUB-TITLE ABOVE REFAX LIST
# (Part 2 of a standard external Query Report)
<REFAXLIST font="^P10 fB">
CRF CORRECTIONS: Initial and date each correction.
Resend all corrected pages without delay.
</REFAXLIST>
# SUB-TITLE ABOVE THE QUESTION AND ANSWER LIST
# (Part 3 of a standard external Query Report)
<QALIST font="^P10 fB">
QUERIES: Print your response below each question and
send back this page.
</QALIST>

QCtitles font specifications

Variables available for use in QCcovers, QCmessages, and QCtitles

DFreportstyle - DFdiscover Report styling

<script type="application/json" id="dfreportjson">{
"palettes": {
"Corporate": ["#26456e", "#3a87b7", "#b4d4da", "#1c5998",
#67add4", "#1c73b1", "#7bc8e2"],
"Diverging" : ["#000066", "#6699ff", "#ffff00", "#3333ff",
"#99ccff", "#ffff99", "#cc9900",
"# 3366ff", "#ccccff", "#ffff66", "#663300"],
"Monochrome": ["#1a1a1a", "#666666", "#b3b3b3", "#333333",
"#808080", "#cccccc", "#4d4d4d",
"#999999", "#e6e6e6"],
"Color blind": ["#006ba4", "#ff80e", "#595959", "#5f9ed1",
"#c85200", "#898989", "#a2c8ec",
"#ffbc79", "#cfcfcf"]
}
}</script>
<style id="dfreportcss">
/* Body font */
body {
font: 100% arial, sans-serif;
}
/* Title font */
.contextText.reportNameText {
font: italic bold 1.5em Helvetica, serif;
}
/* Heading font */
.c3-title {
font: 1.2em Helvetica, serif;
}
/* Axis label font */
.c3-axis-y-label, .c3-axis-x-label {
font: normal 1.2em Helvetica, sans-serif;
}
</style>

The study lut directory

This directory contains all the lookup tables used in the study.

Lookup tables

miss|Please provide a response.
dtformat|Date format is YYYY/MM/DD. Please review and correct
this date.
dtillegal|Date provided is illegal. Legal dates are
2016/01/01-today.
dtbeforeic|Date provided is before date of informed consent.
ongo|End date is provided, but Ongoing is checked.
enddt|End date is prior to Start date.

#N:LL Term|Pref Term|SOC Term|LL Code
#L:Low Level Term|Preferred Term|System Organ Class|Low
Level Code
#F:1|1-4
Abdomen enlarged|Abdominal distension|Gastrointestinal
disorders|10000045
Abdomen mimicking acute|Acute abdomen|Gastrointestinal
disorders|10000047
Abdominal cramp|Abdominal pain|Gastrointestinal
disorders|10000056

a|before
AA|auto accident (MVA)
A&P|anterior & posterior
abd|abdomen
abg|arterial blood gases
ac|before meals
acid phos|acid phosphatase
ACLF|adult congregate living facility
ACTH|adrenocorticotrophic hormone
acute MI|acute myocardial infarction
ad|right ear
Ad|up to
Ad lib|as desired
BaE|barium enema
Ba|barium
BB|blood bank
BCP|biochemical profile
BE|barium enema
BEE|basal energy expenditure

The study work directory

This directory contains summary files created by DF_QCupdate, DFdiscover retrieval files, and temporary files created during the execution of reports and other programs.

DFX_ccycle - cycle conditions met

11020|10|1|2
11020|5|4|98
11020|60|6|0

DFX_ccycle conditional cycle map conditions that were met

DFX_cvisit - visit conditions met

11020|10|1|2
11020|5|4|98
11020|60|6|0

DFX_cvisit conditional visit map conditions that were met

DFX_cplate - plate conditions met

11020|10|1|2
11020|5|4|98
11020|60|6|0

DFX_cplate conditional plate map conditions that were met

DFX_cterm - termination conditions met

11020|10|1|2
11020|5|4|98
11020|60|6|0

DFX_cterm conditional termination map conditions that were met

DFX_keys - key fields for all required plates

1044|3|51|1|7
1044|2|61|1|7
1044|1|92|1|7
1044|2|211|1|7

DFX_keys key fields from all required plates

DFX_schedule - subject scheduling and visit status

1031|1|0|C|S|1|7||||||03/09/21|37884||
1031|1|0|91|X|1|7|||||||03/09/21|37884|||
1031|1|0|92|X|1|1|||||||03/09/28|37891|||
1031|1|1|C|R|1|7||||||~03/10/07|37900||
1031|1|1|51|P|1|7|||||||03/10/05|37898|03/10/05|37898|
1031|1|1|0|B|4|0|-1|10|51||||||||
1031|1|1|61|r|1|1|||||||04/01/06|37991|||
1031|1|1|3|S|2|1|||||||04/01/06|37991|||
1031|1|1|6|S|1|0|||||||04/04/07|38083|||
1031|1|1|9|S|1|0|||||||04/07/07|38174|||
1031|1|1|12|T|1|0|||||||04/10/06|38265|||
1031|1|1|81|E|3|0|||||||||||
1031|1|2|C|E|0|0||||||||||
1031|1|2|80|A|3|0|||||||||||

Field definitions for cycle records in DFX_schedule

Field definitions for visit records in DFX_schedule

DFX_time1 - date and time from receipt to last modification

1|1501|0|1|1|6|1995160154|03/30/95||95/04/24|95/04/24|34787|0|34812|34812||09:43:37|09:43:37|0|583.61|583.61
1|1501|0|1|4|4|1995130086|03/30/95||95/03/30|95/04/24|34787|0|34787|34812||18:41:04|09:43:43|0|1121.07|583.71
1|1501|1|2|1|6|1995200106|04/12/95||95/05/19|95/05/23|34800|0|34837|34841||16:46:37|11:12:18|0|1006.62|672.3
1|1501|1|2|5|5|1995160154|04/12/95||95/04/24|95/05/19|34800|0|34812|34837||09:43:59|16:46:39|0|583.98|1006.65

DFX_time1 timing data for each CRF page

DFX_visit.dates - value in the database of all visit dates

1127|0|03/08/19|37851|3
1127|3|03/11/20|37944|5
1127|6|04/02/18|38034|5
1127|51|03/08/10|37842|10
1127|51|03/08/10|37842|13

DFX_visit.dates the value of all dates using the VisitDate attribute

Shell Level Programs

DFaccess.rpc — Change access to a study database or DFinbound.rpc, or query their current access status.

DFattach — Attach one or more external documents to keys in a DFdiscover study

DFaudittrace — Used by the DF_ATmods report to read study journal files. DF_ATmods produces an audit trail report showing database modifications for the specified study.

DFbatch — Process one or more batch edit check files

DFcompiler — Compile study-level edit check programs and output any warnings and/or errors encountered in the syntax.

DFdisable.rpc — Disable a study database server or incoming daemon to make them unavailable to clients and incoming images

DFenable.rpc — Enable a study database server or incoming daemon after it is disabled

DFencryptpdf — Protect a PDF file by encrypting it with the specified password

DFexport — Client-side, command-line interface for exporting data by plate, field or module; exporting scheduling and query reports; exporting change history; or exporting components of study definition

DFexport.rpc — Export data records from one or multiple plates from a study data file

DFfaxq — Display the members of the outgoing transmission queue

DFfaxrm — Remove transmissions from the outgoing queue

DFget — Get specified data fields from each record in an input file and write them to an output file

DFgetparam.rpc — Retrieve and evaluate the value of the requested configuration parameter

DFhostid — Display the unique DFdiscover host identifier of the system

DFimageio — Request a study CRF image from the database

DFimport.rpc — Import database records to a study database from an ASCII text file

DFjournal — Run by DFsqlload to extract data changes that were made since the last time DFsqlload was run from the study journal files.

DFlistplates.rpc — List all plate numbers used in the study

DFlogger — Re-route error messages from non-DFdiscover applications to syslog, which in turn writes the messages to the system log files, as configured in /etc/syslog.conf.

DFpass — Locally manage user credentials for client-side command-line programs.

DFpdf — Generate bookmarked PDF documents of CRF images

DFpdfpkg — Generate multiple bookmarked PDF files for specified subject IDs or sites

DFprint_filter — Format input file(s) for printing to a PostScript® capable printer, and print to a specified printer

DFprintdb — Print case report forms merged with data records from the study database

DFpsprint — Convert one or more input CRF images into PostScript®

DFqcps — Convert a Query Report, previously generated by DF_QCreports, into a PDF file with barcoding, prior to sending the report to a study site

DFreport — Client-side, command-line interface for executing reports

DFsas — Prepare data set(s) and job file for processing by SAS®.

DFsendfax — Transmit a plain text, PDF, or TIFF file to one or more recipients via email or fax

DFsqlload — Create table definitions and import all data into a relational database

DFstatus — Display database status information in plain text format

DFtextps — Convert one or more input files into PDF

DFuserdb — Perform maintenance operations on the user database

DFversion — Display version information for all DFdiscover executables (programs), reports, and utilities

DFwhich — Display version information for one or more DFdiscover programs, reports and/or utilities

Introduction

The DFdiscover distribution includes several programs that can be executed on their own from a command line, or combined in shell scripts to build other programs (e.g. to export data sets for systems, or create study specific reports). All of the programs described in this section can be found in the DFdiscover executables directory, /opt/dfdiscover/bin with the exception of DFmkdrf.jnl and DFmkdrf.ec which are in /opt/dfdiscover/ecbin.

Permission to run these programs is controlled by UNIX file permissions that are set on DFdiscover installation to execute for user 'datafax' and users in group 'studies', but not for 'other' users.

The remainder of this chapter is a reference for each shell program.

User Credentials

DFdiscover includes many programs. Some operate only on a database server computer (referred to as server-side) while others operate on a user's desktop computer (referred to as client-side). The application of user login and permissions varies slightly in these two environments.

Interactive client-side programs like DFexplore, DFsetup, DFsend and DFadmin require the user to interactively provide a valid username and password each time they connect to a database server.

Conversely, the majority of the shell programs require a UNIX-like shell environment to function, and hence they may only be executed directly on a DFdiscover database server machine. Access to specific database information is granted based upon the DFdiscover permissions assigned to the user, where the user is identified by their UNIX login name.

Starting with release 2014.0.0, DFdiscover includes a third type of program. Programs of this type require a command line environment to execute and can be run from either the user's desktop computer (client-side) or from the database server computer (server-side). These programs are DFbatch, DFsas, DFpdfpkg and DFexport. To authenticate, they require the same credentials that an interactive program does, specifically the username and password required to connect to a database server. Since there is no visual login dialog, these programs require a different interaction for authenticating, and, except for DFsas, there are two solutions for the user:

1. command-line options, the user can specify -S servername, -U username and -C password when the program is run, or

2. environment variables, the user can set the variables DFSERVER servername, DFUSER username and DFPASSWD password before the program is run.

For DFsas, only the environment variable method is available.

Specification of command-line options takes priority if both command-line options and environment variables are supplied. It is not possible to mix solutions using a subset of command-line options and a complimentary set of environment variables. If any command-line option is specified, the expectation is that the command-line solution is preferred; otherwise, the environment variable solution is required.

Good Password Management

In many circumstances, it is poor practice to display or store a plain text password. Including a password in a shell script or in a cron job is not secure behavior, and is not recommended. Hence we strongly recommend that users do not use either the -C password command-line option or the DFPASSWD password environment variable. Instead, we encourage users to make use of the DFpass program to locally manage their DFdiscover passwords. After creating a matching entry with DFpass, a user does not need to subsequently specify a password with either -C password or DFPASSWD password.

The recommended practice then is to create/manage passwords locally using DFpass and subsequently use either the two command-line options, -S servername and -U username, or the two environment variables DFSERVER servername and DFUSER username.

Use of locally managed passwords is limited to the DFattach, DFbatch, DFpdfpkg, DFexport, DFreport and DFuserPerms programs.

Order of Evaluation

When determining the authentication credentials to use for one of DFbatch, DFpdfpkg or DFexport, the order of evaluation is as follows and only the first matching combination, if any, is used:

1. If all three command line options -S servername, -U username and -C password are specified, use them.

2. If all three environment variables DFSERVER servername, DFUSER username and DFPASSWD password are defined, use them.

3. If both of the command line options -S servername and -U username are specified, use them together with the matching password entry previously stored via DFpass.

4. If both of the environment variables DFSERVER servername and DFUSER username are defined, use them together with the matching password entry previously stored via DFpass.

Organization of Reference Pages

The description of each program in this reference is divided into sections.

• Name The official name of the program.

  The program name is case sensitive and must be typed exactly as specified. This section also provides the brief purpose of the program.

• Synopsis Lists all of the valid options for the program invocation.

  Each option is either optional or required. Most options are, not surprisingly, optional, a few are required, and yet others may require one option from a specific subset. Each option type is indicated with a particular notation. Except for some required options, a program option begins with - and is followed by an option letter. In some cases this may be enough to select the option; in other cases, the option letter will be further followed by an option string.

NOTE: An option letter may be given different meanings in different programs. There is no requirement that the same option letter infers the same meaning when used across different programs.

Notation for program options

• Description Describes the program purpose in greater detail.

  This section describes exactly how the program works in terms of environment, input requirements, output format, dependencies, etc. It may also include detailed information about the program behavior for commonly used combinations of options.

• Options Detailed listing of available options, their use, and their meaning.

• Examples Illustrates the program options and output by way of example(s).

• See Also Lists other reference materials, typically other programs, that are relevant to the program. This in an optional section.

• Limitations Describes any limitations in the use or capabilities of the program. This in an optional section.

DFaccess.rpc

DFaccess.rpc — Change access to a study database or DFinbound.rpc, or query their current access status.

Synopsis

DFaccess.rpc { [-s study] | [-inbound] } [ [-ro]| [-rw] | [-enable] | [-q] ] [ [-restrict reason string] | [-disable reason string] ]

Description

DFaccess.rpc provides the same functionality from the command line that is available in the DFadmin Status View. Like all shell level programs, DFaccess.rpc can be executed by datafax or any other user with a UNIX login account who is in group studies.

Read-Only Mode While a study is in read-only mode DFexplore and DFsetup can be used to view but not change study data and setup information. Studies are typically put into Read-Only mode because data collection is complete, all queries have been resolved, and the data has been verified against source documents. Statistical analysis may be ongoing or an FDA audit may be in progress thus the study cannot be disabled, but no further changes are expected or allowed.

By design, read-write access to a study database is only granted to tools that write new data records or update existing data records. All other tools have read-only access. Thus when DFaccess.rpc is used to switch a study sever to read-only mode only those tools that normally have read-write access will be affected. This includes the following:

Software Components Affected by Read-Only Mode

When a study is put into read-only mode it will remain in that mode until reset using DFadmin or DFaccess.rpc with the -rw option.

While a study is in read-only mode, any incoming pages barcoded for that study will be moved to DFrouter's identify directory.

Restricted Access Access to a study can be restricted for DFdiscover and study administrators only using the -restricted option. While a study is restricted other users are not allowed to login, even in view-only mode.

When a study is restricted using DFaccess.rpc or DFadmin users who are logged in will be able to continue working until they log off, but only administrators will be able to make new login connections.

While a study is restricted, any incoming pages barcoded for the study will be processed normally and sent to the study new image queue, but only DFdiscover and study administrators will be able to view or process them.

A study may be made both restricted and read-only, but these options cannot be used together on the same command line, it requires 2 separate executions of DFaccess.rpc; the order is not important.

When a study is put into restricted mode it will remain in that mode until reset using DFadmin or DFaccess.rpc with the -enable option or until the study server is restarted.

Disabled Access.  A study database may be disabled, making it unavailable to all users, using the -disable option.

While a study is disabled, any incoming pages barcoded for the study will be held until the study is enabled, and will not show up in the study new image queue until processing is triggered by the arrival of the next document received by the DFdiscover server. It is not necessary for the next document to contain pages barcoded for the study in question or for any study.

When a study is disabled it will remain in that mode until reset using DFadmin or DFaccess.rpc with the -enable option.

To complete a change in access mode, DFaccess.rpc must be the only client accessing the server. Thus DFaccess.rpc fails if there are any tools with open connections to the study server.

Options

Exit Status

DFaccess.rpc exits with one of the following statuses:

Examples

Enable read-only access to study #123, do lengthy operation, and then reinstate read-write access

# /opt/dfdiscover/bin/DFaccess.rpc -s 123 -ro

...some lengthy operation...

# /opt/dfdiscover/bin/DFaccess.rpc -s 123 -rw

Restrict study #123 in read-only access to DFdiscover and study administrators only, while they analyze and run reports on a static database, and then reinstate normal access to all users

# /opt/dfdiscover/bin/DFaccess.rpc -s 123 -restrict analysis
# /opt/dfdiscover/bin/DFaccess.rpc -s 123 -ro

...administrators perform their analyses

# /opt/dfdiscover/bin/DFaccess.rpc -s 123 -rw
# /opt/dfdiscover/bin/DFaccess.rpc -s 123 -enable

Query the current status of DFinbound.rpc

% /opt/dfdiscover/bin/DFaccess.rpc -inbound -q

DFattach

DFattach — Attach one or more external documents to keys in a DFdiscover study

Synopsis

DFattach [-S server] [-U username] [-C password] [ [-doc docname] | [-subject # -visit # -plate # -doc docname] | [-idrf input_drf] | [-dir directory] ] [-odrf output_drf] [-log logfile]{study#}

Description

DFattach is a command-line utlity that mimics, and extends, the Plate > Attach Subject Document facility in DFexplore. It's primary function is to permit one or more documents to be attached to one or more record keys from a command-line interface.

In DFexplore, it is possible to attach a document to the current data record; using DFattach it is possible to attach documents to many records with simple commands.

Permitted document types are the same as those allowed by DFexplore.

For each document successfully attached to a key:

• an entry is added to the system work/fax_log,

• a DRF entry is appended to output_drf if -odrf output_drf is given as an option. The DRF entry contains 5 fields: the key of the record in the first 3 fields, the image ID assigned to the document, and the original document name.

Attaching Documents

There are 4 different methods for attaching documents.

1. In it simplest form, DFattach attaches a single document to a single key. Specify the document with -doc docname. The key is determined from the filename of the document. The document filename must have the format subject_visit_plate[_other][.extension] where subject, visit and plate are each valid, numeric identifiers for the key.

2. Attach a single document to the key given as arguments. In this method, each of -subject # -visit # -plate # -doc docname must be explicitly specified. This has the advantage that the filename of the document to be attached does not have to change to follow a format.

3. Read the input DRF. For each record in the DRF, the key is in the first three fields and the document name is in the fourth field. Attach the named document to the key. Repeat for each record.

4. Process each file and subdirectory in -dir directory. For each file, if the filename has the format subject_visit_plate[_other][.extension], treat the file as a document to attach to the key. Repeat for all matching files in the directory. Then recursively repeat for each sub-directory.

Permissions

Permissions are enforced for the user identified by the supplied credentials. Minimally, the user must have a study role permission that permits Server - Attach document and DFexplore - Data - Attach subject document. Additional permissions may be required dependent upon the action

Database Actions

For each document and key, [id, visit, plate], to be attached, the steps are:

1. Confirm that the combination of visit and plate are defined in the visit map. If it is not, output an error message and skip this document.

2. Lock the key. If the key cannot be locked, output an error message and skip this document.

3. If the key does not exist in the database, a record with pending status is created. The document becomes the primary image. Create Data permission is required.

4. If the key matches an existing missed record, the missed record is deleted and a record with pending status is created. The document becomes the primary image. Delete Missed record and Create Data permission are required.

5. If the key exists, and there is already a primary image, the document is attached as a secondary image; otherwise, the document is attached as the primary image. Modify Data permission is required.

Options

Exit Status

DFattach exits with one of the following statuses:

Examples

In each of the following examples, the options -S server, -U username and -C password are not shown but are required.

Attach 1234_1_12.pdf to subject 1234, visit 1, plate 12 in the database for study 100

% DFattach -doc 1234_1_12.pdf 100

Attach radiograph.pdf to subject 30001, visit 10, plate 200 in the database for study 22

% DFattach -subject 30001 -visit 10 -plate 200 -doc radiograph.pdf 22

Attach all of the documents in directory /tmp/newdocs to the matching keys in study 200 and record the successes in /tmp/results.drf

% ls /tmp/newdocs
250001_0_1.pdf
250002_0_1.pdf
350001_0_1.pdf
350001_0_2.pdf
350001_1_10.pdf
% DFattach -dir /tmp/newdocs -odrf /tmp/results.drf 200

DFaudittrace

DFaudittrace — Used by the DF_ATmods report to read study journal files. DF_ATmods produces an audit trail report showing database modifications for the specified study.

Synopsis

DFaudittrace {-s #} [-d date1[-date2]] [-q] [-r][-N] [-A] [-I subjectID] [-P #] [-V #] [-ffieldlist] [-v vfence] [-x output.xlsx]

Description

DFaudittrace reads a record from the study journal files, calculates any changes from the previous record for those keys and then outputs the differences to DF_ATmods.

The output from DFaudittrace consists of one or more ASCII records, each having 20 fields. Each field is separated by a pipe-delimiter (|). Field contents depend on:

1. whether DFaudittrace is describing a new record (type N), a changed field (type C), or a deleted record (type D). This information is found in the first field of each record.

2. whether DFaudittrace is describing a data record, query record or a reason record. This information is found in the 8th field of each record.

The tables describe each of the 20 fields for each record type (data, query, reason) for each type of change (new record, changed field, deleted record). Fields 1 to 7 are consistent across all records output by DFaudittrace and have been described in the first table. Fields 8 to 20 depend on the record type and the type of change, and are described in tables 2 to 4.

DFaudittrace Output: Fields 1 to 7

New Records

Changed Field

Deleted Record

It is also important to note the following about DFaudittrace:

1. If a record or a query is deleted and then re-created, the re-created record or query is considered to be new record (N), not a change (C) to the previous record.

2. In queries, the user is sometimes, but not always set when a query is created by the user. The user is never set when the query is modified or deleted, or when it is created by the report DF_QCupdate. Unknown user names appear in the output as unknown for dates before June 1, 1995 and as ?? thereafter.

3. In queries, the validation level is set when the record is created and reset when the record is modified. The validation level of a query does not change when only the record's validation level changes.

4. If the -v vfence option is used when running DFaudittrace, the effect is different for data records and queries. Data changes are output for records journaled after the case reached or surpassed the vfence validation level. Query changes are output for queries journaled after the query was modified at or beyond the vfence validation level.

Options

Exit Status

DFaudittrace exits with one of the following statuses:

Examples

Execute DFaudittrace to output journal information for study 254 between the dates of January 1, 2001 and January 15, 2001.

% DFaudittrace -s 254 -d 20010101~20010115

Execute DFaudittrace to output journal information for study 254 between the dates of January 1, 1998 and today, for field numbers 10-13.

% DFaudittrace -s 254 -d 19980101-today -f 10-13

Execute DFaudittrace to output all journal information for study 254, for records that have existed at or above a validation level of 2 at some point during the study.

% DFaudittrace -s 254 -v 2

Execute DFaudittrace to generate an Excel file containing all changes for data related to subject 44002.

% DFaudittrace -s 254 -I 44002 -x 44002history.xlsx

DFbatch

DFbatch — Process one or more batch edit check files

Synopsis

DFbatch[-S server] [-U username] [-Cpassword] [-b batchnames] [-p stylesheet] [-exec] [[-o outhtml_file] | [-O outhtml_dir] ] [-x out_xlsx][-X out_dir_name] [-e error_file] {-i control_file}{#}

Description

DFbatch is a framework for edit check execution in command-line or unattended mode. The framework specifies which data records are to be operated upon by edit checks and what is to happen when edit checks are invoked. It uses the same edit checks that are used in interactive edit checks through DFexplore and introduces no changes to the language.

Complete reference documentation for DFbatch can be found in Batch Edit Checks.

Options

Exit Status

DFbatch exits with one of the following statuses:

See Also

DFcompiler

DFcompiler — Compile study-level edit check programs and output any warnings and/or errors encountered in the syntax.

Synopsis

DFcompiler [-o compiled_outputfilename] {-s #} {filename}

Description

DFcompiler is used for syntax checking of edit check files. It can be run in the study DFsetup tool or from the command-line.

In DFsetup, selecting Edit checks from Study and selecting the Check Syntax, button in the edit checks window, runs DFcompiler. Results are output to the bottom Outputs section of the edit checks window. A user requires permission to use DFsetup in order to run DFcompiler in this way.

DFcompiler may also be run from the command-line using the options described below. The study number and edit checks file name (with a correct pathname) are both required arguments.

DFexplore has the ability to compile and run edit checks. Edit checks can be compiled (to DFedits.bin) by selecting Publish in DFsetup or from the command line using DFcompiler as follows:

% DFcompiler -s study# -o DFedits.bin DFedits

Edit checks are loaded automatically when DFexplore starts but can be reloaded following a new compile from the File > Reload - Edit checks option.

Options

Exit Status

DFcompiler exits with one of the following statuses:

Examples

Compile and check the syntax for edit checks defined in DFedits for study 254

% DFcompiler -s 254 /opt/studies/254/ecsrc/DFedits

DFdisable.rpc

DFdisable.rpc — Disable a study database server or incoming daemon to make them unavailable to clients and incoming images

Synopsis

DFdisable.rpc { [-s #] | [-i #] } [-f] [-w #] {key}

Description

The study database server must be disabled when a user requires "off-line" access to the database and wishes to ensure that no client connections to the server are permitted.

Equivalent functionality can be obtained using the Studies dialog in DFadmin.

A disable request will fail if there are other users with client tools connected to the study server at the time of the request.

When -i is used to disable an incoming daemon, the daemon is allowed to complete any image processing that it might be engaged in, but as soon as this is completed and the daemon exits, it will not be allowed to process another incoming document.

Following execution of DFdisable.rpc, a study server or incoming daemon can be re-enabled using DFenable.rpc. The DFenable.rpc command must be executed by the user who executed DFdisable.rpc, and the same key must be provided.

When a study server is disabled, the default requirement is that the server shutdown as quickly as possible. In the interest of efficiency, it removes only those index entries that are for obsolete records and sorts only those index files that are unsorted. This is sufficient for correct operation of the server. However, the server does not remove those indices or records that are marked for deletion nor does it perform garbage collection on data records that are obsolete. The result is that it is subsequently possible to export a record that is still scheduled for deletion. This may be undesirable for archival purposes. Use of -f will force the server to remove all records marked for deletion and perform garbage collection of obsolete data records. This option is primarily useful when shutting down a database in preparation for archiving.

Options

Exit Status

DFdisable.rpc exits with one of the following statuses:

Examples

Disable study #123 so that database maintenance can be performed

% key=`date`
% DFdisable.rpc -s 123 -r "maintenance" $key

See Also

DFenable.rpc

DFenable.rpc — Enable a study database server or incoming daemon after it is disabled

Synopsis

DFenable.rpc { [-s #] | [-i] } {key}

Description

When a study database server has been disabled, use DFenable.rpc to enable the server again and make it available for client connections. Equivalent functionality can be obtained using the Studies dialog in DFadmin.

When an incoming daemon is re-enabled it once again becomes available for processing incoming images.

Study servers and incoming daemons can only be enabled by the user who disabled them, and then only if the correct key is provided.

Options

Exit Status

DFenable.rpc exits with one of the following statuses:

Examples

Enable study #123

In this case, the assumption is that the study was previously disabled and the disable key was saved in a shell variable.

% DFenable.rpc -s 123 $key

See Also

DFencryptpdf

DFencryptpdf — Protect a PDF file by encrypting it with the specified password

Synopsis

DFencryptpdf { [-c password] | [-C] } [-i string] [-o string]

Description

PDF supports password protection beginning with version 5.0 of Acrobat Reader and version 1.4 of the PDF standard. A PDF document has at least two levels of password: the owner password (which permits the owner to perform actions that another user cannot) and the user password. Since PDF is typically a plain text format, a password protected PDF document is encrypted and stored in binary format. The owner/user password is used as a salt for the encryption.

For DFdiscover purposes, the password is needed to ensure that the document can only be opened and read by known users (those with the password). Since DFdiscover supports delivery of study data/reports by PDF and email, it makes sense that the PDF document be protected from viewing by an unintended recipient. The password is not intended to limit what a user with the password can do with the document, so the owner and user password are the same and no restrictions are applied to any one of the actions available on the document.

DFencryptpdf returns zero upon success, otherwise returns one. DFencryptpdf may fail if the input PDF file is invalid PDF, has multiple cross-reference tables, such as linearized or updated PDF files, or is already encrypted. If neither -c nor -C is specified, it also returns an error.

Options

Exit Status

DFencryptpdf exits with one of the following statuses:

Examples

Encrypt a set of CRFs stored in crfs.pdf and protect with the password "DoNotUse".

% DFencryptpdf -c DoNotUse -i crfs.pdf -o protected_crfs.pdf

See Also

DFexport.rpc

DFexport.rpc — Export data records from one or multiple plates from a study data file

Synopsis

DFexport.rpc [-A] [-X] [ [-w] | [-a] ] [ [-c] | [-j] ] [-d] [-e] [-J] [-m] [-p] [-q] [-h] [-k] [-z] [-s status_list] [-v #, #-#] [-n #, #-#] [-I #, #-#] [-V #, #-#] [-C yy/mm/dd-yy/mm/dd] [-M yy/mm/dd-yy/mm/dd] [-L lostcode] [ [-U fieldname_list] | [-G fieldname_list] ] [-f fieldnum_list] [-H header_list] {study} {plate(s)} {outfile}

Description

DFexport.rpc can be used to export whole data records or selected fields, from specified plates of a specified study database. Data records are exported in ASCII text format. DFexport.rpc does not provide support for joining fields from different plates or studies. To use DFexport.rpc users must have 'Export Data' permission, which is granted by an administrator on a study-by-study basis (see System Administrator Guide, Roles). Users with export permission will only receive records for which they have get permission, which may be restricted by level, site, subject, assessment and plate. DFexport.rpc provides no warning that some records cannot be exported as this would itself provide information about the existence of restricted data records.

DFexport.rpc may appear in a shell script used to create reports stored in the study reports directory, to be run in DFexplore from the Reports View, or in a shell script run using dfexecute in an edit check. In these cases the permissions of the DFexplore user running the report or edit check will be applied. Thus the behavior and output may differ depending on the permissions of the user who runs the script.

If the same script is run from the UNIX shell, the permissions associated with the UNIX login name apply. Thus, if a user has different UNIX and DFexplore login accounts with different permissions, the user may get different results depending on whether they run the script in the UNIX shell or DFexplore environments.

Options

Required Options

The following three options are required and must appear in order at the end of the option list:

Record Selection Options

The following options allow specific records to be selected from the output. With no options specified, all records are output. With options specified, only records matching the selection criteria are output. If multiple options are specified, only those records that match all criteria are output.