Programmer Guide

Release 5.2.0

All rights reserved. No part of this publication may be re-transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of DF/Net Research, Inc.. Permission is granted for internal re-distribution of this publication by the license holder and their employees for internal use only, provided that the copyright notices and this permission notice appear in all copies.

The information in this document is furnished for informational use only and is subject to change without notice. DF/Net Research, Inc. assumes no responsibility or liability for any errors or inaccuracies in this document or for any omissions from it.

All products or services mentioned in this document are covered by the trademarks, service marks, or product names as designated by the companies who market those products.

Google Play and the Google Play logo are trademarks of Google LLC. Android is a trademark of Google LLC.

App Store is a trademark of Apple Inc.

Nov 01, 2019


Table of Contents

Preface
1. Getting Help
2. Conventions
1. Introduction
1.1. About This Guide
1.2. DFdiscover Programming Limits
2. DFdiscover Study Files
2.1. Introduction
2.2. DFdiscover study directories
2.3. Study File Permissions
2.4. Format used to describe files
2.5. DFdiscover Retrieval Files (DRF)
2.6. The study data directory
2.6.1. Temporary data files
2.6.2. Plate data files
2.6.3. Query data files
2.6.4. Reason for change data files
2.6.5. Newly arrived data file
2.6.6. Journal files
2.6.7. Index files
2.7. The study ecsrc directory
2.8. The study lib directory
2.9. The study lut directory
2.10. The study work directory
3. Shell Level Programs
3.1. Introduction
3.2. User Credentials
3.2.1. Good Password Management
3.2.2. Order of Evaluation
3.3. Organization of Reference Pages
3.4. Alphabetical Listing
DFaccess.rpc — Change access to a study database or , or query their current access status.
DFattach — Attach one or more external documents to keys in a DFdiscover study
DFaudittrace — Used by the DF_ATmods report to read study journal files. DF_ATmods produces an audit trail report showing database modifications for the specified study.
DFbatch — Process one or more batch edit check files
DFcompiler — Compile study-level edit check programs and output any warnings and/or errors encountered in the syntax.
DFdisable.rpc — Disable a study database server or incoming fax daemon to make them unavailable to clients and incoming faxes
DFenable.rpc — Enable a study database server or incoming fax daemon following a previous
DFencryptpdf — Protect a PDF file by encrypting it with the specified password
DFexport — Client-side, command-line interface for exporting data by plate, field or module; exporting change history; or exporting components of study definition
DFexport.rpc — Export data records from one or multiple plates from a study data file
DFfaxq — Display the members of the outgoing fax queue
DFfaxrm — Remove faxes from the outgoing fax queue
DFget — Get specified data fields from each record in an input file and write them to an output file
DFgetparam.rpc — Retrieve and evaluate the value of the requested configuration parameter
DFhostid — Display the unique DFdiscover host identifier of the system
DFimageio — Request a study CRF image from the database
DFimport.rpc — Import database records to a study database from an ASCII text file
DFlistplates.rpc — List all plate numbers used in the study
DFlogger — Re-route error messages from non-DFdiscover applications to syslog, which in turn writes the messages to the system log files, as configured in /etc/syslog.conf.
DFpass — Locally manage user credentials for client-side command-line programs.
DFpdf — Generate bookmarked PDF documents of CRF images
DFpdfpkg — Generate multiple bookmarked PDF files for specified subject IDs or sites
DFprint_filter — Format input file(s) for printing to a PostScript® capable printer, and print to a specified printer
DFprintdb — Print case report forms merged with data records from the study database
DFpsprint — Convert one or more input CRF images into PostScript®
DFqcps — Convert a Query Report, previously generated by DF_QCreports, into a PDF file with barcoding, prior to sending the report to a study site
DFreport — Client-side, command-line interface for executing reports
DFsas — Prepare data set(s) and job file for processing by SAS®.
DFsendfax — Fax or email a plain text, PDF, or TIFF file to one or more recipients
DFsqlload — Create table definitions and import all data into a relational database
DFstatus — Display database status information in plain text format
DFtextps — Convert one or more input files into PDF
DFuserdb — Perform maintenance operations on the user database
DFversion — Display version information for all DFdiscover executables (programs), reports, and utilities
DFwhich — Display version information for one or more DFdiscover programs, reports and/or utilities
4. Utility Programs
4.1. Introduction
4.2. Alphabetical Listing
DFaddHylaClient — Create the symbolic links necessary for accessing HylaFAX on a DFdiscover server
DFcertReq — Request an SSL certificate signing for DFedcservice.
DFclearIncoming — Clean out the fax receiving directory, processing all newly arrived faxes
DFcmpSchema — Apply the data dictionary rules against the study database
DFcmpSeq — Determine the appropriate values for each .seqYYWW file
DFisRunning — Determine if the DFdiscover master program is currently running on the licensed DFdiscover machine
DFmigrate — Upgrade study setup and configuration files from an old DFdiscover version to the current version.
DFras2png — Convert Sun raster files in the study pages directory into PNG files
DFshowIdx — Show the per plate index file(s) for a specific study
DFstudyDiag — Report (diagnose) the current status of a study database server
DFstudyPerms — Report, and correct, the permissions on all required DFdiscover sub-directories and files for a study
DFtiff2ras — Convert a TIFF file into individual PNG files
DFuserPerms — Import and update users and passwords, and optionally import roles, role permissions, and user roles
5. Edit checks
5.1. Introduction
5.1.1. DFopen_study and DFopen_patient_binder
5.2. Language Features
5.3. Database Permissions
5.4. Language Structure
5.4.1. return and exit statements
5.5. Variables
5.5.1. Variables and Types
5.5.2. Database Variables
5.5.3. Positional Variables
5.5.4. Local Variables
5.5.5. Global Variables
5.5.6. Variable Groups
5.5.7. Date Variables
5.6. Missing/Blank Data
5.6.1. dfblank
5.6.2. dfmissing
5.6.3. dfmissval
5.6.4. dfmisscode
5.6.5. dfmissingrecord
5.6.6. dflostcode
5.6.7. dflosttext
5.6.8. Missing Records
5.6.9. Examples
5.7. Arithmetic Operators
5.7.1. Addition
5.7.2. Subtraction
5.7.3. Multiplication/Division/Modulus
5.7.4. Exponentiation
5.7.5. Assignment
5.8. Conditional Execution
5.8.1. Comparison Operators
5.8.2. Logical Operators
5.8.3. if/else
5.9. Built-in Functions and Statements
5.9.1. Edit check Function Compatibility Notes
5.9.2. dfaccess
5.9.3. dfaccessinfo
5.9.4. dfask
5.9.5. dfbatch
5.9.6. dfcapture
5.9.7. dfcenter
5.9.8. dfclosestudy
5.9.9. dfdate2str
5.9.10. dfday
5.9.11. dfdirection
5.9.12. dfentrypoint
5.9.13. dfexecute
5.9.14. dfgetfield
5.9.15. dfgetlevel/dflevel
5.9.16. dfgetseq
5.9.17. dfhelp
5.9.18. dfillegal
5.9.19. dfimageinfo
5.9.20. dflegal
5.9.21. dflength
5.9.22. dflogout
5.9.23. dfmail
5.9.24. dfmatch
5.9.25. dfmessage/dfdisplay/dferror/dfwarning
5.9.26. dfmetastatus
5.9.27. dfmode
5.9.28. dfmoduleinfo
5.9.29. dfmonth
5.9.30. dfmoveto
5.9.31. dfneed
5.9.32. dfpageinfo
5.9.33. dfpassword
5.9.34. dfpasswdx
5.9.35. dfplateinfo
5.9.36. dfpref
5.9.37. dfprefinfo
5.9.38. dfprotocol
5.9.39. dfrole
5.9.40. dfsiteinfo
5.9.41. sqrt
5.9.42. dfstay
5.9.43. dfstr2date
5.9.44. dfstudyinfo
5.9.45. dfsubstr
5.9.46. dftask
5.9.47. dftime
5.9.48. dftoday
5.9.49. dftool
5.9.50. dftrigger
5.9.51. dfvarinfo
5.9.52. dfvarname
5.9.53. dfview
5.9.54. dfvisitinfo
5.9.55. dfwhoami
5.9.56. dfyear
5.9.57. int
5.10. Query operations
5.10.1. dfaddqc
5.10.2. dfaddmpqc
5.10.3. dfanyqc
5.10.4. dfanyqc2
5.10.5. dfanympqc
5.10.6. dfdelmpqc
5.10.7. dfeditqc
5.10.8. dfreplyqc
5.10.9. dfresqc/dfunresqc
5.10.10. dfqcinfo
5.10.11. dfqcinfo2
5.11. Reason operations
5.11.1. dfaddreason
5.11.2. dfanyreason
5.11.3. dfautoreason
5.11.4. dfreasoninfo
5.12. Lookup Tables
5.12.1. Pre-requisites
5.12.2. dflookup
5.13. Looping
5.13.1. while
5.13.2. break
5.13.3. continue
5.14. User-Defined Functions
5.14.1. Sharing edit check files with the #include directive
5.15. Examples and Advice
5.16. Optimizing Edit checks
5.16.1. Saving Time for the User
5.16.2. Limitations in the Language
5.16.3. Maximize Cache
5.16.4. Simplify Conditional Testing
5.16.5. Reduce the Number of Function Calls
5.16.6. Shortcut, and Order of, Evaluation
5.16.7. Delay Message Construction
5.17. Creating generic edit checks
5.17.1. More Examples
5.18. Debugging and Testing
5.18.1. Debugging
5.18.2. Testing
5.18.3. Compiling and Reloading Edit checks
5.19. Language Reference
5.19.1. Identifiers (edit check and variable names)
5.19.2. String Constants
5.19.3. Maximum number of instructions per edit check execution
5.19.4. Reserved Words
6. Batch Edit checks
6.1. Introduction
6.1.1. Overview
6.1.2. About this chapter
6.2. DFbatch Basics
6.2.1. The DFbatch Layer
6.2.2. Do you need DFbatch?
6.2.3. How does DFbatch work?
6.2.4. Getting Started
6.2.5. Summary
6.3. Using DFbatch
6.3.1. General Control File Layout
6.3.2. Invoking DFbatch
6.3.3. Strategies for using DFbatch
6.4. Limitations
6.4.1. Default actions for interactive functions
6.4.2. Not possible with DFbatch
6.4.3. Not recommended with DFbatch
6.5. Example Control Files
6.6. Common Pitfalls and System Messages
6.6.1. Common Pitfalls
6.6.2. System Messages
6.7. BATCHLIST Element Reference
6.7.1. BATCHLIST Document Type Definition
6.7.2. Organization of Reference Pages
6.7.3. Reference Pages
6.8. BATCHLOG Element Reference
6.8.1. BATCHLOG Document Type Definition
6.8.2. Element Reference
6.9. XML Language Basics
6.9.1. The Rules of XML
6.9.2. Companions to XML
6.9.3. Recommended Reading
7. Writing Your Own Reports
7.1. General Guidelines
7.2. Installing Your Reports
7.3. Input Data Files
7.4. Programming Tools
7.5. An Example
7.6. Writing Documentation for Study Specific Reports
8. DFsas: DFdiscover to SAS®
8.1. An Example
8.1.1. Global Specifications
8.1.2. Data Retrieval Specifications
8.1.3. SAS® Procedures
8.1.4. Running DFsas
8.2. Creating a DFsas job file
8.2.1. Impact of SAS® limits
8.2.2. Creating an initial DFsas job file
8.2.3. String splitting
8.2.4. String truncation
8.2.5. Date exporting
8.2.6. A sampling of other options
8.3. Creating SAS® job and data files
8.3.1. Force Option
8.3.2. Export Script Option
8.3.3. Use Field Alias Option
8.3.4. Syntax Checks
8.4. Date Fields
8.4.1. Global Statements
8.4.2. Qualified Dates
8.4.3. Time Qualifiers
8.4.4. Default Actions Performed When Creating a DFsas Job File
8.5. String Fields
8.5.1. String Splitting
8.5.2. Extracting Sub-Strings
8.5.3. Retaining Quotes in String Fields
8.6. DFsas Job File Syntax
8.6.1. Global Specifications
8.6.2. Data Retrieval Specifications
8.6.3. SAS® Procedures
8.7. Creating a Normalized Data Set
8.7.1. Merge
8.7.2. Specifying Data Fields
8.7.3. String Fields in Normalized Data Sets
8.7.4. Case Selection
8.7.5. Value codes or labels
8.7.6. Variable Description
8.7.7. Specifying Normalized Records
8.7.8. Sorting a Normalized Data Set
8.7.9. Example Data File
9. DFsqlload: DFdiscover to Relational Database Tables
9.1. Introduction
9.1.1. Overview
9.1.2. About DFsqlload
9.2. DFsqlload and Relational Database Concepts
9.2.1. Why Relational Databases?
9.2.2. Why is DFsqlload a one-way street?
9.2.3. Relational Database Concepts
9.3. Using DFsqlload
9.3.1. DFsqlload defaults - a quick tutorial
9.3.2. DFsqlload in Detail
A. Copyrights - Acknowledgments
A.1. External Software Copyrights
A.1.1. DCMTK software package
A.1.2. Jansson License
A.1.3. Mimencode
A.1.4. RSA Data Security, Inc., MD5 message-digest algorithm
A.1.5. mpack/munpack
A.1.6. TIFF
A.1.7. PostgreSQL
A.1.8. OpenSSL License
A.1.9. Original SSLeay License
A.1.10. gawk
A.1.11. Ghostscript
A.1.12. MariaDB and FreeTDS
A.1.13. QtAV
A.1.14. FFmpeg
A.1.15. c3.js
A.1.16. d3.js

Preface

1. Getting Help

DF/Net Research, Inc.'s Technical Support Department can be contacted Monday to Friday between 9:00 am and 5:00 pm Eastern Time via any of the following methods:


55 Head Street, Suite 403
DundasOntario  L9H 3H8
Telephone: (905) 522-3282
Email: 
URL: www.dfnetresearch.com

2. Conventions

A number of conventions have been used throughout this document.

Any freestanding sections of code are generally shown like this:

# this is example code
code = code + overhead;

If a line starts with # or %, this character denotes the system prompt and is not typed by the user.

Text may also have several styles:

  • Emphasized words are shown as follows: emphasized words.

  • Filenames appear in the text like so: dummy.c.

  • Code, constants, and literals in the text appear like so: main.

  • Variable names appear in the text like so: nBytes.

  • Text on user interface labels or menus is shown as: Printer name, while buttons in user interfaces are shown as Cancel.

  • Menus and menu items are shown as: File > Exit.

Chapter 1. Introduction

1.1. About This Guide

This guide is for programmers, and for those who aspire to be. It covers DFdiscover file formats and those tools that are executed at the shell level. As a result, some familiarity with the UNIX operating system and UNIX shell level programming is assumed.

If this is your first encounter with UNIX we strongly recommend that you look for a good UNIX programming book in your local bookstore. We would not hesitate to recommend The UNIX Programming Environment by Kernighan and Pike, and The Awk Programming Language by Aho, Kernighan and Weinberger. The former is an excellent introduction to writing UNIX shell scripts, while the latter describes awk, a simple C-like language which is ideal for manipulating data files. All of the tools described in these 2 books come as standard components of the UNIX operating system.

In the remaining chapters of this guide we will describe:

  • DFdiscover Study Files A description of the location and format of all study files (including the configuration and data files) used in all DFdiscover studies

  • Shell Level Programs Over a dozen shell level commands for doing such things as:

    • exporting data records from a study database

    • selecting individual data fields

    • reformatting exported data records

    • importing data records from other sources into a DFdiscover study database

    • sending and managing faxes

    • getting study configuration parameters

    These programs can be used in shell scripts to create your own programs. When combined with standard UNIX commands (like awk, sed, grep, etc.) they provide an extremely powerful and flexible way of creating study report programs.

  • Utility Programs Descriptions of several infrequently used, yet useful, utility programs that can simplify some DFdiscover management tasks.

  • Edit checks A programming language for writing and executing edit checks that occur in real-time during data validation.

  • Batch Edit checks An extension of the edit checks language that permits execution of edit checks in batch, non-interactive mode.

  • DFsas: DFdiscover to SAS® An environment for generating SAS® job and data files from a DFdiscover study database and study schema.

  • DFsqlload: DFdiscover to Relational Database Tables A program that creates relational database tables from a DFdiscover study database and study schema.

1.2.  DFdiscover Programming Limits

The following cribsheet is for DFdiscover programmers and summarizes all relevant DFdiscover database limits and formats. This cribsheet is to be used in conjunction with the information that follows in this chapter.

Description Limit Comments
DFdiscover Study Number 1-999 The suggested range for study numbers is 1-249 as study numbers of 250-255 are reserved for DFdiscover test and validation studies (e.g. ATK = 254). With the appropriate software license, study numbers 256-999 are available for defining EDC studies.
Plate Number 0-501, 510, 511 Plates 501 and 511 are reserved by DFdiscover for Query Reports and can not be re-defined at the user level. Plate 0 references the new record queue. Plate 510 is reserved by DFdiscover for Reason records.
Visit/Sequence Number (barcoded) 0-511
Visit/Sequence Number (first data field) 0-65535 Any data field representing the visit/sequence number must be defined in the database as field #6 using DFdiscover schema numbering.
Site Number 0-21460 This limit applies to the site number only. A subject identifier is concatenated to the site number to obtain the subject ID.
Subject ID Number 0-281474976710655 For subject IDs that are composed of site # + ID #, this limit applies to the concatenated value of the two. This field could contain 15 digits at maximum.
Any numeric value -2147483647-2147483647 Any numeric field, except the subject ID field, can contain 10 digits at maximum, which include any leading sign and decimal point. This limit applies to the following DFdiscover field types, which have a base numeric value: numeric, visual analog scale (VAS), choice codes, check codes.
Query Use 0 = none 1 = external 2 = internal
Query Type 0 = none 1 = Q&A (clarification) 2 = refax (correction)
Query Category Code 1=missing, 2=illegal, 3=inconsistent, 4=illegible, 5=fax noise, 6=other, 21=missing page, 22=overdue visit, 23=edit check missing page, 30-99=user-defined problem type
Query Status 0=pending review, 1=new, 2=in unsent report, 3=resolved, NA, 4=resolved, irrelevant, 5=resolved, corrected, 6=in sent report
Query Detail Field max 500 characters
Query Note Field max 500 characters
Missed Data Log Explanation Field max 500 characters
Default Date Format YY/MM/DD
Validation Level (system) 0-7 Level 0 represents new, not yet entered, records
Validation Level (user) 1-7 A user cannot assign a validation level of 0 to a data record.
Maximum Data Record Length16384 ASCII (4096 UNICODE) charactersThis is the maximum length that the system can accept and includes 55 characters of overheard maintained by the system. Therefore, the length of data record available for user-defined fields is 55 characters less.
Maximum DFsas Record Length2048 charactersDFsas is unable to process input files for SAS® greater than this size.

Chapter 2. DFdiscover Study Files

2.1. Introduction

This chapter describes the files maintained under each DFdiscover study directory. It starts with a brief overview of the top-level directories and then describes the files kept under each directory in detail. A similar chapter, System Administrator Guide, DFdiscover System Files, describes the files located under a DFdiscover installation directory.

2.2. DFdiscover study directories

The following directories are part of a standard DFdiscover study definition.

  • bkgd This directory holds CRF background images created by DFsetup. Three files are created for each study plate. These include the plate background used by DFsetup (plt###), the data entry screen used by DFexplore (DFbkgd###), and a background used when printing CRFs containing database values from DFprintdb or DFexplore (DFbkgd###.png).

    [Note]Note

    This directory must not be used to store any other files. Extraneous files will be deleted when CRF images are published from a development study to its production study, or when reverting a development study to the current state of its production study.

  • data This directory holds the study data files, queries data file, and journal files. The following files are described in detail later in this chapter in The study data directory:

  • dfsas This directory contains any stored SAS® jobs that were created from DFexplore.

  • dfschema This directory contains any stored DFschema files which are used to track changes to the study setup. These are used by DFaudittrace to generate records for DF_ATmods.

  • drf This directory contains any .drf files (DFdiscover Retrieval Files) created by server-side tools, including reports and the program DFmkdrf.jnl. The common .drf files created by DFdiscover reports include:

    • DupKeys.drf Lists any duplicate primary keys found in the database by the last execution of DF_XXkeys.

    • VDillegal.drf Lists any illegal visit dates found in the database by the last execution of DF_XXkeys.

    • VDincon.drf Lists any inconsistent visit dates found in the database by the last execution of DF_XXkeys.

    • DFunexpected.drf Lists any unexpected data records found in the database by the last execution of DF_QCupdate.

  • ecbin Any study specific scripts or programs that are run by edit check function dfexecute() and any Plate Arrival Trigger scripts defined in the DFsetup Plates View, must be stored in the study level ecbin directory. Executables can also be stored in the DFdiscover level ecbin directory for use in all studies. The study level ecbin directory has priority if the same program or script is stored in both locations.

  • ecsrc This directory contains the edit check files: DFedits and any other edit check source files referenced in 'include' statements. Include files can also be stored in the DFdiscover level ecsrc directory. The study level ecsrc directory has priority if the same include file is stored in both locations.

  • lib All study configuration files, created through both the DFadmin and DFsetup tools, are located in this directory. The following files are described in detail later in this chapter in The study lib directory:

    • DFccycle_map - conditional cycle mapDefines cycles that may be required, unexpected or optional, depending on specified database conditions (optional).

    • DFcenters - sites database Study sites database includes information on all clinical sites participating in the study.

    • DFcplate_map - conditional plate mapDefines plates that may be required, unexpected or optional, depending on specified database conditions (optional).

    • DFcterm_map - conditional termination mapDefines database conditions that signal termination of subject follow-up (optional).

    • DFcvisit_map - conditional visit mapDefines visits that may be required, unexpected or optional, depending on specified database conditions (optional).

    • DFCRFType_map - CRF type mapDefines categories for different CRF background types (e.g. translations) (optional).

    • DFcrfbkgd_map - CRF background mapDefines visits with CRF backgrounds used across multiple visits (optional).

    • DFedits.bin - published edit checksThe "published" equivalent of the edit checks - for use by DFexplore clients only (optional).

    • DFfile_map - file mapSpecifies each unique CRF plate used in the study.

    • DFlut_map - lookup table mapDefines and associates lookup table names with directory names of lookup tables. Also includes definition for query lookup table, if it exists (optional).

    • DFmissing_map - missing value mapMissing value codes (and labels) used in the study (optional).

    • DFpage_map - page mapUsed to specify descriptive labels to replace the visit/sequence and plate numbers that are used by default to identify problems on the Query Reports (optional).

    • DFqcsort - Query and CRF sort orderSpecifies a sort order for queries as written on Query Reports. The default is to sort by field number within plate number within visit/sequence number within subject ID (optional).

    • .DFreports.dat - reports historyPersonal file of DFdiscover report commands (and options) corresponding to reports which the user runs in the reports tool (optional).

    • DFschema - database schemaStudy database schema (or dictionary) describes all variables on all plates, as specified in the setup tool.

    • DFschema.stl - database schema stylesStudy database schema (or dictionary) describes all variable styles defined for the study in the setup tool.

    • DFserver.cf - server configurationConfiguration file for the study database server.

    • DFsetup - study definitionThis is a JSON format file maintained and used by DFsetup to record all study setup information, including all plate, style and variable definitions.

    • DFtips - ICR tipsThis file contains the coordinates, field type and legal values for all variables defined on all plates. It is used by the ICR software to create the initial data record from new CRF pages that arrive by fax.

    • DFvisit_map - visit mapThis file describes the scheduling of subject assessments during the trial and the CRF pages expected at each assessment.

    • DFwatermark - watermarkThis file describes watermarks used for printed output assigned by role.

    • QCcovers - Query cover pagesThis file contains formatting information to be included in a Query Report cover sheet. To include a cover sheet for a Query Report, QCcovers must first be defined.

    • QCmessages - Query Report messagesThis file contains messages to be included in Query Report cover sheets. More than one message may be included in a single Query Report, however, DF_QCreports expects the cover sheet information and all messages to fit on a single page.

    • QCtitles - Query Report titlesThis file describes how the report title and each title of the 3 sections of a DFdiscover Query Report are to be customized. DF_QCreports checks for the existence of this file and will use it if it exists, otherwise standard titles will be produced.

    • DFqcproblem_map - Query category code mapThis file contains all the system-defined and user-defined query category codes.

    • DFreportstyle - DFdiscover Report styling This file contains styling "instructions" for reports, excluding Legacy reports. Styling includes font choices and colors used in graphs and charts.

  • lut This directory contains all study specific lookup tables. Lookup tables can also be stored in directory lut at the DFdiscover level. Study level files have priority if a lookup table with the same file name is stored in both locations. Lookup table file names must be associated with an edit check name in DFlut_map (found in the study /lib directory) before they can be used in edit checks.

  • pages This directory holds all standard (SD) resolution (100 dpi) CRF image and supporting document files for all CRF pages received or uploaded during the study. Each CRF page received as a fax or PDF is stored as a PNG file (Sun Rasterfile in pre-2014 DFdiscover releases), and requires about 25K bytes of disk space (i.e. 40 CRF pages per MB). Supporting documents can be audio, video, DICOM or PDF files and will be in their native format.

    DFdiscover creates subdirectories, below the study pages directory, in which to store the CRF image files. These subdirectories are named by the year and week (in yyww format) in which the CRFs arrived. For example, a study which started on January 1, 1995 would have subdirectories named: 9501, 9502, 9503, etc. through to the end of the trial.

    Within each of the yyww subdirectories, the individual CRF page files are named by the concatenation of the sequence number of the fax (starting at 0001 each week) in which they arrived during that week, and their page number within the fax. The file-naming format is ffffppp. For example, if the first fax to arrive in the week of 9501 contained 3 pages, it would be represented by the following files beneath the study pages directory: 9501/0001001, 9501/0001002, and 9501/0001003. The ffff part of the file name is a sequential value constructed from 4 characters, each character taken from the alphabet:

    0 1 2 3 4 5 6 7 8 9 B C D F G H J K L M N P Q R S T V W Y Z

    This is essentially the digits 0 through 9 followed by the uppercase letters A through Z, with the exception of the letters A, E, I, O, U, and X. This part of the file name thus lies between 0001 and ZZZZ, which allows for a total of 30x30x30x30-1 = 809,999 faxes per week. Example file names within a week include:

    • 0001005 5th page of 1st fax

    • 000B001 1st page of 10th fax

    • 000Z011 11th page of 29th fax

    • 0010002 2nd page of 30th fax

    • 01C0004 4th page of 1230th fax

  • pages_hd This directory holds all higher (HD) resolution (300 dpi) CRF images and supporting document files received or uploaded during the study. DFdiscover creates subdirectories and stores the files in the same manner as the pages directory.

  • reports It is recommended that all study specific reports, i.e. those programs which have been created specifically for a particular clinical trial, are stored in the study reports directory. This helps to standardize maintenance across studies.

    Documentation for study specific reports must also be stored in this directory in a file named .info, which uses the same format used to document the standard DFdiscover reports. The specific requirements of this file and its format are described in Writing Documentation for Study Specific Reports.

  • reports/QC This directory is used to store the standard DFdiscover Query Reports, created by DF_QCreports. The files and directories maintained under this directory are briefly described below.

    ccc-yymmdd

    The naming convention for Query Reports is a zero padded 3 digit site ID (if the site ID is greater than 999, it will be a 4 digit number), followed by a dash, and then the date on which the report was created. For example: 055-150815 would identify a Query Report created for site 55 on Aug 15, 2015. Thus you can tell from a Query Report name, both whom it was created for and when.

    Although this is helpful it does have one disadvantage, namely, it means that you cannot create more than one Query Report per day for an individual clinical site. If you do, the earlier one will be over-written. However, since Query Reports are sent to the clinical sites, and this isn't something you should do more than about once a week, and certainly not more than once a day, this approach to naming Query Reports works.

    Query Reports remain under QC until they are sent to the clinical sites. Thus any reports that you see at this level have not been transmitted to the study clinics.

    QC/sent

    Query Reports are moved to this directory after they have been successfully sent to their respective clinical sites.

    QC/internal

    DF_QCreports is also able to create named, internal Query Reports, which can include subjects from more than one site. These reports are written directly to this directory.

    QC_LOG

    This is a plain ASCII file that lists any error messages reported during the last execution of DF_QCreports.

    QC_NEW

    This is a plain ASCII file that lists the Query Reports created by the most recent execution of DF_QCreports.

    SENDFAX.log

    This is a plain ASCII file in which DF_QCfax records the success or failure of its attempts to fax Query Reports to the clinical sites.

    SENDFAX.qup

    This is a plain ASCII file that lists all Query Reports which have been queued up for transmission to the clinical sites.

    other files

    Occasionally, DF_QCreports might be halted in mid-execution, by a power failure, or some other problem. In such cases it may not have time to remove the temporary files that it creates in the process of generating the Query Reports. These files generally contain a process ID#; for example you might see files that look like this 18273_N.refax, 18273_N.qalist. These temporary files can, and should, be removed.

  • work The work directory contains temporary files, DFdiscover retrieval files and summary data files. It is important to be aware of the files that DFdiscover creates and maintains in this directory so that programmers do not inadvertently overwrite them.

    • Temporary Files.  The work directory is used for temporary files created by various reports and is the recommended location for temporary files created by study specific reports and for data files exported using DFexport.rpc. If DFexport.rpc is used to export study data files to this directory, or create temporary files here in the process of generating study specific reports, you need to be very careful in your selection of temporary file names, so that simultaneous execution of different programs does not result in temporary file name conflicts. A useful strategy is to include the process ID plus some part of the program name in all temporary file names, and also to check for and create a lock directory for any programs that should not be executed simultaneously by more than one user.

      Because the work directory is used for temporary file creation by various DFdiscover programs, you might find temporary files that were not removed because a program failed to complete due to a power failure or some other problem. Any file that is several days old, and has what looks like a process id number as part of it's name, is probably a temporary file left over when a program failed to complete successfully. Such files can be removed. This should be done periodically to clean up the study work directory.

    • Summary Files.  The work directory contains a number of summary data files created by DF_XXkeys and DF_QCupdate and used by other programs including DF_QCreports and DF_PTvisits. The following files are described in detail later in this chapter in The study work directory:

2.3. Study File Permissions

The data, pages, and pages_hd directories must be owned by user datafax, with read, write and execute permissions for owner, and read and execute permissions for group.

The bkgd, dde, frame, lib, reports, and work directories must have read, write and execute permissions set for all users of the study group (typically this will be UNIX group studies).

These permissions are set by DFadmin on those directories which it creates when the study is first registered. The utility program DFstudyPerms is available for diagnosing and correcting study permission problems. A detailed list of study files and directories checked by this program can be found at System Administrator Guide, Maintaining study filesystem permissions

2.4. Format used to describe files

Each file documented in this section is described with the following attributes:

Table 2.1. File attributes

HeadingDescription
Usual Namethe file name that is usually given to files having this format. Some files are kept at the DFdiscover directory level while others are kept separately with each study directory.
Type one of: "clear text" or "binary". Clear text files can be reviewed with any text editor.
Created By the name of the DFdiscover program(s) that create and modify this file. If you need to edit the contents of the file, use the program listed here.
Used By the name of the DFdiscover program(s) that reference or read this file.
Field Delimiter how fields within a record are delimited. Typically, the delimiter is a single character.
Record Delimiter how records within the file are delimited. Typically, the delimiter is a single character.
Comment Delimiter how comments within the file are delimited. If comments are not permitted within the file, "NA" is indicated.
Fields/Record the expected number of fields per record. If the number of fields varies across records, the minimum number is given, followed by a +.
Description a detailed description of the meaning of each field.
Example one or more example records from the file.


2.5. DFdiscover Retrieval Files (DRF)

A DFdiscover Retrieval File (DRF) is an ASCII file in which each record identifies the subject ID, visit or sequence number and CRF plate number (in that order, | delimited) and optionally the image ID corresponding to a record in the study database.

DFdiscover retrieval files are created by DFexplore, DFdiscover reports, and the DFdiscover programs DFmkdrf.jnl and DFexport.rpc.

DFdiscover retrieval files created by DFexplore and DFdiscover reports and programs reside in the study drf directory. It is also recommended that DRFs created using DFexport.rpc also be saved to the study drf directory. All DRFs must end with the .drf extension. Exercise caution when selecting DRF names to avoid conflicts with those DRF file names generated by DFdiscover applications. Below is a description of the DRF file format.

Table 2.2. Data Retrieval Files (DRF)

Usual Namefilename.drf
Typeclear text
Created ByDFexport.rpc, DFmkdrf.jnl, DFexplore
Used ByDFexplore
Field Delimiter|
Record Delimiter\n
Comment Delimiter#
Fields/Record3+
Description

Data records have the common fields and characteristics defined in Table 2.3, “DFdiscover Retrieval File field descriptions”.

Example

This is an example of a DRF listing all primary records having one or more queries attached to them. The DRF filename ext_qcs.150813.drf was created on August 13, 2015 and has the description external qc notes.150813.

99001|1|2
99001|1011|9
99002|1|2
99002|22|5
99002|1011|9
99004|1|2


Table 2.3. DFdiscover Retrieval File field descriptions

Field #ContainsDescription
1subject/case IDthis is a number in the range of 0-281474976710655 used to uniquely identify each subject/case in the study database
2visit/sequence numbera number in the range of 0-21460, that uniquely identifies each occurrence of a visit number for a subject ID
3plate numbera number in the range of 1-501 that uniquely identifies the plate to be included in the DRF.
4image identifierthe value in this optional field is the unique identifier of the CRF image identified by the keys in the first 3 fields. This field is used to identify a specific instance of a CRF when there are one or more secondaries having the same key fields.
5optional textthis field can be used to provide a short descriptive message to DFexplore users. The text is displayed in the message window at the bottom of DFexplore when the data record is selected in the DFexplore record list.

2.6.  The study data directory

The study data directory holds all study data files, including the plate files that correspond to CRF plates, the query data file and the database transaction journals. These files are managed by the study database server and should not be accessed directly. Instead the records required should be exported from the study database to another file, as described in DFexport.rpc, and then read from the exported file.

Data records should never be added to these database files without doing so through the study database server. The program DFimport.rpc is available and is the recommended method of sending new or revised data records to the study database server.

Occasionally you may need to make extensive changes to an entire database file. The typical scenario is a change to one or more of the study CRFs to add or remove fields. In this case the plate definitions originally entered in the setup tool also need to be changed to match the new structure of the corresponding data files. This is a major operation. It requires that the study server be disabled while the database and setup definitions are being changed, and that the effected data files be re-indexed before the study database server is re-enabled.

2.6.1. Temporary data files

The following sections describe data and index files ending with .dat and .idx suffixes. Very rarely, you may notice files having the same name but with .tad or .xdi suffixes. These files are temporary files created and managed by the database server. They are present while the server performs sorting and garbage collection on a particular data file. As such they should be treated in the same manner as the other data and index files - basically, do not touch them. Typically, this is not a problem because the files are present for only a second or two.

2.6.2. Plate data files

Table 2.4. plt###.dat - per plate data files

Usual Nameplt###.dat
Typeclear text
Created ByDFserver.rpc
Used ByDFserver.rpc
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record9+
Description

After first validation, new records are moved from to the study database files, plt###.dat. There is a separate data file for each unique CRF plate number defined for the study. For plate N, the data file is named pltN.dat, where N is leading zero-padded to 3 digits in length. Each record in one of these data files corresponds to the data entered from one CRF page with that plate number. Thus, each data file holds the data recorded for all instances of that CRF page (i.e., across all subjects and all visits).

All database records are stored in free-field format with a | delimiter and \n delimiter. All database records have 3 CRF information fields followed by 4 key fields. Within all database files for each study, the study number key should always be the same; and within each plt###.dat data file, the plate number key should always be the same.

[Warning]Do not edit data files

Data files must not be edited by conventional means. Doing so will surely corrupt the database. Each data file has a corresponding binary index file that encodes the information of the data file. Editing any data file will change the contents so that they are inconsistent with the index file.

Another reason not to read or edit the data files explicitly is that each record may appear in the database more than once. This arises because each time a record is modified, the new version is appended to the end of the database file. Only the index file knows which version of the record is the most recent. There is no indication in the data record. DFexport.rpc should always be used to read data files because it consults the index files to identify the correct version of each record to be exported. Old versions of modified records are removed by the study database server the next time the database is resorted.

Data records have the common fields and characteristics described in Table 2.5, “plt###.dat field descriptions”.

Example

Here is an example of a plate 4 data record after level 1 validation to database file plt004.dat

1|1|9145/0045001|099|4|1|0123|egb|92/01/01|high blood pressure|
Dr. Smith|92/01/10 10:23:23|92/01/10 10:23:23|

The text fields have been entered and record status has been set to final.


Table 2.5. plt###.dat field descriptions

Field #ContainsDescription
1 (DFSTATUS)record statusEnumerated value from the list 1=final, 2=incomplete, 3=pending, 4=FINAL, 5=INCOMPLETE, 6=PENDING, 0=missed
2 (DFVALID)validation levelEnumerated value from the list 1, 2, 3, 4, 5, 6, 7
3 (DFRASTER)raster name the fax image id from which this data record was derived, if there was a fax image. The image id is always in the format YYWW/FFFFPPP or YYWWRFFFFPPP, where YY is the year (minus the century) in which the image id was created, WW is the week of the year, FFFF is a sequential base 30 value, in the range 0001-ZZZZ, representing the fax arrival order within the week, and PPP is the page number within the fax (also see pages). If the image id contains a slash (/) in the 5th character position, then the image id is for a fax. Pre-pending this image id with the value of the PAGE_DIR variable for the study creates a unique pathname to the file containing the fax image. If the image id contains R in the 5th character position, then the image id is for a raw data entered record, and in fact there is no image.
4 (DFSTUDY)study numberthe DFdiscover study number (must be constant across all records for the same study). Legal limit is 1-999.
5 (DFPLATE)plate numberthe plate number as identified in the barcode of the CRF. Within each data file, this field has a constant value. Legal limit is 1-501.
6 (DFSEQ[a])visit/seq numberthe visit or sequence number of this occurrence of a plate for a subjects id. Legal limit is 0-511 if defined in the barcode, and 0-65535 if defined as the first data field on the page.
7subject IDthe subject identifier. Legal limit is 0-281474976710655. subject identifiers are composed of site # + subject ID. This limit applies to the concatenated value of the two, where the site # legal limit is 0-21460.
8 to N-3datadata fields from the corresponding CRF page
N-2 (DFSCREEN)record statusEnumerated value from the list 1=final, 2=incomplete, 3=pending. This record status mimics the status recorded in the first field except that there is no distinction between primary and secondary.
N-1 (DFCREATE)creation datethe creation date and time stamp in the format yy/mm/dd hh:mm:ss. The creation date and time stamp are added to the record the first time it is validated to a level higher than 0. It is never modified after initial addition to the record.
N (DFMODIFY)last modification datethe last modification date and time stamp in the format yy/mm/dd hh:mm:ss The initial value for this field is the same as the creation date. Each time any data within the record is modified, the modification stamp is updated. It is not updated if only the record's validation level has changed.

[a] Field 6 has this name only if the field is defined to be in the barcode; otherwise, the name is user-defined.


Table 2.6.  plt###.dat - missed records

Usual Nameplt###.dat
Typeclear text
Created ByDFserver.rpc
Used ByDFserver.rpc
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record11
Description

If a CRF is reported missing, and is not expected to ever arrive (e.g. because the subject missed a visit, or refused a test), it can be registered in the study database using DFexplore. These records have a status of lost and are added to the study database file with the same plate number, plt###.dat, similar to actual data records. The contents of the record fields are described in Table 2.7, “Missed record field descriptions”.

Example

Here is an example of a missed data record

0|7|000/0000000|099|4|1|0123|1|subject was on vacation|92/01/10 10:23:23|92/01/10 10:23:23|


Table 2.7. Missed record field descriptions

Field #ContainsDescription
1 (DFSTATUS)record statusEnumerated value - always contains a 0 for missed records, which is equivalent to the record status lost
2 (DFVALID)validation levelEnumerated value from the list 1, 2, 3, 4, 5, 6, 7
3 (DFRASTER)image namealways contains 0000/0000000 for missed records. This is simply a placeholder value as there is no CRF image.
4 (DFSTUDY)study numberthe DFdiscover study number (must be constant across all records for the same study)
5 (DFPLATE)plate numberthe plate number
6visit/seq numberthe visit or sequence number
7subject IDthe subject identifier
8reason codeEnumerated value - the reason that the CRF was missed, selected from the following list: 1=Subject missed visit, 2=Exam or test not performed, 3=Data not available, 4=Subject refused to continue, 5=Subject moved away, 6=Subject lost to follow-up, 7=Subject died, 8=Terminated - study illness, 9=Terminated - other illness, 10=Other reason
9reason textan additional, optional explanation as to why the CRF was missed
10 (DFCREATE)creation datethe creation date and time stamp in the format yy/mm/dd hh:mm:ss
11 (DFMODIFY)last modification datethe last modification date and time stamp will always be the same as the creation date and time stamp as missed records are never modified once created (changes, if necessary, are made by deleting the existing missed record and creating another one)

2.6.3. Query data files

Table 2.8. DFqc.dat - the Query database

Usual NameDFqc.dat
Typeclear text
Created By 
Used ByDFserver.rpc, DFexplore, DFbatch, DF_CTqcs, DF_ICqcs, DF_ICrecords, DF_PTcrfs, DF_PTqcs, DF_QCreports, DF_QCsent, DF_QCstatus, DF_QCupdate
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record22
Description

The query data file, DFqc.dat, is maintained by the study server in the same manner as the CRF data files, DFin.dat and plt*.dat. All query records have exactly the same format, across all DFdiscover studies. Each query record has 22 fields, of which 5 (fields 4-8) are the keys needed to uniquely identify each record. Multiple queries are permitted per field provided their category codes are unique.

As for study data files, you should always use DFexport.rpc to export the query database before using it. For this purpose, DFqc.dat has been assigned a DFdiscover reserved plate number of 511.

Table 2.9, “DFqc.dat field descriptions” describes each field in a query record.

Example

An example of a newly added query that is to be included in the next Query Report sent to the originating site (the line break is for presentation purposes only):

1|1|0000/0000000|20|10|999|2100101|6|21|000000|0||1. Sample collection date
|11/10/05|1|2|||user1 06/05/27 11:35:57|user1 06/05/27 11:35:57||1|

An example of a query that was sent out in a Query Report and has now been resolved:

5|4|0000/0000000|20|397|2030|1415412|11|14|060424|31|1|A2. Whole Blood 1ml ID #
||1|2|||brian 06/03/23 12:57:20|brian 06/03/23 12:57:20|barry 06/05/11 14:10:04|1|


Table 2.9. DFqc.dat field descriptions

Field #ContainsDescription
1 (DFSTATUS)record statuscurrent status of the query. Legal values are: 0=pending (review), 1=new, 2=in unsent report, 3=resolved NA, 4=resolved irrelevant, 5=resolved corrected, 6=in sent report, 7=pending delete
2 (DFVALID)validation levelvalidation level at which the query was created or last modified. The query validation level matches the level of the data record upon export only when the DFexport.rpc -m option is used.
3 (DFRASTER)raster namemust contain "0000/0000000". Any references to actual raster names will result in the deletion of those rasters if the referencing query is deleted.
4 (DFSTUDY)study numberthe DFdiscover study number. Again, this number is constant across all records for one study.
5 (DFPLATE)plate numberthe plate number of the record that this query is attached to
6 (DFSEQ)visit/seq numberthe visit or sequence number of the record that this query is attached to
7 (DFPID)subject IDthe subject identification number
8 (DFQCFLD)query field numberthe data entry field to which this query is attached
[Warning]Query Field Number = Database Field Number - 3

For historical reasons this number is 3 less than the database field number. For example, a query on the ID field, which is always database field 7, will have a query field number of 4.

9 (DFQCCTR)site IDthe site ID that the query was sent to or will be sent to. This is determined when the query is created. Also, it is checked, and updated if necessary to account for a subject move, each time DF_QCupdate is executed.
10 (DFQCRPT)report numberthe Query Report that the query was written to. The Query Report number is always in the format yymmdd, for the year, month, and day the report was created. If the query has not yet been written to a report, the value is 0.
11 (DFQCPAGE)page numberthe page number of the Query Report, on which the query appears, or 0 if it has not yet been written to a report.
12 (DFQCREPLY)reply to querythe reply entered by an DFexplore user in the format name yy/mm/dd hh:mm:ss reply The maximum length of this field is 500 characters. It is blank when a new query is created, or when an old query is reset to new. This field cannot be created or edited in DFexplore by users who do not have adequate permissions. When a new reply is entered, the query status is changed to pending.
13 (DFQCNAME)namea description of the data field that is referenced by this query. Although this field can be edited when the quality control report is being added, its default value is taken from the "Description" part of the variable definition entered in the Setup tool. The maximum length of this field is 150 characters, but only the first 30 appear on quality control reports.
14 (DFQCVAL)valuethe value held in the data field when the query was added. If the query is subsequently edited and the value of the data field has changed, this field is updated with the new value. The maximum length of this field is 150 characters. For overdue visit queries, this field contains a julian representation of the date on which the visit was due (expressed as the number of days since Jan 1,1900).
15 (DFQCPROB)category codethe numeric category code. Legal values are: 1=missing value, 2=illegal value, 3=inconsistent value, 4=illegible value, 5=fax noise, 6=other problem, 21=missing plate, 22=overdue visit, 23=EC missing plate, 30-99=user-defined category code.
16 (DFQCRFAX)refax coderefax request code. Should the CRF page, which this query references, be re-sent? Legal values are: 1=no (clarification query type), 2=yes (correction query type).
17 (DFQCQRY)queryany additional text needed to clarify the query to the investigator who will receive the Query Reports. The maximum length of this field is 500 characters. Note: the category code label from field 15 is included automatically and is often all that is needed.
18 (DFQCNOTE)noteany additional text needed to describe the problem when it is resolved. The maximum length is 500 characters.
19 (DFQCCRT)creation datethe creator, creation date and time stamp of the query in the format name yy/mm/dd hh:mm:ss The creation date and time stamp are added when the query is first created.
20 (DFQCMDFY)last modification datethe modifier, last modification date and time stamp in the format name yy/mm/dd hh:mm:ss The initial value for this field is the same as the creation date. Each time the query is modified, the modification stamp is updated.
21 (DFQCRSLV)resolution datethe resolver, resolution date and time stamp in the format name yy/mm/dd hh:mm:ss Until the query is resolved, this field is blank
22 (DFQCUSE)usage codeLegal values are: 1=send to site (these queries are formatted into a Query Report and sent to the appropriate study site), 2=internal use only (these queries do not appear in site Query Reports).

2.6.4. Reason for change data files

Table 2.10. DFreason.dat - reason for change records

Usual NameDFreason.dat
Typeclear text
Created ByDFserver.rpc
Used ByDFserver.rpc
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record12
Description

Data fields may require that a change to the value in the field be supported by a reason for the change. This reason information is recorded in . The contents of the record fields are described in Table 2.11, “Reason for data change record field descriptions”.

Example

Here is an example of a reason data record

1|3|000/0000000|254|4|1|0123|9|phone|information provided by physician over the phone|nathan 03/01/10 10:23:23|nathan 03/01/10 10:23:23

It indicates that field 9 was changed because of the reason "information provided by physician over the phone".


Table 2.11. Reason for data change record field descriptions

Field #ContainsDescription
1 (DFSTATUS)record statusvalid status codes include: 1=approved, 2=rejected, 3=pending
2 (DFVALID)validation levelEnumerated value from the list 1, 2, 3, 4, 5, 6, 7. This is the record's last validation level when the reason was created or modified.
3 (DFRASTER)image namealways contains 0000/0000000 for reason for data change records. This is simply a placeholder value as there is no CRF image.
4 (DFSTUDY)study numberthe DFdiscover study number (must be constant across all records for the same study)
5 (DFPLATE)plate numberthe plate number
6 (DFSEQ)visit/seq numberthe visit or sequence number
7 (DFPID)subject IDthe subject identifier
8 (DFRSNFLD)reason field numberthe data entry field number this reason note refers to. This numbering does not correspond directly to the DFdiscover schema numbering, but instead is offset by 3, so for example, the ID field which is always database field number 7 will always report the reason field number as 4. This is identical to the behavior of the field number reported in queries.
9 (DFRSNCDE)reason codean optional coding of the reason for data change. Although this code field contains textual data, it should be possible to use it as a categorical variable. The code will typically come from the first field of the REASON lookup table, if it is defined.
10 (DFRSNTXT)reason textrequired text that provides the reason for the data change. The maximum length of this field is 500 characters.
11 (DFRSNCRT)creation datethe creator, creation date and time stamp in the format name yy/mm/dd hh:mm:ss. This field is completed when the reason for data change is first created.
12 (DFRSNMDF)last modification datethe modifier, last modification date and time stamp in the format name yy/mm/dd hh:mm:ss. The initial value for this field is the same as the creation date. Each time the reason note is modified, the modification stamp is updated.

2.6.5. Newly arrived data file

Table 2.12. DFin.dat - the new records database

Usual NameDFin.dat
Typeclear text
Created ByDFserver.rpc
Used ByDFserver.rpc
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record5+
Description

Each initial data record, created by the ICR software when it scans a CRF page, is passed to the study database server which appends it to this file. These are called "new records".

All new records have the common fields and attributes identified in Table 2.13, “DFin.dat field descriptions”.

The first 5 fields are added by the process that ICRed the page. The 4th and 5th fields were read from the barcode at the top of each CRF page. Dependant upon the success of the ICR algorithm, subsequent fields in each new record may or may not be filled in. If a fax image is particularly difficult to read, it is possible that only the first five fields will appear in the new record.

The first three fields, in addition to being delimited, are also in fixed column positions. The record status begins in the first column and is one column wide. The validation level begins in the third column and is one column wide. The raster name begins in the fifth column and is 12 columns wide. Columns two and four contain the field delimiter.

Example

Here is an example of a new data record from

0|0|9145/0045001|099|4|1|0123||92/01/01|||

This record has new record status, has not yet been validated, contains the data from fax image $(PAGE_DIR)/9145/0045001, and belongs to study 99, plate 4, visit 1, and subject ID 123.


Table 2.13. DFin.dat field descriptions

FieldContainsDescription
1 (DFSTATUS)record statusEnumerated value - always contains a 0 for new records, which is equivalent to the record status new
2 (DFVALID)validation levelEnumerated value - always contains a 0 for new records, which indicates that the record has only been validated by the ICR software
3 (DFRASTER)image namethe fax image from which this data record was derived. The value for this field will always be in the format yyww/ffffppp. Prepending this name with the value of the PAGE_DIR variable for the study creates a unique pathname to the file containing the fax image
4 (DFSTUDY)study numberthe DFdiscover study number (must be constant across all records for the same study)
5 (DFPLATE)plate numberthe plate number as identified in the barcode of the CRF

2.6.6. Journal files

Table 2.14. YYMM.jnl - monthly database journal files

Usual NameYYMM.jnl
Typeclear text
Created ByDFserver.rpc
Used ByDF_ATfaxes, DF_ATmods, DF_WFcrfs, DF_WFqcs
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record11+
Description

The study database server adds a record to the current study database journal file, each time that a data record is written to the study database, including when:

  • a new record arrives in the study database from the incoming daemon

  • an existing record is modified in DFexplore

  • the validation level of an existing record is modified in DFexplore

  • the status of an existing record is modified in DFexplore

  • an existing record is deleted in DFexplore

  • a query is added, modified, resolved, or deleted in DFexplore

  • a record is imported into the study database using DFimport.rpc

  • the database is structured by DFsetup

Separate journal files are kept for each study, and within a study, a new journal file is created for each month. The naming scheme for journal files is YYMM.jnl where YY is the last two digits of the year (e.g. 92) and MM is the two digit month of the year, January being 01 and December being 12.

The fields within each journal record are defined as described in Table 2.15, “YYMM.jnl field descriptions”.

Example Following is an example of a complete journal record. The line breaks are for presentation purposes only.
980312|132426|valid1|d|1|1|9807/0047008|254|5|24|99001|ABC|06/07/97|
171|097|172|096|2|055.1|121.5|2||1143|00|*|*|*|1|1|*|1|1|
98/03/12 13:24:26|98/03/12 13:24:26|

Table 2.15. YYMM.jnl field descriptions

Field #ContainsDescription
1date stampA date stamp, in YYMMDD format, identifying when the data record was written to the database
2time stampa time stamp, in HHMMSS format, identifying when the data record was written to the database. Hours are reported in 24-hour notation.
3usernamethe username of the person who wrote the record to the database. This is the login name of the user who modified the record.
4record typeEnumerated value - indicates the type of the journal record. Possible values are: d for data record, q for query record, r for reason record, s for the beginning of a setup restructuring, and S for the end of a setup restructuring.
5-11record keysfields 5 through 11 contain the first 7 fields from the data record.
12+record data fieldsthe remaining data fields (8 to the end of the record) follow.

2.6.7. Index files

Table 2.16. plt###.ndx - per plate index files

Usual Nameplt###.ndx
Typebinary
Created ByDFserver.rpc
Used By, DFshowIdx
Field DelimiterNA
Record DelimiterNA
Comment DelimiterNA
Fields/RecordNA
Description

Every study data file has a corresponding index file that the study server uses to track the current status and location of each record in the data file. The index entry for a particular data record includes the value of the key fields id, plate, and visit/sequence number, the record status, the validation level, the offset of the beginning of the record into the data file, and the length of the data record. When searching for a data record by keys, it is much more efficient for the database server to search the index file for matching keys and then use the offset and length to extract the data record from the data file.

Each time an existing data record is modified or a new record is added, a new entry is made at the end of the index file for the new, modified copy of that record, and the status of the old index entry (if there was one) is changed to indicate that it has been superseded by a new entry.

Index and data files are sorted (on id and visit/sequence number) by the study database server, each time the server exits. Before sorting, an index file has M sorted entries at the top of the file, and N unsorted entries at the bottom of the file. When searching, the unsorted index entries are searched first in a linear fashion and then, if necessary, a binary search is performed on the sorted entries.

The first 32 bytes of an index file are header information, consisting of four 4-byte numbers that identify attributes of the file as a whole, as described in Table 2.17, “plt###.ndx header bits”, followed by 16 bytes of padding. Before the study database server exits, it checks this header information of each index file. If the number of unsorted entries and the number of pending deletes are both 0, then the file is already sorted and does not need further attention.

The 32 bytes of header information are followed by the actual index entries. Each index entry is 32 bytes in size and is described in Table 2.18, “plt###.ndx record bits”.

Each index entry contains 8 bytes for the subject ID, 2 bytes for the visit number, 2 bytes for the plate number, 4 bytes for the offset into the data file, 2 bytes for the record length, a status byte and 13 bytes of padding.

The status byte encodes three pieces of information: the record status (equivalent to the numeric value of the first byte of the data record), the record validation level (equivalent to the numeric value of the third byte of the data record), and the status of the index entry. This is encoded as illustrated in Figure 2.1, “Encoding for bits within the status byte”.

Figure 2.1. Encoding for bits within the status byte

Encoding for bits within the status byte

The record status contains the same values as those allowed in the record status field of the data record, namely 0 through 7 (binary 111). Similarly, the validation level will take on the same values as those allowed in the validation level of the data record, again 0 through 7. The index status contains the value 2 if this is a new index entry, 1 if the index entry has been superseded by a newer one, and 0 otherwise.

The size of all index files should always be a multiple of 32 bytes.


Table 2.17. plt###.ndx header bits

First ByteLast ByteContains
14magic number indicating that this is an index file. Should always have the fixed value 0xdf6464df.
58the number of sorted index entries
912the number of unsorted index entries
1316the number of pending deletions
1732reserved for future use

Table 2.18. plt###.ndx record bits

SizeContains
8 bytessubject ID
2 bytesvisit/sequence number
2 bytesplate number
4 bytesoffset in bytes from beginning of data file to first byte of record (0 based)
2 byteslength of record in bytes
1 bytestatus bits
13 bytesreserved for future use

2.7.  The study ecsrc directory

This directory contains the edit check source files.

Table 2.19. DFedits - edit checks

Usual NameDFedits
Typeclear text
Created ByDFsetup
Used ByDFexplore
Field DelimiterNA
Record DelimiterNA
Comment Delimiter#
Fields/RecordNA
Description

This file contains the edit checks that are defined for this study. The edit check language is fully described in Edit checks.

Example
edit SetInit()
{
    if ( dfblank( init ) && !dfblank( init[,0,1] ) )
        init = init[,0,1];
}
edit AgeOk()
{
    number      age;
    if ( !dfblank( p001v03 ) && !dfblank( p001v04 ) )
    {
        age = ( p001v03 - p001v04 ) / 365.25;
...

2.8.  The study lib directory

This directory contains the study configuration files, those files that make this study unique from every other DFdiscover study.

Table 2.20. DFcenters - sites database

Usual NameDFcenters
Typeclear text
Created ByDFsetup
Used Byall study tools and the standard reports including: DF_CTqcs, DF_CTvisits, DF_PTcrfs, DF_QCfax, DF_QCfaxlog, DF_QCreports, DF_QCupdate, DF_SScenters, DF_XXtime, and DF_qcsbyfield
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record11+
Description

Each DFdiscover study has a sites database that records where each participating site is located, who the contact person is, and what subject ID number ranges are covered by each site ID.

Each site typically corresponds to a different clinical site, but this is not required. If necessary, a single clinical site may be defined as 2 or more sites, each corresponding to a different participating clinical investigator at that site. The sites database consists of one record per site ID. The field definitions are described in Table 2.21, “Field definitions for DFcenters.

Each site ID must be a number in the range 0 to 21460 and uniquely identifies the site within the database. All site numbers printed on DFdiscover reports are leading zero-padded to 3 digits but they may be up to 5 digits in length if the site number is greater than 999. The contact person is required as it is the person that outgoing faxes for the site are addressed to. A primary fax is not required and may be left blank for sites that are not meant to receive Query Reports. The primary fax number, if specified, should exactly match the number, including long distance, country, and area codes, that would be dialed from a numeric keypad. Most punctuation characters are ignored so they can be used for improving readability. Valid punctuation characters include: +#*-,()0-9 For example, 1-(416)521-9800 is equivalent to 14165219800.

[Warning]Whitespace characters

Whitespace characters (space and tab) may not be used as punctuation as they are interpreted as delimiters between multiple entries.

Each comma inserts a one-second pause in the dialing sequence. This can be helpful when leaving a local PBX or waiting to dial an extension.

A valid email address may also be supplied in place of or together with a primary fax number. The email address must be specified using the notation mailto:email_address. The prefix mailto: is fixed and required while email_address must be a valid email address (there is no validity checking performed on email addresses, so be careful to enter them correctly).

If more than one email and/or fax number is specified, each must be separated by a single space. Comma separators are not permitted.

Subject ID ranges are specified in fields 11 through the end of the record. There is no limit to the number of range fields that can be given. Each range field contains a minimum subject ID and maximum subject ID separated by exactly one space character. For example,

|101 199|

indicates that subject IDs 101 through 199 inclusive are to be included for the site. Individual subject IDs that are disjoint from any range are indicated by setting both the minimum and maximum ids to the actual subject ID. For example,

|244 244|301 399|401 420|

includes subject IDs 244, 301 through 399 inclusive, and 401 through 420 inclusive.

In the event that subject IDs are incorrectly entered in data records, there should always be a 'catch-all' site listed that receives all subject IDs that do not fall into any of the other subject ID ranges. This site is indicated by the phrase ERROR MONITOR in the 11th field and no other subject ID range fields.

[Note]Note

DFcenters may be modified at any time, via DFsetup only, for an active study. However, if quality control reports are being created for the study, the DFdiscover report DF_QCupdate must always be run after DFcenters modification and prior to Query Report creation. DF_QCupdate ensures that the most up-to-date DFcenters file will be used when generating subject status and scheduling information for the Query Reports.

Example A single sites database record for a site that is responsible for subject IDs 1101 through 1149, and 1151 through 1199 inclusive.
011|Lisa Ondrejcek|DF/Net Research, Inc|140 Lakeside Avenue, Suite 310, Seattle, Washington, 98122|1-206-322-5932|country:USA;enroll:100|1-206-322-5931|Lisa Ondrejcek|1-206-322-5931||1101 1149|1151 1199

Table 2.21. Field definitions for DFcenters

Field #DescriptionRequiredMaximum Size
1site IDyes5 digits
2Contactyes30 characters
3Nameyes40 characters
4Addressno80 characters
5Faxno4096 characters
6Attributes including Start Date, End Date, Enroll Target, Protocol Effective Date (x5), Protocol Version (x5)

This field contains zero of more semi-colon ( : ) delimited pairs, where each pair is a keyword and value where the keyword is from the list country, beginDate, endDate, enroll, protocol1, protocol1Date, protocol2, protocol2Date, protocol3, protocol3Date, protocol4, protocol4Date, protocol5, protocol5Date, testSite.

no4096 characters
7Telephone (site)no30 characters
8Investigatorno30 characters
9Telephone (investigator)no30 characters
10Reply to email addressno80 characters
11+Subject ID rangesyes30 digits,1 space

Table 2.22. DFccycle_map - conditional cycle map

Usual NameDFccycle_map
Typeclear text
Created ByDFsetup
Used ByDF_QCupdate
Field Delimiter|
Record Delimiter\n
Comment Delimiter#
Fields/Record2+
Description

This table describes the conditional cycle map file structure and provides an example. It does not describe all of the syntax and rules related to this feature. Usage instructions for all 4 conditional maps is fully described in Study Setup User Guide, Conditional Maps.

The file contains one or more specifications, each consisting of a condition definition followed by one or more actions to be applied if the condition is met. Entries in the file have the general appearance of:

IF|Visit List|Plate|Field|Value
AND|Visit List|Plate|Field|Value
[+-~]|List of conditional cycles

Each condition may be followed by one or more action statements. Each of these statements begins with: '+' to indicate that the cycles are required, '-' to indicate that the cycles are unexpected, or '~' to indicate that the cycles are optional, when the condition is met.

There is no limit to the number of condition/action entries that may be included but the order in which the conditions appear may be important, because in the event of a conflict, the action specified by the last entry, applicable to each cycle, is the action that will be applied. This point is illustrated in the following example.

Example
IF|0|1|22|6
+|2,5,8
-|3,6,9
~|4,7,10

IF|0|1|22|5
AND|0|9|13|>0
AND|0|9|36|!1
+|11

IF|1|3|9|~^A
-|11

This example, consists of 3 conditional specifications. They are applied in the order in which they are defined. The first specification indicates that, if field 22 on plate 1 at visit 0 equals 6, then cycles 2, 5 and 8 are required; cycles 3, 6 and 9 are not expected; and cycles 4, 7 and 10 are optional. The second specification indicates that, if field 22 on plate 1 at visit 0 equals 5, and field 13 on plate 9 at visit 0 is greater than zero, and field 36 on plate 9 at visit 0 is not equal to 1, then cycle 11 is required. The third specification indicates that, if field 9 on plate 3 at visit 1 begins with the capital letter "A", then cycle 11 is not expected.

If both conditions 2 and 3 are met cycle 11 will be considered unexpected because, when a conflict occurs, the last condition wins.


Table 2.23. DFcvisit_map - conditional visit map

Usual NameDFcvisit_map
Typeclear text
Created ByDFsetup
Used ByDF_QCupdate
Field Delimiter|
Record Delimiter\n
Comment Delimiter#
Fields/Record2+
Description

This table describes the conditional visit map file structure and provides an example. It does not describe all of the syntax and rules related to this feature. Usage instructions for all 4 conditional maps is fully described in Study Setup User Guide, Conditional Maps.

The file contains one or more specifications, each consisting of a condition definition followed by one or more actions to be applied if the condition is met. Entries in the file have the general appearance of:

IF|Visit List|Plate|Field|Value
AND|Visit List|Plate|Field|Value
[+-~]|List of conditional visits

Each condition may be followed by one or more action statements. Each of these statements begins with: '+' to indicate that the visits are required, '-' to indicate that the visits are unexpected, or '~' to indicate that the visits are optional, when the condition is met.

There is no limit to the number of condition/action entries that may be included but the order in which the conditions appear may be important, because in the event of a conflict, the action specified by the last entry, is the action that will be applied. This point is illustrated in the following example.

Example
IF|0|1|22|6
+|10-19
-|20-29
~|30

IF|0|1|22|5
AND|0|9|13|>0
AND|0|9|36|!1
+|40

IF|1|3|9|~HIV
-|40

This example, consists of 3 conditional specifications. They are applied in the order in which they are defined. The first specification indicates that, if field 22 on plate 1 at visit 0 equals 6, then visits 10 to 19 are required, visits 20 to 29 are unexpected, and visit 30 is optional. The second specification indicates that, if field 22 on plate 1 at visit 0 equals 5, and field 13 on plate 9 at visit 0 is greater than zero, and field 36 on plate 9 at visit 0 is not equal to 1, then visit 40 is required. The third specification indicates that, if field 9 on plate 3 at visit 1 contains the literal string "HIV", then visit 40 is not expected.

If both conditions 2 and 3 are met, visit 40 will be considered unexpected because, when a conflict occurs, the last condition wins.


Table 2.24. DFcplate_map - conditional plate map

Usual NameDFcplate_map
Typeclear text
Created ByDFsetup
Used ByDF_QCupdate
Field Delimiter|
Record Delimiter\n
Comment Delimiter#
Fields/Record2+
Description

This table describes the conditional plate map file structure and provides an example. It does not describe all of the syntax and rules related to this feature. Usage instructions for all 4 conditional maps is fully described in Study Setup User Guide, Conditional Maps.

The file contains one or more specifications, each consisting of a condition definition followed by one or more actions to be applied if the condition is met. Entries in the file have the general appearance of:

IF|Visit List|Plate|Field|Value
AND|Visit List|Plate|Field|Value
[+-~]Visit List|List of conditional plates

Each condition may be followed by one or more action statements. Each of these statements begins with: '+' to indicate that the plates are required, '-' to indicate that the plates are unexpected, or '~' to indicate that the plates are optional, at the specified visits, when the condition is met.

There is no limit to the number of condition/action entries that may be included but the order in which the conditions appear may be important, because in the event of a conflict, the action specified by the last entry, applicable to each plate, is the action that will be applied. This point is illustrated in the following example.

Example
IF|0|1|22|6
+10,20|50,51
-10,20|40,41
~10,20|15

IF|0|1|22|5
AND|0|9|13|>0
AND|0|9|36|!1
+91-95|16

IF|1|3|9|yes
-91|16

This example, consists of 3 conditional specifications. They are applied in the order in which they are defined. The first specification indicates that, if field 22 on plate 1 at visit 0 equals 6, then at visits 10 and 20: plates 50 and 51 are required, plates 40 and 41 are not expected, and plate 15 is optional. The second specification indicates that, if field 22 on plate 1 at visit 0 equals 5, and field 13 on plate 9 at visit 0 is greater than zero, and field 36 on plate 9 at visit 0 is not equal to 1, then at visits 91-95 plate 16 is required. The third specification indicates that, if field 9 on plate 3 at visit 1 contains exactly the string "yes", and nothing more, then plate 16 is not expected at visit 91.

If both conditions 2 and 3 are met plate 16 will be considered unexpected at visit 91, but required at visits 92-95, because, when a conflict occurs, the last condition wins.


Table 2.25. DFcterm_map - conditional termination map

Usual NameDFcterm_map
Typeclear text
Created ByDFsetup
Used ByDF_QCupdate
Field Delimiter|
Record Delimiter\n
Comment Delimiter#
Fields/Record1+
Description

This table describes the conditional termination map file structure and provides an example. It does not describe all of the syntax and rules related to this feature. Usage instructions for all 4 conditional maps are fully specified in Study Setup User Guide, Conditional Maps.

The file contains one or more specifications, each consisting of a condition definition followed by one action to be applied if the condition is met. Entries in the file have the general appearance of:

IF|Visit List|Plate|Field|Value
AND|Visit List|Plate|Field|Value
A or E

Each condition is followed by either the letter 'A' (abort all follow-up), or 'E' (early termination of the current cycle). The termination date is defined as the visit date of the visit that triggered the condition, specifically the visit specified in the IF statement.

Example
IF|0|1|22|6
A

IF|6|1|22|5
AND|6|9|13|>0
AND|6|9|36|!1
E

This example, consists of 2 conditional specifications. The first specification indicates that, if field 22 on plate 1 at visit 0 equals 6, then all follow-up terminates as of the visit date for visit 0. Visits scheduled to occur before this date are still expected, but visits scheduled following this date are not.

The second specification indicates that, if field 22 on plate 1 at visit 6 equals 5, and field 13 on plate 9 at visit 6 is greater than zero, and field 36 on plate 9 at visit 6 is not equal to 1, then the current cycle terminates, i.e. the cycle in which visit 6 is defined; with the termination date being the visit date of visit 6. Any visits in this cycle (or in previous cycles) that were scheduled to occur before the termination date are still expected, but visit within this cycle scheduled following this date are not. On termination of a cycle, subject scheduling proceeds to the next cycle in the visit map, if there is one.


Table 2.26. DFCRFType_map - CRF type map

Usual NameDFCRFType_map
Typeclear text
Created ByDFsetup
Used ByDFbatch, DFprintdb, DFimport.rpc, DFexport.rpc, DFcmpSchema DFcmpSchema
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record2
Description

Each record in the CRF type map has two fields, an acronym or short form (1st field), and a descriptive label (2nd field). The CRF type acronym and label appear in the Import CRFs dialog. The CRF type label appears in the DFexplore Preferences dialog.

This file is optional. If it does not exist, CRF Types are not used for this study.

Example
PAPER|Print Version
CHINESE|Chinese
ENGLISH|English
SWEDISH|Swedish
FRENCH|French
SPANISH|Spanish

Table 2.27. DFcrfbkgd_map - CRF background map

Usual NameDFcrfbkgd_map
Typeclear text
Created ByDFsetup
Used ByDFprintdb
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record3
Description

Each record in the CRF background map has three fields: the visit number with the background to be repeated (1st field), the list or range of visit numbers where the background will be repeated (2nd field), and an optional comment field (3rd field).

This file is optional. If it does not exist, the default CRF will be displayed for visits not tagged during the Import CRFs step. If no default CRF has been imported, the CRF background will be blank for untagged visits.

Example
10|11,13-14,16-17,19-20|Quarterly visits
12|15,18|Annual visits
22|25,28|Annual lab work

Table 2.28. DFedits.bin - published edit checks

Usual NameDFedits.bin
Typebinary
Created ByDFsetup, DFcompiler
Used ByDFexplore
Field DelimiterNA
Record DelimiterNA
Comment DelimiterNA
Fields/RecordNA
Description

This file contains an internal, binary equivalent of the edit checks stored in the plain text DFedits file. This binary format contains no external references to other included files and is a more compact representation that can be more efficiently transmitted to and interpreted by DFexplore clients.

The DFedits.bin file is created when Publish is selected in DFsetup's edit checks dialog.

This file is the only edit checks file used by DFexplore.


Table 2.29. DFfile_map - file map

Usual NameDFfile_map
Typeclear text
Created ByDFsetup
Used ByDFserver.rpc
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record4
Description

The file map lists all of the unique plate numbers used in a particular study database. The study server uses this file at start-up to determine which database files it may need to initialize and allocate file descriptors for. In addition to the listed plate numbers, the study server also allocates file descriptors for the new records file () and the query data file ().

The format of this file is one record for each plate defined in the study setup. Each record has 4 fields. The first field of the record is the plate number, leading zero-padded to three digits. The second field is a textual description of the plate from the user-supplied definition in DFsetup. The textual part is displayed in the plate selection dialogs that can be found in various study tools. The third field identifies how the visit/sequence number key field is coded for that plate. A code of 1 means that the visit/sequence number is in the barcode; a code of 2 means that it is the first data field on the data entry screen for that plate. Any other code is an error. The fourth field indicates whether the arrival of this plate signals early termination of study follow-up for the subject, as would for example the arrival of a death report. A code of 1 appears if the plate signals early termination, otherwise the code is 2.

The records in the file map do not have to be sorted in increasing plate number order as the file is internally sorted by the study database server at start-up time.

Example
001|Dosage of Study Drug (DOST-2)|2|2
002|Concomitant Medication (COM)|1|2
003|Written Consent (CONS-w)|1|2
005|Diagnosis (DSM-III-R)|1|2
006|Psychiatric History (PSH)|1|2
200|Death Report|1|1

Table 2.30. DFlogo.png - study logo, for reports

Usual NameDFlogo.png
TypePNG
Created ByNA
Used ByDFexplore
Field DelimiterNA
Record DelimiterNA
Comment DelimiterNA
Fields/RecordNA
Description

A study logo is a small PNG file, maximum dimension is 64 pixels tall and 128 pixels wide. DFdiscover reports can display the study logo in the top-left corner of the report output. If no study logo is available, the study name is written to the header of each report output.

The study logo must be created outside of DFdiscover - there is no interface for creating it. Once the file is created, it must be copied to the study folder and saved as lib/DFlogo.png.


Table 2.31. DFlut_map - lookup table map

Usual NameDFlut_map
Typeclear text
Created ByDFsetup
Used ByDFexplore, DFbatch
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record2
Description

Lookup tables are used to select results from a list of pre-defined values and insert them into DFdiscover data fields. Lookup tables can also be defined for the query, query note, and reason fields in DFexplore to allow users to select pre-specified text for these fields. Lookup tables are described in Lookup tables.

A study setup may use multiple lookup tables and so the lookup table map is used to associate simple table names with more complex and lengthy full pathnames of the actual lookup tables.

Each record in the lookup table map has a lookup table name, followed by a |, and a filename containing the lookup table. DFdiscover will search for the lookup table filename first in the $(STUDY_DIR)/lut directory and if that fails in the /opt/dfdiscover/lut directory. The table name is a symbolic name that can be referenced in edit checks.

The special table names, QC, QCNOTE and QCREPLY are used to associate a lookup table with the detail, note and reply fields, respectively, in the DFexplore Field > Add/Edit/Reply Query dialogs. The special table name REASON is used to associate a lookup table with the reason code and text for data changes in DFexplore.

Example
QC|DF_QClut
QCNOTE|DF_QCnotelut
QCREPLY|DF_QCreplylut
REASON|DF_Reasonlut
MEDS|meds.dict
COSTART|costart.dict
WHO|who.dict

Table 2.32. DFmissing_map - missing value map

Usual NameDFmissing_map
Typeclear text
Created ByDFsetup
Used ByDFbatch, DFprintdb, DFimport.rpc, DFexport.rpc, DFcmpSchema, DF_QCupdate, DF_XXkeys, DF_stats, DFsas
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record2
Description

Each record in the missing value map has two fields, the missing value code (1st field), and a descriptive label for the code (2nd field). The missing value code appears verbatim in any data field which has been identified as missing with that code.

The missing value map lists all missing value codes used in the study. These codes can be entered into any data field by selecting the desired code from the list available under Field > Mark Field Missing in DFexplore.

Note that the field delimiter, |, which is used in all DFdiscover databases, can not be used as a missing value code (for obvious reasons). Any other single (or multiple) character is acceptable.

This file is optional. If it does not exist, a single missing value of * (asterisk) - Not Available is available by default. If it does exist but is empty, then no missing values are permitted.

[Warning]Changes to the missing value map for an in-progress study

DFdiscover determines if each data value is a missing value code by comparing the value with each of the missing value codes in the map. If there is a match, DFdiscover handles that data value as a missing value. If there is no match, the data value is handled as a normal data value. This is important to remember because changes to the missing value map after a study has started and data has been entered can result in DFdiscover handling data values in a manner which is different from the handling when the value was originally entered. For example, defining .A as a missing value code at study start and then subsequently deleting that code from the missing value map will result in DFdiscover treating each occurrence of .A in the current database as a normal data value.

Example
*|Not Available
.|Not Applicable
-|Temporarily Unavailable
.U|Unknown

Table 2.33. DFpage_map - page map

Usual NameDFpage_map
Typeclear text
Created ByDFsetup
Used ByDF_QCreports, DFexplore
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record3
Description

This file is optional for a study. If defined, it must be located in $(STUDY_DIR)/lib/DFpage_map. The page map allows one to specify non-default labels, to identify visits and CRF pages, on Query Reports and in DFexplore's subject binder view. If a page map file is not defined, queries and pages in the binder are identified by the visit number and plate number keys of the CRF.

The first record in a page map contains only a single field which specifies a text label to be used as a title over the queries. The default label is:

PLATE       SEQNO

This label appears at the top of the Fax/Refax List and Question & Answer List sections of the Query Reports.

The remaining records describe the text labels to be used in place of the default plate and visit number identifiers.

The first field is the plate number, the second field is the visit/sequence number, and the third field is the substitution to be used for that plate and visit/sequence combination. The third field can contain %P and %S which are replaced with the plate number and the visit/sequence number fields from the query, respectively. It may also contain parts of the plate number and/or sequence number fields by using the notation %{n.P}, %{P.n}, %{n.S}, or %{S.n}. %{n.P} which returns the leading n digits of the plate number while %{P.n} returns the trailing n digits of the plate number. Similarly, %{n.S} and %{S.n} produce n leading and trailing digits of the sequence number. The third field may also contain the notation %# which is replaced with the value stored in field n of the data record matching the specified plate and sequence number. Additionally, when using the %# notation, and for data fields that have a data type of choice or check, it is possible to request that the reported value be decoded by using the %n:d notation. This substitutes the label associated with the value, instead of the value itself. [a]

When a Query Report is created, those queries whose plate and visit numbers match the first and second fields, will be identified on the Query Report by the label which appears in the third field.

If either the first field or second field contains a *, all values for that field that have not yet matched a previous rule will use the format in the third field.

For more information, see Study Setup User Guide, Page Map.

Example
PAGE (PLATE-SEQ)
018|001|MED VISIT1
*|001|VISIT1 (%P-%S)
025|*|TERM (%P-%S)
*|*|(%P-%S)
Note how the last rule catches all plate and visit/sequence pairs that were not previously matched.

[a] To implement %# or %n:d, the data record must be available. This is always true for Query Reports. However, in DFexplore only the currently visible data record is available at any moment. The result is that in the subject binder, for any record other than the current record, use of %# or %n:d is substituted with ???. This limits the usefulness of this notation in DFexplore.


Table 2.34. DFqcsort - Query and CRF sort order

Usual NameDFqcsort
Typeclear text
Created ByDFsetup
Used ByDF_QCreports
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record3
Description

A query sort order map can be specified to control the order in which queries appear on the reports created by DF_QCreports.

If DFqcsort is not defined, queries will appear on Query Reports sorted by ascending subject ID, then visit number, then plate, and finally data field number. A different sort order can be specified to reorder queries from particular visits and/or plates. For example, one might want all queries from adverse event reports to appear at the top of each subject's list. This can be done with DFqcsort.

The file contains one or more records in the following format:

plate#|visit#|sort_order#

where the sort_order# is an integer-valued sort priority such that a lower value indicates an earlier sort order. There is no minimum or maximum value for sort_order#, hence negative numbers can be used to give higher priority to sort_order#s of zero or greater. If two or more entries assign the same sort_order#, those entries are sorted in the default manner of ascending plate number within ascending visit number.

Within each record the plate number, or the visit number, can be replaced by * to signify all plates for a specified visit, or all visits for a specified plate. In such cases queries are sorted numerically on the unspecified field within the specified field. For example:

12|*|1

specifies that, for each subject, all queries on plate 12 are to be given a sort priority of 1 as compared to other queries. Since no visit number is specified, if queries appear on plate 12 at more than one visit for a given subject, they will appear on the Query Report sorted by visit/sequence number. * may also be specified for both plate and visit numbers (to catch all plate and visit combinations that were not otherwise specified), but this is not necessary as this is given the lowest sort priority by default.

Note that while sort order control is provided over plates and visits, no control is provided at the individual field level. Queries on a given plate are always sorted in field number order. Also queries and records are always sorted by ascending subject ID. Thus the control provided here does not allow for all possible sort orders. It merely provides a mechanism for controlling query ordering by plate and visit numbers.

Example
12|*|1  (1)
*|1|2   (2)
25|2|3  (3)
13|2|4  (4)
5|2|5   (5)
*|2|6   (6)
*|*|7   (7)

(1)

Queries on plate 12 of any visit appear first

(2)

Queries for visit 1 appear next (in plate number order)

(3)

Queries on plate 25 of visit 2 appear next

(4)

Queries on plate 13 of visit 2 appear next

(5)

Queries on plate 5 of visit 2 appear next

(6)

Queries on all remaining plates of visit 2 appear next

(7)

All the rest appear in the default order (i.e. plate# within visit#)


Table 2.35. DFqcproblem_map - Query category code map

Usual NameDFqcproblem_map
Typeclear text
Created ByDFsetup
Used ByDFbatch, DFprintdb, DFimport.rpc, DFexport.rpc, DFcmpSchema, DF_QCupdate, DF_XXkeys, DF_stats, DFsas, DFexport
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record4
Description

Each record in the Query category code map has four fields, the category code (1st field), the descriptive label (2nd field), the auto-resolve value (3rd field), and the sort value (4th field). The query category code appears verbatim in any data field which has been assigned a query with that code.

The pre-defined categories are assigned integer codes from 1-6 and 21-23 (inclusive). User-defined categories must be assigned integer codes ranging from 30-99 (inclusive). Category codes must be unique.

The problem labels must have a length ranging from 1-20 characters (inclusive), must be unique, and are case insensitive.

The auto-resolve field may be set to No (0) or Yes (1). With auto-resolve set to Yes, an outstanding query with this category code will be automatically resolved if a new reason is added to the corresponding data value.

The sort value may be set to any integer value between -2147483648 and 2147483647. Category types are sorted in ascending order by sort value, then by code.

The query category code map lists all query category codes in the study. A query with one of these types can be entered into any data field by selecting the desired type from the category code list available under Field > Add Query... in DFexplore.

When a new study is started, the file is created. By default, the file is populated with the following:

1|Missing|1|0
2|Illegal|1|0
3|Inconsistent|1|0
4|Illegible|1|0
5|Fax Noise|1|0
6|Other|1|0
21|Missing Page|0|0
22|Overdue Visit|0|0
23|EC Missing Page|0|0

Example
1|Missing|1|0
2|Illegal|1|0
3|Inconsistent|1|0
4|Illegible|1|0
5|Fax Noise|1|0
6|Other|1|0
21|Missing Page|0|0
22|Overdue Visit|0|0
23|EC Missing Page|0|0
30|Clinical QC|1|1
50|Needs Review|0|1
37|Refuted|1|2

Table 2.36. .DFreports.dat - reports history

Usual Name.DFreports.dat
Typeclear text
Created ByDFexplore
Used ByDFexplore
Field Delimiter|
Record Delimiter\n
Comment Delimiter#
Fields/Record9+
Description

Users create and save report history lists (including options) in much the same way that they create and save task lists.

The reports history list for ail study users is stored in this file in the study lib directory. Each history list is represented by a single record in this file. Each user may have more than one history list.

History lists are created in DFreport by executing the desired reports and then saving the history list. There is no text-based interface to this file or its contents.


Table 2.37. DFschema - database schema

Usual NameDFschema[a]
Typeclear text
Created ByDFsetup
Used ByDF_QCupdate, DF_ICschema, DF_ICrecords, DF_SSvars, DF_SSschema, DFcmpSchema, DFimport.rpc, DFsas, DFsqlload
Field Delimiter\n
Record Delimiter\n\n
Comment DelimiterNA
Fields/Record5+
Description

The schema, or data dictionary, is generated/updated automatically by DFsetup whenever a save is required for the study definition. Each field is composed of a key letter (with a leading % character) to indicate the field type, a space character, and the value of the field. This is essentially UNIX 'refer' format.

The defined key letters and corresponding field types are listed in the Table 2.39, “Schema codes and their meaning”.

The T code requires additional explanation. Its value is the variable type (one of choice, check, int, date, string) followed by the setup style name. If the variable type is date, the style name is followed by the variable pivot year and the date imputation method and whether the date is used for scheduling purposes or not. The pivot year represents the first possible year in those dates where the year is only 2 digits. The imputation method is one of:

0 never - no imputation is allowed
1 start - impute dates to the beginning of the month or year
2 middle - impute dates to the middle of the month or year
3 end - impute dates to the end of the month or year

The scheduling attribute is unused by most programs except for DF_QCupdate which searches the schema file for date fields which use the Visit Date attribute. These date fields are used to determine subject visit scheduling. DFsas also uses this information to determine the correct year value to include in the YEARCUTOFF option statement.

Each study schema begins, in order, with records defining values for the S, B, N, Y and then Z codes. If study is linked with another study then a and b fields are included identifying production and development study numbers. Thereafter, each new plate definition begins with a record defining values for the P, t, n, E, and R codes. This is followed by records for each variable in that plate. Each variable has a record that begins with an I field, followed by at least i, V, D, W, w, T, and A fields. The v, F, L, H, J, j, K, k, g, h, s, c, and C fields are included only if applicable.

Example
%S 254
%N 12
%B 1990
%Y 0 0
%Z 40

%P 1
%p Blood Pressure Screening Visits
%n 35
%t simple
%E 1
%R PrintCRF.shx

%I 1
%i 10
%V DFstatus1
%v DFSTATUS
%D DFdiscover Record Status
%T choice SimpleChoice
%A required
%W 1
%L 0~6
%C 0 lost
%C 1 final
%C 2 incomplete
%C 3 pending
%C 4 FINAL
%C 5 INCOMPLETE
%C 6 PENDING

%I 2
%i 11
%V DFvalid1
%v DFVALID
%D DFdiscover Validation Level

[a] This file is present for users and programs that require backwards compatibility and are not able to read the JSON study definition. This file may be removed in a future release.


Table 2.38. DFschema.stl - database schema styles

Usual Name[a]
Typeclear text
Created ByDFsetup
Used ByDFsetup
Field Delimiter\n
Record Delimiter\n\n
Comment DelimiterNA
Fields/Record2+
Description

This file is very much like the study schema, DFschema - database schema, but instead of variable definitions it simply catalogs all of the variable styles used in DFsetup to define new variables. Like DFschema the file is generated/updated automatically by DFsetup whenever a save is required for the study definition.

Note that the style schema lists only field types that are defined by the style itself. It omits those definitions that are specified at the variable level.

Example
%S 254
%N 51

%I 1
%T int SimpleNumber
%A optional

%I 2
%T date SimpleDate 1940 0
%A optional
%W 8
%F dd/mm/yy

%I 3
%T string SimpleString
%A optional

%I 4
%T choice SimpleChoice
%A optional
%L $(choices)
%l 0
%c 0

[a] This file is present for users and programs that require backwards compatibility and are not able to read the JSON study definition. This file may be removed in a future release.


Table 2.39. Schema codes and their meaning

CodeRelevanceMeaning
SStudyDFdiscover study number
aStudyproduction study number
bStudydevelopment study number
BStudyyear in which study database was defined
eStudyRun edit checks in View mode
NStudythe total number of user-defined plates in the setup (this excludes plate 0 (new records), plate 510 (reasons for data change), and plate 511 (queries). In DFschema.stl file this represents total number of styles defined in the study
UStudyDFdiscover version used to create setup
uStudyDFdiscover version used on last setup change
YStudyreason is required for data change for specified fields (0=per field), for all fields (2=always), or for no field (1=never). These values are followed by the value for the 'Only if changing a non-blank value' setting; 0=no, 1=yes.
ZStudyfield description maximum length (25 or 40)
MStudynumber of new patients to be listed in patient binder
mStudyenable Start New Subject dialog
PPlateplate number
nPlatetotal number of fields per plate; this includes the 6 DFdiscover default fields present on all records (i.e., DFstatus, DFcreate, DFmodify, etc.)
pPlateplate label
EPlateis the sequence number predefined (code 1) or is it the first data field (code 2)?
RPlateplate arrival trigger. This tag identifies an executable shell script or program, located in the study ecbin or DFdiscover ecbin directories, which is executed on plate arrival.
tPlatedoes plate trigger early termination; "simple" = plate does not trigger early termination, "term" = plate triggers early termination
IField/Stylethe field order number or the style index in the DFschema.stl file
iFieldthe number that uniquely identify fields within a study
VField/Stylealias
vField/Stylename
DField/Styledescription
gField/Stylethe minimum validation level after which any changes to the data value will require a reason. This code is optional, or may be present and have the value 0, in which case a reason for a data change is never required. If a minimum validation code between 1-7 is present, it will be followed by the value for the 'Only if changing a non-blank value' setting; 0=no, 1=yes.
hField/Stylefield visibility
LField/Stylelegal values; where $(ids) has been used as a legal range definition for patient id variables, the literal $(ids) will be reported.
AField/Stylefield optionality
FField/Styleformat
HField/Stylehelp message
WField/Stylemaximum number of stored characters
wField/Stylemaximum number of displayed characters
JField/Styleedit check(s) on field entry
KField/Styleedit check(s) on field exit
jField/Styleedit check(s) on plate entry
kField/Styleedit check(s) on plate exit
cField/Stylecode value and label definition for no choice
CField/Stylecode value and label definition
sField/Stylenumber of fields to skip and condition
TField/Stylea compound value containing, in order:
  • data type, one of string, int, date, choice, check, time

  • style name (which may contain spaces)

  • for dates only, the cutoff year

  • for dates only, the imputation method

  • for dates only, one of NonSched or VisitDate

rField/Stylefield's module name, instance number and description.
XField/Styleis field constant (0=no, 1=yes) and constant value (if constant)

Table 2.40. DFsetup - study definition

Usual NameDFsetup
TypeJSON
Created ByDFsetup
Used ByDFsetup, DFexplore, DFprintdb
Field DelimiterNA
Record DelimiterNA
Comment DelimiterNA
Fields/RecordNA
Description

The setup file contains the study definition in JSON format that can be read and written efficiently by DFsetup. It keeps the internal state of the program together with all of the information about study, page, and variable definitions. Many other study configuration files, such as the ICR tips file and study database schema, are constructed from this internal representation.

[Warning]Internal use only

This file format is intended to be written only by DFsetup only. It may be read and processed by other programs that need to determine setup information but its internal structure is subject to change without notice.


Table 2.41. DFserver.cf - server configuration

Usual NameDFserver.cf
Typeclear text
Created ByDFadmin
Used ByDFserver.rpc
Field Delimiter=
Record Delimiter\n
Comment Delimiter#
Fields/Record2
Description

Each study database server is configured by this file. The study server configuration keywords and their meanings are given in the Table 2.42, “DFserver.cf configuration keywords”.

Example
STUDY_NUMBER=254
STUDY_NAME=DFdiscover Acceptance Test Study
STUDY_DIR=/opt/studies/val254
PAGE_DIR=$(STUDY_DIR)/pages
WORKING_DIR=$(STUDY_DIR)/work
PRINTER=hp4000
THRESHOLD=500000
STUDY_URL=http://www.ourstudy.com/index.html
VERSION_STRICT=0
AUTO_LOGOUT=5|30
ADD_IDS=1||
VERIFY_IDS=0|

Table 2.42. DFserver.cf configuration keywords

KeywordMeaning
STUDY_NUMBERthe unique study number identifier. Legal values are in the range 1 to 999 inclusive.
STUDY_NAMEa descriptive name or acronym for the study. This name appears, among other places, in the toolbox frame label. Its recommended length should be no more than 20 characters.
STUDY_DIRthe study root (highest-level) directory. All other study related directories should be sub-directories of this directory and should accomplish this by referencing $(STUDY_DIR) in their value. This directory, by default, is writable by both user and group and is owned by datafax.
PAGE_DIRthe root directory for all CRF image files. This directory has sub-directories named by year and week within year. Within each of these sub-directories each CRF page image is numbered sequentially by page number within fax number. By default, this directory is defined as $(STUDY_DIR)/pages and is writable only by user datafax.
WORKING_DIRthe root directory for all study specific temporary or work files. Any temporary files created by report programs should be written in this directory. By default, this directory is defined as $(STUDY_DIR)/work and is writable only by user datafax and group.
PRINTER or PRINT_CMDdefault printer for the study.
THRESHOLDthe minimum number of Kilobytes of free disk space in the partition containing the study data directory. During normal operation, the study server will exit when the free disk space in this partition drops below the threshold value. Further connections to the study server will fail until additional disk space is made available or the threshold is reduced.
AUTO_LOGOUTa compound value expressed as two numbers delimited with a |. Both numbers represent a time interval expressed in minutes. The first number is the minimum number of minutes that any study user can choose for their auto logout interval; the second is the maximum number of minutes.
VERSION_STRICTa true (1) or false (0) value specifying whether the study server accepts client connections from the current minor release version only, or client connections with any minor release version. The major release version must always match.
STUDY_URLfor future use only, and is currently not used.
ADD_IDSfor future use only, and is currently not used.
VERIFY_IDSfor future use only, and is currently not used.

Table 2.43. DFtips - ICR tips

Usual NameDFtips
Typeclear text
Created ByDFsetup
Used ByDFinbound.rpc
Field Delimiter\n
Record Delimiter.\n
Comment Delimiter#
Fields/Record3+
Description

The purpose of this file is to document the location, size, and type of each data field on each study plate. This information is parsed and used by during its ICR processing of each CRF image file.

The tips file begins with 4 comment lines which identify the study number, name, number of CRF plates that have been defined and the date on which the tips file was created. This is followed by the keyword DELIMITER which identifies the single character delimiter used to separate fields in data records. The current implementation of DFdiscover accepts only the | delimiter.

The above header information is followed by N plate definitions, where N is the number of unique plates defined for the study. Each plate definition begins with plate specific information followed by M variable definitions, where M is the number of variables defined on that plate. Each variable definition includes: the data field number and unique name, the data type, legal values (if any were specified), the size of the boxes, or VAS line, used to record the data field, and its location on the CRF image that was used to define the plate in DFsetup.

Each field in the tips file has a leading keyword separated from the value by a \t (TAB) character.

The plate specific keywords are listed in Table 2.44, “DFtips plate specific keywords”. The keywords recognized for variable definitions are listed in Table 2.45, “DFtips variable specific keywords”.

Example
# Study Number: 253
# Study Name: Demo Study 253
# Total Pages: 14
# Created: Wed Dec  1 12:33:29 2004
DELIMITER       |
PAGE    1
PLATE   1       # Blood Pressure Screening Visits
SEQ_CODE        1
PREOP   echo "$image $data" >> /tmp/test
DO_ICR  2
# FAX: plt001
FIELD   1 # ID001
TYPE    int
LEGAL   $(ids)
PART    106 84 2 50 25
PART    167 84 3 74 25
.
FIELD   2 # PINIT001
TYPE    string  skip
PART    380 84 3 74 25
.
FIELD   3 # AGE
TYPE    int
LEGAL   21-90
PART    79 182 2 50 25
.
FIELD   4 # SEX
TYPE    choice
LEGAL   0,1,2, blank
PART    0 0 0 0 0 # blank
PART    277 189 1 1 15 # male
PART    277 212 1 2 15 # female
.
FIELD   5 # RACE
TYPE    choice
LEGAL   0-65535
PART    0 0 0 0 0 # no choice
PART    464 188 1 1 15 # Caucasian
PART    464 213 1 2 15 # African American
PART    464 238 1 3 15 # Asian
PART    464 263 1 4 15 # Other
.
FIELD   6 # RACEOTH
TYPE    string  skip
PART    588 257 1 107 26
.
FIELD   7 # S1DATE
TYPE    date
FORMAT  dd/mm/yy
DATES   1940 0
LEGAL   01/01/01-today
PART    172 330 2 49 25
PART    232 330 2 50 25
PART    295 330 2 49 25

Table 2.44. DFtips plate specific keywords

KeywordDescription
PAGEan internal page number that simply sequences the plate definitions
PLATEthe unique plate number that is being defined
SEQ_CODEa code to indicate whether the visit/sequence number is in the barcode or the first data field. Legal values are: 1=barcoded, 2=first data field.
PREOPa shell command that is to be executed before the ICR processing for any occurrences of this plate begins. This keyword is optional.
POSTOPa shell command that is to be executed after the ICR processing for any occurrences of this plate ends. This keyword is optional.
DO_ICRa code that indicates whether or not ICR processing should be performed on occurrences of this plate. Legal values are: 1=yes perform ICR, 2=no do not perform ICR.

Table 2.45. DFtips variable specific keywords

KeywordDescription
FIELDthe data field number as it is sequenced on the form. This will not be the same number as the field number that the data value is eventually stored in. The first data field is the visit/sequence number if SEQ_CODE=2; otherwise, the first data field is the subject ID. Hence to determine the actual field number in the database record, add 5 if SEQ_CODE=2; and add 6 if SEQ_CODE=1.
TYPEdata field type. Legal values are: int=integer, string=text fields, date=date, choice=choice, check=check, vas=visual analog scale, and numeric=a rectangular box containing 2 or more digits. The numeric data type is internal to the tips file and is not used anywhere else in DFdiscover. They are defined in DFsetup as strings with pre-printed numerals, although they can also be used for hand-written numbers. The type record ends in the keyword 'skip' if the field is not to be ICRed. Strings and hidden fields (i.e. fields which do not appear on the paper CRF) will be automatically marked as ICR skips by DFsetup.
FORMATthe format, if any, that is to be applied to the ICRed value before it is added to the database record. Typical values include: mm/dd/yy (which inserts the '/' delimiters into dates), nn.n (which inserts a decimal point).
DATESthe pivot year and imputation method to be used for interpreting 2-digit years (applicable only if the current field is of type date). The imputation method is one of:
0 never - no imputation is allowed
1 start - impute dates to the beginning of the month or year
2 middle - impute dates to the middle of the month or year
3 end - impute dates to the end of the month or year
LEGALthe legal value definition for the field. This is the same definition defined in the study schema. If an ICRed value is illegal, the value for that field is left blank in the data record generated by the ICR software.
PART defines the location, size and coding for each part of the data field. String, check, numeric and VAS fields consist of just one part but choice fields have a part definition for each choice box and dates and ints may be comprised of more than one separate parts. Each part record consists of 5 or 6 space delimited components. For all field types the first 2 components are the x and y location of the field on the CRF plate, with x increasing from left to right and y increasing from top to bottom. The location is given in units corresponding to standard fax resolution (i.e. 102 units per inch horizontally and 98 units per inch vertically). For each part the x value is 8 units left of the first box or VAS line, and the y value gives the location of the middle of the box or VAS line. The meaning of the remaining components depends on the field type. For int, date, and string fields they are: the number of boxes in the part, the total length of the part (enclosing all boxes) and the height of the part. For numeric fields they are: the maximum number of digits in the data field, the total length of the rectangular box and height of the box. For VAS scales they are: the length of the scale, the minimum and maximum values at the ends of the scale, and the precision (i.e. number of decimal places) in the stored value. For check fields they are: the number of boxes (always 1), the code value to be stored in the database if the box is not check, the code if checked, and the edge dimension of the box. And for choice fields they are: the number of boxes (always 1), the code value to be stored in the database if the box is checked, and the edge dimension of the box. For check and choice fields each box is assumed to be square. Choice fields also have a special PART record in which all components are zero expect for the 4th one which specifies the code value stored in the database if none of the choice boxes are checked.

Table 2.46. DFvisit_map - visit map

Usual NameDFvisit_map
Typeclear text
Created ByDFsetup
Used ByDF_QCupdate, DF_ICvisitmap, DF_SSvisitmap
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record12
Description

The visit map file describes the subject assessments to be completed during the study, the timing of these assessments, and the pages of the study CRFs which must be completed at each assessment.

Each assessment is identified by a visit type. There must always be a baseline visit which is typically the date on which the subject qualified for entry to the trial and was randomized. There must also be a termination visit which ends study follow-up. Between baseline and termination there are often several scheduled visits, subject diaries, laboratory tests, and perhaps a few unscheduled visits. At each of these visits there will be a set of required (and possibly optional) forms to be completed.

Each visit is defined in a single record of the visit map. The fields in each record are described in Table 2.47, “DFvisit_map field descriptions”.

For additional information regarding visit maps, refer to Standard Reports Guide, DF_QCupdate and Study Setup User Guide, Visit Map.

Example

A simple 4-visit visitmap:

0|B|Baseline|1|9 (mm/dd/yy)|0|0| 1 2 3 4 5 6 7 8||99|3 4 1 2 5 6 8 7
10|S|One Week Followup|9|9 (mm/dd/yy)|7|0| 9 10 14||||
20|S|Two Week Followup|9|9 (mm/dd/yy)|14|0| 9 10||||
30|T|Termination Visit|9|9 (mm/dd/yy)|21|0| 11 12||||


Table 2.47. DFvisit_map field descriptions

Field #ContainsDescription
1visit numberthe unique visit number, which can be any number between 0 and 65535 inclusive. For all scheduled visits, (types P, B, S, T), the numerical value of visit numbers must correspond to the sequential ordering of the visits in time.
2visit typea 1 character code for the type of visit. Legal values are enumerated in Table 2.48, “Visit codes and their meaning”.
3visit labela short (maximum 40 character) textual description of the visit that will be used in quality control reports to identify the visit when it is overdue.
4visit date platethe plate on which the visit date can always be found. This must be one of the required plates listed in field 8. Obviously other plates will have visit dates, however, this is the one that will be used when visit dates (potentially conflicting) appear on several pages of the same visit.
5visit date field & formatthe data field number of the visit date on the plate identified in field 4 and its format. The allowable date formats include any combination of yy (year), mm (month), and dd (day) so long as each occurs exactly once. Delimiter characters are optional between the 3 parts. Note: this date field must be defined using the VisitDate attribute.
6visit due daythe number of days before/after the baseline visit that the visit is scheduled. The baseline visit must have a value of 0, and pre-baseline visits must have negative values.
7visit overdue allowancethe number of days that a scheduled visit is allowed to be late. Visits are considered overdue if they have not arrived within this number of days following the visit due day.
8required platesa space character delimited list of plate numbers for CRFs that are required to be completed on this visit (maximum 1200 characters).
9optional platesa space character delimited list of plate numbers for CRFs that may be completed on this visit, but are not required (maximum 1200 characters).
10missed visit notification platea plate number which if received, indicates that the visit number coded on that plate was missed, and hence that Query Reports should not complain that this visit is overdue, or that it has missing pages.
11termination windowfor visit type W, a termination window is required and may be one of the following forms:
on yy/mm/dd
before yy/mm/dd
after yy/mm/dd
between yy/mm/dd-yy/mm/dd fraction
In each case, the date value must use the format that is defined as the VisitDate attribute's format (and is also recorded in field 5).
12display ordera comma- or space-delimited range list of required and optional plate numbers. The order specified here determines the order in which pages within a given visit are to be displayed in the DFexplore subject binder.

Table 2.48. Visit codes and their meaning

CodeMeaningScheduledWhen Required
Ccycleno|yescycles are used to group visits. For additional information regarding cycle entries, refer to Study Setup User Guide, Cycle Specifications.
Xscreeningnoif subject enters the trial (baseline arrives)
Pscheduled pre-baseline visityesbefore arrival of baseline visit
Bbaselineyescan be scheduled from a pre-baseline visit
Sscheduled follow-upyesscheduled from the baseline visit
Ooptional follow-upnonot required
rrequired by time of next visitnobefore arrival of the next visit
Tcycle termination visityesscheduled from the baseline visit
Rrequired by time of termination visityeson termination if scheduled pre-termination
Eearly termination of current cyclenoif early termination event occurs
Aabort all cyclesnoif abort event occurs
Ffinal visit (terminates all cycles)noterminate multi-cycle visit maps
Wstudy termination windowyesabsolute date scheduling of final visit

Table 2.49. DFwatermark - watermark

Usual NameDFwatermark
Typeclear text
Created ByDFadmin
Used ByDFexplore, DFpdf, DFpdfpkg
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record9
Description

The DFwatermark file stores a description of the watermarks available for use in a study. The file contains zero or more records. The fields in each record are described in Table 2.50, “DFwatermark field descriptions”.

In determining which watermark to use, the records are searched in sequential order from the beginning of the file. The first record to match, based upon the current user's role and the role(s) in the record, is selected.

Example

draft|Draft watermark|DRAFT|20|255,0,0|50|1|1|unrestricted,Sites


Table 2.50. DFwatermark field descriptions

Field #ContainsDescription
1nameThe unique name for the watermark.
2description A description of how and why the watermark is used for this role.
3watermark text The text of the watermark.
4size The point size of the font.
5color The color of the watermark in RGB space in the format 0-255,0-255,0-255.
6transparency The transparency of the watermark as a percentage from 0 (opaque) to 100 (transparent).
7position The placement of the watermark on the page - 1=top, 2=center or 3=bottom.
8frame A flag to indicate if the watermark is framed (1) or not (0).
9rolesA comma delimited list of roles that may use this watermark.

Table 2.51. QCcovers - Query cover pages

Usual NameQCcovers
Typeclear text
Created Bytext editor
Used ByDF_QCreports
Field DelimiterNA
Record DelimiterNA
Comment Delimiter#
Fields/RecordNA
Description

External Quality Control report cover sheets can be included by defining them in the file QCcovers. They may then be requested by running DF_QCreports with no -i option or running DF_QCreports with -i c. If no options are specified with DF_QCreports, a full 3-part Query Report and cover sheet will be generated. If -i is used, you must also specify other report parts to be generated (e.g. s, r, q, c). The -i c option may be specified with the site selection option, -c #. This allows the user to generate cover sheets only for those site IDs specified. The -i c option by itself will generate a cover sheet and nothing else for all sites.

A cover sheet begins with a

<FOR center="#list">

tag to identify the site(s) that are to receive the cover sheet. It is legal for some sites to receive cover sheets while others do not, and for different sites to receive different cover sheets.

Cover sheets are defined using a <TEXT> block. Within <TEXT> blocks, variable substitution may be performed as described in QCtitles for customized report titles. Aside from variable substitution, all other text will appear exactly as formatted within the text block. Blank lines used to double space text will also be preserved. The default font for cover sheet text is the constant width font, fCW.

If more than one cover sheet is defined and a site ID is included in the <FOR center="#list"> for more than one cover sheet, all <TEXT> blocks from all covers sheets specified for the site ID will be added to that site's cover sheet. The <TEXT> blocks will appear in the order they appear in QCcovers, and the site will still only receive one cover sheet.

Sites not specified in either QCcovers or QCmessages do not get a cover sheet. If a site is not defined in QCcovers, but that site is named in QCmessages, a blank cover sheet is generated onto which the message(s) can be written.

[Note]Note

DF_QCreports does not perform line wrapping on cover sheets. Each text block is printed exactly as it appears in QCcovers and QCmessages. Care must be taken when choosing fonts and using variable substitutions to ensure that text lines do not exceed the width of the page.

DF_QCreports also expects that the cover sheet and all messages will fit on a single cover page for each site. It will not create automatic page breaks.

DFdiscover version 3.7-003 and earlier requires that font specifications in QCcovers include the control-P (^P) character; for example <TEXT font="^P12 fB">. DFdiscover version 3.7-004 and greater accept the ^P but it is no longer required. For example, <TEXT font="12 fB"> is a valid font specification and may be defined using the Query Covers view in DFsetup.

Example

The following is an example of an English and French version of a cover sheet for an international trial, where sites 20 to 29 inclusive use French, and all other sites use English.

<FOR centers="1~19,30~300">
<TEXT font="^P12 fB">
--------------------------------------------------------------
PLEASE DELIVER THIS FAX TO THE FITNESS STUDY COORDINATOR
TO: <WHO>
Site <CENTER>: <WHERE>
<DATE>
</TEXT>
</FOR>
<FOR centers="20~29">
<TEXT font="^P12 fB">
--------------------------------------------------------------
DONNEZ CE RAPPORT AU COORDINATOR DE RECHERCHE POUR FITNESS, S'IL VOUS PLAIT
TO: <WHO>
Site <CENTER>: <WHERE>
<DATE>
</TEXT>
</FOR>


Table 2.52. QCmessages - Query Report messages

Usual NameQCmessages
Typeclear text
Created Bytext editor
Used ByDF_QCreports
Field DelimiterNA
Record DelimiterNA
Comment Delimiter#
Fields/RecordNA
Description

Cover sheets may include messages for all or specified site IDs. Messages are stored in the file QCmessages and follow the same rules as described for QCcovers. While messages can be added to cover sheets by placing them in <TEXT> blocks in the QCcovers file, the use of a separate file for messages makes it easier to keep the fixed (probably unchanging) cover sheet header separate from the broadcast messages, which will change frequently throughout the trial.

The file QCmessages may include more than one message, and each site may receive none, some or all of them, as indicated by <FOR centers="#list"> which must appear at the beginning of each message. If a site is to receive more than one message, the messages will be printed on the cover sheet in the order in which they are defined in QCmessages.

It is unnecessary to define a cover sheet in order to use messages. If a message is specified for a site which has not been defined in QCcovers, a blank cover sheet will be created, onto which the message(s) will be printed. Cover sheets and messages must be requested by running DF_QCreports with the -i c option or by running DF_QCreports with no -i at all.

[Note]Note

DF_QCreports does not perform line wrapping on cover sheets or messages. Each text block is printed exactly as it appears in QCmessages. Care must be taken when choosing fonts and using variable substitution to ensure that text lines do not exceed the width of the page.

DF_QCreports also expects that the cover sheet and all messages will fit on a single cover page for each site. It will not create automatic page breaks.

Messages will continue to go on each report until the QCmessages file is removed. If you want to keep a message for use later on, but do not want to send it to anyone in the next batch of Query Reports, leave the site number string blank. e.g. use <FOR center="">.

DFdiscover version 3.7-003 and earlier requires that font specifications in QCmessages include the control-P (^P) character; for example <TEXT font="^P12 fB">. DFdiscover version 3.7-004 and greater accept the ^P but it is no longer required. For example, <TEXT font="12 fB"> is a valid font specification and may be defined using the Query Messages view in DFsetup.

Example

In the following example all site IDs (1 to 300 inclusive) receive the announcement of an upcoming meeting. For site IDs 1, 4, 22 to 24, and 127, this is followed by an important message regarding their lack of response to data queries.

<FOR center="1~300">
<TEXT font ="^P12 fB">
--------------------------------------------------------------
PLEASE NOTE:
</TEXT>
<TEXT font="^P10 fCW">
The next FITNESS Study investigator's meeting will be held on October 9-10,
1999 in Orlando, Florida.
Location and times will follow at a later date.
</TEXT>
</FOR>
<FOR center="1,4,22~24,127">
<TEXT font="^P12 fB">
--------------------------------------------------------------
IMPORTANT!
</TEXT>
<TEXT font="^P10 fCW">
We have not received any corrections related to the Quality Control reports we
have sent to your site over the last 2 months. Please make every attempt to
resolve the outstanding data queries as soon as possible.
By protocol and agreement with the FDA, all adverse events must be reviewed by
us within 3 days of each subject assessment.
If you are having problems of any kind, please let us know.
</TEXT>
</FOR>


Table 2.53. QCtitles - Query Report titles

Usual NameQCtitles
Typeclear text
Created Bytext editor
Used ByDF_QCreports
Field DelimiterNA
Record DelimiterNA
Comment Delimiter#
Fields/RecordNA
Description

The external or internal report title and sub-titles for each of the 3 sections (Subject Status, Refax List, Q & A List) of a DFdiscover Quality Control report may be specified in this file. DF_QCreports checks for the existence of this file and will use it if it exists, otherwise standard titles will be produced.

Title specifications must be formatted exactly as shown in the examples to follow. The opening and closing tags must appear on new lines by themselves, with no leading or trailing text outside of the tag. Anything entered outside of a tagged block is ignored. The # symbol may be used to indicate a comment line but the # is not strictly needed, and has no special meaning inside a tagged block. For example, it is legal to include a line of # as part of a title inside a tagged block.

Tags are only recognized if they begin a new line. Nothing is to precede the tag symbol '<'. No space is allowed within the opening and closing tag except before the word "font". The "font" value must be enclosed in double quotes.

Limited font specifications may be included within the opening tag as listed in Table 2.54, “QCtitles font specifications”.

The font specification is optional. By default, font ^P10 fCW is used for external and internal report titles, and font ^P10 fB is used for the 3 report section sub-titles. Note that in DFdiscover versions 3.7-003 and earlier, font specifications must include the control-P (^P) character; for example <EXTERNAL font="^P12 fB">. DFdiscover versions 3.7-004 and greater no longer require the ^P specification, but it is accepted if present. For example <EXTERNAL font="12 fB"> is a valid font specification.

The variables enumerated in Table 2.55, “Variables available for use in QCcovers, QCmessages, and QCtitles” may be included in the external and internal report titles which appear at the top of each page, but not in the sub-titles for the 3 parts of the report (status, refax and question & answer lists).

[Note]Note

DF_QCreports does not perform line wrapping on report titles. Each report title and section sub-title is printed exactly as it appears in QCtitles. Care must be taken when choosing fonts and using variable substitutions to ensure that text lines do not exceed the width of the page.

Example

Here is an example of a QCtitles file that contains formatting information for an external and internal report title and the 3 section sub-titles.

# TITLE AT THE TOP OF EACH PAGE OF AN EXTERNAL QUERY REPORT
<EXTERNAL font="^P12 fB">
FITNESS STUDY REPORT <NUM> for: <MON> <DAY>, <YEAR> at
<TIME>
TO: <WHO>, <WHERE>
</EXTERNAL>
# TITLE AT THE TOP OF EACH PAGE OF AN INTERNAL QUERY REPORT
<INTERNAL font="^P12 fB">
INTERNAL QC REPORT: <NAME> page <PAGE> created:
<YEAR>-<MON>-<DAY>
</INTERNAL>
#SUB-TITLE ABOVE THE SUBJECT STATUS LIST (Part 1 of a standard external Query Report)
<STATUSLIST font="^P10 fB">
SUBJECT VISIT SCHEDULE: Note: *identifies subjects with data queries.
</STATUSLIST>
#SUB-TITLE ABOVE REFAX LIST (Part 2 of a standard external Query Report)
<REFAXLIST font="^P10 fB">
CRF CORRECTIONS: Initial and date each correction. Refax all corrected pages without delay.
</REFAXLIST>
# SUB-TITLE ABOVE THE QUESTION AND ANSWER LIST (Part 3 of a standard external Query Report)
<QALIST font="^P10 fB">
QUERIES: Print your response below each question and fax back this page.
</QALIST>


Table 2.54. QCtitles font specifications

SymbolDescription
^Pcontrol P, not circumflex P, sets the point size (^P10 and ^P12 are recommended).
fBbold
fHHelvetica
fCWconstant width (monospace) font

Table 2.55. Variables available for use in QCcovers, QCmessages, and QCtitles

VariableUse in ExternalUse in InternalMeaning
<STUDYNAME>yesyesstudy name
<SITE> [a]yesnosite ID
<WHO>yesnoprimary site contact
<WHERE>yesnosite affiliation
<MAIL>yesnosite mailing address
<PRIMEFAX>
<FAX1>
yesnoprimary fax number (to which Query Reports are sent)
<FAX2>yesnosecondary fax number
<PHONE1>yesnoprimary contact's phone number
<PI>yesnoprincipal investigator
<PHONE2>yesnoprincipal investigator's phone number
<REPLYTO>yesnoemail address to which replies made to emailed Query Reports will be sent
<NUM>yesno external report name composed of site ID, date (yymmdd), and page, e.g. 025-080115-01.
<NAME>yesyes external report name composed of site ID and date only, e.g. 025-080115
<PAGE>yesyespage number of Query Report
<DAY>yesyestwo-digit day of month
<MON>yesyesthree-character month of year
<YEAR>yesyesfour-digit year
<WKDAY>yesyesthree-character day of week
<TIME>yesyestime of day (hh:mm:ss AM/PM), e.g. 01:12:22 PM
<DATE>yesyesdate (ddd mmm dd hh:mm:ss yyyy), e.g. Tue Jan 15 14:34:07 2018

[a] The variable <CENTER> is also supported for backwards compatibility


Table 2.56. DFreportstyle - DFdiscover Report styling

Usual NameDFreportstyle
Typeclear text
Created Bytext editor
Used ByDFdiscover reports, excluding Legacy reports
Field DelimiterNA
Record DelimiterNA
Comment DelimiterNA
Fields/RecordNA
Description

The file contains two optional sections, a script section and a style section. Both sections are optional. The script section contains an array, in JSON array notation, of colors for use in chart rendering. These values define the Palette picker. The style section contains CSS that is added to the head of the report output. This CSS can style the text and general appearance of reports.

Example

<script type="application/json" id="dfreportjson">{
"palettes": {
    "Corporate": ["#26456e", "#3a87b7", "#b4d4da", "#1c5998", "#67add4", "#1c73b1", "#7bc8e2"],
    "Diverging" : ["#000066", "#6699ff", "#ffff00", "#3333ff", "#99ccff", "#ffff99", "#cc9900",
    "# 3366ff", "#ccccff", "#ffff66", "#663300"],
    "Monochrome": ["#1a1a1a", "#666666", "#b3b3b3", "#333333", "#808080", "#cccccc", "#4d4d4d",
    "#999999", "#e6e6e6"],
    "Color blind": ["#006ba4", "#ff800e", "#595959", "#5f9ed1", "#c85200", "#898989", "#a2c8ec",
    "#ffbc79", "#cfcfcf"]
}
}</script>
<style id="dfreportcss">
/* Body font */
body {
    font: 100% arial, sans-serif;
}
/* Title font */
.contextText.reportNameText {
    font: italic bold 1.5em Helvetica, serif;
}
/* Heading font */
.c3-title {
    font: 1.2em Helvetica, serif;
}
/* Axis label font */
.c3-axis-y-label, .c3-axis-x-label {
    font: normal 1.2em Helvetica, sans-serif;
}
</style>


2.9.  The study lut directory

This directory contains all the lookup tables used in the study.

Table 2.57. Lookup tables

Usual NameNone
Typeclear text
Created Bytext editor
Used ByDFexplore, DFbatch
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record1+
Description

A lookup table is a simple ASCII file. If it is anything other than this, an error message will appear on the command-line upon starting DFbatch.

Each lookup table file must be associated with a symbolic table name in the lookup table map in order to be used by the dflookup function or in DFexplore. The lookup table map is described in DFlut_map - lookup table map.

Within a lookup table, each record has at least 1 field. The first field is a short acronym (the search field), and the subsequent fields are the lookup text. If the record has only one field, it is the search and the result field (this is useful for spell checking). If the record has 2 fields, the first field is the search key and the second is the return value.

An example query lookup table with two fields is illustrated below.


miss|Please provide a response.
dtformat|Date format is YYYY/MM/DD. Please review and correct this date.
dtillegal|Date provided is illegal. Legal dates are 2016/01/01-today.
dtbeforeic|Date provided is before date of informed consent.
ongo|End date is provided, but Ongoing is checked. 
enddt|End date is prior to Start date. 

Records may also have 3+ fields (e.g COSTART tables, MedDRA), where the first field is the search key and all other fields are returned as a single delimited string that can be parsed using the dfgetfield function.

Within the lookup table structure, the search field is typically short but has no maximum length. The lookup text fields have no maximum length; however, they are truncated to the maximum length of the data field into which they are inserted.

Multi-field lookup tables can also support a more flexible structure as illustrated below.


#N:LL Term|Pref Term|SOC Term|LL Code
#L:Low Level Term|Preferred Term|System Organ Class|Low Level Code
#F:1|1-4
Abdomen enlarged|Abdominal distension|Gastrointestinal
disorders|10000045
Abdomen mimicking acute|Acute abdomen|Gastrointestinal
disorders|10000047
Abdominal cramp|Abdominal pain|Gastrointestinal disorders|10000056

These lookup tables contain 3 header rows which define the structure of the table.

  • The first row begins with '#N:' followed by a short column name for each field, which appears above the scrolling list of entries in the lookup dialog displayed by an edit check.

  • The second row begins with '#L:' followed by a longer descriptive label for each field. This appears in the top section of the lookup dialog where the field values for the current row are displayed, and where the user can indicate which fields to search.

  • The third row begins with "#F:" followed by 2 fields: the 1st lists the fields to be searched by dflookup() for a match and the 2nd lists the fields to be returned when a match is found or selected by the user. If multiple return fields are specified they are returned to the edit check as a '|' delimited string, which can be parsed using dfgetfield().

If a lookup table is defined for a study data field, the defined lookup text can be retrieved in 2 ways. If the user remembers the acronym (first field) the lookup value can be entered by typing the acronym in the current data field and pressing Enter. The acronym is then replaced with the value from the matching lookup table record. Matching on the acronym is performed case-insensitive. Alternatively the user can simply press Enter without giving an acronym, to pop-up a scrolling list of all acronyms and their lookup text, and then click on the desired choice.

Example
a|before
AA|auto accident (MVA)
A&P|anterior & posterior
abd|abdomen
abg|arterial blood gases
ac|before meals
acid phos|acid phosphatase
ACLF|adult congregate living facility
ACTH|adrenocorticotrophic hormone
acute MI|acute myocardial infarction
ad|right ear
Ad|up to
Ad lib|as desired
BaE|barium enema
Ba|barium
BB|blood bank
BCP|biochemical profile
BE|barium enema
BEE|basal energy expenditure

2.10.  The study work directory

This directory contains summary files created by DF_QCupdate, DFdiscover retrieval files, and temporary files created during the execution of reports and other programs.

Table 2.58. DFX_ccycle - cycle conditions met

Usual NameDFX_ccycle
Typeclear text
Created ByDF_QCupdate
Used ByDF_PTvisits
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record4+
Description

This file records the conditions defined in the conditional cycle map that were met for each subject. It is updated each time DF_QCupdate is run.

Table 2.59, “DFX_ccycle conditional cycle map conditions that were met” describes the data recorded for each of these conditions.

Example

The following example shows 3 conditions (1,4 and 6), that were met for subject 11020. These conditions were met at visits 10,5 and 60 respectively, with data values 2, 98 and 0 respectively. To determine the actions triggered by these conditions we would need to consult the conditional cycle map to determine which cycles are affected by each condition.

11020|10|1|2
11020|5|4|98
11020|60|6|0


Table 2.59. DFX_ccycle conditional cycle map conditions that were met

Field #ContainsDescription
1Subject IDsubject ID number
2Visit Numbervisit or sequence number at which the condition was met, taken from the IF statement if the condition includes both IF and AND statements
3Condition Numbercondition number, from the order in which conditions are defined in the conditional cycle map
4+data value(s)the database value(s) that resulted in the condition being met, for each IF and AND statements defined in the condition (in statement order)

Table 2.60. DFX_cvisit - visit conditions met

Usual NameDFX_cvisit
Typeclear text
Created ByDF_QCupdate
Used ByDF_PTvisits
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record4+
Description

This file records the conditions defined in the conditional visit map that were met for each subject. It is updated each time DF_QCupdate is run.

Table 2.61, “DFX_cvisit conditional visit map conditions that were met” describes the data recorded for each of these conditions.

Example

The following example shows 3 conditions (1,4 and 6), that were met for subject 11020. These conditions were met at visits 10,5 and 60 respectively, with data values 2, 98 and 0 respectively. To determine the actions triggered by these conditions we would need to consult the conditional visit map to determine which visits are affected by each condition.

11020|10|1|2
11020|5|4|98
11020|60|6|0


Table 2.61. DFX_cvisit conditional visit map conditions that were met

Field #ContainsDescription
1Subject IDsubject ID number
2Visit Numbervisit or sequence number at which the condition was met, taken from the IF statement if the condition includes both IF and AND statements
3Condition Numbercondition number, from the order in which conditions are defined in the conditional visit map
4+data value(s)the database value(s) that resulted in the condition being met, for each IF and AND statements defined in the condition (in statement order)

Table 2.62. DFX_cplate - plate conditions met

Usual NameDFX_cplate
Typeclear text
Created ByDF_QCupdate
Used ByDF_PTvisits
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record4+
Description

This file records the conditions defined in the conditional plate map that were met for each subject. It is updated each time DF_QCupdate is run.

Table 2.63, “DFX_cplate conditional plate map conditions that were met” describes the data recorded for each of these conditions.

Example

The following example shows 3 conditions (1,4 and 6), that were met for subject 11020. These conditions were met at visits 10,5 and 60 respectively, with data values 2, 98 and 0 respectively. To determine the actions triggered by these conditions we would need to consult the conditional plate map to determine which plates are affected by each condition.

11020|10|1|2
11020|5|4|98
11020|60|6|0


Table 2.63. DFX_cplate conditional plate map conditions that were met

Field #ContainsDescription
1Subject IDsubject ID number
2Visit Numbervisit or sequence number at which the condition was met, taken from the IF statement if the condition includes both IF and AND statements
3Condition Numbercondition number, from the order in which conditions are defined in the conditional plate map
4+data value(s)the database value(s) that resulted in the condition being met, for each IF and AND statements defined in the condition (in statement order)

Table 2.64. DFX_cterm - termination conditions met

Usual NameDFX_cterm
Typeclear text
Created ByDF_QCupdate
Used ByDF_PTvisits
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record4+
Description

This file records the conditions defined in the conditional termination map that were met for each subject. It is updated each time DF_QCupdate is run.

Table 2.65, “DFX_cterm conditional termination map conditions that were met” describes the data recorded for each of these conditions.

Example

The following example shows 3 conditions (1,4 and 6), that were met for subject 11020. These conditions were met at visits 10,5 and 60 respectively, with data values 2, 98 and 0 respectively. To determine the actions triggered by these conditions we would need to consult the conditional termination map to determine which cycles are affected by each condition.

11020|10|1|2
11020|5|4|98
11020|60|6|0


Table 2.65. DFX_cterm conditional termination map conditions that were met

Field #ContainsDescription
1Subject IDsubject ID number
2Visit Numbervisit or sequence number at which the condition was met, taken from the IF statement if the condition includes both IF and AND statements
3Condition Numbercondition number, from the order in which conditions are defined in the conditional termination map
4+data value(s)the database value(s) that resulted in the condition being met, for each IF and AND statements defined in the condition (in statement order)

Table 2.66. DFX_keys - key fields for all required plates

Usual NameDFX_keys
Typeclear text
Created ByDF_XXkeys
Used ByDF_CTvisits, DF_PTcrfs, DF_QCupdate
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record5
Description

This file records the key fields from all required plates found in the database. It is updated each time DF_XXkeys is run.

Table 2.67, “DFX_keys key fields from all required plates” describes the data recorded for each of these conditions.

Example

The following example shows 4 records for subject 1044, for plates 3,2,1 and 2 respectively at visits 51,61,92 and 211 respectively. All plates have status 1=final and are at validation level 7.

1044|3|51|1|7
1044|2|61|1|7
1044|1|92|1|7
1044|2|211|1|7


Table 2.67. DFX_keys key fields from all required plates

Field #ContainsDescription
1Subject IDsubject ID number
2Plate numberCRF plate number
3Visit Numbervisit or sequence number
4Status1-7
5Validation level1-7

Table 2.68.  DFX_schedule - subject scheduling and visit status

Usual NameDFX_schedule
Typeclear text
Created ByDF_QCupdate
Used ByDF_QCreports, DF_PTmissing, DF_PTvisits
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record16 for cycle records, and 18 for visit records
Description

This file records the current scheduling and status of all cycles and visits defined in the study visit map, for each subject. It is updated each time DF_QCupdate is run.

Table 2.69, “field definitions for cycle records in DFX_schedule describes each field in the cycle records, and Table 2.70, “field definitions for visit records in DFX_schedule describes each field in the visit records.

Example

The following example shows the current status for one subject (id=1031) in a study having 3 cycle. The screening cycle consists of visits 91 and 92. The in-study cycle is required and has 8 visits beginning with pre-baseline visit 51 and ending with early termination visit 81. The end cycle contains a single abort visit (visit number 80).

1031|1|0|C|S|1|7||||||03/09/21|37884||
1031|1|0|91|X|1|7|||||||03/09/21|37884|||
1031|1|0|92|X|1|1|||||||03/09/28|37891|||
1031|1|1|C|R|1|7||||||~03/10/07|37900||
1031|1|1|51|P|1|7|||||||03/10/05|37898|03/10/05|37898|
1031|1|1|0|B|4|0|-1|10|51||||||||
1031|1|1|61|r|1|1|||||||04/01/06|37991|||
1031|1|1|3|S|2|1|||||||04/01/06|37991|||
1031|1|1|6|S|1|0|||||||04/04/07|38083|||
1031|1|1|9|S|1|0|||||||04/07/07|38174|||
1031|1|1|12|T|1|0|||||||04/10/06|38265|||
1031|1|1|81|E|3|0|||||||||||
1031|1|2|C|E|0|0||||||||||
1031|1|2|80|A|3|0|||||||||||


Table 2.69. field definitions for cycle records in DFX_schedule

Field #ContainsDescription
1Subject IDsubject ID number
2Site IDclinical site number
3Cycle Numbercycles are numbered sequentially within the visit map, using 0 for screening, if present, and starting at 1 for the first in-study cycle.
4Cthe capital letter C appears in this field to distinguish cycle records from visit records
5Cycle Typecycles are defined in the visit map as one of: S=screening, R=in-study required, O=in-study optional, C=in-study conditional, and E=end
6Cycle Needcycle need, after applying all conditional and termination rules, can be one of: 0=unknown, 1=required, 2=next required cycle, 3=optional, and 4=excluded
7Cycle Statusthe current status of the cycle (as of the date on which DF_QCupdate was last run) can be one of: 0=not done, 1=overdue, 7=done, 8=cycle has terminated, 9=cycle was aborted
8Condition needcycle need set by a condition in the conditional cycle map; one of: 0=optional, 1=required, -1=excluded
9Condition numberthe number of the condition as defined within the conditional cycle map
10Condition sequence numbervisit or sequence number at which the condition was met (from IF statement)
11Start datescheduled start date for the cycle
12Start dayscheduled start day (days since Jan 1,1900) for the cycle
13Baseline datebaseline visit date for the cycle
14Baseline daybaseline visit day (days since Jan 1,1900) for the cycle
15Termination datetermination date for the cycle
16Termination daytermination day (days since Jan 1,1900) for the cycle

Table 2.70. field definitions for visit records in DFX_schedule

Field #ContainsDescription
1Subject IDsubject ID number
2Site IDclinical site number
3Cycle Numbercycles are numbered sequentially within the visit map, using 0 for screening, if present, and starting at 1 for the first in-study cycle.
4Visit/Sequence Numbera unique subject assessment number in the range 0-65535
5Visit Typeone of the 12 supported visit types: X=screening, P=pre-baseline, B=baseline, S=scheduled follow-up, T=cycle termination, W=cycle termination windows, F=final, O=optional, E=early cycle termination, A=abort all follow-up, R=required on termination, and r=required on next scheduled visit
6Visit Needvisit need, after applying all conditional and termination rules, can be one of: 0=unknown, 1=required, 2=next required visit, 3=optional, and 4=excluded
7Visit Statusthe current status of the visit (as of the date on which DF_QCupdate was last run) can be one of: 0=not done, 1=overdue, 2=recorded as missed, 7=done, 8=done and cycle terminates, 9=done and aborts all follow-up
8Condition needvisit need set by a condition in the conditional visit map; one of: 0=optional, 1=required, -1=excluded
9Condition numberthe number of the last condition met within the conditional visit map
10Condition sequence numbervisit or sequence number at which the condition was met (from IF statement)
11Missed Visit Plateplate number of missed visit plate received for this visit
12Early Termination Plateplate number of early cycle termination plate received for this visit
13Conditional Termination Numberthe number of the last condition met in the conditional termination map
14Start datescheduled due date
15Start dayscheduled due day (days since Jan 1,1900)
16Visit dateactual date on which the visit was performed
17Visit dayactual day (days since Jan 1,1900) on which the visit was performed
18Days Post-Terminationa positive number indicates that the visit was performed this many days following termination

Table 2.71. DFX_time1 - date and time from receipt to last modification

Usual NameDFX_time1
Typeclear text
Created ByDF_XXtime
Used ByDF_CTcrfs, DF_PTqcs, DF_WFfaxes
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record21
Description

This file records the date and time each CRF page was processed from fax arrival to last record modification. It is updated each time DF_XXtime is run. A related file, named DFX_dfin, contains the same information as DFX_time1, for records that are still in the DFdiscover new queue. These records have not been validated and entered into the study database and thus may contain ICR errors in the key fields.

Table 2.72, “DFX_time1 timing data for each CRF page” describes the data recorded for each of these conditions.

Example

The following example shows 4 records for subject ID 1501. The first 2 records are for plate 1 at visit 0, and the second 2 records are for plate 2 at visit 1. In both cases the first record is the primary and the second is a secondary record. In all 4 records the fax arrival date is missing. This will be the case if the record was not submitted by fax or email, but instead entered using raw data entry or ASCII record import, or if the fax_log is missing or has been trimmed.

1|1501|0|1|1|6|1995160154|03/30/95||95/04/24|95/04/24|34787|0|34812|34812||09:43:37|09:43:37|0|583.61|583.61
1|1501|0|1|4|4|1995130086|03/30/95||95/03/30|95/04/24|34787|0|34787|34812||18:41:04|09:43:43|0|1121.07|583.71
1|1501|1|2|1|6|1995200106|04/12/95||95/05/19|95/05/23|34800|0|34837|34841||16:46:37|11:12:18|0|1006.62|672.3
1|1501|1|2|5|5|1995160154|04/12/95||95/04/24|95/05/19|34800|0|34812|34837||09:43:59|16:46:39|0|583.98|1006.65


Table 2.72. DFX_time1 timing data for each CRF page

Field #ContainsDescription
1Site IDclinical site number
2Subject IDsubject ID number
3Visit Numbervisit or sequence number
4Plate NumberCRF plate number
5Record Status1-6,0: 1=final, 2=incomplete, 3=pending, 4=FINAL, 5=INCOMPLETE, 6=PENDING, 0=missed
6Record Validation Level1-7
7fax nameyyyywwssss: yyyy=year, ww=week of the year, ssss=order number of fax within the week
8visit datedate the visit occurred, from DFX_visit.date, in yy/mm/dd format
9fax datedate the fax was received in yy/mm/dd format
10creation datedate the data record was created, in yy/mm/dd format
11modification datedate the data record was last modified, in yy/mm/dd format
12visit julian daydate the visit occurred, from DFX_visit.date, in days since Jan 1,1900
13fax julian daydate the fax was received in days since Jan 1,1900
14creation julian daydate the data record was created, in days since Jan 1,1900
15modification julian daydate the data record was last modified, in days since Jan 1,1900
16fax timetime the fax was received in hh:mm:ss format
17creation timetime the data record was created in hh:mm:ss format
18modification timetime the data record was last modified in hh:mm:ss format
19fax minutestime the fax was received, in number of minutes into the day (0-1440)
20creation minutestime the data record was received, in number of minutes into the day (0-1440)
21modification minutestime the data record was last modified, in number of minutes into the day (0-1440)

Table 2.73.  DFX_visit.dates - value in the database of all visit dates

Usual NameDFX_visit.dates
Typeclear text
Created ByDF_XXkeys
Used ByDF_CTvisits, DF_PTcrfs, DF_QCupdate
Field Delimiter|
Record Delimiter\n
Comment DelimiterNA
Fields/Record5
Description

This file records the value of all dates in the database that use the VisitDate attribute. Only one date using the VisitDate attribute may appear on each CRF plate, but different plates for the same visit may contain VisitDate fields, and thus it is possible to have more than one record in for each visit for any given subject. A related file, named , records the preferred visit date for each visit, as defined in the study visit map. These 2 files are updated each time DF_XXkeys is run.

Table 2.74, “DFX_visit.dates the value of all dates using the VisitDate attribute” describes the data recorded for each visit date.

Example

The following example shows 5 records for subject ID 1127, for visits 0,3,6,51. Note that there are 2 entries for visit 51, one from plate 10 and one from plate 13.

1127|0|03/08/19|37851|3
1127|3|03/11/20|37944|5
1127|6|04/02/18|38034|5
1127|51|03/08/10|37842|10
1127|51|03/08/10|37842|13


Table 2.74. DFX_visit.dates the value of all dates using the VisitDate attribute

Field #ContainsDescription
1Subject IDsubject ID number
2Visit Numbervisit or sequence number
3datethe date value in the format used by the VisitDate attribute
4julian valueThis is a modified julian value for the date, calculated as the number of says since Jan 1,1900.
5Plate NumberThe plate on which the visit date is recorded.

Chapter 3. Shell Level Programs

Table of Contents

3.1. Introduction
3.2. User Credentials
3.2.1. Good Password Management
3.2.2. Order of Evaluation
3.3. Organization of Reference Pages
3.4. Alphabetical Listing
DFaccess.rpc — Change access to a study database or , or query their current access status.
DFattach — Attach one or more external documents to keys in a DFdiscover study
DFaudittrace — Used by the DF_ATmods report to read study journal files. DF_ATmods produces an audit trail report showing database modifications for the specified study.
DFbatch — Process one or more batch edit check files
DFcompiler — Compile study-level edit check programs and output any warnings and/or errors encountered in the syntax.
DFdisable.rpc — Disable a study database server or incoming fax daemon to make them unavailable to clients and incoming faxes
DFenable.rpc — Enable a study database server or incoming fax daemon following a previous
DFencryptpdf — Protect a PDF file by encrypting it with the specified password
DFexport — Client-side, command-line interface for exporting data by plate, field or module; exporting change history; or exporting components of study definition
DFexport.rpc — Export data records from one or multiple plates from a study data file
DFfaxq — Display the members of the outgoing fax queue
DFfaxrm — Remove faxes from the outgoing fax queue
DFget — Get specified data fields from each record in an input file and write them to an output file
DFgetparam.rpc — Retrieve and evaluate the value of the requested configuration parameter
DFhostid — Display the unique DFdiscover host identifier of the system
DFimageio — Request a study CRF image from the database
DFimport.rpc — Import database records to a study database from an ASCII text file
DFlistplates.rpc — List all plate numbers used in the study
DFlogger — Re-route error messages from non-DFdiscover applications to syslog, which in turn writes the messages to the system log files, as configured in /etc/syslog.conf.
DFpass — Locally manage user credentials for client-side command-line programs.
DFpdf — Generate bookmarked PDF documents of CRF images
DFpdfpkg — Generate multiple bookmarked PDF files for specified subject IDs or sites
DFprint_filter — Format input file(s) for printing to a PostScript® capable printer, and print to a specified printer
DFprintdb — Print case report forms merged with data records from the study database
DFpsprint — Convert one or more input CRF images into PostScript®
DFqcps — Convert a Query Report, previously generated by DF_QCreports, into a PDF file with barcoding, prior to sending the report to a study site
DFreport — Client-side, command-line interface for executing reports
DFsas — Prepare data set(s) and job file for processing by SAS®.
DFsendfax — Fax or email a plain text, PDF, or TIFF file to one or more recipients
DFsqlload — Create table definitions and import all data into a relational database
DFstatus — Display database status information in plain text format
DFtextps — Convert one or more input files into PDF
DFuserdb — Perform maintenance operations on the user database
DFversion — Display version information for all DFdiscover executables (programs), reports, and utilities
DFwhich — Display version information for one or more DFdiscover programs, reports and/or utilities

3.1. Introduction

The DFdiscover distribution includes several programs that can be executed on their own from a command line, or combined in shell scripts to build other programs (e.g. to export data sets for systems, or create study specific reports). All of the programs described in this section can be found in the DFdiscover executables directory, /opt/dfdiscover/bin with the exception of DFmkdrf.jnl and DFmkdrf.ec which are in /opt/dfdiscover/ecbin.

Permission to run these programs is controlled by UNIX file permissions that are set on DFdiscover installation to execute for user 'datafax' and users in group 'studies', but not for 'other' users.

The remainder of this chapter is a reference for each shell program.

3.2. User Credentials

DFdiscover includes many programs. Some operate only on a database server computer (referred to as server-side) while others operate on a user's desktop computer (referred to as client-side). The application of user login and permissions varies slightly in these two environments.

Interactive client-side programs like DFexplore, DFsetup, DFsend and DFadmin require the user to interactively provide a valid username and password each time they connect to a database server.

Conversely, the majority of the shell programs require a UNIX-like shell environment to function, and hence they may only be executed directly on a DFdiscover database server machine. Access to specific database information is granted based upon the DFdiscover permissions assigned to the user, where the user is identified by their UNIX login name.

Starting with release 2014.0.0, DFdiscover includes a third type of program. Programs of this type require a command line environment to execute and can be run from either the user's desktop computer (client-side) or from the database server computer (server-side). These programs are DFbatch, DFpdfpkg and DFexport. To authenticate, they require the same credentials that an interactive program does, specifically the username and password required to connect to a database server. Since there is no visual login dialog, these programs require a different interaction for authenticating, and there are two solutions for the user:

  1. command-line options, the user can specify -S servername, -U username and -C password when the program is run, or

  2. environment variables, the user can set the variables DFSERVER servername, DFUSER username and DFPASSWD password before the program is run.

Specification of command-line options takes priority if both command-line options and environment variables are supplied. It is not possible to mix solutions using a subset of command-line options and a complimentary set of environment variables. If any command-line option is specified, the expectation is that the command-line solution is preferred; otherwise, the environment variable solution is required.

3.2.1. Good Password Management

In many circumstances, it is poor practice to display or store a plain text password. Including a password in a shell script or in a cron job is not secure behavior, and is not recommended. Hence we strongly recommend that users do not use either the -C password command-line option or the DFPASSWD password environment variable. Instead, we encourage users to make use of the DFpass program to locally manage their DFdiscover passwords. After creating a matching entry with DFpass, a user does not need to subsequently specify a password with either -C password or DFPASSWD password.

The recommended practice then is to create/manage passwords locally using DFpass and subsequently use either the two command-line options, -S servername and -U username, or the two environment variables DFSERVER servername and DFUSER username.

Use of locally managed passwords is limited to the DFattach, DFbatch, DFpdfpkg, DFexport, DFreport and DFuserPerms programs.

3.2.2. Order of Evaluation

When determining the authentication credentials to use for one of DFbatch, DFpdfpkg or DFexport, the order of evaluation is as follows and only the first matching combination, if any, is used:

  1. If all three command line options -S servername, -U username and -C password are specified, use them.

  2. If all three environment variables DFSERVER servername, DFUSER username and DFPASSWD password are defined, use them.

  3. If both of the command line options -S servername and -U username are specified, use them together with the matching password entry previously stored via DFpass.

  4. If both of the environment variables DFSERVER servername and DFUSER username are defined, use them together with the matching password entry previously stored via DFpass.

3.3. Organization of Reference Pages

The description of each program in this reference is divided into sections.

  • Name.  The official name of the program.

    The program name is case sensitive and must be typed exactly as specified. This section also provides the brief purpose of the program.

  • Synopsis.  Lists all of the valid options for the program invocation.

    Each option is either optional or required. Most options are, not surprisingly, optional, a few are required, and yet others may require one option from a specific subset. Each option type is indicated with a particular notation. Except for some required options, a program option begins with - and is followed by an option letter. In some cases this may be enough to select the option; in other cases, the option letter will be further followed by an option string.

    [Note]Option Letter

    An option letter may be given different meanings in different programs. There is no requirement that the same option letter infers the same meaning when used across different programs.

    Table 3.1. Notation for program options

    NotationMeaning

    {study}

    Required. The program will not function without the specification of this option. The option must be provided at the specified location in the list of options.

    [-o]

    Optional. The option may be omitted. The program will function, albeit differently, with or without this option.

    [-o string ]

    Optional, with an additional option string. The option may be omitted, however when it is used, the option letter must be immediately followed by a space and the option string. The option string generally ends at the next white-space character, however some option strings may contain spaces, and may or may not require that such strings be delimited by ".

    { [-s] | [-u] }

    One of the options from the specified subset is required.

    [ [-s #] | [-u #] ]

    Several options are possible but only zero or one from the specified subset may be used.


  • Description.  Describes the program purpose in greater detail.

    This section describes exactly how the program works in terms of environment, input requirements, output format, dependencies, etc. It may also include detailed information about the program behavior for commonly used combinations of options.

  • Options.  Detailed listing of available options, their use, and their meaning.

  • Examples.  Illustrates the program options and output by way of example(s).

  • See Also.  Lists other reference materials, typically other programs, that are relevant to the program. This in an optional section.

  • Limitations.  Describes any limitations in the use or capabilities of the program. This in an optional section.

3.4. Alphabetical Listing

DFaccess.rpc

DFaccess.rpc — Change access to a study database or , or query their current access status.

Synopsis

DFaccess.rpc { [-s study] | [-inbound] } [ [-ro] | [-rw] | [-enable] | [-q] ] [ [-restrict reason string] | [-disable reason string] ]

Description

DFaccess.rpc provides the same functionality from the command line that is available in the DFadmin Status View. Like all shell level programs, DFaccess.rpc can be executed by datafax or any other user with a UNIX login account who is in group studies.

Read-Only Mode.  While a study is in read-only mode DFexplore and DFsetup can be used to view but not change study data and setup information. Studies are typically put into Read-Only mode because data collection is complete, all queries have been resolved, and the data has been verified against source documents. Statistical analysis may be ongoing or an FDA audit may be in progress thus the study cannot be disabled, but no further changes are expected or allowed.

By design, read-write access to a study database is only granted to tools that write new data records or update existing data records. All other tools have read-only access. Thus when is used to switch a study sever to read-only mode only those tools that normally have read-write access will be affected. This includes the following:

Table 3.2. Software Components Affected by Read-Only Mode

ProgramWhen Database Server is in Read-Only Mode
DFbatchexits without executing any edit checks
DFimport.rpcexits without importing any records
DFinbound.rpcincoming pages barcoded for the study are moved to the system-wide identify directory
DFqcsent.rpcwill not run because the Query database cannot be updated
DFexplorethe study can be used in view-only mode but pages cannot be sent to the study from the unidentified fax router
DFsetupcan be used in view-only mode
DFadmincan be used by DFdiscover and study administrators to view and change study access
DF_ICqcsthe -r (repair) option cannot be used
DF_QCfaxfaxes can be sent but post-processing to move external reports to the sent directory and to mark queries as having been sent will fail
DF_QCreportsexternal reports that include refax (correction) and/or Q&A (clarification) queries cannot be created because the Query database cannot be updated
DF_QCsentwill not run because the Query database cannot be updated
DF_QCupdatewill not run because the Query database cannot be updated


When a study is put into read-only mode it will remain in that mode until reset using DFadmin or with the -rw option.

While a study is in read-only mode, any incoming pages barcoded for that study will be moved to DFrouter's identify directory.

Restricted Access.  Access to a study can be restricted for DFdiscover and study administrators only using the -restricted option. While a study is restricted other users are not allowed to login, even in view-only mode.

When a study is restricted using or DFadmin users who are logged in will be able to continue working until they log off, but only administrators will be able to make new login connections.

While a study is restricted, any incoming pages barcoded for the study will be processed normally and sent to the study new fax queue, but only DFdiscover and study administrators will be able to view or process them.

A study may be made both restricted and read-only, but these options cannot be used together on the same command line, it requires 2 separate executions of ; the order is not important.

When a study is put into restricted mode it will remain in that mode until reset using DFadmin or with the -enable option or until the study server is restarted.

Disabled Access.  A study database may be disabled, making it unavailable to all users, using the -disable option.

While a study is disabled, any incoming pages barcoded for the study will be held until the study is enabled, and will not show up in the study new fax queue until processing is triggered by the arrival of the next fax received by the DFdiscover server. It is not necessary for the next fax to contain pages barcoded for the study in question or for any study.

When a study is disabled it will remain in that mode until reset using DFadmin or with the -enable option.

To complete a change in access mode, must be the only client accessing the server. Thus fails if there are any tools with open connections to the study server.

Options

-s study

the study number

-inbound

used when changing the status of , which processes incoming faxes and PDF files

-ro

change the database server to read-only mode

-rw

change the database server to read-write mode

-enable

reverses -disable or -restrict to return the study or to normal access

-q

query the state of the database server or

-restrict reason string

restrict the database server making it available to DFdiscover and study administrators only, and displaying the reason string in study selection dialogs

-disable reason string

disable the database server making it unavailable to all users, and displaying the reason string in study selection dialogs

Exit Status

DFaccess.rpc exits with one of the following statuses:

0

The mode of the database server was successfully changed to the requested mode, or the mode was successfully queried.

1

The mode of the database server was not successfully changed. The server is currently in a mode that is incompatible with the change.

2

The study server could not be contacted or there was an error in the command-line arguments.

Examples

Example 3.1. Enable read-only access to study #123, do lengthy operation, and then reinstate read-write access

# /opt/dfdiscover/bin/DFaccess.rpc -s 123 -ro

...some lengthy operation...

# /opt/dfdiscover/bin/DFaccess.rpc -s 123 -rw

Example 3.2. Restrict study #123 in read-only access to DFdiscover and study administrators only, while they analyze and run reports on a static database, and then reinstate normal access to all users

# /opt/dfdiscover/bin/DFaccess.rpc -s 123 -restrict analysis
# /opt/dfdiscover/bin/DFaccess.rpc -s 123 -ro

...administrators perform their analyses

# /opt/dfdiscover/bin/DFaccess.rpc -s 123 -rw
# /opt/dfdiscover/bin/DFaccess.rpc -s 123 -enable

Example 3.3. Query the current status of

% /opt/dfdiscover/bin/DFaccess.rpc -inbound -q


DFattach

DFattach — Attach one or more external documents to keys in a DFdiscover study

Synopsis

DFattach [-S server] [-U username] [-C password] [ [-doc docname] | [-subject # -visit # -plate # -doc docname] | [-idrf input_drf] | [-dir directory] ] [-odrf output_drf] [-log logfile] {study#}

Description

DFattach is a command-line utlity that mimics, and extends, the Plate > Attach Subject Document facility in DFexplore. It's primary function is to permit one or more documents to be attached to one or more record keys from a command-line interface.

In DFexplore, it is possible to attach a document to the current data record; using DFattach it is possible to attach documents to many records with simple commands.

Permitted document types are the same as those allowed by DFexplore.

For each document successfully attached to a key:

  • an entry is added to the system work/fax_log,

  • a DRF entry is appended to output_drf if -odrf output_drf is given as an option. The DRF entry contains 5 fields: the key of the record in the first 3 fields, the image ID assigned to the document, and the original document name.

Attaching Documents

There are 4 different methods for attaching documents.

  1. In it simplest form, DFattach attaches a single document to a single key. Specify the document with -doc docname. The key is determined from the filename of the document. The document filename must have the format subject_ visit_ plate [_other] [.extension] where subject, visit and plate are each valid, numeric identifiers for the key.

  2. Attach a single document to the key given as arguments. In this method, each of -subject # -visit # -plate # -doc docname must be explicitly specified. This has the advantage that the filename of the document to be attached does not have to change to follow a format.

  3. Read the input DRF. For each record in the DRF, the key is in the first three fields and the document name is in the fourth field. Attach the named document to the key. Repeat for each record.

  4. Process each file and subdirectory in -dir directory. For each file, if the filename has the format subject_ visit_ plate [_other] [.extension], treat the file as a document to attach to the key. Repeat for all matching files in the directory. Then recursively repeat for each sub-directory.

Permissions

Permissions are enforced for the user identified by the supplied credentials. Minimally, the user must have a study role permission that permits Server - Attach document and DFexplore - Data - Attach subject document. Additional permissions may be required dependent upon the action

Database Actions

For each document and key, [id, visit, plate], to be attached, the steps are:

  1. Confirm that the combination of visit and plate are defined in the visit map. If it is not, output an error message and skip this document.

  2. Lock the key. If the key cannot be locked, output an error message and skip this document.

  3. If the key does not exist in the database, a record with pending status is created. The document becomes the primary image. Create Data permission is required.

  4. If the key matches an existing missed record, the missed record is deleted and a record with pending status is created. The document becomes the primary image. Delete Missed record and Create Data permission are required.

  5. If the key exists, and there is already a primary image, the document is attached as a secondary image; otherwise, the document is attached as the primary image. Modify Data permission is required.

Options

-S server

DFdiscover server name

-U username

DFdiscover login username

-C password

login password. Refer to Section 3.2, “User Credentials” for recommended/better solutions for safe password handling.

-doc docname

determine the key encoded in docname and attach docname to that key

-subject # -visit # -plate # -doc docname

attach docname to the given key

-idrf input_drf

read input_drf, which is in DRF format, and for each record add the document to the given key

-dir directory

read directory and recursively each sub-directory, and for each filename that matches the required pattern attach filename to the given key

-odrf output_drf

for each document that is successfully attached, append a DRF record to output_drf. Subsequently a task can be built from the DRF to review the attached documents.

-log logfile

write any log messages to logfile

study#

the study number where the documents are to be attached; required

Exit Status

DFattach exits with one of the following statuses:

0

The command is successful.

1

One or more errors occurred, and the command has failed. Error messages are written to standard error on the command-line.

Examples

In each of the following examples, the options -S server, -U username and -C password are not shown but are required.

Example 3.4. Attach 1234_1_12.pdf to subject 1234, visit 1, plate 12 in the database for study 100

% DFattach -doc 1234_1_12.pdf 100

Example 3.5. Attach radiograph.pdf to subject 30001, visit 10, plate 200 in the database for study 22

% DFattach -subject 30001 -visit 10 -plate 200 -doc radiograph.pdf 22

Example 3.6. Attach all of the documents in directory /tmp/newdocs to the matching keys in study 200 and record the successes in /tmp/results.drf

% ls /tmp/newdocs
250001_0_1.pdf
250002_0_1.pdf
350001_0_1.pdf
350001_0_2.pdf
350001_1_10.pdf
% DFattach -dir /tmp/newdocs -odrf /tmp/results.drf 200


DFaudittrace

DFaudittrace — Used by the DF_ATmods report to read study journal files. DF_ATmods produces an audit trail report showing database modifications for the specified study.

Synopsis

DFaudittrace {-s #} [-d date1[-date2]] [-q] [-r] [-N] [-I subjectID] [-P #] [-V #] [-f fieldlist] [-v vfence] [-x output.xlsx]

Description

DFaudittrace reads a record from the study journal files, calculates any changes from the previous record for those keys and then outputs the differences to DF_ATmods.

The output from DFaudittrace consists of one or more ASCII records, each having 20 fields. Each field is separated by a pipe-delimiter (|). Field contents depend on:

  1. whether DFaudittrace is describing a new record (type N), a changed field (type C), or a deleted record (type D). This information is found in the first field of each record.

  2. whether DFaudittrace is describing a data record, query record or a reason record. This information is found in the 8th field of each record.

The tables describe each of the 20 fields for each record type (data, query, reason) for each type of change (new record, changed field, deleted record). Fields 1 to 7 are consistent across all records output by DFaudittrace and have been described in the first table. Fields 8 to 20 depend on the record type and the type of change, and are described in tables 2 to 4.

Table 3.3. DFaudittrace Output: Fields 1 to 7

FieldDescription
1Record type (N=new, C=changed field, D=deleted record)
2Date (yyyymmdd)
3Time (hhmmss)
4User
5Subject ID
6Visit
7Plate

Table 3.4. New Records

FieldData RecordsQuery RecordsReason Records
80>0: unique field ID of field on which query is located<0: unique field ID of field on which reason is located
90 (summary), or unique field IDfield number of Query recordfield number of Reason record
10status 1-6, 0=new missed recordstatus 1-6status 1-6
11validation levelvalidation levelvalidation level
12maximum validation level reachedmaximum validation level reachedmaximum validation level reached
13missed record reason code or blankQuery category code 1-6, 21-23 + DFqcproblem_map codesreason code
14missed record reason text or blankQuery usage code 1=internal, 2=externalreason text
15blankblankblank
16blankblankblank
17blank (summary), ordinal field positionordinal field position of data fieldordinal field position of data field
18blank (summary), field namefield name of data fieldfield name of data field
19blank (summary), old decoded label for choice/checkblankblank
20blank (summary), new decoded label for choice/checkblankblank

Table 3.5. Changed Field

FieldData RecordsQuery RecordsReason Records
80>0: unique field ID of field on which query is located<0: unique field ID of field on which reason is located
90 (summary), or unique field IDfield number of Query recordfield number of Reason record
10status 1-6status 1-6status 1-6
11validation levelvalidation levelvalidation level
12maximum validation level reachedmaximum validation level reachedmaximum validation level reached
13blankQuery Category code 1-6, 21-23 + DFqcproblem_map codesreason code
14blankQuery usage code 1=internal, 2=externalreason text
15old valueold valueold value
16new valuenew valuenew value
17ordinal field positionordinal field position of data fieldordinal field position of data field
18field namefield name of data fieldfield name of data field
19blank, old decoded label for choice/checkblankblank
20blank, new decoded label for choice/checkblankblank

Table 3.6. Deleted Record

FieldData RecordsQuery RecordsReason Records
80>0: unique field ID of field on which query is located<0: unique field ID of field on which reason is located
90 (summary), or unique field IDfield number of Query recordfield number of Reason record
10status 7status 7status 7
11validation levelvalidation levelvalidation level
12maximum validation level reachedmaximum validation level reachedmaximum validation level reached
131=missed record, 0=data recordQuery category code 1-6, 21-23 + DFqcproblem_map codesreason code
14blankQuery usage code 1=internal, 2=externalreason text
15blankblankblank
16blankblankblank
17blank (summary), ordinal field positionordinal field position of data fieldordinal field position of data field
18blank (summary), field namefield name of data fieldfield name of data field
19blankblankblank
20blankblankblank

It is also important to note the following about DFaudittrace:

  1. If a record or a query is deleted and then re-created, the re-created record or query is considered to be new record (N), not a change (C) to the previous record.

  2. In queries, the user is sometimes, but not always set when a query is created by the user. The user is never set when the query is modified or deleted, or when it is created by the report DF_QCupdate. Unknown user names appear in the output as unknown for dates before June 1, 1995 and as ?? thereafter.

  3. In queries, the validation level is set when the record is created and reset when the record is modified. The validation level of a query does not change when only the record's validation level changes.

  4. If the -v vfence option is used when running DFaudittrace, the effect is different for data records and queries. Data changes are output for records journaled after the case reached or surpassed the vfence validation level. Query changes are output for queries journaled after the query was modified at or beyond the vfence validation level.

Options

-s #

DFdiscover study number.

-d date1[-date2]

date of changes. This option restricts the output to changes made at date1, unless date2 is also specified, in which case the output is the list of changes made between date1 and date2. Dates are specified in the format yyyymmdd and may include the word today to represent the current date.

-q

include query details.

-r

include reason for change details.

-N

include all field details for type N (new) records.

-I #

Subject ID.

-P #

DFdiscover plate number.

-V #

DFdiscover visit or sequence number.

-f fieldlist

field number(s). This option restricts the output to the field numbers specified in the list. This may include a single field number, or a range or list of numbers. Range and list specifications may include a dash (-), tilde (~), or comma(,).

-v vfence

validation level. In order to be considered for output, the keys on this record must have attained or surpassed the specified validation level at least once. This is useful if the user wants to output all changes that were made to records that have existed at or above the specified validation level. Once the record keys have made it to the vfence level, further changes will be output even if the record subsequently drops to a validation level below the vfence value.

-x output.xlsx

Excel output file. Write the output, in Excel format, to the named file. The Excel file contains the same output, 20 columns per row, as the standard output. It includes one additional header row, with column names.

Exit Status

DFaudittrace exits with one of the following statuses:

0

The command was successful.

1

The command failed because the command-line arguments were not present or were incorrectly specified, the database server could not be contacted, or communication with the database server failed.

Examples

Example 3.7.  Execute DFaudittrace to output journal information for study 254 between the dates of January 1, 2001 and January 15, 2001.

% DFaudittrace -s 254 -d 20010101~20010115

Example 3.8.  Execute DFaudittrace to output journal information for study 254 between the dates of January 1, 1998 and today, for field numbers 10-13.

% DFaudittrace -s 254 -d 19980101-today -f 10-13

Example 3.9.  Execute DFaudittrace to output all journal information for study 254, for records that have existed at or above a validation level of 2 at some point during the study.

% DFaudittrace -s 254 -v 2

Example 3.10.  Execute DFaudittrace to generate an Excel file containing all changes for data related to subject 44002.

% DFaudittrace -s 254 -I 44002 -o 44002history.xlsx


DFbatch

DFbatch — Process one or more batch edit check files

Synopsis

DFbatch [-S server] [-U username] [-C password] [-b batchnames] [-p stylesheet] [ [-o outhtml_file] | [-O outhtml_dir] ] [-e error_file] {-i control_file} {#}

Description

DFbatch is a framework for edit check execution in command-line or unattended mode. The framework specifies which data records are to be operated upon by edit checks and what is to happen when edit checks are invoked. It uses the same edit checks that are used in interactive edit checks through DFexplore and introduces no changes to the language.

Complete reference documentation for DFbatch can be found in Batch Edit checks.

Options

-S server

DFdiscover server name

-U username

DFdiscover login username

-C password

login password. Refer to Section 3.2, “User Credentials” for recommended/better solutions for safe password handling.

-b batchnames

selected batches by name from the control file, and/or re-order selected batches for processing.

-p stylesheet

style the output with the specified XSL stylesheet before generating the HTML output. By default, batchlog.xsl is applied.

-O outhtml_dir

will make a default output HTML file in your local outhtml_dir for each batch. The default output file name will be 'outhtml_dir' + 'batch_name' + '_out.html'.

-o outhtml_file

'-o outhtml' output html file name is local and must include the full path. If '-o outhtml' is specified and there are more than one batches within a control file, only the last log will be saved to the given name.

-e error_log

will write any errors to the full patch name of error_log. By default, errors are written to stderr.

-i control_file

The input file of instructions (required). These instructions control the behavior of the program including how records are selected, and what actions, if any, are applied.

study#

The study database number (required).

Exit Status

DFbatch exits with one of the following statuses:

0

The command was successful.

>0

An error has occurred. The text of the error will appear in one of the last elements of the log file.


DFcompiler

DFcompiler — Compile study-level edit check programs and output any warnings and/or errors encountered in the syntax.

Synopsis

DFcompiler [-o compiled_outputfilename] {-s #} {filename}

Description

DFcompiler is used for syntax checking of edit check files. It can be run in the study DFsetup tool or from the command-line.

In DFsetup, selecting Edit checks from Study and selecting the Check Syntax, button in the edit checks window, runs DFcompiler. Results are output to the bottom Outputs section of the edit checks window. A user requires permission to use DFsetup in order to run DFcompiler in this way.

DFcompiler may also be run from the command-line using the options described below. The study number and edit checks file name (with a correct pathname) are both required arguments.

DFexplore has the ability to compile and run edit checks. Edit checks can be compiled (to DFedits.bin) by selecting Publish in DFsetup or from the command line using DFcompiler as follows:

% DFcompiler -s study# -o DFedits.bin DFedits

Edit checks are loaded automatically when DFexplore starts but can be reloaded following a new compile from the File > Reload - Edit checks option.

Options

-s #

DFdiscover study number

-o DFedits.bin

the output executable file used to run production edit checks in DFexplore and DFbatch. If no output file name is specified, syntax checking is performed but no output file is created.

filename

the source file in which the edit check programs reside. Typically this will be $STUDY_DIR/ecsrc/DFedits containing the edit checks used by DFexplore and DFbatch

Exit Status

DFcompiler exits with one of the following statuses:

0

The command was successful.

1

The command is not supported by the user's current DFdiscover release.

2

The command failed because the database server could not be contacted, the study is not defined, the input file could not be located, or the server is out of memory.

Examples

Example 3.11.  Compile and check the syntax for edit checks defined in DFedits for study 254

% DFcompiler -s 254 /opt/studies/254/ecsrc/DFedits


DFdisable.rpc

DFdisable.rpc — Disable a study database server or incoming fax daemon to make them unavailable to clients and incoming faxes

Synopsis

DFdisable.rpc { [-s #] | [-i #] } [-f] [-w #] [-r string] {key}

Description

The study database server must be disabled when a user requires "off-line" access to the database and wishes to ensure that no client connections to the server are permitted.

Equivalent functionality can be obtained using the Studies dialog in DFadmin.

A disable request will fail if there are other users with client tools connected to the study server at the time of the request.

When -i is used to disable an incoming daemon, the daemon is allowed to complete any fax processing that it might be engaged in, but as soon as this is completed and the daemon exits, it will not be allowed to process another incoming fax.

Following execution of DFdisable.rpc, a study server or incoming fax daemon can be re-enabled using DFenable.rpc. The command must be executed by the user who executed DFdisable.rpc, and the same key must be provided.

When a study server is disabled, the default requirement is that the server shutdown as quickly as possible. In the interest of efficiency, it removes only those index entries that are for obsolete records and sorts only those index files that are unsorted. This is sufficient for correct operation of the server. However, the server does not remove those indices or records that are marked for deletion nor does it perform garbage collection on data records that are obsolete. The result is that it is subsequently possible to export a record that is still scheduled for deletion. This may be undesirable for archival purposes. Use of -f will force the server to remove all records marked for deletion and perform garbage collection of obsolete data records. This option is primarily useful when shutting down a database in preparation for archiving.

Options

-s #, -i #

the study number or the incoming daemon number of the server to be disabled

-f

force clean-up of database. This results in a complete garbage clean-up by the server of the database. Usage of this flag will slow down the execution time of DFdisable.rpc and is not required for proper server operation.

-w #

an optional number of seconds for the command to delay while waiting for the server to shutdown. The default is 60 and the legal range is 10 to 1000. Very large databases may not shutdown in 60 seconds and so should delay longer.

-r string

an optional reason string indicating why the study server is being disabled. This reason will appear in the error log file whenever a user attempts to connect to the server while it is disabled.

key

a required string, which must be subsequently provided by the same user from the same machine when executing to enable the server again. The key must be the last argument on the command line.

Exit Status

DFdisable.rpc exits with one of the following statuses:

0

The command was successful.

1

The command failed because the required command-line arguments were not present or were incorrectly specified.

> 1

The command failed because the database server could not be contacted, communication with the database server failed, or the database server has already been disabled.

Examples

Example 3.12. Disable study #123 so that database maintenance can be performed

% key=`date`
% DFdisable.rpc -s 123 -r "maintenance" $key

See Also

DFenable.rpc

DFenable.rpc

DFenable.rpc — Enable a study database server or incoming fax daemon following a previous

Synopsis

DFenable.rpc { [-s #] | [-i] } {key}

Description

When a study database server has been disabled, use DFenable.rpc to enable the server again and make it available for client connections. Equivalent functionality can be obtained using the Studies dialog in DFadmin.

When an incoming daemon is re-enabled it once again becomes available for processing incoming faxes.

Study servers and incoming fax daemons can only be enabled by the user who disabled them, and then only if the correct key is provided.

Options

-s #

the study number to be enabled

-i

enable incoming daemon

key

a required string, which must be subsequently provided by the same user from the same machine when executing to enable the server again. The key must be the last argument on the command line.

Exit Status

exits with one of the following statuses:

0

The command was successful.

1

The command failed because the required command-line arguments were not present or were incorrectly specified.

> 1

The command failed because the database server could not be contacted, communication with the database server failed, or the database server has already been enabled.

Examples

Example 3.13. Enable study #123

In this case, the assumption is that the study was previously disabled and the disable key was saved in a shell variable.

% DFenable.rpc -s 123 $key

See Also

DFdisable.rpc

DFencryptpdf

DFencryptpdf — Protect a PDF file by encrypting it with the specified password

Synopsis

DFencryptpdf { [-c password] | [-C] } [-i string] [-o string]

Description

PDF supports password protection beginning with version 5.0 of Acrobat Reader and version 1.4 of the PDF standard. A PDF document has at least two levels of password: the owner password (which permits the owner to perform actions that another user cannot) and the user password. Since PDF is typically a plain text format, a password protected PDF document is encrypted and stored in binary format. The owner/user password is used as a salt for the encryption.

For DFdiscover purposes, the password is needed to ensure that the document can only be opened and read by known users (those with the password). Since DFdiscover supports delivery of study data/reports by PDF and email, it makes sense that the PDF document be protected from viewing by an unintended recipient. The password is not intended to limit what a user with the password can do with the document, so the owner and user password are the same and no restrictions are applied to any one of the actions available on the document.

DFencryptpdf returns zero upon success, otherwise returns one. DFencryptpdf may fail if the input PDF file is invalid PDF, has multiple cross-reference tables, such as linearized or updated PDF files, or is already encrypted. If neither -c nor -C is specified, it also returns an error.

Options

-c password, -C

If -c is specified, the command line password is used. If -C is specified, DFencryptpdf reads the password from password file ~/.dfpdfpasswd. If neither -c nor -C is specified, the program exits with an error.

The password file ~/.dfpdfpasswd may contain multiple lines, but only the first non-empty line will be used. The leading spaces and trailing spaces will be removed. The maximum length of the password is 32 characters.

-i string

The name of the input PDF file that will be encrypted. If no input file is specified, DFencryptpdf reads from standard input.

-o string

The name of the encrypted output PDF file. If no output file is specified, DFencryptpdf writes to standard output. DFencryptpdf will exit with an error if the input filename and output filename are the same.

Exit Status

DFencryptpdf exits with one of the following statuses:

0

The command was successful.

1

The command failed because the required command-line arguments were not present or were incorrectly specified, the input file could not be read, or the output file could not be created/written.

Examples

Example 3.14.  Encrypt a set of CRFs stored in crfs.pdf and protect with the password "DoNotUse".

% DFencryptpdf -c DoNotUse -i crfs.pdf -o protected_crfs.pdf

See Also

DFpdf

DFexport

DFexport — Client-side, command-line interface for exporting data by plate, field or module; exporting change history; or exporting components of study definition

Synopsis

DFexport [-S server] [-U username] [-C password] [ [-Mname moduleName] | [-Mnum moduleID] | [-Pnum [ [#] | [qc] | [reason] | [new] | [missed] ] ] ] [-status #, #-#] [-level #, #-#] [-visit #, #-#] [ [-site #, #-#] | [-subject #, #-#] ] [-plate #, #-#] [-create list] [-modify list] [-resolve list] [ [-Pattern string] | [-pattern string] ] [-expr string] [ [-ALL criteria] | [-ANY criteria] ] [ [-Fnum #, #-#] | [-Fname fieldname_list] | [-Falias fieldalias_list] ] [ [-joinALL plt#[moduleFields]...] | [-joinANY plt#[moduleFields]...] ] [ [-c] | [-j] ] [-d] [-p] [-z] [-h] [-D] [-H header_list] [-L lostcode] [ [-w] | [-a] ] [-o [ [outfile] | [-] ] ] [-e errlog] {study#}

DFexport also has a descriptive mode for the output of schema information.

DFexport [-S server] [-U username] [-C password] [-listfields] { [ [-listmodules] | [-listplates] ] [ [plt#,plt#~plt#] | [all] ] } {study#}

DFexport has a history mode for the output of change history related to a subject, visit, or plate.

DFexport [-S server] [-U username] [-C password] {-history} {-subject #} [-visit #] [-plate #] {study#}

Options and Description

DFexport exports data from an individual study database. Minimally, the user must specify the study database and also at least one of the plates or modules containing the data of interest. Data records are exported in ASCII plain text format to a specified file, or the command-line output.

Differences and Similarities compared to DFexport.rpc

DFexport and DFexport.rpc share many features yet they are also unique in several important ways.

  • DFexport is available as both a server-side and client-side application; DFexport.rpc is server-side only.

  • DFexport is able to export descriptive information from a study schema with the -listfields, -listmodules and -listplates options.

  • DFexport is able to export, with the -history option, the history of changes for a subject, visit within subject, or plate within visit of a subject.

  • DFexport includes support for joining data fields from the same module definition that are instanced over one or more plates.

  • DFexport is able to export all records marked as missed in one command execution using the -Pnum missed option. In DFexport.rpc this would require looping through each of the plates individually, selecting missed records at each step of the loop.

  • User permissions for selecting by validation level are enforced in DFexport.

  • Selecting queries and reasons by creation and modification date is supported in DFexport; in DFexport.rpc only data records may be selected in this way.

  • Many options are specified using the same notation as DFexport.rpc, while others are specified using unique notation.

  • It is possible for DFexport and DFexport.rpc to produce different results for the same selection criteria. DFexport enforces user permissions for the user specified with the -U username option or the DFUSER environment variable. DFexport.rpc uses the UNIX login name of the command-line.

General Approach

DFexport can export data in a plate-centric manner, like DFexport.rpc, using -Pnum, or in a module-centric manner that is unique to DFexport using -Mname or -Mnum. Many options are available to create data sets for a wide variety of needs. To get the most out of DFexport it can be useful to approach it's use with the following in mind:

  1. Preparation.  Access to exported data is available only to authenticated users with appropriate database permissions, Authentication and Database Permissions. Plate, module and field name information is available from DFexport itself, Schema Listings.

  2. Where is the data?  Specifying the source of the data, whether it is plate-based or module-based, is required, Data Source.

  3. Do the data records need to be filtered?  In many cases not all of the rows in a data set are of interest and need to be filtered further, Record Selection by Keys and Filters.

  4. Data Fields.  By default, all of the data fields in the specified data source will be exported. It is possible to specify a subset of the fields to export, Extracting/Combining/Concatenating Fields for Output. It is also possible to add field, module, plate, visit, image, site, and study metadata (including user-defined properties) to the export data, Including Metadata in Output.

  5. Output Formatting.  For export, the appearance of data fields can be modified without modification of the data source, Output Formatting.

  6. Output Destination.  Finally, the export data is written to a destination, Output Options.

Authentication and Database Permissions

Authentication.  To authenticate, DFexport requires the username and password for connection to a specific database server. These may be supplied as:

  1. command-line options, the user can specify -S servername, -U username and -C password when the program is run, or

  2. environment variables, the user can set the variables DFSERVER servername, DFUSER username and DFPASSWD password before the program is run.

Specification of command-line options takes priority if both command-line options and environment variables are supplied. Refer to User Credentials for more information.

Database Permissions.  Subsequent to authentication with a specific database server, use of DFexport is allowed (or disallowed) by the Server - Export Data permission, or the intersection of both DFexplore - List view and DFexplore - Print/Save Data permissions. These permissions are granted to a role (and then user) by an administrator on a study-by-study basis (see System Administrator Guide, Server). An authenticated user with one of these role permissions will then be able to export data records from the set of site and subject IDs granted by their user permissions, which may further be limited by visit, plate and level restrictions associated with the user's role. DFexport provides no warning that some records cannot be exported due to permission restrictions as this would itself provide information about the existence of restricted data records. For roles without 'Show Hidden Fields' permission and fields with the Hidden or Masked property enabled, DFexport will output an empty (NULL) value when exporting such data fields, and it will also skip over reasons and queries attached to such fields.

Schema Listings

This feature is unique to DFexport and is not available in DFexport.rpc. DFexport is able to output database schema information. The available information includes plate numbers and module names, and may optionally also include field number, name, alias and data type for data fields in the requested plates or modules. This descriptive information can be useful reference material when a user wishes to further refine an export.

To obtain a listing of all defined plate numbers, use -listplates all. [1] To obtain a listing of all defined module names, use -listmodules all. In both cases:

  • the output is a space-delimited list of values (either plate numbers or module names) matching the requested criteria,

  • a subset can be specified by replacing all with individual, lists, or ranges of plate numbers in the format #, #-#.

Example 3.15. List all module names used in plates 100 through 200

%  DFexport -listmodules 100-200 10
Chemistry Eligibility Enrollment Header LabData PhysicalExam SocialImpact Urinalysis

The output is sorted by module name and each module is listed exactly once, even when instanced more than once. Modules that are not instanced on one of the selected plates are not included in the output.


Information about data fields in the specified plates or modules can be requested by additionally including the -listfields option. This option may only be used in conjunction with the -listmodules and -listplates options.

Example 3.16. List schema information for data fields of all plates

%  DFexport -listfields -listplates all 253
Plate 001: Blood Pressure Screening Visits
 Field  Name                       Alias                      Type
 -----  -------------------------  -------------------------  -------
    1   DFSTATUS                   DFstatus1                  Choice
    2   DFVALID                    DFvalid1                   Choice
    3   DFRASTER                   DFraster1                  String
    4   DFSTUDY                    DFstudy1                   Number
    5   DFPLATE                    DFplate1                   Number
    6   DFSEQ                      DFseq1                     Number
    7   ID                         ID001                      Number
    8   PINIT                      PINIT001                   String
    9   AGE                        AGE                        Number
   10   SEX                        SEX                        Choice
   11   RACE                       RACE                       Choice
   12   RACEOTH                    RACEOTH                    String
   13   S1DATE                     S1DATE                     Date
...

This is the beginning of the output - actual output would be much longer as the user has requested this information for all defined plates. Field information is included both for user-defined fields and the fixed DFdiscover fields.


Example 3.17. List schema information for data fields of all modules

%  DFexport -listfields -listmodules all 253
Module: BloodPressure (ID=5022): Blood Pressure Readings
 Field  Name                  Alias                 Type     Used in Plates
 -----  --------------------  --------------------  -------  ------------------
    1   DFSTATUS              DFSTATUS              Choice   
    2   DFVALID               DFVALID               Choice   
    3   DFRASTER              DFRASTER              String   
    4   DFSTUDY               DFSTUDY               Number   
    5   DFPLATE               DFPLATE               Number   
    6   DFSEQ                 DFSEQ                 Number   
    7   DFPID                 DFPID                 Number   
    8   DFMNAME               DFMNAME               String   
    9   DFMID                 DFMID                 Number   
   10   DFMREF                DFMREF                Number   
   11   SBP1                  SystolicReading1      Number   1,5
   12   SBP2                  SystolicReading2      Number   1,5
   13   SBP3                  SystolicReading3      Number   1
   14   DBP1                  DiastolicReading1     Number   1,5
   15   DBP2                  DiastolicReading2     Number   1,5
   16   DBP3                  DiastolicReading3     Number   1
   17   SDAT                  ReadingDate           Date     1
   18   BPARM                 BPwhichArm            Choice   5
...

This is the beginning of the output - actual output would be much longer as the user has requested this information for all defined modules.

The module listing provides the module name, BloodPressure (useful in conjunction with -Mname), the module number, 5022 (useful in conjunction with -Mnum) and the field numbers and names defined in the module, DFSTATUS, ... (useful in conjunction with -Fname or -Fnum).


History of all Changes

This feature is unique to DFexport and is not available in DFexport.rpc. DFexport is able to output the history of all changes made to the data for a specific subject, visit or plate. The available information includes when the change was made, who made the change, what was changed, any reason associated with the change and any queries related to the data. The output is always to a file, in Excel format, and hence option -o outfile.xlsx is required.

To obtain a history listing use -history. The history of changes is specific to a single subject, and includes all data, query and reason changes for all data elements in records idenitfied by the subject ID. The output can be further filtered by visit, -visit #, and plate, -plate # .

DFexport is designed to export history of changes for exactly one subject. To export history for multiple subjects, DFaudittrace is available.

Data Source

Specifying the data source for the records to export is required. Data can be exported by plate number, using the -Pnum option, or by module, using either the -Mname or -Mnum option. Within any of these data sources, the default behavior is to export all of the defined fields. It is possible to further select the exported fields using one of the -Fname, -Falias or -Fnum options. Field name, alias and number information can be obtained from a previous invocation that includes the -listfields option.

-Pnum #

Plate number. This is equivalent to the required plate number in DFexport.rpc. Exactly one plate number value may be specified. The value is the actual plate number or from this list of keywords:

  • qc: export all queries

  • reason: export all reason records

  • new: export all records awaiting first validation

  • missed: export all records marked as missed across the entire database

If a plate selector is not given, the data source must be specified by module name or module number.

-Mname or -Mnum

Module name or module number. Export data from the specified module. Exactly one module name or module number may be specified. Schema listing options -listfields -listmodules may be used to determine the module name or number of interest.

If a module selector is not given, the data source must be specified by plate number.

Record Selection by Keys and Filters

There are many options available for selecting subsets of data records for export. Several of these options are similar or identical to their counterparts in DFexport.rpc. [2] In all cases, record selection and filtering is from the set of records allowed by the user's database permissions.

The available selection mechanisms and filters are:

-level #, #-#

validation level. By default, records at all validation levels permitted by the user permissions are exported. This option further selects a subset of those records, namely those having a validation level that falls within the specified range.

-site #, #-#

site number. This option selects records by site number, site numbers, site number range or any combination thereof. The site number of any data record is derived by dereferencing the subject ID in the study's sites database.

This option may not be combined with -subject #, #-# - doing so will cause DFexport to exit with an error message.

-subject #, #-#

subject ID. This option selects records by subject ID, subject IDs, subject ID range or any combination thereof.

This option may not be combined with -site #, #-# - doing so will cause DFexport to exit with an error message.

-visit #, #-#

visit/sequence number. This option selects records by visit number, visit numbers, visit number range or any combination thereof.

-status status

record status. Select records by status keyword or status numeric value. The recognized status keywords are final, incomplete, pending, missed, all. [3] The default is all statuses, using keyword all, or by excluding this selector. A single status keyword or multiple, comma-delimited status keywords can be specified but it is not possible to specify a range of statuses. It is also possible to reference a status by it's numeric value, and in that case a range of numeric values may be specified.

If plate 511 is exported, the available status keywords are the same but their meaning (as related to query status) is translated by the table:

Table 3.7. Numeric value, record status keyword, query status and reason status equivalence

Numeric valueRecord status keywordQuery status equivalentReason status equivalent
1finalnewapproved
2incompletein unsent reportrejected
3pendingresolved NApending
4secondary [a] resolved irrelevant-
5secondary [b] resolved corrected-
6secondary [c] in sent report-

[a] For backwards compatibility, old keyword CLEAN is supported.

[b] For backwards compatibility, old keyword DIRTY is supported.

[c] For backwards compatibility, old keyword ERROR is supported.


-plate #, #-#

plate number. Select from module instances, queries, new, missed or reason records by plate number.

It is important to note that this is not the same as the required plate number selector in DFexport.rpc. In that environment, the plate number selects the data file source for the export. In DFexport, this option filters the data source which may be modules, complete data records, queries, reason, new or missed records.

Example 3.18. Export all missed records in plates 1 through 10

%  DFexport -S server -U username -Pnum missed -plate 1-10 -o - 254

Example 3.19. Export all ESIG modules in plates 100 through 110 and 200

%  DFexport -S server -U username -Mname ESIG -plate 100-110,200 -o esigmodules.txt 123

The output, from study 123, is written to the file esigmodules.txt in the current directory.


-create yy/mm/dd-yy/mm/dd

creation date. Select data, query or reason records by their creation date. The keyword today can be used to include records that were created today.

-modify yy/mm/dd-yy/mm/dd

modification date. Select data, query or reason records by their last modification date. The keyword today can be used to include records that were last modified today.

-resolve yy/mm/dd-yy/mm/dd

resolution date. Select query records by their resolution date. Queries that are not yet resolved will not have a resolution date and hence will always be excluded if this selection filter is used. The keyword today can be used to include query records that were resolved today.

-Pattern|pattern string

Filter records to export only those that match the specified pattern string. For inclusion in the export, a fragment of the record must exactly match the entire pattern string. The -Pattern option requests a case sensitive match, while -pattern requests a case insensitive match. [4]

-expr

Include only records that match the expression. The expression has the same format and meaning as the expression builder in DFexplore. For a complete description of the available expression syntax refer to DFexplore User Guide, List ViewSearching Data Records SearchingDataRecordsSearching Data Records.

-ALL, -ANY

Specify one or more, simplified filter criteria. Each filter references fields from one plate and multiple filters may be combined to include fields from other plates. The result of these selection criteria is a set of subject IDs matching all of the criteria (-ALL) or at least one of the criteria (-ANY). That set of subject IDs is then used to further filter the output.

Each criterion must be specified using the notation:

plate#:$(fieldname) op value

where:

  • plate# is a plate number from the set of plate numbers defined for the study, followed by a : separator,

  • $(fieldname) is the name of a user data field defined for the plate number,

  • op is an operator from the set of operators: >, >=, ==, !=, <, <=, and

  • value is a single value consistent with the data type of the field (string and data values should be enclosed in "" to prevent interpretation by the command environment).

To specify multiple criteria use | as a delimiter. The -ANY and -ALL options specify the resulting match behavior: with the -ALL option, every criterion must evaluate to true for a match to occur; otherwise, at least one criterion must evaluate to true.

Example 3.20. Export eSignature records for eligible, african-american males

The SEX (1=male) and RACE (2=african-american) are defined on plate 1 while ELIG (1=yes) is defined on plate 2.

-Mname eSignature -ALL '1:$(SEX) == 1 | 1:$(RACE) == 2 | 2:$(ELIG) == 1'

The fields used in the criteria are, by default, not part of the output, nor are they required to be. In fact, unless the criteria are all from the same source, this is not possible with this option.

Extracting/Combining/Concatenating Fields for Output

Frequently, only a subset of the fields in the data source are of interest. These can be selected using one of the -Fnum, -Fname or -Falias options. Special handling of data fields that are defined in a module which is then instanced across more than one plate is required.

-Fnum #, #-# Select by field number. From each exported record, select for output only those fields requested by their number. Output fields can be re-ordered by specifying the field numbers in the desired order. Field numbers can be repeated. It is possible to request field numbers using NF notation which selects by field number relative to the last field of the record. Some examples of NF usage are:

  • 2-NF.  field numbers 2 to the last field inclusive

  • NF-1.  the second last field

  • 5-NF-1.  fields 5 to the second last field, inclusive

  • NF-5-NF-1.  fields fifth from the last to the second last field, inclusive

This option may not be combined with -Fname or -Falias - doing so will cause DFexport to exit with an error message.

-Falias fieldname_list, -Fname fieldname_list Select by field name or field alias. From each exported record, select for output only those fields requested by their name or alias. Output fields can be re-ordered by specifying the field names or aliases in the desired order. Field names/aliases can be repeated. This option may not be combined with -Fnum - doing so will cause DFexport to exit with an error message. Similarly, only one of -Falias or -Fname may be specified - any other specification or combination will cause DFexport to exit with an error message.

-joinANY, -joinALL The default behavior of DFexport is to export data from one source, a plate or a module. However special handling is needed in the cases where a module instance is distributed across multiple plates. With the -joinALL and -joinANY options it is possible to combine and export data from a data source that is defined across plates in one invocation and into one output file. The join keys (the "join key triple") are always the subject ID, the sequence number and the module reference instance number and must match across data sources. These keys are always exported as the first three fields in each output record when joining occurs. Multiple data sources are specified using | as a delimiter. The specification for a single data source uses the notation:

plate#[fieldlist]

where:

  • plate# is a plate number from the set of plate numbers defined for the study,

  • fieldlist is a list of one or more comma-separated field numbers or names, enclosed with [ and ].

The -joinANY and -joinALL options specify the resulting output behavior: with the -joinALL option, the database must contain a data record for every data source; otherwise, no data is output for that join key triple. Conversely, specifying -joinANY will export data so long as the join key triple appears in at least one data source.

Example 3.21. Re-construct the demographics module for output

The DEMOGRAPHICS module is instanced with fields on plates 1 and 2. Export the data, with a specific field ordering and some output formatting.

-Mname DEMOGRAPHICS -joinANY '1[PINIT, SEX] | 2[RACE, RACE:d, RACEOTH] | 1[AGE, DOB:c]'


Concatenation with + By default, each selected field is output with optional formatting (see the section called “Output Formatting” and is separated from adjacent selected fields by the delimiter. Alternatively, selected fields can be concatenated, output with no delimiter at all. To specify concatenation, use the special concatenation symbol, +, between fields to concatenate. For example, to concatenate the site identifier and the subject identifier, use

DFSITE_ID+SUBJID

or to concatenate fields 9 and 12, use

9+12

It is also possible to concatenate a range of fields. In this case, concatenation has precedence over field range so,

8+9-12

or

8-11+12

or

8+9+10+11+12

are all equivalent. Note that there is no notation to concatenate a range of fields by specifying only the minimum and maximum field numbers; at least one of the numbers must be inidividually specified so that the concatenation operator can be used.

Including Metadata in Output

Study metadata is available for export as columns in each output record. Metadata is specified for inclusion in the output by inserting metadata keywords in the list of fields for export. In all cases, the exported value has the string data type.

The available metadata keywords are grouped by 6 categories: field, page, visit, image, site and study.

  1. field.  These metadata keywords reference field definition properties of a data field. For these keywords only, the keyword is appended to a field name to request a specific property of that specific data field. For example, MHENDAT.DFVAR_PROMPT requests the DFVAR_PROMPT property (the field prompt from the study definition) of the MHENDAT data field. User-defined

    The field properties that are available are:

    • DFVAR_DESC: field description

    • DFVAR_TYPE: field type

    • DFVAR_PROMPT: field prompt

    • DFVAR_UNITS: field units

    • DFVAR_COMMENT: field comment

    • DFVAR_MODNUM: module instance number containing field (Module Instance in setup definition)

    • DFVAR_MODNAME: module name containing field (Module Name in setup definition)

    • DFVAR_MODDESC: module description containing field (Module Description in setup definition)

    • DFVAR_USER#: field user-defined property value, where # is 1-20. User-defined property tags may be used in place of the default name

  2. module.  These metadata keywords reference properties of the module instance.

    • DFMODULE_NAME: module name (Module Name in setup definition)

    • DFMODULE_DESC: module description (Module Description in setup definition)

    • DFMODULE_USER#: module user-defined property value, where # is 1-20. User-defined property tags may be used in place of the default name

  3. plate.  Metadata keywords for a plate include DFPLATE_DESC (plate description) and DFPLATE_USER# (plate user-defined property value, where # is 1-20. User-defined property tags may be used in place of the default name).

  4. page.  These metadata keywords reference page properties of the plate associated with the current data record. There is currently one page property available: DFPAGE_LABEL. The value is the descriptive label of the page. This label is taken from DFpage_map if there is a matching entry for the combination of visit and plate (field number substitution is also performed if requested with the # or #:d notation); otherwise, it is the plate description taken from the study database definition.

  5. visit.  These metadata keywords reference visit properties of the visit associated with the current visit. The current visit is determined by the visit number of the data record, even if the visit number is not included in the export. The visit properties that are available are: DFVISIT_DATE, DFVISIT_TYPE, DFVISIT_ACRONYM, DFVISIT_LABEL, DFVISIT_DUE, DFVISIT_OVERDUE, DFVISIT_REQUIREDPLATES, DFVISIT_OPTIONALPLATES, DFVISIT_ORDERPLATES and DFVISIT_MISSEDPLATE. The value of each property is detailed in Study Setup User Guide, Visit Map.

  6. image.  Each data record has an image attribute, which will have a value if there is an image associated with the data record. For those records, the image properties that are available are:

    • DFIMAGE_ARRIVAL: arrival date/time of the image

    • DFIMAGE_FIRSTARRIVAL: arrival date/time of the image; if there were multiple images over time, the chronologically first image

    • DFIMAGE_LASTARRIVAL: arrival date/time of the image; if there were multiple images over time, the chronologically last (most recent) image

    • DFIMAGE_FORMAT: image format

    • DFIMAGE_SENDER: sender ID for the document that included this image

    • DFIMAGE_PAGES: number of pages in the document that included this image

  7. site.  The site metadata associated with each data record can be determined by connecting the subject ID of the data record with the site record that includes the subject ID in the list of subjects. The site properies that are available are: DFSITE_ID, DFSITE_NAME, DFSITE_CONTACT, DFSITE_ADDRESS, DFSITE_FAX, DFSITE_PHONE, DFSITE_INVESTIGATOR, DFSITE_SUBJECTS, DFSITE_TEST, DFSITE_COUNTRY, DFSITE_BEGINDATE, DFSITE_ENDDATE, DFSITE_ENROLL, DFSITE_PROTOCOL1, DFSITE_PROTOCOLDATE1, DFSITE_PROTOCOL2, DFSITE_PROTOCOLDATE2, DFSITE_PROTOCOL3, DFSITE_PROTOCOLDATE3, DFSITE_PROTOCOL4, DFSITE_PROTOCOLDATE4, DFSITE_PROTOCOL5, DFSITE_PROTOCOLDATE5 and DFSITE_REPLYTO. The value of each property is detailed in Programmer Guide, DFcenters - sites database.

  8. study.  The study metadata is static for the entire study, it is defined at the study database level and does not change for different data record keys. The study properies that are available are: DFSTUDY_NUMBER (study number), DFSTUDY_NAME (study name) and DFSTUDY_YEAR (4-digit year of study start, defined by study admin as a global setting). In addition, DFSTUDY_USER# provides the study user-defined property value, where # is 1-20. User-defined property tags may be used in place of the default name.

Output Formatting

Date Modifiers.  By default, date fields are output "as is" in their defined format. The options -c and -j may be used to cause imputation and to alter/normalize that defined format for all date fields that are exported. The -c (calendar format) option requests that any 2-digit year be replaced with a 4-digit year and imputation be applied. The -j (julian format) option requests that imputation be applied and dates be exported as their equivalent julian value - this is primarily useful for mathematical calculations and comparisons with other date values. On an individual field basis, it is also possible to override the defined format with field level date modifiers. By following the field number, name or alias specification with a :c or :j modifier, it is possible to export the same calendar or julian format result for a single field. The :o modifier requests the "original" date value, and can be useful where -c or -j has been previously specified. DateField and DateField:o produce the same result. Similarly, DateField:c also produces the same result if the field's format already specifies 4-digit years and no imputation is needed.

Example 3.22. Calendar and Julian Date Modifiers

For the field named VisitDate, export the value in default, original, calendar and julian formats.

-Fname "VisitDate, VisitDate:o VisitDate:c, VisitDate:j"

If VisitDate is defined with "imputation to the beginning" and an instance of that field in a data record has the data value 16/01/00, this specification will produce 16/01/00|16/01/00|2016/01/01|2456963.


A field level date modifier cannot be applied to a non-date field nor can it be applied to a range of field numbers, names or aliases; it may only be applied to a single field number, name or alias at a time.

Label Decoding.  For any field that is defined with coding, it is possible to output the decoded label for a field's value by appending the :d modifier to the field number, name or alias.

Example 3.23. Output coded and decoded values

-Falias "sex, sex:d"
2|female


Any coded value that cannot be decoded is output as is. The modifier cannot be applied to a field that has no coding nor can it be applied to a range of field numbers, names or aliases; it must be applied to a single field number, name or alias at a time.

Header Record.  Specifying -h forces the inclusion of a descriptive record, the header, as the first row of output. If -h is specified with -D, the header contains each field's description. Otherwise, if -h is specified with -Fname, the header contains each field alias. If -h is combined with -Fnum, or no field selection criteria are needed:

  • the header contains each field alias if the data source is a plate, as specified with -Pnum

  • the header contains each field name if the data source is a module, as specified with -Mname or -Mnum.

During export it is possible to create one or more new data fields by including options to string split (see String Splitting), extract substrings (see Extracting Sub-Strings) or include fixed, literal values (see Literal, constant values). If -h is specified then these new fields also need names to appear in the header record. This is done with the -H header_list option. This option specifies the variable names to appear in the header record for any new fields. If header_list contains fewer names than there are new fields, the remaining fields are assigned names identical to their original field names.

String Splitting.  It is possible to split a string field into multiple, shorter string fields by appending a :numxcharscw modifier to a field number, name or alias selection. The modifier includes a specification of the number of fields to split the input string into (num), the literal x as a separator, the maximum width in characters of each output field (chars), and whether or not splitting should be done on character (c) boundaries or word (w) boundaries. If word boundaries are used, a word is delimited by any combination of space or tab characters. Word splitting will always split at the word boundary closest to but less than the maximum width. If there is no word boundary in the substring, the string will be split at the character boundary.

Example 3.24. Split a string field into 5 fields

This example splits the comments field into 5 200-character fields at character boundaries:

-Fname "comments:5x200c"


If a field to be split contains a missing value code, the first output field will contain the same missing value code and the remaining fields will be blank. If a field is split to create additional fields and a header record is requested with -h, field names for the new fields are identical to the original field name unless -H is also specified. If -H header_list is given, the field names for the new fields are assigned in order from the header_list.

Extracting Sub-Strings.  New fields can be created from substrings while exporting fields. The modifier notation :xS.L indicates that a substring is being extracted (x), starting at character position S (the first position is 1) and having length L characters. The modifier can be applied to any field number, name or alias selection.

New fields can be created from substrings while exporting fields. The modifier notation :xS.L indicates that a substring is being extracted (x), starting at character position S (the first position is 1) and having length L characters. The modifier can be applied to any field number, name or alias selection. Substrings can be extracted from any field type: string, date, time or numeric. Substrings are extracted from the string value equivalent of the field. Numeric field values are leading zero-padded to their store width before substring extraction is performed. [5]

Example 3.25. Digit extraction from subject ID

In some settings, the subject ID is a concatenation of site ID and a numeric subject identifier. In this example, the leading three digits are the site ID and the trailing four digits. Given PID values of 50001, 60401 and 1232003, the extraction would yield these results.

-Fname "PID:x1.3,PID:x4.4"
005|0001
006|0401
123|2003


Literal, constant values.  A constant value may be inserted into the data export by including it in one of the -Fnum, -Fname or -Falias specifications. The value must always be enclosed with single-quotes, as in 'AB', to prevent any confusion with a matching field name or alias. To insert a blank field into the export, it must be represented by 2 adjacent single quotes, specifically ''.

Example 3.26. Insert AD into the data export

-Falias "EnrollDate,'AD',SubjInit"
2014/12/12|AD|STN
2013/06/15|AD|PRP
2015/03/12|AD|KKW

The constant value AD is inserted between the EnrollDate and SubjInit fields in every exported record.


Enclosed by single-quotes, DFexport ignores the special meaning of any characters that would otherwise be delimiters. For example, 'a,b' is the constant value a,b, not two separate constants. If -h is present, the name used in the header record is the value itself. If -H header_list is also present, the name used in the header record for each constant value will be the next value from header_list, if one is available; otherwise, the value itself will again be used.

Example 3.27. Specify a field name for the header record

-h -H "When" -Falias "EnrollDate,'AD',SubjInit"
EnrollDate|When|SubjInit
2014/12/12|AD|STN
2013/06/15|AD|PRP
2015/03/12|AD|KKW


Trailing Field Delimiter.  For backwards compatibility, exported data records can be terminated by a trailing | if one is not already present. This is specified with the -p option. [6] With the trailing delimiter, exported data records maintain the original DFdiscover record format so that they can be, for example, easily re-imported into DFdiscover in a subsequent step.

Missed Record Handling.  Missed records are include in the export if either -status all or -status missed are specified. To export missed records, there are two output options:

  1. Internal to DFdiscover, missed records are handled in a different structure than other data records (see plt###.dat - missed records and Table 2.7, “Missed record field descriptions” for further information). Exporting them in this structure, when intermingled with plate- or module-based data records, does not result in normalized records - most records have the data structure, while some have the missed record structure. The result is that these records can be difficult to work with post-export.

  2. By specifying the -L lostcode option, DFexport will export missed records by matching the structure of the corresponding plate and inserting lostcode into all of the user-defined data fields. This results in exported records that are normalized and all share the same structure. Post-export, missed records can be identified by the missed status in the status field, or the lostcode in all of the user-defined data fields.

When missed records are exported along with one of the field selection options -Fnum, -Fname, or -Falias, the second output option is always used. If the option -L lostcode has not been specified, the standard DFdiscover missing value code (*) is automatically inserted in each user-defined field of the exported record.

The lostcode specified with -L must be comprised of one or more alphanumeric characters. It is not permissible to use control characters. Also, the DFdiscover delimiter (|) may not be used, unless data is also being exported in CSV format with the -z option. Note that specifying the -L lostcode option will not, by itself, cause missed records to be exported. That is controlled entirely by the -status option. The -L option only indicates that the second, "normalized" output format is desired and is to be used with the specified substitution code.

Output Options

Output File.  By default, output from DFexport is written to standard output (the command-line output). To write the output to a specific file, specify the filename with -o outfile. The user must have filesystem permission to create or modify the file. If outfile is specified as a relative pathname, the file is created relative to the DFexport command invocation directory.[7] It is also possible to be explicit that standard output is the output destination by specifying -o - although this is not necessary.

Output Mode.  Options -w and -a control the output mode. The output is written (-w) or appended (-a) to the output file. This option is ignored if exported records are written to standard output rather than a file. If the output file does not exist, it is first created. If the output file already exists, it is either overwritten or appended to, depending upon the output mode.

Error Output.  Re-direct any error messages to a specific file with the -e errorfile option. By default, error messages are written to standard error and hence would get intermixed with data if the data is being exported to standard output.

Comma-Separated Values (CSV) Format.  CSV is a popular format used for sharing data records between different software programs. By including the -z option, DFexport will generate output that is compliant with this format. The unique requirements of the format are:

  • Each field within a record is separated by a comma.

  • Leading and trailing spaces adjacent to comma separators are ignored.

  • Fields containing commas as part of their value are enclosed in double quotes.

  • Fields containing double quotes are enclosed within double quotes, and the embedded double quotes themselves are each represented by a pair of consecutive double quotes.

  • Fields with leading or trailing spaces are enclosed in double quotes.

  • The record delimiter is a newline character (this is also true of records exported in the default, non-CSV format).

Exported records, in CSV or traditional (non-CSV), format are always delimited by a newline character.

If the -h option is also specified, the CSV requirements are applied to the header record and all of the values in the header.

Exit Status

DFexport exits with status 0 if the command was successful. Exit status 1 indicates that an error occurred - errors are written to standard error or the errlog file if -e was specified.

Examples

In the following examples, only the options unique to the example are specified. For clarity, required options such as authentication credentials, output destination and study number are omitted.

Example 3.28. Select and filter using data from two different plates

Export complete data records from plate 6. Filter those records to include only instances where records with matching keys on plate 1 have a date variable, S1DATE with a value of 01/01/06.

-Pnum 6 -ANY "1:$(S1DATE) == 01/01/06"


Example 3.29. Module-based export

The structure of data exported from a module varies dependent upon whether all of the fields are instanced on a single plate, or on multiple plates.

Consider a study containing a module, BASE, that collects baseline data in fields HEIGHT, WEIGHT, the data across two records as in:

-Mname BASE
73|180||
||120|70
65|145||
||110|65

To generate the previous single record output requires a little more work, specifically:

-Mname BASE -joinALL "1[HEIGHT,WEIGHT] | 2[SYSTOLIC,DIASTOLIC]"
73|180|120|70
65|145|110|65


Example 3.30. Export eSignature records with a specific name

The eSignature name is stored in the eSignature module. Export records signed by 'Jack'. Several specifications are possible as illustrated here.

-Mname eSignature -expr '$(SigName) == "Jack"'

This is the most specific filter - the signature name field must contain exactly "Jack" and nothing else.

-Mname eSignature -expr 'tolower($(SigName)) ~ "jack"'

This is a less specific filter - the signature name field can contain a case insensitive variation of "Jack" somewhere in the value. Note that this also selects values like "Jackson", "Alex Jack" and "Wojack".

-Mname eSignature -pattern "jack"

This is the least specific filter and selects each eSignature module record that contains a case insensitive match for "jack" anywhere in the record.




[1] This output format matches the output from DFlistplates.rpc.

[2] DFexport typically uses a word to specify an option while DFexport.rpc uses an option letter.

[3] For backwards compatibility, the legacy status keywords (clean, dirty, error, CLEAN, DIRTY, ERROR, missed, primary, secondary, all) are currently supported but will be removed in a future release. Do not rely upon the availability of the legacy status keywords.

[4] Matching occurs before any output formatting (such as variable decoding or string splitting) is applied.

[5] Choice and check field values are NOT zero-padded and hence substring extraction for those data types may yield unexpected or unreliable results.

[6] This option has no effect when applied to exported query or reason records.

[7] If DFexport is invoked from a cron facility, absolute pathnames are highly recommended.


DFexport.rpc

DFexport.rpc — Export data records from one or multiple plates from a study data file

Synopsis

DFexport.rpc [ [-w] | [-a] ] [ [-c] | [-j] ] [-d] [-e] [-J] [-m] [-p] [-q] [-h] [-k] [-z] [-s status_list] [-v #, #-#] [-n #, #-#] [-I #, #-#] [-V #, #-#] [-C yy/mm/dd-yy/mm/dd] [-M yy/mm/dd-yy/mm/dd] [-L lostcode] [ [-U fieldname_list] | [-G fieldname_list] ] [-f fieldnum_list] [-H header_list] {study} {plate(s)} {outfile}

Description

DFexport.rpc can be used to export whole data records or selected fields, from specified plates of a specified study database. Data records are exported in ASCII text format. DFexport.rpc does not provide support for joining fields from different plates or studies. To use DFexport.rpc users must have 'Export Data' permission, which is granted by an administrator on a study-by-study basis (see System Administrator Guide, Server). Users with export permission will only receive records for which they have get permission, which may be restricted by level, site, subject, assessment and plate. DFexport.rpc provides no warning that some records cannot be exported as this would itself provide information about the existence of restricted data records.

DFexport.rpc may appear in a shell script used to create reports stored in the study reports directory, to be run in DFexplore from the Reports View, or in a shell script run using dfexecute in an edit check. In these cases the permissions of the DFexplore user running the report or edit check will be applied. Thus the behavior and output may differ depending on the permissions of the user who runs the script.

If the same script is run from the UNIX shell, the permissions associated with the UNIX login name apply. Thus, if a user has different UNIX and DFexplore login accounts with different permissions, the user may get different results depending on whether they run the script in the UNIX shell or DFexplore environments.

Options

-w, -a

output mode. The output is written (-w) to or appended (-a) to the output file. If the output file does not exist, it is created and written to for either output mode. If the output file already exists, it is overwritten with the -w mode or appended to with the -a mode. The output mode is ignored if the output file is standard output (indicated by -).

-c, -j

default date format. Without this option, date values are output in the format specified by their variable definition, and for partial date values, without any date imputation. With this option, date values are output in calendar (-c) format or julian (-j) format. In calendar format, the existing variable format is preserved except that 4-digit years are substituted for occurrences of 2-digit years. If the date value is not present, a blank value, "", is output, or if the date value cannot be imputed, the value * is output. In julian format, the date value is output as the number of days since a fixed point in the past. The fixed point in the past is the same for all invocations of the command allowing for date arithmetic. If the date value is not present, the value 0 is output, or if the date value is illegal, the value -1 is output. In both types of formatting, date rounding is also performed as specified by the variable definition.

-d

decode coded variables. With this option, all requested variables that are coded are output with their decoded labels. The default behavior, without this option, is to output the code itself. This option can be invoked at the individual variable level by appending the :d qualifier to the variable.

-e

outfile extension. With this option, the output filename specified will have a text file extension (.txt) added to the end of the user defined outfile name. Outfile extension is ignored if the output file is standard output indicated by '-'. The outfile extension is also ignored if only a single plate is specified. In the case of a single plate, the outfile name will be exactly as specified.

If this option is used along with the csv (-z) option, the csv extension (.csv) is added to the end of the specified output instead of the text extension (.txt).

-J

Outer join with requested fields that may not exist for current plate. With this option, if the user specifies one or more fields that do not exist for a plate that is requested, data for that plate will still be exported but non-existent fields will have a data field containing the missed subsitution code and a header containing "NOT_DEFINED". If this option is omitted, plates that do not contain one or more of the fields requested will not be exported.

-m

match data record validation level. This option is only relevant when exporting metadata records: reasons (plate 510) and queries (plate 511); and is ignored when exporting data records.

Meta-data records may have different validation levels from the underlying data records to which they are attached. When metadata records are exported using the -m option, the validation level of each exported metadata record is changed to match the validation level of the data record to which it is attached.

-p

trailing pipe. For backwards compatibility, exported data records can be terminated by a trailing |, if one is not already present. This allows exported records to maintain the original DFdiscover record format so that they can be, for example, easily imported. A trailing pipe is only needed to make data records compatible.

[Important]Important

DFdiscover does not expect a trailing pipe to be present in query records (plate 511) or reason for change records (plate 510). Hence, the -p option should not be used when exporting query records or data change records if the intention is to re-import them into the database using DFimport.rpc.

-q

quiet mode. This option instructs the program to execute in quiet mode, silencing all warning messages. The default, without this option, is to write warning messages to standard error. Warning messages are generated upon a request to export a field that does not exist in the record.

-h

header. Include as the first record in the output file, a delimited record of variable names for the columns in the data records. The variable names are, by default, the alias variable names defined in the study data dictionary combined with -H. If fields are extracted using -G, the header contains the variable names combined with -H.

This option cannot be used with an export of the new record queue, plate 0, as the variable names are not fixed. It can however, be used with the export of query records (plate 511) and reason for data change records (plate 510). The headers for query and reason for data change records are defined in the study schema.

The trailing character is different in data records and queries. Data records are terminated by a final | while queries are not. This also applies to the header records. Thus, when exporting data records the header record is terminated by a trailing | while this is omitted when exporting queries.

If multiple plates are requested and the -h option is used, a warning message will appear if the output is written to standard output. The warning message will notify users that the per plate headers will be interleaved with multiple plate output data.

-k

keys only. Output only the key fields for each data record. This includes, in order: id, plate, visit, status, and validation level, as | delimited fields. This option is a shorthand notation for the equivalent option -f "7,5,6,1,2". This option cannot be used in conjunction with -f, -G, or -U.

-z

Comma Separated Variables (CSV) format. This option exports all records so that they are compliant with this popular format. If this option is used in conjunction with the file extension option, the extension added to the end of the outfile name is the csv extension (.csv).

Required Options

The following three options are required and must appear in order at the end of the option list:

study

the DFdiscover study number, from which data records are to be exported.

plate(s) #, #-# all

plate numbers of the data files to export. A single plate can be specified or a list of plates can be specified to run in one invocation. If the same plate is listed more than once, the plate data is only exported once. The keyword 'all' can be used to export all existing plates within the specified study including the special reserved plates.

Special reserved plate numbers include: 0 for new records (received but not yet reviewed), 501 for returned Query reports, 510 for reason for data change records, and 511 for queries. If plate 511 is exported, the validation level of all exported records is updated to reflect the current validation level of the record that the notes are attached to.

outfile

output file that the exported records are written to. The user executing DFexport.rpc must have permission to create or modify this file. In the case of multiple plates being exported, the outfile name is used as the base filename from which a unique filename is constructed by appending each three digit, zero-padded plate number. If the output file is given as -, exported records are written to standard output rather than a file.[8]

Record Selection Options

The following options allow specific records to be selected from the output. With no options specified, all records are output. With options specified, only records matching the selection criteria are output. If multiple options are specified, only those records that match all criteria are output.

-s status

record status. The recognized status keywords consist of the legacy terminology: clean, dirty, error, CLEAN, DIRTY, ERROR, missed, primary, secondary, all and the current terminology: final, incomplete, pending, missed, all. Any combination of legacy and current terminology can be given as a quoted string. The default is records of all statuses.

If plate 511 is exported, the available status keywords are the same but their meaning (as related to query status) is translated by the table:

Table 3.8. Record and query status equivalence

record statusdeletefinalincompletependingsecondary [a] secondary [b] secondary [c]
query statusdeletenewin unsent reportresolved NAresolved irrelevantresolved correctedin sent report

[a] For backwards compatibility, old keyword CLEAN is supported.

[b] For backwards compatibility, old keyword DIRTY is supported.

[c] For backwards compatibility, old keyword ERROR is supported.


-v #, #-#

validation level. By default, records at all validation levels are exported, independent of the maximum validation level defined for the user. Specifying this argument causes only those records that match a validation level or are within a validation level to be exported.

-I #, #-#

subject ID. If this option is specified, only those records which have a subject ID matching the selection criteria are output. It is not valid to select records using -I in the same invocation as -n (selection by site). If this is done, DFexportrpc; will exit with an error message.

-n #, #-#

site ID. This option allows selection by site ID, site range or any combination thereof. However, it is not valid to specify -n in the same invocation as -I (subject ID). If this is done, DFexport.rpc will exit with an error message.

-V #, #-#

visit/sequence number. Specifying this option selects records to output by the visit number. Only those records matching the visit selection criteria are output.

-C yy/mm/dd-yy/mm/dd

creation date. If this option is specified, only those records which have a creation date matching the selection criteria are output.

Note that the selection criteria apply only to the creation date of data records and not queries.

-M yy/mm/dd-yy/mm/dd

modification date. If this option is specified, only those records which have a modification date matching the selection criteria are output.

Note that the selection criteria apply only to the modification date of data records and not queries.

For both -C and -M, the term today can be used to represent today's date.

-L lostcode

missing value code for missed records.

Missed records are exported if the keywords 'missed', 'lost', or 'all' are specified with the status option, -s. Missed records can be exported in 2 different formats. The first format is comprised of the missed data fields (i.e. including the missed category code and the text field containing the reason the record was classified as missed). The second format is comprised of all of the user defined data fields that appear on the CRF for that plate. The first format is the default and will be used whenever missed records are requested and the -L option is not used. If missed records are requested and the -L option is specified, missed records are exported with the lostcode value inserted into each user data field after field 7 (subject ID) - the trailing 3 system fields are exported with status 0, and the missed record creation and modification timestamps. The lostcode will also be used if the user specified fields using the -f and/or -U|G options along with the -J option and if one or more of those fields do not exist.

The only exception to this is when missed records are requested along with one of the field selection options (-f, -G, or -U). In this case, the second format is always used. If a substitution code has not been specified with the -L option, the generic DFdiscover default missing value code (*) is used, and a warning message is written to standard error, each time the substitution is made.

The lostcode specified with the -L option may consist of one or more alphanumeric characters. It is not permissible to use control characters. Also, the DFdiscover delimiter (|) cannot be used, unless data is being exported in CSV format with the -z option.

Note that specifying the -L lostcode option will not, by itself, cause missed records to be exported. This decision is controlled entirely by the status option. The -L option only indicates that the second format is desired and is to be used with the specified substitution code.

[Note]Argument parsing

Argument parsing in DFexport.rpc ignores extra, repeated delimiters. Any combination of space and comma is collapsed to one delimiter. For example:

DFRASTER,,,DFVALID = DFRASTER,DFVALID
DFRASTER  DFVALID = DFRASTER,DFVALID
DFRASTER ,,  ,,DFVALID = DFRASTER,DFVALID
,,DFRASTER ,,  ,,DFVALID = DFRASTER,DFVALID

are equivalent.

Field (variable) Selection Criteria

The following options allow fields (variables) to be selected from the output. With no options specified, all fields are output, unless -k has already been specified. With options specified, only the requested fields are output. If multiple options are specified, all of the requested fields are output, in the order requested.

-f #, #-#

field number. Specifying this option selects for output from each record only those fields requested by their number. Output fields can be re-ordered by specifying the field numbers in the desired order. Fields can be repeated within the option. This option cannot be repeated in the command-line.

Command-line referencing of field numbers using NF (last field) or relative to NF is also possible. Some examples of NF usage are as follows:

  • 2-NF.  include fields 2 to the last field inclusive

  • NF-1.  include the second last field

  • 5-NF-1.  include fields 5 to the second last field, inclusive

  • NF-5-NF-1.  include fields fifth from the last to the second last field, inclusive

-U fieldname_list, -G fieldname_list

field name. Specifying this option selects for output from each record only those fields requested by their field name. If -U is used, fields are selected by their alias variable name. If -G is used, fields are selected by their variable name. Field names can be repeated within the option. These options cannot be mixed or repeated.

-H header_list

field name headers. This option specifies the variable names to appear in the header (-h must also be specified) for new fields that are created as a result of one or more split modifiers.

Wherever a field number or field name is expected in field selection criteria, a constant value may also be referenced by inserting it in single quotes, as in 'AB'. The purpose of this is to insert the constant value in the specified field location into the output records. The 'value' notation allows constant values to be exported in much the same way that a variable value can be. Since it is possible for a constant value and variable name to be the same, a constant value must always use the 'value' notation. For example, the specification:

-G "date,'AD',DFRASTER"

requests the variable with the name date, followed by the constant value AD and the raster image ID number. The output might appear as:

99/06/17|AD|9924/00001001

If a blank field is being exported, it must be represented by 2 single quotes with nothing inside.

DFexport.rpc will ignore the special meaning of any characters that would otherwise be delimiters. For example, 'a,b' is the constant value a,b, not two separate constants.

If -h is present, the column name for each constant value will be the value itself, as in the following specification and output:

% DFexport.rpc -h -G "date,'AD',DFRASTER" 254 1 - | head -2
date|AD|DFRASTER
99/06/17|AD|9924/0001001

If -H is also present, the column name for each constant value will be replaced with the next -H value, if one is available. For example:

% DFexport.rpc -h -G "date,'AD',DFRASTER" -H "when" 254 1 - | head -2
date|when|DFRASTER
99/06/17|AD|9924/0001001

Date Modifiers

By default, date fields are output in the format defined for the field in the study data dictionary. If -c or -j is specified, the default output format is changed to 4-digit year format with imputation (-c) or julian format (-j). It is possible to override the default format for selected fields by following the field number or field name specification with a :c, :j or :o modifier. For example, the specification: -G "VisitDate, VisitDate:c, VisitDate:j" requests the variable with name VisitDate to be output in its default format, in calendar date format, and finally in julian format. The :c modifier applied to a field that already uses a 4 digit year format applies date imputation only. When using -c or -j to change the default output format for dates, note that there is no field level modifier that restores the original date format defined in the study data dictionary.

The :o modifier can be used in an analogous fashion to :c and :j. The effect of the :o modifier will be to cause the original date value to be output without any imputation.

A modifier cannot be applied to a non-date variable nor can it be applied to a range of field numbers or field names; it must be applied to a single field number or field name at a time.

Variable Decoding

It is possible to output the decoded label for a variable's value by appending the :d modifier to any variable that is defined with coding.

Example 3.31. Output the coded and decoded values for the sex variable

-G "sex, sex:d"

Any coded value that cannot be decoded is output as is.

The modifier cannot be applied to a non-coded variable nor can it be applied to a range of field numbers or field names; it must be applied to a single field number or field name at a time.

String Splitting

It is possible to split a string field into multiple, shorter string fields by appending a :numxcharscw modifier to the field number or field name specification. The modifier includes a specification of the number of fields to split the input string into (num), the maximum width in characters of each output field (chars), and whether or not splitting should be done on character (c) boundaries or (w) boundaries. If word boundaries are used, a word is delimited by any combination of space or tab characters. Word splitting will always split at the word boundary closest to but less than the maximum width. If there is no word boundary in the substring, the string will be split at the character boundary.

Example 3.32. Split a string field into 5 fields

This example splits the comments field into 5 200-character segments at character boundaries:

-G "comments:5x200c"


If a field to be split contains a missing value code, the first output field will contain the complete missing value code and the remaining fields will be blank.

If a field is split to create additional fields and a header record is requested with -h, field names for the new fields are identical to the original field name unless -H is specified. If -H is given, the field names for the new fields are assigned in order from the option list. If the option list contains fewer names than there are new fields, the remaining fields are assigned names identical to their original field names.

Extracting Sub-Strings

New fields can be created consisting of substrings of database fields. The following example creates a data field from the 4th and 5th characters of field 22.

-f 22:x4.2

Substrings can be extracted from any field type: strings, dates, or numbers. Substrings are extracted from the string value of the field. Numeric fields will be zero-padded to their store width before the substring extraction is done. In the following example, the first three digits of the subject ID field are extracted.

-f 7:x1.3

Comma Separated Variables (CSV) Format

CSV is a popular format used for sharing data records between different software programs. By selecting the -z option, DFexportrpc; will generate output records that are compliant with this format. The requirements of the format are:

  • The record delimiter is a newline character (this is also true of records exported in the default, non-CSV format).

  • Each field within a record is separated by a comma.

  • Leading and trailing spaces adjacent to comma separators are ignored.

  • Fields containing commas as part of their value are enclosed in double quotes.

  • Fields containing double quotes are enclosed within double quotes, and the embedded double quotes themselves are each represented by a pair of consecutive double quotes.

  • Fields containing embedded line breaks are enclosed within double quotes.

  • Fields with leading or trailing spaces are enclosed in double quotes.

The first record in the CSV output file may consist of a header record containing field names. Each field in the header will also be comma-delimited and follow the requirements above.

Exit Status

DFexport.rpc exits with one of the following statuses:

0

The command was successful.

24

The user does not have the necessary permission to execute the command.

31

The requested plate does not exist in the database.

36

The required command-line arguments were not present or were incorrectly specified.

> 0

The command failed because the database server could not be contacted, or communication with the database server failed.

Examples

Example 3.33. Export all records from plate 1 of study 255 to standard output

% DFexport.rpc -s all -h 255 1 -
DFSTATUS|DFVALID|DFRASTER|DFSTUDY|DFPLATE|DFSEQ|PID|INIT|VDATE|DFSCREEN|DFCREATE|DFMODIFY|
1|1|9807/1234567|255|1|0|99001|SCL|98/01/25|1|98/02/10 12:34:12|98/02/12 12:34:12|
2|4|9811/0005001|255|1|1|99002|RRN|98/02/12|2|98/02/10 15:03:34|98/03/01 11:23:14|
5|2|9831/0004012|255|1|1|99002|RRN|98/12/12|2|98/07/02 13:45:20|98/07/05 09:21:44|
1|3|9809/0044002|255|1|0|99003|*|98/02/03|1|98/02/10 14:23:01|98/02/10 14:23:01|

Example 3.34. Export, with header, all primary records at validation levels 1-2 from plate 1 of study 255 to standard output

% DFexport.rpc -h -s primary -v "1-2" 255 1 -
DFSTATUS|DFVALID|DFRASTER|DFSTUDY|DFPLATE|DFSEQ|PID|INIT|VDATE|DFSCREEN|DFCREATE|DFMODIFY|
1|1|9807/1234567|255|1|0|99001|SCL|98/01/25|1|98/02/10 12:34:12|98/02/10 12:34:12|
1|3|9809/0044002|255|1|0|99003|*|98/02/03|1|98/02/10 14:23:01|98/02/10 14:23:01|

Example 3.35. Export the first 3 and the 7th fields from plate 1 to standard output

% DFexport.rpc -f "1-3,7" 255 1 -
1|1|9807/1234567|99001
2|4|9811/0005001|99002
5|2|9831/0004012|99002
1|3|9809/0044002|99003

Example 3.36. Export the VDATE date field in 4-digit year format and export the subject initials in 3 single-character fields. Include the header record and define names for the newly created fields

% DFexport.rpc -c -G "VDATE,INIT:3x1c" -H "middle,last" 255 1 -
VDATE|INIT|middle|last
1998/01/25|S|C|L
1998/02/12|R|R|N
1998/12/12|R|R|N
1998/02/03|*||

Example 3.37. Export the primary records for subject IDs 99001 and 99002 selecting the fields from DFSTUDY to VDATE inclusive

% DFexport.rpc -s primary -I "99001,99002" -G "DFSTUDY-VDATE" 255 1 -
255|1|0|99001|SCL|98/01/25
255|1|1|99002|RRN|98/02/12

Example 3.38. Show all forms of manipulation for a partial date value in the variable DateCompleted1 for study 251

% DFexport.rpc -j -G "DateCompleted1,DateCompleted1:c,DateCompleted1:o" 251 1 -
2450845|1998/02/01|98/02/00
2451297|1999/04/29|99/04/29

Example 3.39. Create a DFdiscover Retrieval File (ID99001_plate10.drf) for study 254, all primary records for plate 10, for subject ID 99001

% DFexport.rpc -f "7,6,5" -I 99001 254 10 ID9901_plate10.drf

The following is the contents of the file ID9901_plate10.drf.

99001|1|10
99001|2|10
99001|3|10
99001|6|10
99001|9|10
99001|12|10

Example 3.40. Export all primary records for plates 1-3, 7 and 8 to standard output

% DFexport.rpc -f "7,6,5,3" 254 "1-3,7,8" -
99001|0|1|0915/000T001
99003|0|1|0915R000S001
99004|0|1|0915/000V001
99001|1|2|0915/000T002
99004|1|2|0915/000V002
99001|1|3|0915/000T003
99004|1|3|0915/000V003
99005|1|3|0000/0000000
99001|30|7|0915/000T009
99004|30|7|0915/000V009
99001|51|8|0915/000T010

When exporting multiple plates in one invocation, the outputs are always sorted in ascending order of the plate numbers, for example if the plate argument is changed to "7,8,3-1", the same output will be seen.


Example 3.41. Export all primary records for all plates in a study and create separate text files for each set of plate data.

% DFexport.rpc -e -f "7,6,5,3" 254 all Study254_

The following are the output files created by the above command

Study254_000.txt
Study254_001.txt
Study254_002.txt
Study254_003.txt
Study254_004.txt
Study254_005.txt
Study254_006.txt
Study254_007.txt
Study254_008.txt
Study254_009.txt
Study254_010.txt
Study254_011.txt
Study254_020.txt
Study254_501.txt
Study254_510.txt
Study254_511.txt

As seen above, all plates are exported in one invocation of the command. Special reserved plates are also included when the keyword 'all' is specified for the plates argument. A three digit, zero padded, plate number is also appended to the end of the outfile name, and is optionally followed by a file extension.


Example 3.42. Export, in CSV format, a subset of fields for plate 3 of study 254 and write the results to standard output

% DFexport.rpc -s all -z -f 1-7,59,63-66 254 3 -
2,1,9807/0047003,254,3,1,99001,,0,0,"knee surgery, hip replacement",2
1,1,0347R0012001,254,3,1,99101,"""other"" surgery",1,1," carotid   endarterectomy  ",2

Note that field values containing commas (knee surgery, hip replacement) are enclosed within double quotes. Fields containing double quotes (such as "other" surgery) are enclosed within double quotes and the embedded double quotes themselves are represented by a pair of consecutive double quotes.


Example 3.43. Export only missed records for plate 1, inserting the missing value code "NA" into the fields of each missed record exported. Export the results to standard output.

% DFexport.rpc -s missed -L "NA" 254 1 -

0|7|0000/0000000|254|1|0|20100|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|0|2018/01/15 12:35:23|2018/01/15 12:35:23
0|7|0000/0000000|254|1|0|20101|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|NA|0|2018/01/15 12:35:23|2018/01/15 12:35:23

Note that missing value code insertion is only performed for user-defined fields after field 7 (subject ID).


Limitations

DFexport.rpc and DFexport are the recommended ways to export data records from the study database for subsequent examination or analysis. Never read the data files directly as they may contain old copies of records scheduled for removal.

Since DFexportrpc; may only be run server-side by an authenticated user, it does not enforce user permissions as strictly as DFexport does. For example, DFexportrpc; ignores the Hidden/Masked field property while exporting field values.



[8] Data written to standard output will also include "inline" header metadata if either of the -h or -x options are specified.


DFfaxq

DFfaxq — Display the members of the outgoing fax queue

Synopsis

DFfaxq [#, #-#] [username...]

Description

DFfaxq reports on the status of requested faxes, either by their fax IDs, or by all faxes owned by the user(s) specified. When invoked with no arguments, DFfaxq reports on all faxes in the queue. For each outgoing fax, DFfaxq reports the unique fax ID number (by which it is referred to when using DFfaxrm), the user name of the fax owner, the scheduled time of sending, the file to be sent, the current status of the entry in the queue, the number of attempts at sending, and the phone number (or email address) to be sent to.

In the case of a fax that is being sent to multiple recipients, the fax ID, owner, scheduled time, and file to be sent, are all displayed on the first line together with the status, attempt, and phone number/email address of the first recipient. For the second and subsequent recipients, only the status, attempt and phone number/email address are displayed. The possible status values are:

  • New: in queue and waiting for scheduled time to arrive

  • Sending: in process of being sent to recipient

  • Retry: in queue as previous tries have failed and waiting for retry delay to expire

  • Sent: successfully sent to recipient

  • Failed: failed and maximum number of retries was exceeded

If the recipient is an email address, the only possible status values are Sending or Sent. It is not possible to reliably track the delivery status of an email message and so the other statuses cannot be reported.

DFfaxq displays the queue information sorted by increasing fax ID#.

Options

#, #-#

report on the status of faxes with the requested fax id numbers.

username

report on the status of faxes owned by username. Multiple usernames can be specified.

Exit Status

DFfaxq exits with one of the following statuses:

0

The command was successful.

> 0

The required command-line arguments were not present or were incorrectly specified.

Examples

Example 3.44. Scheduled but not yet sent fax

% DFfaxq
FaxId Owner Scheduled Time  File       Status Try To
  307 usr1  Nov 30 16:18:23 /etc/fstab New    0   1234567

Example 3.45. In-progress fax to multiple recipients

Here is a queue entry for an in-progress fax to multiple recipients:

% DFfaxq
FaxId Owner Scheduled Time  File       Status Try To 
  202 root  Nov 30 16:18:23 /etc/magic New    0   5432198
                                       Sent   1   abc@hotmail.com
                                       Retry  1   9876543


See Also

DFfaxrm

DFfaxrm

DFfaxrm — Remove faxes from the outgoing fax queue

Synopsis

DFfaxrm { [-] | [FaxId#...] | [-a] }

Description

DFfaxrm removes faxes from the outgoing fax queue, that is, those faxes that have been scheduled for sending but have not yet been sent.

A specific fax can be removed by supplying its faxid#, which can be obtained using DFfaxq.

Example 3.46. Removing a fax by its faxid

% DFfaxq
FaxId Owner Scheduled Time  File       Status Try To 
  307 usr1  Nov 30 16:18:23 /etc/fstab New    0   1234567
  308 usr1  Nov 30 16:18:23 /etc/fstab New    0   555-1212
% DFfaxrm 307


DFfaxrm reports the names of any faxes it removes, and is silent if there are no applicable faxes to remove.

When DFfaxrm removes a queue entry, faxes that are scheduled to be sent (status New) or retried (status Retry) in the future will not be sent or retried. However, any entries that have a status Sending are currently being transmitted and cannot be aborted. These faxes may be transmitted successfully, but any success command arguments specified to DFsendfax will not be executed. If a Sending status transmission fails, it will not be retried if it has been removed.

Options

Exactly one of the options is required. It is an error to mix these options. It is also an error to not supply any option.

-

remove all faxes owned by the user executing the command. Fax ownership is determined by the user's login name on the machine where the fax was originally queued for sending.

faxid#

remove the fax specified by faxid#. Multiple faxid#s can be specified.

-a

remove all faxes from the outgoing fax queue. This option can only be invoked by users datafax and root.

Exit Status

DFfaxrm exits with one of the following statuses:

0

The command was successful.

> 0

The required command-line arguments were not present or were incorrectly specified.

Examples

Example 3.47. Removing a fax by its faxid

% DFfaxq
FaxId Owner Scheduled Time  File       Status Try To 
  314 usr1  Dec 02 12:04:07 /etc/magic New    0   1234567
                                       New    0   9876543
% DFfaxrm 314
STATUS  FaxId Owner File       Status Try To 
DELETED   314 usr1  /etc/magic New    0   1234567
                               New    0   9876543

See Also

DFfaxq

DFget

DFget — Get specified data fields from each record in an input file and write them to an output file

Synopsis

DFget [-w] [-i delim] [-o delim] [-f #] {#, #-#}

Description

DFget reads records from the standard input and writes records to the standard output. Each input record is assumed to contain one or more fields delimited by the input delimiter. For each input record DFget applies the field specification string to create a new output record that is written to the standard output.

The field specification is constructed from one or more field specifiers. Each field specifier can be a single field number, a range of field numbers or a string.

One or more constant string fields can be inserted in output records. String fields can contain numbers and characters, and can also contain blank spaces. If the constant string field is a number (or begins with a number) it must be enclosed in a pair of single-double quotes (e.g. '"55"') or double-single quotes (e.g. "'55'"). This prevents DFget from interpreting those numeric strings as field specifiers.

Only those fields included in the field specification will be written to the new output record. Fields in the output record are delimited by the output delimiter and are written in the order given in the field specification.

The field number NF can be used to reference the last field in a record, and NF-n can be used to reference the nth from last field in a record.

DFget is a convenient way to extract fields from exported database records. Remember to first export the desired data files using DFexport.rpc.

Combining the tools DFexport.rpc, DFget, and DFimport.rpc is a powerful way to manipulate data.

Options

-w

warn about records that are too short to contain one or more of the requested fields. By default, no warnings are written and the requested fields that are not available are filled with blanks.

-i delim

the specified character is the field delimiter in the input records. The default is |.

-o delim

use the specified character as the field delimiter in the output records. The default is |.

-f #

skip any records that do not contain the specified number of fields.

#, #-#

the fields to retrieve from each record, and any constant values that are to be inserted (required).

Exit Status

DFget exits with one of the following statuses:

0

The command was successful.

36

The required command-line arguments were not present or were incorrectly specified.

Examples

Example 3.48. Retrieving specified fields from input records

Consider the following two line input file:

102|2|2|0101|gh|2|mellaril|t|100|1|90/03/01|0|
102|2|9|0101|bc|2|noctec|c|500|1|90/07/12|0|

To retrieve fields 1, 4 through 7, a constant string and the last 2 fields from each record, with a colon as the output delimiter, DFget could be used as follows:

% DFget -i '|' -o ':' 1 4~7 "'12 gm'" NF-1 NF < infile
102:0101:gh:2:mellaril:12 gm:90/03/01:0
102:0101:bc:2:noctec:12 gm:90/07/12:0


See Also

DFexport.rpc

DFgetparam.rpc

DFgetparam.rpc — Retrieve and evaluate the value of the requested configuration parameter

Synopsis

DFgetparam.rpc [-n] [-s #] {param}

Description

DFgetparam.rpc is the recommended way of determining the value of the master daemon's or a study server's configuration parameter. The current value of the parameter is printed on the standard output without a trailing new line character, unless -n is given. This is a format suitable for command line substitution by the shell.

When DFgetparam.rpc is running as user datafax, if the environment variable DFUSER is set, DFgetparam.rpc runs as that user. Thus for example requesting USER_PERMISSION will get the permissions associated with DFUSER, not the permissions for user datafax. This is important in DFexplore for shell scripts run by dfexecute in edit checks, and in study reports run under the Reports View.

The following configuration parameters can be retrieved from the master configuration:

AUTO_SLAVES

hostnames on which slaves are started when DFdiscover is started

MAILEE

email address for delivery of important system messages

PRINTER

default printer for DFadmin application

PASSWORD_RULES

a tuple of values in the following format: length,complexity,expiry,reuse,lockout,failures,email,reset. The meaning of these values can be found in System Administrator Guide, DFmasterDFmaster.cf

ROUTER_USERS

list of users that have permission to use the router

The following parameters can be retrieved from a study configuration:

AUTO_LOGOUT

a compound value containing two numbers that represent, in minutes, the minimum and maximum settable automatic logout intervals

SITES

database of participating clinical sites (CENTERS has been deprecated)

DATABASE_DIR

study database directory

FILE_MAP

lists all plates defined for the study

PAGE_DIR

CRF page images root directory

PRINTER

default printer for the study and all study applications

REPORTS_DIR

study reports directory

SCHEMA

study database schema

SETUP

study setup defintion (JSON format)

STUDY_DIR

study home directory

STUDY_HINTS

information for ICR to locate data fields

STUDY_NAME

descriptive study title, or acronym

STUDY_NUMBER

DFdiscover study number

USER_PERMISSION

the restrictions, if any, on which records the current user can access. The output is:

  • the word unrestricted if the user has no access restrictions, or

  • one or more concatenated 5-tuples describing the access restrictions. In each | delimited 5-tuple, the ordered fields identify the restrictions: 1st field is site ID, 2nd is subject ID, 3rd is visit number, 4th is plate number, and 5th is validation level.

VISIT_MAP

subject assessment schedule

WORKING_DIR

study work directory

Options

-n

append a newline character to the end of the output. The default is to not emit a trailing newline.

-s #

the DFdiscover study number. If this argument is missing, the parameter request is made of the master daemon's configuration.

param

the name of the configuration parameter to be retrieved and evaluated (required).

Exit Status

DFgetparam.rpc exits with one of the following statuses:

0

The command was successful.

36

The required command-line arguments were not present or were incorrectly specified.

> 0

The command failed because the master/server could not be contacted, or communication with the master/server failed.

Examples

Example 3.49. Determine the working directory for study 123

In this case, the value is assigned to a new shell variable, work.

% work=`DFgetparam.rpc -s 123 WORKING_DIR`


Example 3.50. Echo the value of the default printer for the master daemon

% echo `DFgetparam.rpc PRINTER`

Example 3.51. Determine the access permissions for study 253 for the current user

%  DFgetparam.rpc -s 253 USER_PERMISSION
1-10|||||||0,1||||||1-44,46-90||

In this example, 3 access permission rules apply to the user:

  • 1-10|||||: all records for sites 1 through 10, inclusive

  • ||0,1|||: all records for visits 0 or 1

  • ||2|1-44,46-90||: all records for plates 1 through 44 and 46 through 90, inclusive, at visit 2

The user will be granted access to data records that meet at least one of these permission rules.



DFhostid

DFhostid — Display the unique DFdiscover host identifier of the system

Synopsis

DFhostid

Description

DFhostid prints the unique DFdiscover host identifier of the system on the standard output. The output is a 20-character/digit identifier displayed in 5 blocks of 4. Note that the identifier will never contain any of 0 (zero), 1 (one), I (capital i), or O (capital o).

The host identifier of the DFdiscover master machine is required for licensing purposes.

Options

None.

Exit Status

DFhostid always exits with status 0, indicating that the command was successful.

Examples

Example 3.52. Determine the DFdiscover host identifier

% DFhostid
SG8D-QM2L-V8TK-PFB5-DDEG


DFimageio

DFimageio — Request a study CRF image from the database

Synopsis

DFimageio [-hd] [-f string] {-s #} {imageID}

Description

DFimageio fetches a specific, single CRF image from the requested study database and writes it in its native format to a file, or standard output.

The study number and the unique CRF image identifier are required arguments. The CRF image identifier must be one of:

  • a database image identifier, written in the YYWW/SSSSPPP notation,

  • a DFsetup background identifier, written using the notation $plt###.png [9] where ### is the plate number, or

  • an DFexplore background identifier, written using the notation $DFbkgd###.png [9] where ### is the plate number

Study CRF images are stored in the study file system and hence can also be accessed using standard shell commands. However, shell commands will fail to access the CRF image if:

  • the CRF image resides on a file system which is not visible to the current computer

  • filesystem permissions prevent the file from being read

  • the CRF image was previously deleted when the database record which it references was deleted

DFimageio is the correct, accepted method for accessing CRF image contents from the study database. Future releases of DFdiscover will further tighten the permissions on the CRF image files so that DFimageio will be the only method for retrieving them reliably.

Options

-hd

Request the HD version of the database image. If available, the HD version is returned; if not available, or if HD is not requested, the standard definition image is returned. This is applicable to database images only - the DFsetup and DFexplore backgrounds are available in only one definition. [10]

-f string

write the retrieved CRF image to the named file. Without this option, the retrieved CRF image is written to standard output where it should be re-directed using shell syntax.

-s #

the study number (required).

imageID

the unique identifier for the CRF image (required).

Exit Status

DFimageio exits with one of the following statuses:

0

The command was successful.

36

The required command-line arguments were not present or were incorrectly specified.

1

The requested CRF image could not be located using the supplied image identifier.

Examples

Example 3.53. Get and write a requested CRF image to a file

% DFimageio -f image1 -s 254 1525/0008001

The CRF image with the unique identifier 1525/0008001 is written to the file named image1 in the current directory.


Example 3.54. Get and re-direct a requested CRF image to a file

% DFimageio -s 254 1525/0008001 > image1

The CRF image with the unique identifier 1525/0008001 is written to the file named image1 in the current directory. The result is identical to that in the previous example but uses shell re-direction to achieve it.


Example 3.55. Get and write a DFsetup background image to a file

% DFimageio -f image1 -s 254 \$plt001.png

The DFsetup background image with the unique identifier plt001 is written to the file named image1 in the current directory.




[9] The $ character may need to be escaped with a backslash to avoid interpretation as a shell variable.

[10] Combining -hd with $plt###.png or $DFbkgd###.png will silently ignore the -hd argument.


DFimport.rpc

DFimport.rpc — Import database records to a study database from an ASCII text file

Synopsis

DFimport.rpc [-v] [-N] [-R] [ [-n] | [-q] | [-c] ] { [-a] | [-m] | [-r] } [-s] {#} {string}

Description

DFimport.rpc imports records from the input file into a DFdiscover study database. To use DFimport.rpc users must have permission to import data, defined by the study administrator in role permissions. To import new records (using the -a or -m options) users must have create permission and to replace existing records (using the -r or -m options) users must have modify permission. Remember that create and modify permission may be specified by level, site, subject, assessment and plate. DFimport.rpc will reject any records it cannot import with an error message indicating that the user does not have the necessary permissions.

When reading records from the input file, each record must be delimited by a newline ('\n') character. Additionally, each record can be at most 4095 characters in length. If 4095 characters are read without encountering a newline, the record is truncated at 4095 characters. It will subsequently fail record checking and be rejected.

DFimport.rpc may be used in a shell script stored in the study reports directory and run in the DFexplore Reports View, or in a shell script stored in the study ecbin directory and run dfexecute in an edit check. In these cases the permissions of the DFexplore user running the report or edit check will be applied. Thus the behavior and output may differ depending on the permissions of the user who runs the script.

If the same script is run from the UNIX shell the permissions associated with the UNIX login name apply. Thus, if a user has both UNIX and DFdiscover login accounts, and these accounts have different permissions, the user may get different results when they run the script in the UNIX and DFexplore environments.

DFexplore uses the environment variable DFUSER to implement user permissions. It is possible to set and export this variable in the shell script to fix the permissions to those of any specified user, thus rendering the results constant for all users despite their specific DFexplore permissions.

Input records can be new data records that need to be reviewed in Fax View, new or previously entered data records for direct entry to the study database, or metadata records (data queries or reasons). It is not possible to mix these different record types in a single import operation. The type of records being imported is specified using one of the following options:

  • -n : new records to be added to the new fax queue

  • -q : queries for specified data fields

  • -c : reasons for specified data fields

If none of these options is specified, the records must be new or replacement data records to be imported into the study database according to the plate number specified in the fifth field of each data record.

In addition to the record type options, there are 3 import mode options (add, replace and merge) to specify whether records are new and being added to the database (-a), replacement records (-r), or a combination requiring a merge operation during which new records are added to the database and existing records are replaced (-m).

To locate existing records in the database, DFimport.rpc searches for a record with matching keys (id, visit, plate) and image ID. This will always result in the identification of a record uniquely, if it exists, independent of whether the record has primary or secondary status. Dependent upon the record's presence in the database, the requested import mode will result in either an 'Add' or 'Replace' operation as shown in Table 3.9, “Deriving operations from import mode”.

Table 3.9. Deriving operations from import mode

Import modeExistsDoes not exist
-a **error Add
-r Replace **error
-m Replace Add

If the operation is not in error, then the primary or secondary status of the import record is considered subject to the following rules:

Table 3.10. Add operation

Import record status Existing record status Ending State
primary none imported record becomes primary
secondary none **error
primary primary existing primary becomes secondary; imported record becomes primary
secondary primary imported record is added as secondary
primary missing **error


Table 3.11. Replace operation

Import record status Existing record status Ending State
primary secondary **error
secondary secondary imported record replaces existing
primary primary imported record replaces existing
secondary primary **error


Imported records are verified against the study data dictionary if -v is given. Without this option, imported records are only verified for the correct number of fields.

[Important]Formatting Records for Import

Records must be correctly formatted for the DFdiscover plate to which they will be imported. The standard field delimiter, |, must appear between data fields. The first 7 fields must include valid values for: status, level, image ID, study, plate, visit and subject ID respectively, and the last 3 fields must have valid values for: status, creation and modification. The only exception is for data records being imported to the new fax queue (using the -n option), for which the subject ID and visit keys may be blank.

The third field, image ID, must be unique for all data records in the study database. This field contains the image identifier if the record has an associated image file, or a raw record identifier if there is no image. When importing new records that have no image (e.g. lab data from an external source) this field should contain a placeholder image ID, 0000/0000000. Specifying -R, will instruct DFimport.rpc to generate a new raw record identifier and insert it in the record, replacing the placeholder.

When importing metadata records (queries and reasons) the image ID field must also contain 0000/0000000, but this value is not converted to a unique identifier by DFimport.rpc (0000/0000000 is valid in metadata records).

A trailing delimiter is required at the end of all data records but must not be present on metadata records, i.e. queries (plate 511) and reasons (plate 510). Thus, care should be taken not to use the -p option, which ensures there is a trailing pipe on exported records, when using DFexportrpc; to export queries and reasons if the intention is to re-import them into the database.

If the validation level of an imported record is higher than the user's maximum write level, it is adjusted down to the maximum, a warning message is generated, and the record import proceeds.

If an input record has a # or a newline in the first column position, DFimport.rpc treats the entire record as a comment and skips over it.

If DFimport.rpc completes without error, it echoes the number of records imported and the number of warning messages and exits with status 0. If DFimport.rpc encounters errors, which are unrelated to the records being imported, it exits with a message to stderr and a positive exit status. If DFimport.rpc encounters errors with the records it is importing, it writes each offending record to stdout, a problem description to stderr, and increments a counter of failed records. Upon exit, DFimport.rpc sets its exit status to the failed record counter. If all records are successfully imported, the exit status is 0.

[Note]Note

  • DFimport.rpc does not complete successfully if the study server has been disabled or put into read-only access mode.

  • If the study server is in exclusive access mode, only the user who put it into this mode can import data.

  • If the study is in restricted access mode, only study and DFdiscover administrators are able to import data.

Options

-v

Verify that the imported records match the database record format as specified in the study data dictionary. It is highly recommended that this argument be used under most circumstances. Without this option, minimal record checking is performed on the number of fields (must be at least 7), the record status, and the record validation level. The verification performed by -v is equivalent to the consistency checking performed by the DFdiscover utility program DFcmpSchema when run with the -v option.

-N

Test that the data records verify correctly but do not import them.

-R

Convert 0000/0000000 in field 3 to a new raw record identifier before importing data records to the study database. This option cannot be used when importing metadata records (queries and reasons).

-n, -q, -c

Imported record type: if none of these options is specified the input file must contain data records to be imported into the study database according to the plate number field in each data record. Specifying:

  • -n, New: add records to the new record queue, DFin.dat. When importing to DFin.dat, the import mode must be -a or -r.

  • -q, queries: add records to the Query database, DFqc.dat. When importing to DFqc.dat, the import mode must be -a or -r.

  • -c, Reason for change notes: add records to the reasons database, DFreason.dat. When importing to DFreason.dat, the import mode must be -a or -r.

-a, -m, -r

Import mode (required) specifies one of three possible actions:

  • -a, Add: fail if a record with matching keys and image ID already exists in database.

  • -m, Merge: if a primary record with matching keys exists in the database, replace it if the image ID is the same, make it secondary if the image ID is different, otherwise proceed as if adding.

  • -r, Replace: fail if a record with matching keys and image ID does not exist in the database, otherwise replace it.

-s

Skip plate arrival triggers. This option can only be specified in conjunction with the -n (new record) import option. By default, plate arrival triggers (defined in Plate View in DFsetup) are executed when data records are imported to the new record queue. The combination of -n -s options allows new records to be imported without executing plate arrival triggers.

Plate arrival triggers are only executed if the record import is successful. If a trigger fails, a warning message is written to the DFdiscover error log.

#

The study database number into which the records are to be imported (required).

string

The input file containing the records to be imported (required), or -, in which case the input records are read from standard input.

Exit Status

DFimport.rpc exits with one of the following statuses:

0

The command was successful.

24

The user does not have the necessary permission to execute the command.

36

The required command-line arguments were not present or were incorrectly specified.

> 0

The command failed because the study schema or the input file could not be read, the database server could not be contacted, or communication with the database server failed.

Limitations

DFimport.rpc attempts to enforce database integrity as best it can. As a result, the following constraints are enforced:

  • DFimport.rpc does not allow the import of type 21 (missing page), 22 (overdue visit) and 23 (edit check missing page) queries for which there is a primary record (missing, final, incomplete) in the database. Attempts to do this will result in a message that the record already exists.

  • DFimport.rpc will delete any corresponding type 21 (missing page), 22 (overdue visit) and 23 (edit check missing page) queries when a primary record (missing, final, incomplete) is imported into the database.

  • DFimport.rpc does not allow the import of new query records with category codes that are not defined in the Query Category Map in DFsetup. However, it does allow modification or deletion of records with undefined category codes.

  • DFimport.rpc can be used to import data records that contain fields from an e-signature module but each of the e-signature fields will be blanked during the import. It is not possible to programmatically add/replace data records that have pre-filled e-signature fields. Such records must be reviewed in DFexplore after the import so that they can be e-signed.

  • DFimport.rpc does not allow the import of queries for which the primary record is locked. Attempts to do this will result in a message that the record is currently in use.

  • DFimport.rpc does not allow the import of data values for choice and check fields that do not exactly match one of the codes defined for those fields in the study schema.

  • DFimport.rpc will not import secondary records with a raw or zeros image ID (e.g. 0922R00023001 or 0000/0000000). The only supported function for secondary records is to reference additional images attached to the primary data record. Thus secondary records must have an image ID that points to an image file.

However, there are situations where a knowledgeable user may want to import records out of order, for example, so that referential integrity is initially violated, and then subsequently restored. The following operations are permitted as they allow a knowledgeable user to fully utilize DFimport.rpc.

  • DFimport.rpc allows the import of queries for which there is no primary record in the database. This is to allow an out-of-sequence import of queries followed by their data records in a subsequent import. The correct sequence of steps is to first import the primary records followed by the queries.

  • DFimport.rpc allows the primary record to be imported with a delete status, effectively deleting the primary record and all secondary records, queries and reasons associated with it. If the primary record is deleted, its secondary records and metadata will also be deleted to avoid having orphaned secondary, query and reason records in the database. During deletion, the record currently in the database is compared with the record being imported. All fields (other than status) must match for the operation to succeed. The number of fields can be different than the current definition for a given plate. This allows for the deletion of possibly corrupt records with field counts that differ from the current plate definition by exporting the record, changing its status to delete, and importing the record in question.

  • DFimport.rpc allows query and reason records to be imported with a delete status, effectively deleting the queries and reasons. All fields (other than status and validation level) must match for the operation to succeed. The deletion of queries and reasons does not delete the primary record to which they are attached.


DFlistplates.rpc

DFlistplates.rpc — List all plate numbers used in the study

Synopsis

DFlistplates.rpc [-n] [-p #, #-#] {-s #}

Description

DFlistplates.rpc creates a list of the plate numbers defined for the study and writes that list to the standard output. The list is sorted in increasing numeric order. Each plate number is output with leading zero-padded to three digits and is delimited from the next plate number by a single space character. The complete list of plate numbers is output in one line without a trailing new line, unless -n argument is given.

Both the plate numbers for user-defined plates, and 501 (the plate number reserved for returned Query Reports), are output by DFlistplates.rpc. However, the special plate numbers, 0, which is reserved for new records in DFin.dat, 510, which is reserved for reason for data change records, and 511, which is reserved for records in the query database DFqc.dat, are not included in the list.

With -p #, #-#, the list of known plates for the study is intersected with the argument list to create an output list. If this option is not specified, the program lists all of the defined plate numbers for the study.

Options

-n

append a new line character to the end of the output.

-p #, #-#

include only the specified plates in the range of plates to output.

-s #

the DFdiscover study number (required).

Exit Status

DFlistplates.rpc exits with one of the following statuses:

0

The command was successful.

> 0

The command failed because the database server could not be contacted, or communication with the database server failed.

Examples

Example 3.56. List the plates defined for study 123 to the standard output

% DFlistplates.rpc -s 123
 001 002 003 004 005 006 007 008 009 010 020 501% 

Notice how the prompt has been affected by the lack of a trailing newline.


Example 3.57. List all plate numbers, one per line, for study 254 - Bourne shell method

% for p in `DFlistplates.rpc -s 254`
? do
? echo $p
? done
001
002
003
004
005
006
007
008
009
010
020
501


DFlogger

DFlogger — Re-route error messages from non-DFdiscover applications to syslog, which in turn writes the messages to the system log files, as configured in /etc/syslog.conf.

Synopsis

DFlogger [-t string] [-f string] {-s #}

Description

DFlogger reads the standard input, or the file specified with -f string, treating each input line as an error message to be forwarded to syslog.

DFlogger should be used, in a pipeline fashion, at the end of commands that might generate error messages that you want to record in the system error log. DFlogger sends each input line to syslog and counts how many lines were written. The number of lines is used as the exit status of DFlogger. This has the benefit that, if DFlogger is used at the end of a pipeline, the exit status of the pipelined command will always be the number of error messages generated.

Options

-t string

the name of the program that DFlogger should prepend each message with. This is the program name that will appear in the header information for the message when it is written to the error log. The default is DFlogger.

-f string

the name of a file that contains error messages to be sent to the error log. Each line of the file is assumed to be an individual error message. If this option is not provided input is assumed to come from the standard input.

-s #

the DFdiscover study number (required).

Exit Status

DFlogger exits with one of the following statuses:

36

The required command-line arguments were not present or were incorrectly specified.

> 0

The command was successful. The exit status is the number of input lines that were transferred to syslog.

Examples

Example 3.58. Capture any error output from to the system error log

% DFtiff2ras /tmp/TIFFfile /tmp/rasterfile_ 2>&1 | \
DFlogger -t DFtiff2ras

Both the standard output and standard error are directed to DFlogger (this is C-shell syntax).



DFpass

DFpass — Locally manage user credentials for client-side command-line programs.

Synopsis

DFpass {[-add] | [-replace] | [-remove]} {servername:username}

Description

Several command-line programs, namely DFattach, DFbatch, DFexport, DFpdfpkg, DFreport and DFuserPerms, connect to study databases and require valid database credentials for the database connection to succeed. Each of these programs already permits the use of command-line options -S, -U and -C for specifying the needed servername, username and password credentials. Similarly, the DFSERVER, DFUSER and DFPASSWD environment variables may be assigned values and used.

DFpass is a convenience program that allows a user to locally store the password part of these credentials in a secure manner. Use of DFpass is not required but it is recommended for command-line users, and is strongly encouraged for users that write shell scripts and/or schedule program execution with facilities like UNIX cron.

DFpass manages credentials for the current user by keeping a local, user-specific database file of servername, username and password triples. Each record in the database is an encrypted representation of the credentials for one unique combination of servername, username and password. With DFpass one can add new records, update existing records with a new password and remove records.

Use of DFpass requires command-line specification of the action to be taken (one of: add, replace or remove) and the servername and username to which the action applies:

  • if the -add action is specified, the combination of servername and username must not match a previously added entry that is already being managed locally by DFpass. The user is prompted to enter their password. DFpass obscures the password as it is typed in and requires the user to confirm by entering the password again. If the passwords match exactly, the password is accepted and saved locally for the user.

  • if the -replace action is specified, the combination of servername and username must match a previously added entry that is already being managed locally by DFpass. The user is prompted to enter their new password. DFpass obscures the password as it is typed in and requires the user to confirm by entering the password again. If the passwords match exactly, the password is accepted and saved locally for the user.

  • if the -remove action is specified, the combination of servername and username must match a previously added entry that is already being managed locally by DFpass.

In all cases, DFpass prints a message confirming that the requested action was action, or an error message if the action could not be completed.

[Important]Important

DFpass does not confirm that the supplied servername, username and password combination is valid. This is the responsibility of the user.

Password management

DFpass is not a replacement for the existing DFdiscover tools for managing user credentials, nor does it offer the same functionality. User credentials must still be created and managed within DFdiscover using standard methods. DFpass simply allows you to write and read those credentials locally in a way that does not expose passwords as clear text.

Adding entries with DFpass does not add credentials for the user to DFdiscover. It is vitally important that the entries made with DFpass match existing DFdiscover credentials, otherwise those entries are of no value. Users must also be aware that updating a password in DFdiscover does not update the local information managed by DFpass; this must be done separately with the -replace action.

Options

-add | -replace | -remove

action to take (required).

servername:username

the specific credentials to add, replace or remove (required). For the -add and -replace actions, the user will be prompted to enter their password and confirm the password by entering it again.

Exit Status

DFpass exits with one of the following statuses:

0

The command was successful.

1

The command was not successful.

Examples

Example 3.59. Add credentials

% DFpass -add testserver:testuser
Password: xxxxxxx
testserver:testuser added


Example 3.60. Remove credentials

% DFpass -remove testserver:testuser
testserver:testuser removed



DFpdf

DFpdf — Generate bookmarked PDF documents of CRF images

Synopsis

DFpdf [ [-c password] | [-C] ] [-d] [-i string] [-o string] [-f string] [-v string] [-g string] [-s string] [-t string] [-j string] [-k string] [-n] [-b string [-a string] [-m #] ] {#}

Description

DFpdf generates an optionally password-protected PDF document containing one or more pages, where each page is a study CRF, bookmarked within the complete PDF. The resulting PDF document follows the Adobe PDF specification, version 1.4.

If a password is provided, the entire file is encrypted and stored in binary format. Otherwise, the file is stored in plain text format.

DFpdf requires a study number argument and an input file. The input file, which must be in either DFdiscover data export or DRF format, lists the images that are to be included in the PDF output document. Sort order in the output matches the order that records appear in the input file. If the -s sortmap option is specified, sort order within subject will follow the rules specified by the contents of sortmap.

Bookmarks are created at the document root, subject, visit, and plate levels. The label for the document root bookmark is ID, Visit, Plate but can be over-ridden through the use of -t. A new bookmark is created as each new ID is encountered in the input and the label is of the form ID #####. A visit bookmark is created for each new visit number within the current ID. The label is created from field 3 of the study visit map if the visit is defined in the visit map; otherwise, it has the form Visit #####. Finally, a plate bookmark is created for each CRF in the input file. The label is created from the matching page map entry, if it is defined; from field 2 of the study DFfile_map if the plate is defined (which it should always be); or, it has the form Plate #####. For secondary records, an asterisk (*) is appended to the end of the page bookmark label. Hence it is easy to scan the list of expanded bookmarks and identify those records that are secondaries.

Using the -j and -k options, custom page header and footer labels, respectively, can be created for each CRF page in the output document. If no labels are specified, the default header label is ID %DFID : %DFPMLBL and there is no footer label. The desired label is specified in a string [11] and may contain any combination of fixed text and zero or more of the following keywords, which are replaced at output creation time with the matching database value:

  • %DFSTATUS - record status (the status label is returned, rather than the status number)

  • %DFVALID - record validation level

  • %DFRASTER - image ID

  • %DFSTUDY - DFdiscover study number

  • %DFPLATE - plate number

  • %DFSEQ - visit or sequence number

  • %DFID - subject ID

  • %DFCREATE - record creation time stamp

  • %DFMODIFY - record modification time stamp

  • %DFPMLBL - page map label for the current keys as defined in DFpage_map

  • %DFVLBL - visit label for the current keys as defined in DFvisit_map

  • %DFPLBL - plate label for the current keys as defined in DFschema

A warning message is written to stderr for each record in the input file that does not reference an image file. No output is written to the PDF file for such records.

For records exported from the new fax queue, bookmarking will only be as good as the accuracy of the ICR on the record keys.

It is possible to override the default labeling for study visits and plates by providing alternative visit map, file map, and page map files with the -v, -f, and -g options respectively.

[Note]PDF document size

The output PDF document contains copies of the CRF images and hence each file can be quite large. The images are compressed so that the space requirements are not as onerous as for the source CRF images, but still, an input file listing 500 CRF images will create a PDF document that is approximately 5MB in size.

DFpdf exits with a status 0 if a PDF output file was successfully created; otherwise, a non-zero exit status is returned.

Options

-c password, -C

If -c is specified, the command line password is used. If -C is specified, DFpdf reads the password from password file ~/.dfpdfpasswd.

The password file ~/.dfpdfpasswd may contain multiple lines, but the first non-empty line will be used. The leading spaces and trailing spaces will be removed. The maximum length of password is 32 characters.

-d

the input file is in DFdiscover Retrieval File (DRF) format

-i string

the input file of data records that reference the CRF image to be included. Data records are assumed to be in data export format, unless -d is also used. If no input file argument is present, standard input is read.

-o string

the output file that will contain the resulting PDF document. If no output file argument is present, the resulting PDF document will be written to standard output.

-f string

an alternative file location for the study filemap. This file is read to determine plate labels for bookmarking. If the argument is not present, the FILE_MAP value of the study configuration will be used.

-v string

an alternative file location for the study visit map. This file is read to determine visit labels for bookmarking. If the argument is not present, the VISIT_MAP value of the study configuration will be used.

-g string

an alternative file location for the study page map. This file is read to determine visit labels for bookmarking. If the argument is not present, the STUDY_DIR/lib/DFpage_map value of the study configuration will be used.

-s string

allows the specification of sort order by an external file. The file is expected to be in query sort order format. Without this option, no sorting of the input file is performed and images appear in the PDF document in the order that records appear in the input file. With this option, consecutive records for the same ID are sorted to follow the rules stated by the sortmap file. Note that no sorting is performed across IDs, only within IDs, and only when the IDs appear consecutively in the input file.

-t string

the title to appear at the top of the bookmark list. If a title is supplied, in addition to setting the text of the bookmark title object, the title is also set in the PDF document information. If the title contains any characters that are meaningful to the shell, including spaces, the title must be enclosed in double quotes. If no title is supplied, the default title of ID, Visit, Plate is used.

-j string

customize the page header label using a combination of fixed text and references to key and meta information. The default header is ID %DFID : %DFPMLBL, where %DFID and %DFPMLBL are replaced by the subject ID and page map label, respectively.

-k string

customize the page footer label using a combination of fixed text and references to key and meta information. The default footer is blank.

-n

reverses the order of nesting of visits and plates. Without this option, the preferred order is to nest plates within visits, unless the visit cannot be found in the visit map, in which case the visit (sequence) is nested within the plate. If -n is used, the preferred order is to nest visits within plates, independent of whether the visit can be found in the visit map or not.

-b string

allows the specification of fields to be blinded (blanked out) in the output PDF file. The string is specified in the following format:

plate:field,field,field;plate:field,field

where plate is a valid plate number and field is a comma-delimited list of field numbers that are to be blinded on that plate (ranges of field numbers are specified by enumerating each field in a comma-delimited list). Field numbers must use DFdiscover schema numbering.

Field numbers that are not valid for the plate are skipped. Where multiple plates are required, plate specifications are separated by semi-colons.

To implement the blindlist, DFpdf must read the study setup file. The study setup file may be provided with -a string. If the setup file is not provided, the study server will be contacted to locate and supply the setup file.

-m # can be used to pad by the specified number of pixels, the blinded area. The default margin is 0 pixels, and the blinded areas are drawn in exactly the dimensions specified by the study setup. Legal values for margin must be > 0 and reasonable values are less than 10. For a margin greater than 0, the margin is added to each side of the setup dimensions and these new dimensions are used to blind the request field.

#

the DFdiscover study number (required)

Exit Status

DFpdf exits with one of the following statuses:

0

The command was successful.

1

The command was successful, but the resulting PDF file contained 0 pages.

2

The user does not have the necessary permission to execute the command.

2

The input file cannot be read, the setup definition cannot be read, the database server could not be contacted, or communication with the database server failed.

36

The required command-line arguments were not present or were incorrectly specified.

Examples

Example 3.61. Use DFpdf without the use of the DFexplore Save As PDF option

This example involves creating a DRF using Save Data Retrieval File from the File menu in DFexplore, and then running DFpdf from the command line to generate a PDF file of the saved record set.

To start the process, DFexplore was used to retrieve a set of records from the study database. The retrieved records were then saved to the DRF dde.drf. Example contents of dde.drf are shown below.

#user1 16/02/22|Records ready for DDE
#Id|Visit|Plate
1002|0|1
1006|0|1
2035|1|2
2035|1|3
2035|1|4
2035|21|5
2035|23|5
2035|24|5

Next, DFpdf was run from the command line to create a PDF document from the DRF having a custom header label for each record in the PDF document. In this example, DFpdf was invoked by the following command:

% DFpdf -d -t "Subjects" -i /opt/studies/val254/drf/dde.drf \
-j "Subject %DFID, Image %DFRASTER" -o /opt/studies/val254/drf/dde.pdf 254

To generate the same file with password protection, the following command was invoked using the password "4aY3gh98B":

% DFpdf -c 4aY3gh98B -d -t "Subjects" -i /opt/studies/val254/drf/dde.drf \
-j "Subject %DFID, Image %DFRASTER" -o /opt/studies/val254/drf/dde.pdf 254

It is also possible to pipe the output from DFexport.rpc directly into DFpdf as illustrated in the following example:

% DFexport.rpc 254 1 - | DFpdf -t "Screening Data" \
-j "Subject %DFID, Image %DFRASTER" -o /opt/studies/val254/work/Screening.pdf 254


See Also

DFencryptpdf


[11] If the string contains characters that are meaningful to the shell, such as space, quote, percent, etc., then the entire string must be enclosed in double quotes. The string can always be enclosed in double quotes, to ensure that no characters are interpreted by the shell.


DFpdfpkg

DFpdfpkg — Generate multiple bookmarked PDF files for specified subject IDs or sites

Synopsis

DFpdfpkg [-S server] [-U username] [-C password] [-A] [-n] [-color] [-hd] [-b blind_spec] [-f filemap] [-v visitmap] [-g pagemap] [-s sortmap] [-t title] [-j header] [-k footer] [ [-P pdfpasswd] ] [-history [-bookmark label] ] { [-data data,missed] [-image primary|all] } {-output out_dir_name } [-prefix file_prefix] { [-subject pid_list] [-site site_list] } [-visit visnum_list] [-plate pltnum_list] [-e errlog] {study#}

Description

DFpdfpkg generates a set of optionally password-protected PDF documents (one per subject) containing one or more pages representing data records (EDC, fax, email, DFsend), and images and audit trail information. The resulting PDF documents follow the Adobe PDF specification, version 1.4, and as a result can be read by Acrobat Reader versions 5.X and greater.

If a password is provided, files are encrypted and stored in binary format. Otherwise, the file is stored in plain text format.

DFpdfpkg requires a study number argument and an output folder name. Records are output ascending numeric order by visit and plate. If the -s option is specified, sort order will follow the rules specified by the DFsortmap file.

Bookmarks are created at the document root, ID, visit, and plate levels. The label for the document root bookmark is ID, Visit, Plate but can be over-ridden through the use of -t. A visit bookmark is created for each new visit number within the current ID. The label is created from field 3 of the study visit map if the visit is defined in the visit map; otherwise, it has the form Visit #####. A plate bookmark is created for each CRF in the visit. The label is created from the matching page map entry, if it is defined; from field 2 of the study DFfilemap if the plate is defined (which it should always be); or, it has the form Plate #####. For secondary records, an asterisk (*) is appended to the end of the page bookmark label. Hence it is easy to scan the list of expanded bookmarks and identify those records that are secondaries.

Using the -j and -k options, custom page header and footer labels, respectively, can be created for each CRF page in the output document. If no labels are specified, the default header label is ID %DFID : %DFPMLBL and there is no footer label. The desired label is specified in a string [12] and may contain any combination of fixed text and zero or more of the following keywords, which are replaced at output creation time with the matching database value:

  • %DFSTATUS - record status (the status label is returned, rather than the status number)

  • %DFVALID - record validation level

  • %DFRASTER - image ID

  • %DFSTUDY - DFdiscover study number

  • %DFPLATE - plate number

  • %DFSEQ - visit or sequence number

  • %DFID - subject ID

  • %DFCREATE - record creation time stamp

  • %DFMODIFY - record modification time stamp

  • %DFPMLBL - page map label for the current keys as defined in DFpage_map

  • %DFVLBL - visit label for the current keys as defined in DFvisit_map

  • %DFPLBL - plate label for the current keys as defined in DFschema

It is possible to override the default labeling for study visits and plates by providing alternative visit map, file map, and page map files with the -v, -f, and -g options respectively.

Options

-S server

DFdiscover server name

-U username

DFdiscover login username

-C password

login password. Refer to Section 3.2, “User Credentials” for recommended/better solutions for safe password handling.

-A

output PDF in A4 size

-n

nest visits within plates

-color

apply field color to completed pages

-hd

output the PDF file with images in higher definition (HD), which is 300dpi while the default standard definition (SD) is 100dpi.

-b string

allows the specification of fields to be blinded (blanked out) in the output PDF file. The string is specified in the following format:

plate:field,field,field;plate:field,field

where plate is a valid plate number and field is a comma-delimited list of field numbers that are to be blinded on that plate (ranges of field numbers are specified by enumerating each field in a comma-delimited list). Field numbers must use DFdiscover schema numbering.

Field numbers that are not valid for the plate are skipped. Where multiple plates are required, plate specifications are separated by semi-colons.

To implement the blindlist, DFpdf must read the study setup file. The study setup file may be provided with -a string. If the setup file is not provided, the study server will be contacted to locate and supply the setup file.

-m # can be used to pad by the specified number of pixels, the blinded area. The default margin is 0 pixels, and the blinded areas are drawn in exactly the dimensions specified by the study setup. Legal values for margin must be > 0 and reasonable values are less than 10. For a margin greater than 0, the margin is added to each side of the setup dimensions and these new dimensions are used to blind the request field.

-f filemap

alternate study filemap

-v visitmap

alternate study visitmap

-g pagemap

alternate study pagemap

-s string

allows the specification of sort order by an external file. The file is expected to be in query sort order format. Without this option, no sorting of the input file is performed and images appear in the PDF document in the order that records appear in the input file. With this option, consecutive records for the same ID are sorted to follow the rules stated by the sortmap file. Note that no sorting is performed across IDs, only within IDs, and only when the IDs appear consecutively in the input file.

-t string

the title to appear at the top of the bookmark list. If a title is supplied, in addition to setting the text of the bookmark title object, the title is also set in the document information (which is accessed from the Reader's "Document Info-General-Title" attribute. If the title contains any characters that are meaningful to the shell, including spaces, the title must be enclosed in double quotes. If no title is supplied, the default title of ID, Visit, Plate is used.

-j string

customize the page header label using a combination of fixed text and references to key and meta information. The default header is ID %DFID : %DFPMLBL, where %DFID and %DFPMLBL are replaced by the subject ID and page map label, respectively.

-k string

customize the page footer label using a combination of fixed text and references to key and meta information. The default footer is blank.

-P password

If -P is specified, the password is used to encrypt the PDF. The recipient will need to supply the same password to decrypt the file before reading.

-history

data and metadata change history. This outputs the audit trail for each record.

-bookmark label

bookmark label for history

-data data,missed

data record options. Data records are included for the record types listed - data for records that include EDC and image-based, or missed which includes any records marked as missed.

-image primary|all

image options. Include only primary images or include all images attached to a record.

-output dirname

output folder for PDFs (required). This must be the full path to an existing directory or folder that is writable by the user.

-prefix file_prefix

PDF file prefix. Each file written to the output directory with a filename comprised of the prefix and the subject ID.

-subject pid_list

subject ID list. Output packages for the subject IDs in this comma-delimited list.

-site site_list

site number list. Output packages for the subject IDs that belong to sites in this comma-delimited list.

-visit visnum_list

visit number list. Include pages with the specified visit numbers.

-plate pltnum_list

plate number list. Include pages with the specified plate numbers.

-e errlog

error log file. Write any error or warning message to this filename. The default is to write any error or warning messages to stderr.

#

the DFdiscover study number (required)

Exit Status

DFpdfpkg exits with one of the following statuses:

0

The command was successful, one or more PDF output files were created.

1

The command was not successful.

Examples

Example 3.62. Use DFpdfpkg without the use of the DFexplore Create Subject Packages... option

The following example creates subject packages for subject ID 2035 and 2049 from Site 2.

% DFpdfpkg -S example.server.com -U example_user -C passwd \
-t "Site 2 Subject Packages" -color -data data -image primary \
-output /opt/studies/demo253/work -prefix "Site2_" \
-subject 2035,2049 253


See Also

DFpdf


[12] If the string contains characters that are meaningful to the shell, such as space, quote, percent, etc., then the entire string must be enclosed in double quotes. The string can always be enclosed in double quotes, to ensure that no characters are interpreted by the shell.


DFprint_filter

DFprint_filter — Format input file(s) for printing to a PostScript® capable printer, and print to a specified printer

Synopsis

DFprint_filter [-p string] [-c #] [-o string] [-f string]

Description

DFprint_filter is the main print interface. DFdiscover programs that send output to a printer may use DFprint_filter to format the output into PostScript® format, if necessary, and then spool the output to a printer.

DFprint_filter is capable of re-formatting PNG and Sun rasterfiles (the image formats used in DFdiscover), ASCII text files, and PostScript® files.

Options

-p string

direct the output to the specified printer. If this argument is missing, and the environment variable PRINTER is defined, the value of the environment variable will be taken as the name of the printer. Otherwise, the default printer is the first printer specified in /opt/dfdiscover/lib/DFprinters, if the file is present and non-empty. If this is not customized locally, the system default printer, lp, is used.

-c #

the number of copies to print. The default is 1.

-o string

pass the user-supplied options to the print pre-processor. The print pre-processor is DFpsprint (for image files) and DFtextps (for all other files). DFtextps accepts additional user options that can be passed via this option. For example, -o -p8 will cause the input text file to be printed in 8-point font rather than the default 10-point font.

-f string

the name of the input file. If the filename is not provided, or is given as -, the input is assumed to come from the standard input.

Exit Status

DFprint_filter exits with one of the following statuses:

0

Always.

Examples

Example 3.63. Print the UNIX password file to the printer "hplj"

% /opt/dfdiscover/lib/DFprint_filter -p hplj -f /etc/passwd


DFprintdb

DFprintdb — Print case report forms merged with data records from the study database

Synopsis

DFprintdb [-d] [-A] [-p string] [-i string] [-o string] [-t string] [-e platelist|all] {-s #}

Description

DFprintdb prints case report forms merged with data records from a study database. One page is printed for each data record in the input file, and is bookmarked by the record keys: subject ID, visit and then plate. Printed output is either sent directly to the printer or the named output file.

DFprintdb uses the high resolution backgrounds previously generated by DFsetup during a Study - Import CRFs operation. A background, named DFbkgd###.png is created for each CRF plate, where ### is the plate number. If tagged backgrounds also exist, e.g. DFbkgd###_all_SPA.png for a Spanish background used at all visits for plate ###, they can be requested using the -t option.

If the high resolution PNG background does not exist for a plate, DFprintdb will print the field widgets and the data they contain on a blank background.

Options

-d

the input file is in DFdiscover Retrieval File (DRF) format where each input record contains only the record keys. Without this option, each record is assumed to be in data export format.

-A

size the output pages for A4 paper rather than the default US-letter size.

-p string

the name of the printer that the output should be sent to. The default is the printer defined by the study configuration.

-i string

the input file of data records that are to be merged for printing. Records are assumed to be in the format output by a previous DFexport.rpc, unless -d has been specified. The default is standard input.

-o string

the name of the output file. Instead of printing directly to the printer the output can be re-directed to a file with this argument. If both -o and -p are specified, only -o is honored. The output is in PostScript® format (with bookmarks) suitable for subsequent printing. If the .pdf or .PDF file extension is used in the output file name, then the output is in PDF format (without bookmarks).

-t string

the CRF background type. If the study setup includes backgrounds tagged with different type names, this option can be used to select the type to be used for printing. For example, -t SPA to use Spanish backgrounds, if a Spanish version of the CRFs was imported and tagged with 'SPA'. If this option is omitted, or used but the specified type does not exist for a particular plate, the default CRF background (i.e. untagged version) will be used if one exists.

-e platelist|all

plates for which text fields may expand beyond their display size. Without this option printed text is truncated if it exceeds the field display size. For example,

-e 10-15,22

allows text field expansion, if necessary, on plates 10 to 15 and 22; while

-e all

allows text expansion on all plates. NOTE: expanded text may cover other data fields on the plate and thus should be used only on plates with empty space into which text fields can expand.

-s #

the DFdiscover study number (required).

Exit Status

DFprintdb exits with one of the following statuses:

0

The command was successful.

36

The required command-line arguments were not present or were incorrectly specified.

> 0

The database server could not be contacted, communication with the database server failed, the study number is not defined, the database setup cannot be read, the input file cannot be read, or the output file cannot be written.

Examples

Example 3.64. Print all primary adverse event report records (plate 50) for study 222

% DFexport.rpc -s primary 222 50 - | DFprintdb -s 222


DFpsprint

DFpsprint — Convert one or more input CRF images into PostScript®

Synopsis

DFpsprint {infile...}

Description

DFpsprint is a utility program that converts input CRF image(s) to PostScript® suitable for printing. When multiple CRF images are given as options, the resulting PostScript® file contains one CRF image per page.

The resulting PostScript® document is always sent to standard output where it can be re-directed to a file.

[Note]Note

To convert CRF images files for immediate printing, DFprint_filter provides a simpler interface.

Options

infile

one or more input CRF images. If only one input file is being printed, the filename argument can be given or the file can be re-directed in via standard input. If there are multiple input files, each input file name must appear as an option. The file name - can be used as a placeholder for standard input.

Exit Status

DFpsprint exits with one of the following statuses:

0

The command was successful.

36

The required command-line arguments were not present or were incorrectly specified.

> 0

The command was successful but one or more input files could not be read. The exit status is the number of input files that could not be read.

Examples

Example 3.65. Simple use of DFpsprint for printing the file /tmp/rasterfile

% DFpsprint /tmp/rasterfile | lp -dlp

This command line subsequently re-directs the output to the printer.


Example 3.66. Concatenating 3 image files where the second is read from standard input

% cat /tmp/ras2 | DFpsprint /tmp/ras1 - /tmp/ras3 > /tmp/ps.out

See Also

DFprint_filter

DFqcps

DFqcps — Convert a Query Report, previously generated by DF_QCreports, into a PDF file with barcoding, prior to sending the report to a study site

Synopsis

DFqcps [-PDF] [-A4] [-new] [-l string] [-f string] [-h string] [-r string] [-n #] [-p #] {-s #}

Description

DFqcps is used by DF_QCfax and DF_QCprint to format the Query Reports created by DF_QCreports, before they are sent or printed. It takes its input from the standard input, and translates it into a PDF document with the standard DFdiscover barcoding. Barcoding permits returned Query Reports to be routed to the correct study database, in the event that they contain queries in DFdiscover Q&A (question and answer) format, or in case an investigator decides to make a comment on the Query Report and fax it back.

Options

-PDF

create a PDF output file. This is the default behavior. The option is present for backwards compatibility only.

-A4

format for A4 size paper. The default is US letter.

-new

use the new format for Query Reports introduced in DataFax version 4.2.0. This option applies only to external Query Reports; internal reports will always use the pre-4.2 format with a single Query Report ID field.

-l string

customize the label associated with the investigator signature line at the top of each page. The string specified must be enclosed in double quotes. The investigator signature line may be omitted entirely by specifying the -l with empty double quotes. The default label for the signature line is "Study Coordinator Sign and Date".

-f string

font to be used. If the specified font is not installed on the DFdiscover server the default fonts (Times, Helvetic, Courier, Century Gothic) will be used.

-h string

the header name to appear in the top-left corner of each formatted page. Typically this is the study name.

-r string

the report name that appears as the report identifier field on each formatted page. This value should be a 3 or 4-digit site ID, followed by a hyphen, followed by the 6-digit Query Report date. The default is no report name.

-n #

the total number of pages in the Query Report. This number appears at the bottom of each formatted page in a Page m of n label.

-p #

the point size to print the report text in. The default is 10 point.

-s #

the DFdiscover study number (required). The study number is barcoded across the top of each page, together with the plate number (always 501) and the page number of the output.

Exit Status

DFqcps exits with one of the following statuses:

0

The command was successful.

> 0

The command failed because the required command-line arguments were not present or were incorrectly specified.


DFreport

DFreport — Client-side, command-line interface for executing reports

Synopsis

DFreport [-S server] [-U username] [-C password] [-JS js] [-CSS css] [-logo] [-e errfile] [-o outfile] {-CMD cmd} {study#}

Options and Description

DFreport runs DFdiscover reports outside of DFexplore, making them available to run from a command-line or scheduled via the UNIX cron facility.

Authentication and Database Permissions

Authentication.  To authenticate, DFreport requires the username and password for connection to a specific database server. These may be supplied as:

  1. command-line options, the user can specify -S servername, -U username and -C password when the program is run, or

  2. environment variables, the user can set the variables DFSERVER servername, DFUSER username and DFPASSWD password before the program is run.

Specification of command-line options takes priority if both command-line options and environment variables are supplied. Refer to User Credentials for more information.

Database Permissions.  Subsequent to authentication with a specific database server, use of DFreport is allowed (or disallowed) by the DFexplore - Reports view or Server - Run reports permissions.

An authenticated user with one of these role permissions will then be able to run reports. The reported data is filtered by user permissions, which may further be limited by visit, plate and level restrictions associated with the user's role.

DFreport provides no warning that some records cannot be exported due to permission restrictions as this would itself provide information about the existence of restricted data records.

Report Options

-CMD command

resource request sent to server, specifying report to execute

-JS command

matching javascript resource for report interactivity

-CSS command

matching stylesheet resource for report styling and presentation

-logo

include the study logo in the top-left corner of the HTML output. If not defined, or not available, the study name is used

Output Options

Output File.  By default, output from DFreport is written to standard output (the command-line output). To write the output to a specific file, specify the filename with -o outfile. The user must have filesystem permission to create or modify the file. If outfile is specified as a relative pathname, the file is created relative to the DFreport command invocation directory.[13]

Output from legacy reports is written as is (plain text) while output from standard DFdiscover reports is in HTML or JSON format, determined by the file extension of the argument file.

Error Output.  Re-direct any error messages to a specific file with the -e errfile option. By default, error messages are written to standard error and hence would get intermixed with data if the data is being exported to standard output.

Exit Status

DFreport exits with status 0 if the command was successful. Exit status 1 indicates that an error occurred - errors are written to standard error or the errfile file if -e was specified.

Examples

DFexplore includes a convenience feature, available from Help > Show Command, to display the command required to run a report. For example, run the Enrollment by Site report within DFexplore and select Help > Show Command to obtain this command (line breaks are for presentation only):

DFreport -S explore.dfdiscover.com -U demo_user1 -CMD "/DFedc?cmd=getreport&name=enrollment&title=Enrollment%20by%20Site&json&compress"
-JS "https://cdn.dfdiscover.com/v5.2/js/DF_Enrollment.min.js" -CSS "https://cdn.dfdiscover.com/v5.2/css/DF_ReportStyles.css"
252 -logo -o DF_Enrollment_20191028140508.html -C xxx

Substitute the username for demo_user1, user password for xxx and provide an output filename in place of DF_Enrollment_20191028140508.html.



[13] If DFreport is invoked from a cron facility, absolute pathnames are essential.


DFsas

DFsas — Prepare data set(s) and job file for processing by SAS®.

Synopsis

DFsas {jobfile} [-f] [-dbx] [ [ [-c] | [-C] ] study# ] [-p platelist] [-l [check] | [choice] ] [-d csjo] [-s #] [-t #] [-I pidlist] [-n cidlist] [-v visitlist] [-w] [-SAS #] [-a] [-r rundir] [-z]

Description

DFsas is an intermediary between DFdiscover and SAS®. It can be used to extract summary data files from a study database and to create the corresponding SAS® job file for subsequent processing by SAS®.

Complete reference documentation for DFsas can be found in DFsas: DFdiscover to SAS®.

Options

jobfile

DFsas job file, specifies details of the translation between DFdiscover and SAS® (required).

-f

force DFsas to include all specified plates, even those that have no data.

-dbx

debug; do not remove the data export script jobfile.dbx after program completion.

-c|C #

create mode; create a default DFsas jobfile for the argument study number. Specifying -c includes all user-defined data fields, while -C includes all user-defined data fields plus DFdiscover header and trailer fields.

-p platelist

in create mode, the list of plates to include.

-l check,choice

export labels instead of codes for check and/or choice fields.

-d csjo

output one or more date fields from each date variable using these specified formats:

  • c=calendar (imputation and 4 digit years)

  • s=convert date to a string field as is (no imputation)

  • j=convert date to a julian number (with imputation)

  • o=use original value and date informat (no imputation)

-s #

split string field data containing more than # characters into 2+ fields on word boundaries.

-t #

truncate string field data containing more than # characters at exactly # characters.

-I pidlist

include only records for the specified subject IDs.

-n cidlist

include only records for the specified site IDs.

-v visitlist

include only records for the specified visit/sequence numbers.

-w

suppress warning messages.

-SAS #

apply SAS® version # rules/limits.

-a

use the DFdiscover field alias, instead of field name, for SAS® variable names.

-r rundir

set RUNDIR, e.g. -r 'C:\mydir\mysas'

-z

use STUDYDIR/dfsas and do not overwrite existing DFsas job files

Exit Status

DFsas exits with one of the following statuses:

0

The command was successful.

1

The command was not successful.

Examples

Example 3.67. Create a default DFsas job file for study 11

% DFsas mytestjobfile -c 11

Example 3.68. Create a DFsas job file for study 11, include only a subset of plates and request labels for check fields

% DFsas mytestjobfile -c 11 -p 1-5 -l check


DFsendfax

DFsendfax — Fax or email a plain text, PDF, or TIFF file to one or more recipients

Synopsis

DFsendfax [-2] [-A4] [-F from] [-w #] [-r #] [-d #] [ [-p password] | [-P] ] [-c string] [-sANY string] [-sEACH string] [-sALL string] [-sFIRST string] [-fANY string] [-fEACH string] [-fALL string] [-fFIRST string] {file} {recipients...}

Description

DFsendfax transmits any plain text, PDF, or TIFF file to one or more recipients.

DFsendfax interacts with a DFdiscover outbound daemon. You must have at least one outbound daemon configured before DFsendfax will work. DFsendfax makes a temporary copy of the file to be sent. By default, this is in /usr/tmp, although it may be your home directory if there is no space available in /usr/tmp. The temporary copy of the file becomes owned by user datafax. DFsendfax places its options into a structure that it then sends to the outbound daemon. Which outbound daemon is used is determined by the DFdiscover master using a round-robin scheduling algorithm. The outbound daemon then makes a queue entry for the fax request in its work directory. It also copies (or remote copies, if necessary) the temporary file to a data file in its work directory. The outbound daemon then manages the queue entry until the scheduled time for sending arrives. At this point, the data file is passed to the system email service, DFprotusfax or HylaFAX, which attempts delivery of the document. If the transmission is successful, that status is forwarded to the outbound daemon which then executes zero or more of the success commands. If the transmission has failed, the outbound daemon either waits the number of minutes specified by the retry delay if one or more retries are still possible, or executes one or more of the failure commands. Finally, if the outbound daemon never receives a reply regarding the disposition of a fax, the outbound daemon de-queues the fax and executes the appropriate failure commands.

When specifying success or failure commands to execute at the completion of a fax session, you can reference the values of various arguments that you supplied in your original DFsendfax command. The values that you can reference are:

$FaxID

the faxid# of the outgoing fax

$DataFile

the name of the file to be sent, <file> argument

$Requestee

the username of the person executing DFsendfax

$Callees

the <recipient list> argument

$Comments

the <comments> argument

$Number

the fax number (or email address) that the success or fail reply was generated from. This will be one of the fax numbers (or email addresses) from the original <recipients list> ($Callees).

If your fax is successfully queued, DFsendfax echoes back the name of the file to be sent, the phone number of the recipients, the scheduled fax time, and the faxid of the queue entry. This faxid is unique within the system and can be used to monitor the status of the fax with DFfaxq, or to de-queue the fax with DFfaxrm. You should be aware that depending on how your outbound daemons are configured, it may take several seconds for a fax queued with DFsendfax to be detected so that it appears in the output of DFfaxq.

[Note]DFsendfax and DF_QCfax

The standard DFdiscover report program, DF_QCfax, uses DFsendfax to send Query Reports to participating sites in a study. If a Query Report is successfully transmitted, DFqcsent.rpc is executed (via the -sANY option) to mark the quality control notes as having being successfully sent to the investigator site. This is needed so that DFdiscover will display the correct status of queries in DFexplore, and is also needed if you intend to use the overdue allowance option when creating Query Reports. Thus you should always use DF_QCfax to send quality control reports and not DFsendfax.

Delayed Sending

DFsendfax can schedule a fax for transmission at a specified time, rather than the default which is immediately. This is specified by the -w option. The possible arguments to this option can have the following syntax:

now | today | tomorrow | <time spec> [<offset>]
                         (1)                (2)

(1)

<time spec> may be any one of noon | midnight | hh[:mm] | <month> <day> [,<year>]

(2)

and <offset> = + # {seconds | minutes | hours | days | weeks}

If the time specified is in the past then this is the same as specifying now.

Options

-2

transmit the fax in fine mode. The default is standard mode.

-A4

transmit the fax using A4 sizing. The default is US letter size.

-F from

identify any files transmitted via email as originating from sender from.

-w #

the time that the file should be sent or emailed. The default is now. The specification of the time is similar to that allowed by the UNIX at.

-r #

the number of attempts to refax the file if the first attempt fails. The default is 2. The total number of attempts is the number of retries plus 1. Multiple attempts do not apply to emails as DFdiscover has no way of determining whether or not an email was delivered.

-d #

time, in minutes, between refax attempts. The default is 10 minutes. If the number of retries is 0, this value is ignored.

-p password, -P

encrypt the file to be emailed using the supplied password or using the password specified in the password file ~/.dfpdfpasswd. This option applies only to PDF attachments. Other file formats are sent unencrypted.

-c string

any comment string that you want to attach to the fax command. You can reference this comment field later when executing success or failure commands.

file

the file to be sent (required). It can be any text, PostScript®, TIFF, or PDF file. For faxing, DFdiscover uses HylaFAX to convert any non-TIFF file into a TIFF file. For emailing, PostScript® files are converted to PDF first (other formats are sent unchanged), and then sent as a MIME attachment to the specified email address. You must have at least read permission on any file to be sent.

recipients

a white-space delimited list of fax numbers or email addresses for recipients of the file (required). For each fax number include all long distance codes, etc. as required. Be careful not to include white-space characters inside a fax number as it will be interpreted as multiple fax numbers. Each email address must be in the format mailto:email_address where mailto: is fixed and required, and email_address is a valid email address.

[Warning]Warning

DFdiscover does not perform validity checking on email addresses. Further, if the outbound service is configured to send documents via email only, any recipients that are not email addresses are quietly ignored.

DFsendfax provides commands that can be executed upon successful or unsuccessful completion of the transmission. Completion occurs after the specified number of retries have been attempted, if necessary. The command is executed in a Bourne shell as a background child process of the outbound daemon on the machine that the DFsendfax command was originally specified. There are four levels of commands that can be specified. They can be specified individually or in any combination. It is the user's responsibility to ensure that any concurrency issues that may arise as a result of more than one command executing simultaneously are planned for.

-sANY string

execute the command specified in string as soon as the file is sent to any recipient in the recipients list. The command is executed only once, immediately after the fax is sent to one recipient.

-sALL string

execute the command as soon as the file is successfully sent to all recipients in the recipients list.

-sEACH string

execute the command each time the file is successfully sent to a recipient.

-sFIRST string

execute the command as soon as the file is successfully sent to the recipient specified first in the recipients list.

-fANY string

execute the command when the file cannot be successfully transmitted to a recipient. The command is executed only once, immediately after the first failure.

-fALL string

execute the command if the file cannot be sent to any of the recipients in the recipients list; that is, all send attempts to all recipients have failed.

-fEACH string

execute the command for each recipient that the file cannot be successfully sent to.

-fFIRST string

execute the command if the file cannot be successfully sent to the recipient first specified in the fax list.

Exit Status

DFsendfax exits with one of the following statuses:

0

The command was successful.

2

The requested fax could not be scheduled for sending.

Examples

Example 3.69. Send the file /etc/printcap to one recipient at 5551212

% DFsendfax /etc/printcap 5551212
Fax /etc/printcap
scheduled for sending to 5551212
at Thu Dec 3 08:45:26 1992
->Faxid is 317

Example 3.70. Schedule a fax for transmission at 11:00pm tonight and send it to two recipients, one via email (with a specific from email address)

% DFsendfax -w 23:00 -F repltyo@company.com /etc/fstab 9876543 mailto:luckyuser@hotmail.com
Fax /etc/fstab
scheduled for sending to 9876543 mailto:luckyuser@hotmail.com
at Thu Dec 3 23:00:00 1992
->Faxid is 318

Example 3.71. Schedule a fax for transmission 30 minutes from now and be informed via email that the fax was either successfully transmitted or failed

The line break is for presentation purposes only.

% DFsendfax -w 'now + 30 minutes' \
-sEACH 'echo $DataFile faxed to $Number | mail $Requestee' \
-fEACH 'echo $DataFile FAILED to $Number | mail $Requestee' \
/etc/magic 1-800-555-1212 9876543


DFsqlload

DFsqlload — Create table definitions and import all data into a relational database

Synopsis

DFsqlload [-flavor oracle|postgresql|mysql|mssql] [-d drfname] [-q] [ [-ignore_mssql_priv] | [-ignore_mysql_priv] ] [-type typed|untyped|both] [-coding code|label|both] [-missing code|label] [-date typed|untyped|both] [-noimpute] [-missed] [-table all|dfcoding[,dfnullvalue]] {param} {study}

Options

-flavor oracle|postgresql|mysql|mssql

Type of target database. The default is -flavor oracle.

-d drfname

A DFdiscover retrieval file to use to record problems encountered during data import.

-q

Quiet mode. Instructs the program to suppress all warning messages. The default, without this option, is to write warning messages to standard error.

-ignore_mssql_priv|ignore_mysql_priv

Ignore administrator privileges. For MS SQLServer or MySQL, allows tables to be loaded without requiring administrator privileges on the database.

-type typed|untyped|both

Type of SQL tables to be created. If set to typed (or by default) then tables use field data types as defined in DFschema. Table names used are DFTABLE_### for regular plates, DFQC for plate 511 and DFREASON for plate 510. The field names use unique names for user data fields (8 ~ NF-3, where NF is the total number of fields defined for a given plate), and generic names for DFdiscover fields. If set to untyped, all user data fields for regular plates are converted to VARCHAR. Table names used are DFPLATE_### for regular plates, DFQC for plate 511 and DFREASON for plate 510. The field names use unique names for user data fields, and generic names for DFdiscover fields. If set to both, both typed and untyped tables are created for regular plates. Only one DFQC for plate 511 and one DFREASON for plate 510 are created. Both of these tables adhere to typed rules.

-coding code|label|both

Coded field specification. If set to code (or by default), then there is no translation for coded fields. if set to label, then the label corresponding to a given code is imported, including QCs and REASONs. If no label is defined for a given value, then the value is used. If both are requested, then two columns are imported for the coded field, one for the value and the other for the corresponding label.

-missing code|label

Missing value specification only applies to untyped tables. If set to code (or by default), then no translation is done for missing codes. If set to label, then codes are translated to their corresponding label. If no label is defined for a given code, the code itself is imported. If the target tables are typed, missing value codes are automatically replaced with NULL values in all cases.

-date typed|untyped|both

Date type specification for user date fields. If set to typed, then the true type defined in DFschema for dates is used (the default setting). If set to untyped, then dates are converted to strings and imported as type VARCHAR. If set to both then two columns are imported for the date field, one for the typed date and one for the untyped string representation.

-noimpute

No imputation for partial dates replaces partial dates with NULL values for typed dates only. The default is to impute partial dates for typed dates according to the imputation method specified in DFschema.

-missed

Include missed records from the database in the output. Typically missed records are not included as they contain no actual data.

-table all|dfcoding[,dfnullvalue]

List of optional tables to create. If set to dfcoding, the optional table DFCODING is created. Value and label pairs are imported to this table. If set to dfnullvalue, the optional table DFNULLVALUE is created. All problem fields that are converted to NULL are imported to this table. If the option -d drfname is specified, this table is created by default. If set to all, then all optional tables are created and imported.

Required Options

The following two options are required and must appear in order at the end of the option list:

param

A set of parameters of the form server:database:schema[.tablespace][:username:password]. The value of these parameters vary depending on the target database, and are detailed in Table 3.12, “Database Parameters”.

study

the DFdiscover study number, from which data records are to be exported.

Table 3.12. Database Parameters

ParameterDescriptionPostgresMySQLOracleMS SQLServer
serverThe server name to connect torequiredrequireda place holder. Oracle will lookup tnsnames.ora based on database namerequired
databaseThe database namerequiredsee Schemarequiredrequired
schemaThe DFdiscover study namerequiredrequiredrequiredrequired (must be the same as the database name)
tablespaceThe alternative storage tablespace for Oracleignoredignoredsee Schemaignored
username:passwordThe database login user name and passwordoptional.
  1. If both username and password specified, login as specified username with specified password.

  2. If only username specified, login as specified username, lookup the the specified user's password from file ~/.pgpass.

  3. If neither username nor password specified, lookup OS user's name and OS user's password from file ~/.pgpass.

  4. If only password is specified, this is an error.

optional.
  1. If both username and password specified, login as specified username with specified password.

  2. If only username specified, discard the specified username and using OS user's name and lookup OS user's password from file ~/.my.cnf. The same applies if neither username nor password are specified.

  3. If only password is specified, this is an error.

optional.
  1. If both username and password specified, login as specified username with specified password.

  2. If only username specified, discard the specified username, using OS user's name and lookup the password from file ~/.orapass identified by database name. If ~/.orapass does not exist or cannot find the password, use Oracle external credential. The same applies if neither username nor password are specified.

  3. If only password is specified, this is an error.

[Note]Oracle external credentials

The user must have an OS account. In the Oracle initialization file, parameter initDB_NAME.ora must be set as follows

remote_os_authent = true

optional.
  1. If both username and password are specified, login as the specified username with the specified password.

  2. If only username is specified, login using the OS user's name and lookup the OS user's password from the file ~/.mssqlpass.

  3. If neither username nor password is specified, login using the OS user's name and lookup the OS user's password from the file ~/.mssqlpass.

  4. If only password is specified, login using the OS user's name and the specified password.


Exit Status

DFsqlload exits with one of the following statuses:

0

DFsqlload was able to (re-)create the schema, create all of the tables, and import all of the data without error.

1

The schema and tables were created and the data were imported, but one or more errors were encountered in the imported data.

2

A more serious error was encountered which prevented some or all data from being imported.

Description

DFsqlload is a command-line solution that creates all of the table definitions and imports all of the data into a relational database. One SQL table is created for each DFdiscover plate. In addition, three tables are added to record logging information, qc and reason for change data. Optional tables are created to note any problem fields or store value and label pairs for coded fields.

When run repeatedly, existing SQL tables are compared with the current DFschema file. Unchanged tables are truncated i.e., the data is removed but the table definitions remain. If any changes were made, the existing DFdiscover-defined table is dropped and re-created. DFsqlload calls DFexport.rpc to export all primary records to a temporary file, and then loads the database tables plate by plate.

SQL Database Setup

The following operations are performed on the target database.

  1. At least one and optionally three meta tables are created in the target database.

    1. DFSQLLOAD: Each DFsqlload run creates an entry in this log table. This table is also the lock table that prevents more than one DFsqlload process from operating on the same schema. A NULL value for column DFFINISH indicates that a DFsqlload process is running. If DFsqlload terminates abnormally, this record must be removed manually before starting a new DFsqlload process. The following SQL statements are used internally to create this table. The statements use syntax specific to PostgreSQL. Similar tables are created for MySQL, MS SQLServer and Oracle implementations of DFsqlload.

      CREATE TABLE <schema>.dfsqlload
      (
        dfuser varchar(30) NOT NULL,   -- the database user
        dfstart timestamp(0) NOT NULL, -- the start date and time
        dffinish timestamp(0),         -- the finish date and time (NULL if process running)
        dfoption varchar(500),         -- the options used with the DFsqlload command
        dfnull int4,                   -- the total number of problem fields converted to null
        dferror int4,                  -- the total number of records discarded or rejected
        dfstatus int2,                 -- the status: 0=process completed, 1=running
        CONSTRAINT dfsqlload_pk PRIMARY KEY (dfstart)
      ) WITH OIDS;
    2. DFNULLVALUE: This is an optional table where all problem fields that are converted to NULL are recorded. The following SQL statements are used internally to create this table.

      CREATE TABLE <schema>.dfnullvalue
      (
        dfpid int4 NOT NULL,            -- study subject id
        dfplate int2 NOT NULL,          -- plate
        dfseq int4 NOT NULL,            -- sequence/visit number
        dftable varchar(30) NOT NULL,   -- name of sql table where substitution was made
        dffield varchar(30) NOT NULL,   -- name of sql field where substitution was made
        dfvalue varchar(###),           -- the original value exported from DFdiscover
        dfproblem varchar(###),         -- the reason the value was converted to NULL
        dfraster varchar(12),           -- the image id of the source CRF page
        CONSTRAINT dfnullvalue_pk PRIMARY KEY (dfpid, dfplate, dfseq, dftable, dffield)
      ) WITH OIDS;

      The ### above represents the maximum length encountered in the data source.

    3. DFCODING: This is an optional table where all value and label pairs for coded fields are stored. The following SQL statements are used internally to create this table.

      CREATE TABLE <schema>.dfcoding
      (
        dfplate int2 NOT NULL,          -- plate
        dffield varchar(30) NOT NULL,   -- sql table column name
        dfcode varchar(###) NOT NULL,   -- code value for this column
        dflabel varchar(###),           -- code label for this column
        CONSTRAINT dfcoding_pk PRIMARY KEY (dfplate, dffield, dfcode)
      ) WITH OIDS;

      The ### above represents the maximum length encountered in the data source.

  2. Verify any existing SQL tables. The following tasks are performed in the verification of existing SQL tables.

    1. All tables with the prefix DF are treated as DFsqlload-defined tables. For any given run of DFsqlload, only the tables relevant to the current job are used.

    2. Existing SQL table definitions are compared to the DFschema file. DFsqlload will truncate any unchanged tables and re-create changed tables. Changes that may cause DFsqlload to drop a table are as follows:

      • add, delete, reorder fields

      • rename field

      • change data type

      • any change to field size or format causing storage changes in the target database

      • missing code or label changes

      • code or label changes to coded fields

      • SQL tables that have been directly user-modified

    3. All privileges (if any) for tables create by DFsqlload are backed up and restored.

    4. If other tables reference any tables created by DFsqlload, the foreign keys will either be dropped (Postgresql, MS SQLServer) or disabled (MySQL, Oracle) and saved to the log file in the form of a database-specific SQL statement. The user can choose to execute these statements to restore the foreign keys manually, as DFsqlload does not do it automatically.

    5. Other database objects - triggers, views, procedures, etc., which depend upon DFsqlload-defined tables are ignored.

Files used by DFsqlload

All DFsqlload created files are in the study/working_dir/DFsqlload_logs directory by default. If this directory does not exist, DFsqlload will create it at the default location. The file names use time stamp in the format yymmdd_hhmiss as a prefix, where mm is two-digit month and mi is two-digit minute. The time stamp, which is also the value of DFSQLLOAD.DFSTART, is the unique identifier of this DFsqlload process.

  1. Log file: yymmdd_hhmiss.log This file records the detailed loading progress including schema setup, SQL table definitions, record count, and rejected records with error messages in the following formats:

    Regular plates: Record (id, seq, plate, image): error message

    QCs and REASONs: Record (id, seq, plate, image, field): error message

  2. DFdiscover Retrieval File: If path is included, this file will be in the specified location. If the file already exists, the existing file will be removed. If the file cannot be created or written, DFsqlload will report an error and continue the loading process. The file is in standard DRF format. The first four fields are Id, Visit, Plate, and Image. The combination of id, visit, and plate is unique. The 5th field records the list of problems. The record level errors appear first, followed by the field level problems. The field level problems are derived from the DFNULLVALUE table, which is created by default if a .drf file is requested. Problem types are summarized below.

    1. Record level errors

      • error - incorrect number of fields: The number of fields of a record in plt###.dat does not match the DFschema definition.

      • error - invalid DFdiscover field: The value of DFdiscover leading (1~7) or trailing (NF-2~NF) field is blank or invalid.

      • error - duplicate primary record: Two or more records have the same id, seq, plate combination.

      • error - rejected by database: Any field error that was not identified by DFsqlload, but rejected by the database.

    2. Field level problems

      • missing value

      • too wide

      • bad format

      • invalid date

      • partial date

      • undefined code

      • data/type conversion

      The format used in the .drf file is Table1Name: FieldName (problem), FieldName (problem),..., Table2Name: F ieldName (problem),...

    3. QC or REASON specific error

      • error - primary record was rejected.

  3. Data file:  yymmdd_hhmiss_plt### (for regular plates)

    yymmdd_hhmiss_DFreason (for plate 510)

    yymmdd_hhmiss_DFqc (for plate 511)

    This is the data source of current loading plate created by DFexport.rpc, and is removed automatically after the data is loaded into the SQL database.

  4. Error files:  yymmdd_hhmiss_plt###.err (for untyped table)

    yymmdd_hhmiss_tbl###.err (for typed table)

    yymmdd_hhmiss_DFreason.err (for plate 510)

    yymmdd_hhmiss_DFqc.err (for plate 511)

    These files record the records rejected by the SQL database along with database generated error messages.

  5. Rejected keys: yymmdd_hhmiss.tmp.  This file records the keys (plate, seq, id) of primary records discarded by DFsqlload or rejected by the SQL database. If a REASON or QC record matches one of the key combinations in this file, that REASON or QC record will not be loaded into the SQL database and a message will be written to the log file:

    Record (id#,seq#,plate#,image,field#): discarded by DFsqlload.

    DFRECORD (error - primary record was rejected).

    This file is removed automatically.

Database login credentials files.  The following database-specific files may be used to store login credentials.

  • Postgres: ~/.pgpass.  This is a Postgres standard file located in the user's home directory and the file permission must be 600; otherwise, Postgres will ignore it. This file specifies the database login credentials in the format:

    host:port:database:username:password

    The username can be any valid database user, for example:

    parkcity:5432:test_db_name:user_a_name:user_a_password
    parkcity:5432:test_db_name:user_b_name:user_b_password

  • MySQL: ~/.my.cnf.  This is MySQL standard file located in user's home directory and the file permission should be 600. This file specifies the program groups and options for each group. A typical group is [client]. In this group the database password for OS user can be specified:

    [client]
    password=my_pass
    ...

    The global options can be specified in the global option file /etc/my.cnf or DATADIR/my.cnf.

  • Oracle: tnsnames.ora, ~/.orapass.  tnsnames.ora is the Oracle standard net services file located in ORACLE_HOME/network/admin directory by default. Oracle client will lookup net service names from this file. This file can also be in ANY_DIR/network/admin and the environment variable ORACLE_HOME is set to ANY_DIR.

    ~/.orapass is NOT an Oracle standard file. It is provided for convenience for DFdiscover users to store their database password. This file should be in the user's home directory and file permission should be 600. The format is:

    db_name:password

    for example,

    test_db_name:test_db_password
    production_db_name:production_db_password

  • MS SQLServer: ~/.mssqlpass.  This is not an MS SQLServer standard file. It is provided as a convenience to DFdiscover users for storing their database password. This file needs to be kept in the user's home directory and the file permission needs to be 600. The format of this file is:

    server_name:password

    There are two types of entries that can be used. Both the server name and the password to use can be specified as follows:

    my_server:my_server_password

    or a password can be specified for any server this user connects to as in the following example:

    *:any_server_password

Tables

The following details apply to tables and fields created by DFsqlload.

  • Table Names.  Table names follow these rules. DFSQLLOAD is a log table and is used to store the details of a given DFsqlload for a particular schema. The table for plate 510 is named DFREASON. The table for plate 511 is named DFQC. All other plates are named DFPLATE_nnn (untyped) or DFTABLE_nnn (typed), where nnn is the plate number. Optional tables created are DFNULLVALUE for the storage of any problem field data conversions and DFCODING for the storage of value/value label pairs. All table names are in uppercase letters. Note that the table names are case sensitive for MySQL on UNIX platforms only.

  • Table Types.  Table types used for each of the supported database products are as follows: For PostgreSQL and Oracle, all tables are type relational table. For MySQL, all tables are type InnoDB table.

  • Field Names for study tables.  Field naming follows these rules. For fields 1 to 7 and the last three fields for a plate, generic variable names are used. For all fields in between, unique variable names are used. Any field names matching an SQL keyword get an _ (underscore) appended. This is target product dependent. Refer to the documentation for the product you are using for a complete list of the relevant keywords. Any non-alphanumeric characters are replaced with _ (underscore). If a field name starts with a digit, DF_ is prepended. Field names are truncated to 30 chars. A sequence number is appended to each non-unique field name.

  • DFdiscover type to SQL type mapping:  DFsqlload will map DFdiscover types to SQL types according to Data typing when creating SQL tables for typed columns. Untyped columns are mapped to VARCHAR (PostgreSQL, MySQL) and VARCHAR2 (Oracle).

    Table 3.13. Data typing

    DFdiscover typePostgreSQLOracleMySQLMS SQL Server
    stringVARCHAR(n)VARCHAR2(n)VARCHAR(n)[a] VARCHAR(n)
    checkINT2NUMBER(p)INT2SMALLINT
    choiceINT2NUMBER(p)INT2SMALLINT
    integerINT4NUMBER(p)INT4INT
    floatNUMERIC(p,s)NUMBER(p,s)DECIMAL(p,s)DECIMAL(p,s)
    vasNUMERIC(p,s)NUMBER(p,s)DECIMAL(p,s)DECIMAL(p,s)
    number(nn:nn)TIMEVARCHAR2(n)TIMEVARCHAR(n)
    dateDATEDATEDATEDATETIME
    timestampTIMESTAMPDATEDATETIMEDATETIME

    [a] MySQL converts VARCHAR(n) to CHAR(n) (n < 4) or TEXT (n > 255).


  • SQL column naming convention:  DFsqlload will follow the rules defined in Fields for typed SQL tables and Fields for untyped SQL tables for DFdiscover field name to SQL column naming convention when creating SQL tables.

    Table 3.14. Fields for typed SQL tables

    FieldField NumberField NameField TypeMissing Code/LabelPartial DateImpute
    Coded field: Code1~7,NF-2~NFGenericTrue TypeNo  
    Coded field: Label1~7,NF-2~NFGenericVARCHARNo  
    Coded field: Code [a] 1~7,NF-2~NFGenericTrue TypeNo  
    Coded field: Label [b] 1~7,NF-2~NFU_GenericVARCHARNo  
    Coded field: Code8~NF-3UniqueTrue TypeNo  
    Coded field: Label8~NF-3UniqueVARCHARYes  
    Coded field: Code[a]8~NF-3UniqueTrue TypeNo  
    Coded field: Label[b]8~NF-3U_UniqueVARCHARYes  
    Date field: Typed8~NF-3UniqueTrue TypeNoNoYes
    Date field: Untyped8~NF-3UniqueVARCHARYesYesNo
    Date field: Typed[a]8~NF-3UniqueTrue TypeNoNoYes
    Date field: Untyped[b]8~NF-3U_UniqueVARCHARYesYesNo
    Other fields8~NF-3UniqueTrue TypeNo  
    Other fields1~7,NF-2~NFGenericTrue TypeNoNoNo

    [a] The first field, if -coding both or -date both is specified.

    [b] The second field, if -coding both or -date both is specified.


    Table 3.15. Fields for untyped SQL tables

    FieldField NumberField NameField TypeMissing Code/LabelPartial DateImpute
    Coded field: Code1~7,NF-2~NFGenericTrue TypeNo  
    Coded field: Label1~7,NF-2~NFGenericVARCHARNo  
    Coded field: Code[a]1~7,NF-2~NFGenericTrue TypeNo  
    Coded field: Label[b]1~7,NF-2~NFU_GenericVARCHARNo  
    Coded field: Code8~NF-3U_UniqueVARCHARYes  
    Coded field: Label8~NF-3U_UniqueVARCHARYes  
    Coded field: Code[a]8~NF-3UniqueVARCHARYes  
    Coded field: Label[b]8~NF-3U_UniqueVARCHARYes  
    Date field: Typed8~NF-3U_UniqueTrue TypeNoNoYes
    Date field: Untyped8~NF-3U_UniqueVARCHARYesYesNo
    Date field: Typed[a]8~NF-3UniqueTrue TypeNoNoYes
    Date field: Untyped[b]8~NF-3U_UniqueVARCHARYesYesNo
    Other fields8~NF-3U_UniqueVARCHARYes  
    Other fields1~7,NF-2~NFGenericTrue TypeNoNoNo


  • Locking:  DFsqlload uses the table DFSQLLOAD to prevent two processes from working on the same database schema. A NULL entry in DFSQLLOAD.DFFINISH indicates that another process is running or terminated abnormally. If DFSQLLOAD does not exist, DFsqlload will create it. Upon completion, DFsqlload updates DFSQLLOAD.DFFINISH to the finish time and the DFSQLLOAD.DFSTATUS to zero (normal exit process).

  • DFdiscover Retrieval File:  The first line in the DRF contains a two-field comment. The first field contains the username and creation timestamp. The second field identifies the creator of the DRF as DFsqlload and lists the parameters used to access the SQL database. The next line is a comment describing the format of the DRF records to follow. One DRF data record is created for each DFdiscover record with data import problems. Each DRF data record is identified by Id, Visit, Plate and Image. The 5th field records the SQL table name, followed by the field name and problem description for each problem encountered. Multiple field/problem descriptions are separated by commas.

  • Date fields:  DFsqlload converts two-digit years to four-digits based on the cut off year specified for each date field in DFschema, or the year the study began (%B) if not specified. DFsqlload imputes partial dates according to the imputation method specified for each date field.

  • Number of fields:  DFsqlload creates a minimum of 10 fields (1~7,NF-2~NF) for each SQL table. The maximum number of fields is limited by the database: PostgreSQL 1600, MySQL 1000, Oracle 1000, MS SQLServer 1024. DFsqlload will report errors for DFdiscover records that do not meet these field requirements and will continue the loading process.

Schema

Postgres.  A schema is a namespace that contains DFdiscover study tables. Schema names are DFdiscover study names. If the schema name specified in the DFsqlload command line does not exist in the Postgres database, DFsqlload will create the schema as specified. DFsqlload will never drop existing schemas.

Oracle.  A schema is a database user who is also the owner of DFdiscover tables that belong to one study. The schema specified in the command line must already exist in Oracle, unless the tablespace name (must exist in database) is also specified. If the schema does not exist, DFsqlload will create the schema and assign the specified tablespace as the schema's default tablespace. The DFdiscover study tables will be created, if specified, in the specified tablespace, otherwise in the schema's default tablespace. DFsqlload will check the schema's quota privilege on storage tablespace and grant unlimited quota privilege on that tablespace. DFsqlload will never drop existing schemas.

MySQL.  There is no schema in MySQL. A MySQL database maps to a DFdiscover study database. In the command line, the schema name must be the same as the database name. Note that the database name is case sensitive whether MySQL is running on a UNIX or Windows platform. If the specified database does not exist in the MySQL database, DFsqlload will create the database as specified. DFsqlload will never drop existing MySQL databases.

MS SQLServer.  There is no schema in MS SQLServer in version 2000 and older. A SQLServer database maps to a DFdiscover study database. In the command line, the schema name must be the same as the database name.

Examples

Example 3.72. Import study 254 into mySQL

Import the validation study val254 into mySQL on host talisman.

% DFsqlload -flavor mysql -q talisman:val254:val254:root:mysql 254



DFstatus

DFstatus — Display database status information in plain text format

Synopsis

DFstatus [-l] [-m] [-h] [-c] [-n #, #-#] [-i #, #-#] [-v #, #-#] [-p #, #-#] [-s #]

Description

This command-line invocation of the DFexplore Status View generates a tabular representation of the database status in plain text format and writes the results to standard output.

Options

-c

execute DFstatus in command-line mode. This option is retained for backward compatibility but is no longer necessary as DFstatus now runs only in command line mode.

-l

displays a list of current user logins with the hostname of the system running their DFdiscover applicataion.

-h

removes the title header when executed with the -l option.

-m

displays current seat usage statistics for the current DFdiscover server.

-n #, #-#

select only the requested site IDs for inclusion in the database status table. Default is all sites.

-i #, #-#

select only the requested subject IDs for inclusion in the database status table. Default is all subject IDs.

-v #, #-#

select only the requested visit numbers for inclusion in the database status table. Default is all visit numbers.

-p #, #-#

select only the requested plate numbers for inclusion in the database status table. Default is all plate numbers in the range 1 to 500, inclusive.

-s #

the study number (required unless -l or -m option is used).

Exit Status

DFstatus exits with one of the following statuses:

0

The command was successful.

36

The required command-line arguments were not present or were incorrectly specified.

> 0

The command failed because the database server could not be contacted, or communication with the database server failed.

Examples

Example 3.73. Display record status and current user connections for study 253

%  DFstatus -s 253
#Database Status of Study 253
#Date           Mon Mar 14 09:45:34 2011
#Sites
#Subjects
#Visits
#Plates
 Level       Pending        Missed    Incomplete         Final         Total
     0             7             0             0             0             7
     1             0             2            32            43            77
     2             0             0             4             2             6
     3             0             0             3             1             4
     4             0             0             1             7             8
     5             0             0             0             1             1
     6             0             0             0             0             0
     7             0             0             0             0             0
 Total             7             2            40            54           103

#Records awaiting validation: 18
#Records being validated: 0

#Connected Users
User             Program          Host
---------------- ---------------- ------------------------------
datafax          DFexplore        vncdemo.datafax.com
datafax          Status Tool      dhcp214.datafax.com

Example 3.74. Display current seat usage on the server

%  DFstatus -m
Seats in Use:
 Admin   2 of 10
 General 122 of 150
 Total   124 of 160


DFtextps

DFtextps — Convert one or more input files into PDF

Synopsis

DFtextps [-PDF] [-A4] [-h] [-n #] [-d] [ [-1] | [-2] | [-4] ] [-p #] {file...}

Description

DFtextps is a utility program that converts plain text input files to PDF. If no input files are named, the input is taken from standard input.

The resulting PDF is sent to standard output where it can be re-directed to a file.

Options

-PDF

create a PDF output file. This is the default behavior. The option is present for backwards compatibility only.

-A4

format for A4 size paper. The default is US letter.

-h

apply a standard header to each page.

-n #

start the page numbering at # instead of 1.

-d

double-space the output.

-1, -2, -4

format the output in 1-up, 2-up, or 4-up mode

-p #

use # as the default point size.

file

one or more input files.

Exit Status

DFtextps exits with one of the following statuses:

0

The command was successful.

1

One or more errors were encountered.

36

The required command-line arguments were not present or were incorrectly specified.

Examples

Example 3.75. Print output from DF_SSvisitmap report

Run the DFdiscover report DF_SSvisitmap for study 1, saving the output as a PDF document.

% /opt/dfdiscover/reports/DF_SSvisitmap 1 | /opt/dfdiscover/bin/DFtextps > visitmap.pdf



DFuserdb

DFuserdb — Perform maintenance operations on the user database

Synopsis

DFuserdb [-help] [-fsck] [-reset] [-stats] [-unlock] [-migrate] [-export filename] [-import filename]

Description

DFuserdb is used to perform maintenance operations such as import/export on the user and role database.

Invoked without arguments DFuserdb works in interactive mode. Allowable commands match the options listed here, but without the leading dash.

Options

-help

print a list of the commands available.

-fsck

perform a consistency check on the database. If there are no consistency errors in the database structure only the message FSCK done will be output. If errors do exist, the database will need to be rebuilt by using the -import option along with a copy of the DFuserdb.log as input.

-reset

used to reset the datafax password and administrator status in cases where it has been accidentally altered.

-stats

display information about the database version and the number of records in the database for each of the indices.

-unlock

reset the database synchronization locks in cases where they may be stuck after system failures.

-migrate

convert a pre-3.8 DFdiscover user and permissions databases to the new database. It must be run as root and can only be run on a newly created database. The -migrate option reads the DFperm file and consults /etc/passwd and the ROUTER_USERS parameter in to create user entries and then traverses all studies listed in , reads the lib/DFaccess_perms and lib/.DFrep_excl files and generates roles, role permissions and user role entries.

-export outfile

export (write) the user and role database information to the argument filename.

-import infile

import (read) user and role database information from the argument filename. The file is expected to be in the DFuserdb.log format. If the entire database is to be rebuilt, the old DFuserdb.log and DFuserdb.idx files should first be removed. In this case, DFuserdb will create a new database and import the file.

Exit Status

DFuserdb exits with one of the following statuses:

0

The command was successful.

2

An error occurred.


DFversion

DFversion — Display version information for all DFdiscover executables (programs), reports, and utilities

Synopsis

DFversion

Description

DFversion invokes the DFwhich command for every DFdiscover executable, DFdiscover report and DFdiscover utility included in a standard installation.

Output is grouped by executables, reports and utilities. Within each group, output is sorted alphabetically by name.

DFversion is the most efficient and accurate way to determine the current state of an installation.

Options

None.

Exit Status

DFversion exits with one of the following statuses:

0

The command was successful.

See Also

DFwhich

DFwhich

DFwhich — Display version information for one or more DFdiscover programs, reports and/or utilities

Synopsis

DFwhich {name...}

Description

DFwhich examines and displays the RCS (revision control system) string of the specified filename(s), in a manner similar to the UNIX what command.

The output from the DFwhich command can be useful in determining exactly what version of the DFdiscover software is currently being used.

Options

name

one or more names of DFdiscover programs, reports, or utilities

Exit Status

DFwhich exits with one of the following statuses:

0

The command was successful.

Examples

Example 3.76. Show the version information for DFedcservice

% DFwhich DFedcservice
DFedcservice: Version: 2016.0.0, Date: Apr 29 2016 DF/Net

Limitations

DFwhich uses standard UNIX commands to locate version information. The content of the output produced will be essentially the same but may differ in appearance due to different implementations of these commands in flavors of the Linux operating systems.

See Also

DFversion


[1] This output format matches the output from DFlistplates.rpc.

[2] DFexport typically uses a word to specify an option while DFexport.rpc uses an option letter.

[3] For backwards compatibility, the legacy status keywords (clean, dirty, error, CLEAN, DIRTY, ERROR, missed, primary, secondary, all) are currently supported but will be removed in a future release. Do not rely upon the availability of the legacy status keywords.

[4] Matching occurs before any output formatting (such as variable decoding or string splitting) is applied.

[5] Choice and check field values are NOT zero-padded and hence substring extraction for those data types may yield unexpected or unreliable results.

[6] This option has no effect when applied to exported query or reason records.

[7] If DFexport is invoked from a cron facility, absolute pathnames are highly recommended.

[8] Data written to standard output will also include "inline" header metadata if either of the -h or -x options are specified.

[9] The $ character may need to be escaped with a backslash to avoid interpretation as a shell variable.

[10] Combining -hd with $plt###.png or $DFbkgd###.png will silently ignore the -hd argument.

[11] If the string contains characters that are meaningful to the shell, such as space, quote, percent, etc., then the entire string must be enclosed in double quotes. The string can always be enclosed in double quotes, to ensure that no characters are interpreted by the shell.

[12] If the string contains characters that are meaningful to the shell, such as space, quote, percent, etc., then the entire string must be enclosed in double quotes. The string can always be enclosed in double quotes, to ensure that no characters are interpreted by the shell.

[13] If DFreport is invoked from a cron facility, absolute pathnames are essential.

Chapter 4. Utility Programs

Table of Contents

4.1. Introduction
4.2. Alphabetical Listing
DFaddHylaClient — Create the symbolic links necessary for accessing HylaFAX on a DFdiscover server
DFcertReq — Request an SSL certificate signing for DFedcservice.
DFclearIncoming — Clean out the fax receiving directory, processing all newly arrived faxes
DFcmpSchema — Apply the data dictionary rules against the study database
DFcmpSeq — Determine the appropriate values for each .seqYYWW file
DFisRunning — Determine if the DFdiscover master program is currently running on the licensed DFdiscover machine
DFmigrate — Upgrade study setup and configuration files from an old DFdiscover version to the current version.
DFras2png — Convert Sun raster files in the study pages directory into PNG files
DFshowIdx — Show the per plate index file(s) for a specific study
DFstudyDiag — Report (diagnose) the current status of a study database server
DFstudyPerms — Report, and correct, the permissions on all required DFdiscover sub-directories and files for a study
DFtiff2ras — Convert a TIFF file into individual PNG files
DFuserPerms — Import and update users and passwords, and optionally import roles, role permissions, and user roles

4.1. Introduction

DFdiscover includes several utility programs that can be executed from a command line. Most of these programs are useful in situations where repairs or corrections within your DFdiscover environment are needed. They are located in the DFdiscover utilities directory, /opt/dfdiscover/utils.

4.2. Alphabetical Listing

DFaddHylaClient

DFaddHylaClient — Create the symbolic links necessary for accessing HylaFAX on a DFdiscover server

Synopsis

DFaddHylaClient

Description

DFdiscover uses HylaFAX to send and receive faxes. However, as installed by the DFdiscover INSTALL program, HylaFAX is not fully configured to run on a fax server workstation. DFaddHylaClient makes the links necessary to allow HylaFAX to operate.

Each workstation that will be used for receiving incoming faxes or sending faxes, must have this utility command run on it. The command must be run by someone with super-user privileges and needs to be run only once per workstation (unless the location of the DFdiscover software is subsequently changed).

As distributed, HylaFAX expects to find all of the fax resources it needs in the /opt/hylafax directory. However, those resources are actually in /opt/dfdiscover/$MACHINE/hylafax. DFaddHylaClient makes symbolic links to resolve this inconsistency. Additionally, DFaddHylaClient creates a HylaFAX work directory in /var/spool/fax. If the directory already exists, the necessary HylaFAX files are copied into the directory.

[Note]Note

This program is generally executed as part of the HylaFAX configuration in a new DFdiscover install, and subsequently never needs to be executed again.

Options

None.

Exit Status

DFaddHylaClient exits with one of the following statuses:

0

The command was successful.


DFcertReq

DFcertReq — Request an SSL certificate signing for DFedcservice.

Synopsis

DFcertReq

Description

DFcertReq is used to generate an SSL key and a signing request for use with DFedcservice. It must be run by root or datafax. DFserveradmin also provides this functionality in a point-and-click visual interface. Most administrators will find DFserveradmin to be the preferred interface for this task.

[Note]Note

There is no requirement to use DFcertReq and DF/Net Research, Inc. as the SSL certificate signing authority. There are many standard, commercial certificate signing authorities (known as CAs) that are internationally recognized. For a small annual fee paid to the CA, they will sign your certificate. Their signed certificate can be used in your DFdiscover installation. Some clients prefer this approach of using an independent CA.

DFexplore, DFsetup, DFadmin and DFsend communicate with DFedcservice using TLS (Transport Layer Security) [14] in the same way that Internet banking sites do. TLS provides an encrypted path through the internet that prevents eavesdropping and modification of your data by third parties. The client applications check that they are communicating with the correct DFdiscover server by means of a certificate that encodes the DFdiscover server's name and ownership. This certificate is generated by choosing a very large random number (specifically a 4096-bit RSA key), adding organizational ownership information to it and then requesting that DF/Net Research, Inc. certify this key as authentic. Subsequently when an DFexplore, DFsetup DFadmin or DFsend client connects to DFedcservice, it asks for this certificate and can then determine whether it is communicating in an encrypted manner with the correct server.

The process of generating a certificate starts with the execution of the DFcertReq script. This script generates a large random number and then prompts for organizational information, including the country, state, organization name and server name.

[Important]Important

It is extremely important that the organizational information is up-to-date and accurate. Client applications may view this information to confirm the server's identity and certificate status.

The organizational information is combined with the large random number to create a unique certificate signing request text file. An email is created for containing a request to certify that this is an authentic DFdiscover server. DF/Net Research, Inc. processes this request and emails back a small file containing the certificate, which is then installed in the DFdiscover system. At the end of these steps, communication between the DFexplore, DFsetup, DFadmin and DFsend client applications is encypted and secure.

DFcertReq will fail to email the signing request if the computer from which it is run is unable to send email via the internet. In this case, it is possible to manually generate the request by:

  1. Transferring the files /tmp/cert.csr.text and /tmp/cert.csr, that are generated by DFcertReq, to an email enabled computer. Remember to perform the transfer in binary mode if using an application like ftp.

  2. Attach the two transferred files to a new email message and send it to .

Impact on Login Dialog Banner

At the time of executing DFcertReq, if there is no /opt/dfdiscover/lib/DFlogin.html file present, DFcertReq will also create the file, adding the organizational information collected from the user's responses. Subsequently, this information appears in the banner of each DFdiscover login dialog. To override this behaviour, before executing DFcertReq, create your own login banner message in /opt/dfdiscover/lib/DFlogin.html.

Options

None.

Exit Status

DFcertReq exits with one of the following statuses:

0

Always.

Examples

Example 4.1. Creating a DFedcservice SSL certificate and signing request

# DFcertReq
*****************************************************************
* When asked for 'DFdiscover Server Name (fully qualified domain name)'
* type the full name of the machine (e.g. dfdiscover.mycompany.com)
* as it is called from the Internet.
*****************************************************************
----------------- Generating Server Key ----------------
Generating RSA private key, 4096 bit long modulus (2 primes)
...............................................++++++
..................++++++
e is 65537 (0x10001)
----------------- Decrypting Server Key ----------------
writing RSA key
----------------- Generating Server Signing Request ----------------

You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [US]:
State or Province Name (full name) []:California
Locality Name (eg, city) []:San Diego
Organization Name (eg, company) []:Pharmadrug Biotech Inc.
Organizational Unit Name (eg, section) [Research]:
DFdiscover Server Name (fully qualified domain name) [dfdiscover.your-company.com]:dfdiscover.pharmadrug.com
Email Address [support@your-company.com]:support@pharmadrug.com

Emailing certificate signing request to DF/Net Research, Inc...

Once the certificate has been signed it will be emailed back to you.



[14] Specifically, TLS v1.2 or v1.3 is used.


DFclearIncoming

DFclearIncoming — Clean out the fax receiving directory, processing all newly arrived faxes

Synopsis

DFclearIncoming

Description

DFclearIncoming uses DFinnotify.rpc to submit all unprocessed faxes in the DFdiscover received fax (incoming) directory to the defined incoming DFdiscover daemons.

Under normal circumstances, this command is not needed as DFdiscover automatically processes each incoming fax as receipt completes.

This command is also automatically invoked by the DFdiscover bootstrap procedure when the software is restarted to process any faxes that were received while DFdiscover was not operational.

If one or more incoming daemons are processing when DFclearIncoming is started, DFclearIncoming will wait until all of the daemons have exited. Then it will empty out the /opt/dfdiscover/work/.dfincoming_work file and submit for processing all of the faxes that appear in the DFdiscover incoming directory.

Options

None.

Exit Status

DFclearIncoming exits with one of the following statuses:

0

The command was successful.

Examples

Example 4.2. Clean out the incoming fax directory

# DFclearIncoming
1. Checking state of incoming daemons...
2. Checking /opt/dfdiscover/work/.dfincoming_work...
  2 stale incoming entries obsoleted
3. Checking /opt/dfdiscover/incoming...
  1 faxes awaiting processing.
  Processing fax00010.tif


DFcmpSchema

DFcmpSchema — Apply the data dictionary rules against the study database

Synopsis

DFcmpSchema [-a] [-v] [-p #] [-d DRF_filename] {-s #}

Description

DFcmpSchema performs a consistency check of the database against the data dictionary for the study. It reports:

  • records that have an incorrect number of fields

  • records that have illegal status, validation levels, or time stamps

  • records that have an impossible CRF image reference

  • key fields that are illegal or inconsistent with the current study or plate

  • values that occupy more columns than the maximum defined for the field

  • values that have an incorrect format

  • fields that have impossible values

  • choice and check fields whose data value does not match the coding defined in the study schema

  • missing delimiter at the end of data records for user defined CRF plates

  • missing or empty required fields on clean primary or secondary data records

  • date values that contain digits, characters, or ?? on clean primary and secondary records, and do not conform to the field imputation or partial rounding variable format

When the data dictionary for an existing study is changed via DFsetup, DFdiscover does not retroactively modify the database to match the new dictionary. DFcmpSchema is useful in this circumstance to identify and locate existing data that now violates the data dictionary.

DFcmpSchema applies two types of checking: basic and exhaustive checking. Basic checking is the default and includes all but the last two bulleted items described above. Exhaustive checking includes everything verified in basic checking plus the last two items for missing/empty required fields and inconsistent date values. Exhaustive checking can be performed by running DFcmpSchema with the -v option.

By default, DFcmpSchema applies basic checks to all primary records in the database, excluding any records in the new record queue. New records are only checked if plate zero is explicitly specified. Because the data fields in new records have not yet been verified, DFcmpSchema only checks fields 1-7, i.e. checking stops at the subject ID field.

For each inconsistency, the report includes the record, plate number, the line number in the exported data file, a synopsis of the inconsistency, the key fields of the record (so it can be retrieved with DFexplore), the data dictionary definition, and the current data value.

Options

-a

Check all primary and secondary records. The default is to check only primary records. [15]

-v

Apply exhaustive checking. Exhaustive checking includes all basic checking plus additional checks on clean records only for

  • missing or empty required fields

  • date values that do not conform to the field's imputation or partial rounding specifications

-p #

Check only the requested plate number. The plate number can be one of the user-specified plates, plate 0 (new record queue), plate 510 (reason for data change), or plate 511 (query).

-d DRF_filename

Create a DFdiscover retrieval file for all problems identified.

-s #

Study number (required).

Exit Status

DFcmpSchema exits with one of the following statuses:

0

The command was successful.

36

The required command-line arguments were not present or were incorrectly specified.

2

The command failed because the study number was not defined, the study schema could not be read, the requested plate number was not defined, or communication with the database server failed.

Examples

Example 4.3. Report on all inconsistencies for study 240

% DFcmpSchema -s 240
2|1|0245R0032001|240|1||1||0|0||||||||2|02/11/11 13:26:52|02/11/11 13:26:52|
E** Plate 1 (Screening Form), line 1: Incorrect field count: 20 should be 21.

Example 4.4. Perform exhaustive checking for inconsistencies on plate 2 for study 254

% DFcmpSchema -v -p 2 -s 254

1|7|0436R0008002|254|2|1|10052|KKL|23/06/04||09/09/24|||180||080|1|1|23/07/04|1|04/09/08 11:08:15|04/11/11 09:33:48|
E** Plate 2 (Patient Entry Form), line 2: Required data field.
        Subject ID='10052', Visit='1'
        Field#10  (status=1, validation=7)
        Name='MEDCODE'
        Desc='Medication Code 1'
        Type=INT      Required  Width=4
        DATA LEN=0 ''

1|7|0436R0006002|254|2|1|10056|POL|00/04/03|1112|07/08/24||111.1|166||070|1|1|05/06/04|1|04/09/08 10:03:14|04/11/11 09:45:32|
E** Plate 2 (Patient Entry Form), line 3: Bad data.
        Subject ID='10056', Visit='1'
        Field#9  (status=1, validation=7)
        Name='EDATE'
        Desc='Entry Date'
        Type=DATE     Required  Width=8   Format='dd/mm/yy'
        Imputation='never'  Range='1940 - 2039'
        DATA LEN=8 '00/04/03'



[15] Missed records are never checked.


DFcmpSeq

DFcmpSeq — Determine the appropriate values for each .seqYYWW file

Synopsis

DFcmpSeq

Description

This command must be run by user datafax or root.

DFcmpSeq verifies the contents of the sequence files maintained by DFdiscover in the /opt/dfdiscover/work directory. It verifies for each file that the number stored in the file is:

  • greater than the highest sequence number assigned to a file in the TIFF archive directory (by any daemon)

  • greater than the highest sequence number referenced in the DFdiscover maintained fax_log file

In unusual circumstances it may be that a sequence file cannot be updated by DFdiscover when it should be. This can subsequently lead to problems processing incoming faxes because new sequence numbers assigned by DFmaster.rpcd overlap with previously used sequence numbers. Incoming fax processing will fail and messages will be logged while this condition persists. DFcmpSeq must be used in this case to determine what the correct values are for the sequence files.

Typically this command is required only in circumstances where a sequence file has been accidentally deleted and its contents need to be re-created.

Options

None.

Exit Status

DFcmpSeq exits with one of the following statuses:

0

The command was successful.

2

The command failed because the configuration of the incoming daemon could not be read.

Examples

Example 4.5. Report on the current status of the sequence files

In this example, DFcmpSeq finds two problems. The solutions are to insert 286 in /opt/dfdiscover/work/.seq9245 to fix the first problem and 3 in /opt/dfdiscover/work/.seq9401 to fix the second problem.

# DFcmpSeq
checking daemon 1, configuration file /df/lib/DFinbound.1...
Checking archive directory /archive/g3...
checking YYWW 9618...
checking YYWW 9621...
checking YYWW 9622...
checking YYWW 9604...
checking YYWW 9605...
checking YYWW 9606...
checking YYWW 9607...
checking daemon 2, configuration file /df/lib/DFinbound.1...
Checking .seq files...
Checking fax_log...
YYYY/WW   Archive   Faxlog  SeqFile  Error
1992/45         0      285      279  FAXLOG > SEQFILE
1994/01         2        0        0  ARCHIVE > SEQFILE


DFisRunning

DFisRunning — Determine if the DFdiscover master program is currently running on the licensed DFdiscover machine

Synopsis

DFisRunning [-q]

Description

DFisRunning is a utility program that determines whether or not DFmaster.rpcd is currently running on the licensed DFdiscover machine. It is most useful in shell scripts and cron jobs that need to determine if DFdiscover is operational before proceeding.

Options

-q

execute in quiet mode. Do not echo any messages to standard output. The result of the command can be determined from the command status, where 0 indicates that the master is not running, and 1 indicates that it is.

Exit Status

DFisRunning exits with one of the following statuses:

0

The DFdiscover master is not running.

1

The DFdiscover master is running.

2

There is an error in the command-line arguments or the hostname executing the DFdiscover master cannot be determined/located.

Examples

Example 4.6. Report on the current status of the DFdiscover master, first in quiet mode and then in non-quiet mode

% DFisRunning -q
% echo $status
1
% DFisRunning
DFmaster is running on 'oberon'.
% echo $status
1


DFmigrate

DFmigrate — Upgrade study setup and configuration files from an old DFdiscover version to the current version.

Synopsis

DFmigrate [-f] [-a] [-s #, #-#] [# study_directory]

Description

Several changes were introduced in release 2014.0.0 that are incompatible with earlier releases of DFdiscover. As a result any ongoing studies created in a release previous to 2014.0.0 must be migrated in order to make them compatible. DFmigrate is provided for this purpose. Until a study is migrated, it is not possible to use it in the current release of the software.

Studies that have been created in release 2014.0.0, or newer, do not need to be migrated. Similarly, studies that were previously migrated to be compatible with 2014.0.0, do not require (re-)migration.

[Important]Important

It is strongly recommended that you create a backup of your original study directories prior to migration. If you have not already backed up all studies prior to the new installation you should do it now. The migration process does not touch study data or faxed images, only study setup files including plate backgrounds are migrated.

The DFmigrate utility must be run by either user datafax or root. DFdiscover does not need to be running to run DFmigrate. If DFdiscover is running at the time of migration, the study to be migrated must not be in use. DFmigrate can be used to migrate all studies at once or one study at a time. The migration process follows the same steps regardless of the number of studies being migrated. The steps of the migration process are documented in the DFmigrate output as follows:

  1. Sanity Checks.  Before migrating any studies DFmigrate checks to ensure that:

    1. no study directory is nested within another study directory,

    2. each study directory and configuration file belongs to exactly one study [16]

    If a failure is detected in either of these requirements, DFmigrate terminates with an error message and no studies are migrated.

  2. Updating DFserver.cf.  This step updates the existing DFserver.cf file found in STUDY_DIR/lib to include new Minimum Version parameters. A copy of the original DFserver.cf file is created as DFserver.cf_oldversion and stored in STUDY_DIR/lib prior to the conversion. The study's minimum version restriction for DFexplore and DFsetup are set to the same version as DFmigrate (i.e. the current DFdiscover version).

  3. Converting DFsetup.  This step converts the existing DFsetup, DFschema, DFschema.stl, DFtips and DFcterm_map files found in STUDY_DIR/lib to the format expected by the current version. Copies of the original files (called DFsetup_oldversion, DFschema_oldversion, DFschema.stl_oldversion, DFtips_oldversion, DFcterm_map_oldversion) are created and stored in STUDY_DIR/lib prior to the conversion. The QC_Report_ID style is converted from type string to type number. If user-defined string styles also exist with the "Treat As Pre-printed Numerals" attribute, these too will be converted to type number during migration. If plate triggered procedures (Pre- and Post-processes) have been defined for any of the plates in DFsetup, a new plate arrival trigger file is created for each Pre- and Post-process. The plate arrival trigger files are named DFPrePost###.sh, where ### represents the plate number on which the process is defined. These files are saved to the study directory STUDY_DIR/ecbin. If a plate contains both a Pre- and Post-process, both processes will be merged into a single plate arrival trigger file.

    This step is also responsible for creating the following directories in the STUDY_DIR parent directory: ecbin, dfsas, drf and dfschema, which did not exist in earlier releases.

    Where they are not already defined, custom level labels are set to the default label values Level 0, Level 1,... , Level 7.

  4. Moving Lookup Tables to lut directory.  All lookup tables must reside in either a study lookup table directory, specifically STUDY_DIR/lut, or in a DFdiscover level lookup table directory, named /opt/dfdiscover/lut. DFdiscover looks for referenced lookup tables first at the study level and then if they cannot be found there at the DFdiscover level. STUDY_DIR/lut is created and lookup tables defined in the study lookup table map file, STUDY_DIR/lib/DFlut_map, which can be found in STUDY_DIR/lib are moved to STUDY_DIR/lut. DFlut_map itself continues to reside in STUDY_DIR/lib/DFlut_map.

    [Note]Note

    Lookup tables which do not reside in $STUDY_DUR/lib, and were instead defined in STUDY_DIR/lib/DFlut_map with an absolute path location, will not be moved by DFmigrate and must be moved manually to either STUDY_DIR/lut or /opt/dfdiscover/lut. Also STUDY_DIR/lib/DFlut_map must be updated to remove any absolute pathnames to lookup tables.

  5. Moving Edit checks code to the ecsrc directory.  Edit checks are stored in the STUDY_DIR/ecsrc directory. If necessary this directory is created and the pre-migration edit check file DFedits is moved to this directory. Published edit check file DFedits.bin, which is used by DFexplore, is not moved and continues to reside in STUDY_DIR/lib.

    [Note]Note

    Other edit check source files (i.e. any include files specified in DFedits) are not moved by DFmigrate and must be moved manually to the ecsrc directory in either STUDY_DIR or /opt/dfdiscover. Like lookup tables, DFdiscover now looks for edit check include files first in the study directory STUDY_DIR/ecsrc and if not found, then in the system location /opt/dfdiscover/ecsrc. All include files must be referenced by their file name alone, and an absolute path name is not allowed.

  6. Converting postscript and old TIFF files to PNG.  A DFbkgdXXX.png file is created for each plate background image found in the study STUDY_DIR/bkgd directory. These PNG files are necessary if you use the DFdiscover DFprintdb program to print CRF backgrounds merged with data records from the study database. These high-resolution PNG files are also used by DFexplore to print CRF backgrounds. The PNG files are created by converting existing high-resolution tiff files. If tiff files do not exist then the PNG files are generated by amalgamating the DFbkgd-prologue.ps file and the individual DFbkgdXXX.ps files for each plate and using Ghostscript (DFgs) to convert the merged PostScript files into high resolution PNG files.

    [Note]Note

    If the PostScript conversion performed in this step fails, the PNG files can be created by re-importing and saving the CRF backgrounds in the DFsetup tool.

If during execution of DFmigrate any of the above steps fail, DFmigrate can be re-run for the same study after the outstanding issues have been resolved.

DFmigrate appends a title (which includes: study number, study directory, date and time) followed by any error or warning messages for each migrated study to a log file named dfmigrate_errors which is created in the directory from which DFmigrate is run if it does not already exist. This file should be checked when DFmigrate ends. It may contain messages describing manual steps that need to be taken to complete the migration of one or more studies.

When migrating pre 4.1 studies, if the -a (migrate all studies) option is used an entry is added to DFstudyspaces.db for each unique study parent directory found in DFstudies.db; and if the -s option is used to migrate a single study an entry is added for that study alone, unless it's study space is already defined.

Once migration has been successfully completed for a study the DFdiscover utility program DFstudyPerms should be run to verify and, if necessary, correct any permission problems that may exist for the study directories and files.

Options

-f

force the conversion of all plate backgrounds if there are no PNG backgrounds already existing in STUDY_DIR/bkgd.

-a

migrate all studies found in /opt/dfdiscover/lib/DFstudies.db

-s #, #-#

migrate the specified study numbers (required). /opt/dfdiscover/lib/DFstudies.db is read to determine the study directory corresponding to each study number.

# study_directory

migrate the specified study number (required) that is located in the specified study directory

Exit Status

DFmigrate exits with one of the following statuses:

0

DFmigrate ran successfully

> 0

DFmigrate failed, Any output error messages are written to the dfmigrate_errors file in the working directory.

Examples

Example 4.7. Migrate study 100 to the current version

DFmigrate gets the correct path for study 100 from its entry in /opt/dfdiscover/lib/DFstudies.db.


% DFmigrate -s 100

**********************************************************
DFmigrate: Study 100 - /opt/studies/test100 - Thu Apr 15 12:58:40 2010
**********************************************************
Step 1: Updating DFserver.cf
------------------------------------------------------------
File Converted: /opt/studies/test100/lib/DFserver.cf
Step 2: Converting DFsetup
------------------------------------------------------------
OLD SETUP FILE <DFsetup v3.7.0>

File Created: /opt/studies/test100/ecbin/DFPrePost001.sh.
File Created: /opt/studies/test100/ecbin/DFPrePost002.sh.
File Created: /opt/studies/test100/ecbin/DFPrePost003.sh.

NOTE: Style type for the following styles has been been changed to
Number as it was previously set to treat data as pre-printed numerals:
        STYLE
        --------------
        QC_Report_ID

File Converted: /opt/studies/test100/lib/DFsetup
File Converted: /opt/studies/test100/lib/DFschema
File Converted: /opt/studies/test100/lib/DFschema.stl
File Converted: /opt/studies/test100/lib/DFtips
File Converted: /opt/studies/test100/lib/DFcterm_map

Step 3: Moving Lookup Tables to lut directory
------------------------------------------------------------
ERROR: Absolute file path found! lookup table file
/opt/studies/test100/lib/DF_QClut not moved.

Step 4: Moving Edit Checks code to ecsrc directory
------------------------------------------------------------
File Moved: from lib/DFedits to ecsrc/DFedits.
WARNING: DFedits contains #include files which must be moved to ecsrc
manually.

Step 5: Converting postscript to png and old tiffs to png
------------------------------------------------------------
All background files converted from DFbkgd###.ps to DFbkgd###.png

Study migration was successful.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
DONE:
Migration has been performed on the following study: 100
Please review dfmigrate_errors for any migration errors
or warnings and/or manual steps that may be required.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

%

Example 4.8. Migrate a non-active study to current version

Study 207 is not active. It does not appear in the DFadmin study list and is not defined in /opt/dfdiscover/lib/DFstudies.db. The study directory is located at /tmp/studies/study207. It might be an old study copied from a backup medium, or a copy of an active study (e.g. cp -pr /opt/studies/study107 /tmp/studies/study207)

%  DFmigrate 207 /tmp/studies/study207

Example 4.9. Migrate all studies found in /opt/dfdiscover/lib/DFstudies.db to current version

%  DFmigrate -a



[16] Study directory paths are evaluated without regard to case, e.g. /opt/studies/study1 and /opt/studies/Study1 are considered identical, and not allowed as each study path must be unique.


DFras2png

DFras2png — Convert Sun raster files in the study pages directory into PNG files

Synopsis

DFras2png {-s study_dir}

Description

DFras2png is used to convert Sun raster files in the study pages directory into PNG files. File names remain the same.

All recent versions of DFdiscover store images in PNG format rather than the Sun raster image format used in older versions. Converting the images to PNG format brings several advantages: smaller file sizes, color capability and better interoperability with other software. Very few packages can deal with the Sun raster file format.

Only files in the pages directory are converted. Any Sun raster image files in any of the other study directories are not touched. Any PNG files that exist in the pages directory are reported but are not re-converted. Any other files that are not Sun raster files are ignored. Study directories that are incorrectly specified or don't exist are reported. All reporting is written to stderr.

Options

-s study_dir

This is a required option. The study_dir is the full pathname to a study directory.

Exit Status

exits with one status:

0

The command was successful.

Examples

Example 4.10. Convert all Sun raster files in the pages directory found in the study directory /opt/studies/demo253 into PNG files

% DFras2png -s /opt/studies/demo253


DFshowIdx

DFshowIdx — Show the per plate index file(s) for a specific study

Synopsis

DFshowIdx [-F] [-q] [-l #] [-p #] {-s #}

Description

DFshowIdx displays the per plate index files as described in plt###.ndx - per plate index files.

Options

-F

Update the count of total entries (if needed), sort the index (so that the sorted count now matches the total count), and reset the unsorted count to 0.

-q

Display header only.

-l

Maximum length of data record.

-p #

Display the index file for a specific plate. The plate number can be one of the user-specified plates, plate 0 (new record queue), plate 510 (reasons for data), or plate 511 (queries). If not specified, index files for all plates are checked.

-s #

Study number (required).

Exit Status

DFshowIdx exits with one of the following statuses:

0

The command was successful.

1

The command failed because of one or more errors with command-line flags.


DFstudyDiag

DFstudyDiag — Report (diagnose) the current status of a study database server

Synopsis

DFstudyDiag {-s #}

Description

There are a variety of consistency checks that must be in place for a study server to operate correctly. If one or more of these consistency checks fails, it may be difficult for a user to determine the failure condition so that it can be remedied. DFstudyDiag is the utility to use in situations like this.

DFstudyDiag checks a variety of places while diagnosing a study server. It consults with the DFdiscover master, reviews the DFdiscover studies database, communicates with the operating system's portmapper, and also interacts with one or more DFdiscover slaves. DFstudyDiag displays its progress which each of these interactions as these proceed.

Options

-s #

the DFdiscover study number to diagnose (required).

Exit Status

DFstudyDiag exits with one of the following statuses:

0

The command was successful and the study server is operational and responding correctly, or the study server has been disabled.

1

The command was successful and the study server is not responding, or the command failed because there was an error in the command-line arguments.

Examples

Example 4.11. What is the status of study 7's database server?

% DFstudyDiag -s 7
Diagnosing study server 7 starting Fri Apr 25 14:57:16 1997...
>> Trying to contact study server directly...
<< Study server is currently operational and responding.

Example 4.12. DFstudyDiag detects a problem with study 8

% DFstudyDiag -s 8
Diagnosing study server 8 starting Fri Apr 25 15:00:52 1997...
>> Trying to contact study server directly...
<< Failed.
>> Trying to load studies database from master...
<< OK.
>> Contacting portmapper on candidate hosts...
<< OK.
>> Contacting slaves on candidate hosts...
<< OK.
>> Checking portmapper entries on candidate hosts...
<< OK.
>> Looking for existing serverstatus file...
<< The file '/opt/dfdiscover/work/.serverstatus8' exists although no study
<< appears to be running.  The file should be removed.
Please show this output to your DFdiscover administrator.


DFstudyPerms

DFstudyPerms — Report, and correct, the permissions on all required DFdiscover sub-directories and files for a study

Synopsis

DFstudyPerms [-v] [-f] [-g string] {#}

Description

Incorrect permissions or file ownerships are the most common causes of problems in UNIX applications, and DFdiscover, being a UNIX application, is no exception.

DFstudyPerms is a useful utility for detecting permission and ownership problems on all required DFdiscover sub-directories and files for a study. It is not able to detect errors in files that are not required by DFdiscover.

DFstudyPerms exits with a status of 0 if no errors are detected and a non-zero status if errors are detected.

DFstudyPerms does not detect problems with ownership or permissions in the DFdiscover software files themselves. To reset ownerships and permissions with the DFdiscover software, use the SETPERMS script that can be found in /opt/dfdiscover.

Options

-v

verbose mode. Choosing this option causes DFstudyPerms to echo the name of each file and sub-directory it examines. The default behavior is to examine directories and files quietly unless an error is detected.

-f

fix mode. Choosing this option causes DFstudyPerms to attempt to fix each permission problem that is detected. In most environments, the user executing DFstudyPerms will need to be root for fix mode to work.

-g string

check group ownership and permissions for the named group instead of the default studies group.

#

the DFdiscover study number (required).

Exit Status

DFstudyPerms exits with one of the following statuses:

0

The command was successful and no permission problems were detected.

1

The command was executed with the -u (usage) option.

2

The command was successful and permission problems were detected, or the command failed because user datafax or group studies was not defined.

Examples

Example 4.13. Check, in verbose mode, the permissions for study 251

% DFstudyPerms -v 251
checking study 251, group studies
...checking '/opt/studies/exemplar251'
...checking '/opt/studies/exemplar251/lib'
...checking '/opt/studies/exemplar251/bkgd'
...checking '/opt/studies/exemplar251/data'
...checking '/opt/studies/exemplar251/data/1602.jnl'
...checking '/opt/studies/exemplar251/data/1603.jnl'
...checking '/opt/studies/exemplar251/data/1604.jnl'
...checking '/opt/studies/exemplar251/pages'
...checking '/opt/studies/exemplar251/pages_hd'
...checking '/opt/studies/exemplar251/reports'
...checking '/opt/studies/exemplar251/work'
...checking '/opt/studies/exemplar251/lib/DFcenters'
...checking '/opt/studies/exemplar251/lib/DFfile_map'
...checking '/opt/studies/exemplar251/lib/DFschema'
...checking '/opt/studies/exemplar251/lib/DFsetup'
...checking '/opt/studies/exemplar251/lib/DFtips'
...checking '/opt/studies/exemplar251/lib/DFvisit_map'
...checking '/opt/studies/exemplar251/lib/DFlut_map'
...checking '/opt/studies/exemplar251/lib/DFmissing_map'
...checking '/opt/studies/exemplar251/lib/DFpage_map'
...checking '/opt/studies/exemplar251/lib/DFqcproblem_map'
% echo $status
0


DFtiff2ras

DFtiff2ras — Convert a TIFF file into individual PNG files

Synopsis

{infile} {outfile}

Description

is used to convert TIFF files into individual PNG files whose names are comprised of the output stem followed by a 3-digit page number. The raster files are converted to 100DPI images but are not image processed in any other way.

Options

infile

the name of the input TIFF file to be split

outfile

the name of the output PNG file

Exit Status

exits with one of the following statuses:

0

The command was successful.

2

The command failed because there was an error in the command-line arguments, or the required input file could not be read.

> 0

The command was successful but one or more output files could not be created. The exit status is the number of output files that could not be created.

Examples

Example 4.14. Convert the file fax12345.tif into PNG files rast001, etc

% DFtiff2ras fax12345.tif rast


DFuserPerms

DFuserPerms — Import and update users and passwords, and optionally import roles, role permissions, and user roles

Synopsis

DFuserPerms [-S server] [-U username] [-C password] [-a] [-f] [-e errorlog] {-i inputfile}

Description

Normally, creating and modifying users, passwords, roles, role permissions, and user roles is done with DFadmin. However, there may be cases where it would be useful to perform these operations from the command line. DFuserPerms is a tool that grants the ability to import these records from the command line. The records must be stored in an input file which conforms to the format used in the DFuserdb.log file (see System Administrator Guide, DFuserdb.logDFuserdb.log), although with the following exceptions:

  • Record Time Stamp and Record Modifier must not be included for any record type.

  • Instead of a Password Hash field for PASS records, the actual password to be imported must be specified.

[Important]Important

The following behaviors of DFuserPerms should be noted:

  • USRL and RLPM records require the corresponding ROLE record to be present in the same import file. However, PASS and USRL records do not require the corresponding USER to be present in the same file.

  • For ROLE records, the role ID specified in the import file does not affect which ID it is actually assigned. If the role's name and study number matches an existing role, the existing role is edited to match the record being imported. If the role's combination of name and study number is new and unique, it is imported as a new role with the next available ID. As a result, role names cannot be edited with DFuserPerms.

  • Though it does not matter to the ROLE record which ID it is given, if a USRL or RLPM record is being imported, the Role ID field is used to match it to a role imported in the same file. So if a user wishes to import USRL or RLPM records for a certain role, they must give the ROLE, RLPM, and USRL records the same role ID in their import file. See the second example.

  • The Expiry field for PASS records is required but irrelevant. DFuserPerms sets the password expiry according to the settings in DFadmin.

  • When there is an error in the record formatting or data, DFuserPerms does not import that specific record, but does import all other valid records in the file. As such, DFuserPerms is considered to have run successfully even if there are invalid records. Thus errors in the record formatting or data are not written to stderr, but rather stdout. Only errors which prevent DFuserPerms from running, such as authentication errors, are written to stderr.

  • If there is an error when reading a PASS record, the password characters are replaced with X's when the error is written to stdout.

Options

-S server

the DFdiscover server name. Must be specified unless the user has the DFSERVER shell variable set appropriately.

-U username

login username. Must be specified unless the user has the DFUSER shell variable set appropriately.

-C password

login password. Must be specified unless the user has the DFPASSWD shell variable set appropriately, or has previously set up authentication using DFpass.

-a

allow ROLE, USRL, and RLPM records to be imported.

-f

check data format only. Does not import records. This option only looks for record formatting errors. Problems in the records that would need to be caught by the study server, such as passwords that do not meet the requirements or importing passwords for non-existent users, are not recognized by this option.

-e errorlog

output critical errors to specified file. Formatting errors are not written to the error log, only errors that prevent DFuserPerms from running. If this option is not specified, errors are written to stderr.

-i inputfile

specify the input file for DFuserPerms to import (required).

Exit Status

DFuserPerms exits with one of the following statuses:

0

The command was executed, DFuserPerms has run and imported all valid records. If there are invalid records but DFuserPerms has still run, the exit status is still 0.

1

DFuserPerms encountered a critical error such as failed authentication or permission errors. Invalid records in the import file are not critical errors since DFuserPerms still runs, and do not cause an exit status of 1.

Examples

Example 4.15. Importing USER and PASS records

In this example, a new user named Jason is imported and a password is assigned to them. The input file contains:

USER|Jason|2|Jason Johnson|A Company Incorporated|||||||||0|2|1|
PASS|Jason|apassword1234|1464208516

Importing the file:

% DFuserPerms -S servername -U username -C password -i inputfile

**********************************************************************
DFuserPerms: Server [servername] User [username] Fri Jun 10 09:40:37 2016
**********************************************************************
Input file: inputfile
Reading file
===============================
Imported Records = 2
Total Rows = 2
===============================

DONE: Importing user-perms complete. Errors (if any) are logged above.
**********************************************************************
    


Example 4.16. Importing ROLE, RLPM, and USRL records

In this example a new role called Company Permissions is imported, role permissions for this role are specified, and user Jason is assigned to have this user role. The input file contains:

ROLE|20|254|2|Company Permissions||*|*|*||*|0|0
RLPM|20|2|2|*|*|1,2,3,4,5|1,2,3,4,5|1,2,3,4|*|*|*|0|0
USRL|Jason|20|1|2|*|*

Importing the file:

% DFuserPerms -S servername -U username -C password -a -i inputfile

**********************************************************************
DFuserPerms: Server [servername] User [username] Fri Jun 10 09:51:27 2016
**********************************************************************
Input file: inputfile
Reading file
===============================
Imported Records = 3
Total Rows = 3
===============================

DONE: Importing user-perms complete. Errors (if any) are logged above.
**********************************************************************

Note that the ROLE, RLPM, and USRL records all used the same Role ID value (20). The DFuserdb.log file shows the the IDs have changed when they were imported, but DFuserPerms ensures that since they were imported with the same ID value, they are assigned to the same role ID (105):

20160610095127|ROLE|datafax|105|254|2|Company Permissions||*|*|*||*|0|0
20160610095127|RLPM|datafax|105|2|2|*|*|1,2,3,4,5|1,2,3,4,5|1,2,3,4|*|*|*|0|0
20160610095127|USRL|datafax|Jason|105|1|2|*|*




[14] Specifically, TLS v1.2 or v1.3 is used.

[15] Missed records are never checked.

[16] Study directory paths are evaluated without regard to case, e.g. /opt/studies/study1 and /opt/studies/Study1 are considered identical, and not allowed as each study path must be unique.

Chapter 5. Edit checks

Table of Contents

5.1. Introduction
5.1.1. DFopen_study and DFopen_patient_binder
5.2. Language Features
5.3. Database Permissions
5.4. Language Structure
5.4.1. return and exit statements
5.5. Variables
5.5.1. Variables and Types
5.5.2. Database Variables
5.5.3. Positional Variables
5.5.4. Local Variables
5.5.5. Global Variables
5.5.6. Variable Groups
5.5.7. Date Variables
5.6. Missing/Blank Data
5.6.1. dfblank
5.6.2. dfmissing
5.6.3. dfmissval
5.6.4. dfmisscode
5.6.5. dfmissingrecord
5.6.6. dflostcode
5.6.7. dflosttext
5.6.8. Missing Records
5.6.9. Examples
5.7. Arithmetic Operators
5.7.1. Addition
5.7.2. Subtraction
5.7.3. Multiplication/Division/Modulus
5.7.4. Exponentiation
5.7.5. Assignment
5.8. Conditional Execution
5.8.1. Comparison Operators
5.8.2. Logical Operators
5.8.3. if/else
5.9. Built-in Functions and Statements
5.9.1. Edit check Function Compatibility Notes
5.9.2. dfaccess
5.9.3. dfaccessinfo
5.9.4. dfask
5.9.5. dfbatch
5.9.6. dfcapture
5.9.7. dfcenter
5.9.8. dfclosestudy
5.9.9. dfdate2str
5.9.10. dfday
5.9.11. dfdirection
5.9.12. dfentrypoint
5.9.13. dfexecute
5.9.14. dfgetfield
5.9.15. dfgetlevel/dflevel
5.9.16. dfgetseq
5.9.17. dfhelp
5.9.18. dfillegal
5.9.19. dfimageinfo
5.9.20. dflegal
5.9.21. dflength
5.9.22. dflogout
5.9.23. dfmail
5.9.24. dfmatch
5.9.25. dfmessage/dfdisplay/dferror/dfwarning
5.9.26. dfmetastatus
5.9.27. dfmode
5.9.28. dfmoduleinfo
5.9.29. dfmonth
5.9.30. dfmoveto
5.9.31. dfneed
5.9.32. dfpageinfo
5.9.33. dfpassword
5.9.34. dfpasswdx
5.9.35. dfplateinfo
5.9.36. dfpref
5.9.37. dfprefinfo
5.9.38. dfprotocol
5.9.39. dfrole
5.9.40. dfsiteinfo
5.9.41. sqrt
5.9.42. dfstay
5.9.43. dfstr2date
5.9.44. dfstudyinfo
5.9.45. dfsubstr
5.9.46. dftask
5.9.47. dftime
5.9.48. dftoday
5.9.49. dftool
5.9.50. dftrigger
5.9.51. dfvarinfo
5.9.52. dfvarname
5.9.53. dfview
5.9.54. dfvisitinfo
5.9.55. dfwhoami
5.9.56. dfyear
5.9.57. int
5.10. Query operations
5.10.1. dfaddqc
5.10.2. dfaddmpqc
5.10.3. dfanyqc
5.10.4. dfanyqc2
5.10.5. dfanympqc
5.10.6. dfdelmpqc
5.10.7. dfeditqc
5.10.8. dfreplyqc
5.10.9. dfresqc/dfunresqc
5.10.10. dfqcinfo
5.10.11. dfqcinfo2
5.11. Reason operations
5.11.1. dfaddreason
5.11.2. dfanyreason
5.11.3. dfautoreason
5.11.4. dfreasoninfo
5.12. Lookup Tables
5.12.1. Pre-requisites
5.12.2. dflookup
5.13. Looping
5.13.1. while
5.13.2. break
5.13.3. continue
5.14. User-Defined Functions
5.14.1. Sharing edit check files with the #include directive
5.15. Examples and Advice
5.16. Optimizing Edit checks
5.16.1. Saving Time for the User
5.16.2. Limitations in the Language
5.16.3. Maximize Cache
5.16.4. Simplify Conditional Testing
5.16.5. Reduce the Number of Function Calls
5.16.6. Shortcut, and Order of, Evaluation
5.16.7. Delay Message Construction
5.17. Creating generic edit checks
5.17.1. More Examples
5.18. Debugging and Testing
5.18.1. Debugging
5.18.2. Testing
5.18.3. Compiling and Reloading Edit checks
5.19. Language Reference
5.19.1. Identifiers (edit check and variable names)
5.19.2. String Constants
5.19.3. Maximum number of instructions per edit check execution
5.19.4. Reserved Words

5.1. Introduction

The DFdiscover edit check language provides a general-purpose programming environment in which to create procedures that can be used to perform logic checks on data fields, generate quality control notes, and calculate and insert values into data fields. Edit checks may reference one or more data fields from any available record in the database, and may include arithmetic, logical and comparison operators. Looping and conditional execution constructs are also provided. The language structure is based on the C programming language and is a loose subset of C.

Edit checks are defined, named and stored in a study edit check file (named DFedits and located in the study lib directory). This file is accessed and edited by selecting View > Edit checks in DFsetup. Once defined, an edit check can be applied to the desired data field(s) using the style or variable definition dialogs, where the edit check can be set to execute on: field entry, field exit, page entry, or page exit.

When choosing among the options of field entry, field exit, plate entry and plate exit, note the following:

  • most edit checks should execute on exit from the last field used in the edit check to ensure that the user has finished all relevant data entry and corrections.

  • if you do not trust users to tab through all data fields you may want to execute edit checks on both field and plate exit to ensure that a data record can not be saved without executing the edit checks.

  • do not attach edit checks to hidden fields if you want them to be executed by users who do not have access to hidden fields.

  • in Image view most edit checks should not be executed until after the duplicate resolution step has completed (which occurs on entry to the first non-key field), because the initial ICR values may be replaced by existing database values during duplicate resolution. In particular note that any edit checks triggered on exit from the key fields (visit and ID) will run to completion before duplicate resolution has a chance to occur, and thus might produce incorrect results.

  • any plate entry edit checks executed in Image view will be re-executed after the existing data record is brought forward by duplicate resolution.

Edit checks are most often executed interactively when entering and reviewing data records in DFexplore, DFcollect and DFweb but can also be executed in batch mode (see Batch Edit checks).

The process used to define edit checks in DFsetup is detailed in Study Setup User Guide, Edit checks.

5.1.1. DFopen_study and DFopen_patient_binder

In addition to user-defined field and plate entry and exit edit checks, there are 2 reserved edit check names:

  • DFopen_study - executed when the study is selected in the login dialog, and

  • DFopen_patient_binder - executed when a subject binder is opened in DFweb, DFcollect or in DFexplore Data View, including when a user switches to Data View from any of the other views, and when a task record is selected for a different subject.

These two reserved edit checks are optional; if they are defined by an edit check programmer they will execute in DFexplore, DFweb and DFcollect. Additionally, DFopen_study executes for the first batch in a batchlist when run in batch mode. They are typically used to set user preferences (using function dfpref), change visit and plate requirements from the default visit map specifications (using function dfneed), and to display messages (using function dfwarning or dferror). Since no data record has the focus when these edit checks run, the edit checks cannot refer to data fields unless the fields are fully qualified. For example, @[1001,0,1,15] references field 15 and VDATE[1001,0,1] references variable VDATE, both of plate 1 at visit 0 for subject 1001. In DFopen_patient_binder, @PID can be used to refer to the subject ID; thus VDATE[@PID,0,1] contains the value of the VDATE field for the subject whose binder is being opened. In DFopen_study, @PID equals 0 since no subject has yet been selected.

5.2. Language Features

The edit check language provides the following features:

  • Support for all DFdiscover data types

  • Read/Write access to variables on the current plate

  • Read access to variables on other plates

  • Local variables

  • Arithmetic, logical, comparison operators

  • Query functions

  • Lookup table support

  • Legal range check functions

  • Message output functions

  • String matching and manipulation functions

  • Information functions

  • User-defined functions

5.3. Database Permissions

In DFexplore the data fields available to edit checks are restricted by user's site, subject, visit, plate and read level access permissions, as specified in the roles the user plays within each study. This helps to ensure that users will not be shown database values in edit checks to which they would normally not have access, but it also means that data records unavailable to the DFexplore user can not be used in edit checks. This restriction does not apply to hidden fields. Thus data values that are needed in an edit check, but should not normally be seen by users with a certain role, can be made available by defining them as hidden fields on plates to which the user has read access permissions.

5.4. Language Structure

As each edit check is defined, it must be given a name. This name can be any sequence of letters, but it is a good idea to use names that are descriptive of what the edit check is to accomplish. Names must start with a letter and may contain letters, numbers and underscores, _. There is no length limit on edit check names. Edit check names are case sensitive, so myeditcheck() and MyEditCheck() refer to different edit checks.

Comments can be placed anywhere an edit check by preceding it with an octothorpe, #.

Edit checks begin with the line containing the keyword edit, the name of the edit check, a (, an optional parameter declaration list and a ). The optional parameter declaration allows arguments to be passed into the edit check. It is important to note that if parameters are declared, they must be supplied when the edit check is invoked. Following this is the body of the edit check, enclosed in a pair of braces, { and }. The body of the edit check contains zero or more statements that perform the desired operations.

Execution of the edit check begins with the first statement in the edit check and proceeds to the next in sequence until the logic flow changes because of an if or while statement. Execution is terminated via an exit or return statement or when control reaches the end of the edit check.

Braces, { and }, are used throughout the edit check language to group two or more statements together to operate as one statement. They become especially important when used with the if and while statements that require a single action statement.

Each statement in the edit check language is terminated by a semicolon, ;. A simple edit check to convert pounds to kilograms could be written as follows:

# This is a comment
edit lbs2kgs()
{
        kgs = lbs / 2.205;
}

Edit checks are placed one after the other in the DFedits file, so the addition of a similar function to convert kilograms to pounds would result in the following DFedits file:

edit lbs2kgs()
{
        kgs = lbs / 2.205;
}
edit kgs2lbs()
{
        lbs = kgs * 2.205;
}

An example of an edit check with parameters would look like this:

edit isbetween(number low, number high)
{
        if ((lbs < low) || (lbs > high)) {
                dferror("Weight is not between ", low, " and ", high, " pounds.");
        }
}

To verify that the weight is in the range of 100-250 pounds we would attach the edit check isbetween(100, 250) on field exit on the variable lbs.

5.4.1. return and exit statements

Statements in the body of an edit check are executed in order. The logic of an edit check typically terminates after the last statement is executed, however, an edit check can be forced to terminate earlier by use of the return or exit statements. These two statements are very similar in purpose; they terminate the logic of an edit check at the point where the statement is executed - no following statements are executed.

The difference in the return and exit statements is in their behavior with respect to a sequence of edit checks that are to be executed at the same location on the current variable. The exit statement terminates the current edit check and does not execute any of the following edit checks on the variable, whereas return terminates the current edit check but then resumes processing with the next edit check defined on the variable, if any.

Example 5.1. Difference between exit and return

In this example, a variable has the following definition for the field enter edit checks:

ageCheck, demog, conmed

The three edit checks are to be executed in the listed order. If an exit statement is executed in the body of ageCheck then edit checks demog and conmed are not executed. However, if a return statement is executed in the body of ageCheck, edit checks demog and conmed would still be executed.

The same would be true for an exit or return statement inside the body of edit check demog, conmed would be skipped if exit executed, while it would not be skipped if return was executed.


Now that you have some idea of what edit checks are about and have seen an example, we will turn to a description of the elements of the edit check language.

5.5. Variables

5.5.1. Variables and Types

Variables are storage spaces that can hold values. Each variable has a data type associated with it, where the data type is from the list:

  • number
  • date
  • string
  • time

The data types have the following properties:

  • number.  an optional leading plus/minus sign, a sequence of one or more digits (whole part of the number), an optional decimal point, and, optionally, one or more digits (fractional part). The range of numbers (10 digits at maximum, except for subject ID which allows 15 digits at maximum) that can be represented is -2147483647 to 2147483647, inclusive.

  • date.  2 or 4 digits for the year, 2 digits or 3 characters for the month, 2 digits for the day, and optional delimiters. The format of the date is taken from the field definition for database variables, or from the format date declaration (see Section 5.5.7, “Date Variables”) for constant dates.

  • string.  a sequence of zero or more alphanumeric characters, plus a small set of two character sequences (see Section 5.19.2, “String Constants”), enclosed in double quotes. The maximum length of a string is taken from the field definition for database variables, or is 16383 bytes for constant strings (4095 UNICODE characters).

  • time.  2 digits for hours, a colon delimiter, two digits for minutes or 2 digits for hours, a colon delimiter, two digits for minutes, another colon delimiter and two digits for seconds. The minimum and maximum values are based on a 24 hour clock with a minimum value of 00:00:00 and a maximum value of 23:59:59.

Field types map to data types in the following way:

Field typeData type
Numbernumber
Datedate
Stringstring
Timetime
Choicenumber
Checknumber
VASnumber

Different types can have different operations performed on them. For example, it makes sense to multiply two numbers together, but not two dates. The edit check language uses type information to decide what operations are permitted and what the results are. Subtracting two dates returns the number of days between them. Adding dates doesn't make sense, so this operation is not permitted and will result in an error message. Adding a number (of days) to a date is permitted and is used to calculate a date in the future.

5.5.2. Database Variables

Variables allow read/write access to data fields on the current plate, read-only access to data fields on another plate in the database, or read/write access to local temporary storage areas. A database field is referenced by its name as defined in the setup tool.

For example, if you have a database field called DOB in your study setup, you would reference the first instance of that field on the current page using:

DOB

References can be refined by specifying which module the variable is in. To reference the DOB field in the current module that the edit check is running in, use:

.DOB

To reference the DOB variable in the first instance of the DEMOGRAPHICS module, use the construct:

DEMOGRAPHICS.DOB

To reference the DOB variable in the module DEMOGRAPHICS with instance ID of 5, use the construct:

DEMOGRAPHICS[5].DOB

The above constructs operate on the current page. It is also possible to reference variables in a read-only context on other pages by including the keys to those pages by adding the id, visit and plate parameters to the variable name using the form variable[id, visit, plate].

To reference the first instance of the DOB variable for subject ID 12345 at visit 1 on plate 2, you would use:

DOB[12345, 1, 2]

The id, visit, and plate fields can each be arithmetic expressions, or may be left blank in which case they default to the current id, visit, and plate. Thus:

DOB[,,]

means, the variable DOB for the current id, on the current visit and plate which is the same as writing DOB by itself. Writing:

DOB[,1,2]

means the first instance of variable DOB for the current subject ID, on visit 1, plate 2.

Similarly, module names can be added to more precisely specify which instance of the DOB variable you wish to reference. To reference the DOB field in the first instance of the DEMOGRAPHICS module for subject ID 12345 at visit 1 and plate 2, use:

DEMOGRAPHICS.DOB[12345, 1, 2]

To reference the DOB field in instance 5 of the DEMOGRAPHICS module for subject ID 12345 at visit 1 and plate 2 use:

DEMOGRAPHICS[5].DOB[12345, 1, 2]

Any string values written to database variables are sanitized (i.e., any '|' characters or control characters are converted to blank) before the data is written out to the database.

The first 6, and last 3, fields in all data records in a DFdiscover database are used for database management by DFdiscover, and are referred to by the same variable names in all DFdiscover studies. They are: DFSTATUS (data record status: final, incomplete, etc.), DFVALID (data record validation level: 1-7), DFRASTER (the name of the file holding the fax image of the CRF page corresponding to the data record), DFSTUDY (the DFdiscover study number), DFPLATE (the data record plate number), DFSEQ (the data record sequence or visit number), DFSCREEN (data record status without consideration for primary/secondary), DFCREATE (the record creation date and time) and DFMODIFY (the record's last modification date and time). Note that DFSEQ is assigned only if the sequence/visit number is in the barcode; otherwise the name is user-defined. A detailed description of these 6 fields can be found in plt###.dat field descriptions.

5.5.3. Positional Variables

In addition to named references to database variables, the edit check language allows access to data via positional notation. We may want to reference the variable we're presently on, without knowing its name. Similarly, we may want to reference the previous variable or the next variable in a record. This is accomplished with the @[expression], @[id,visit,plate,field], @T, @(T-expression), and @(T+expression) constructs.

Except for @[id,visit,plate,field] these positional variables may only reference data on the current record.

The @[expression] form refers to the field number represented by expression. For example, if expression evaluates to 7, then it would refer to the ID field (always the 7th field on a plate). To refer to the current field, the expression @[.] can be used. @[.+1] would indicate the next field on the plate.

A second, older construct also exists which does the same job. This is the @T construct (current field), @(T+1) which refers to the next field, and @(T-1) which refers to the previous field.

[Important]Important

Note that @(T-1) means the previous field, while (@T-1) means one less than the current field. Watch those brackets!

The @[id,visit,plate,field] form can be used to refer to any field on any record in the study database, by it's numeric keys. Those keys which will always be the same as the plate on which the edit check is triggered can be omitted. For example, @[,,55,10] refers to field 10 on plate 55 of the same visit for the same subject, and @[,,,10] is equivalent to @[10], both referring to field 10 on the current plate. While this form can be used to read and test the value of any field in the study database it cannot be used to change the value of fields on other records in the database. The edit check language does not allow changes to be made to records other than the one on which the edit check is triggered.

The use of positional variables is both convenient and necessary when writing reusable generic edit checks. However they depend on field positions remaining constant. Problems can arise if a plate is modified by adding, deleting or renumbering fields. Thus edit checks must be reviewed as part of any such modifications.

5.5.4. Local Variables

Local variables are temporary storage locations that can be used to hold intermediate results of calculations. Local variables are only valid inside the edit check or function in which they are defined. They do not retain their values across invocations of that edit check or function.

Local variables are declared at the beginning of the edit check or function where they will be used. The declaration includes the data type of the variable and it's name. It must precede any other statements inside that function or edit check. Local variables are automatically initialized to a blank value.

Example 5.2. Local variable declaration

edit testdecl()
{
        number average;
        average = (bp1 + bp2) / 2;
}

In this simple edit check, the average of two variables is calculated and stored in the local variable average. Since this variable is local to this edit check, its value is no longer available as soon as the edit check completes.


5.5.5. Global Variables

Global variables are storage locations just like local variables but differ in that they maintain their values across edit checks. Global variables are initialized when the edit checks file is first loaded and maintain their values until an edit check changes it. That new value is then used in any subsequent references until it is changed again.

[Important]Important

It is important to remember that edit checks can be executed in an arbitrary order as the user traverses fields. It is therefore important to consider where the values in global variables really came from when executing in interactive tools such as DFexplore.

Global variables are declared in the same way that local variables are, except that they are declared outside of edit checks.

Example 5.3. Global variable declaration

number pages_traversed = 0;
edit another_page()
{
        pages_traversed = pages_traversed+1;
        dfmessage("Page ", pages_traversed, " PID=",
                PID, " Plate=", DFPLATE, " Visit=", DFSEQ);
}

If this edit check is attached as a plate exit check, it will count how many pages were traversed by a user in the course of a session and record the subject, plate and visit numbers in the edit checks log file.


5.5.6. Variable Groups

There are often cases where it is useful to group variables that are related to each other. The group declaration can be used to build one-dimensional arrays of variables. The group declaration syntax is as follows:

Figure 5.1. Group declaration syntax

group groupname variable1, variable2, ... ;


Once a group has been defined it is referenced by groupname[index].

Example 5.4. Compute the total dose from the individual dosages

edit compute_dose()
{
        number total=0, i=1, blank=0;
        group dgrp dosage1, dosage2, dosage3, dosage4, dosage5;
        while (i<=5) {
                if (dfblank(dgrp[i])) blank = blank+1;
                else total = total + dgrp[i];
                i = i+1;
        }
}

The individual dosages are in the database variables dosage1 through dosage5. The group variable dgrp allows the individual dosages to be referenced through an index.


Group variables can be used anywhere other variables can be used but must be declared after any local variables in that edit check or function have been declared and before the first statement of the edit check's body.

[Note]Missing group variables

If a variable is missing from a group, the missing variable is treated as a missing value. If a variable defined in a group is on a plate that is missing from the database, the variable will also evaluate to a missing value.

5.5.7. Date Variables

Each date variable in a DFdiscover study setup has a date format associated with it. This date format is used to determine which part of the date string contains the year, month and day portions of a date.

Internally, the edit check language uses julian dates and converts them to printable representations as required. Dates that are stored back into the database are stored using the date format associated with that variable. Dates displayed in messages are output using the default date format, YY/MM/DD, which can be overridden via the date format instruction at the beginning of a DFedits file.

Example 5.5. Set the default date output format to MM/DD/YY

date format "MM/DD/YY"

This statement must be added to the DFedits file.


[Note]Date Format

All date constants used throughout the DFedits file are assumed to be in this format. The date format statement may only appear once in each edit check file, but it may appear anywhere within the file.

Example 5.6. Incrementing a date value

date format "yy/mm/dd"
edit main()
{
        date d;
        d = "90/04/07";
        d = d + 14;
}

This example assigns the julian date for April 7th, 1990 to the local variable d, and then add two weeks (14 days) to this date. The two assignment lines cannot be combined into one line because the conversion of the string to a julian date must be completed before the subsequent addition of 14 to this value. Combining them into one line would indicate that the string "90/04/07" and the number 14 were to be added which is clearly meaningless.


5.6. Missing/Blank Data

The DFdiscover edit check language allows the use of missing values in data fields.

Data may be unavailable for one of four reasons:

  • the data field has been left blank,

  • the data field contains a missing value code

  • the CRF page has not yet arrived and thus the data record does not exist in the database, or

  • the CRF page exists as a missed record.

The edit check language provides five functions for dealing with missing values:

  • dfblank,

  • dfmissing,

  • dfmissval,

  • dfmisscode, and

  • dfmissingrecord.

In addition, two functions exist that return information about missed data:

  • dflostcode

  • dflosttext

Table 5.1. Return values for dfblank, dfmissing, dfmissval, dfmisscode, and dfmissingrecord

 dfblankdfmissingdfmissvaldfmisscodedfmissingrecord
Data Record does not existFALSETRUE""""2
Data Record is a missed recordFALSETRUE""""1
Field = missing value codeFALSETRUEmissing value labelmissing value code0
Field = blankTRUETRUE""""0
Field = valueFALSEFALSE""""0


5.6.1. dfblank

The dfblank function is used to determine if a data record exists and the field is blank.

Table 5.2. dfblank usage

Syntax:dfblank(var)
Input Parameters:var is any variable
Return Value:TRUE if variable is blank. A blank string/number/date is one containing no characters or digits, not even a space. Also TRUE for check and choice fields having a code for "no choice".

FALSE in all other cases.

Example:
if (dfblank(DOB)) dfmessage("DOB is missing");
Notes:The record containing var must exist for dfblank to return TRUE.

Previous to 3.8, it was necessary to use, as a work-around, the construct

if ( @T == 99 )...

for check fields that coded "not checked" using a code other than 0, (e.g. 99 in this example). This work-around is no longer correct or supported. For check and choice fields any test for the code used for no choice, or unchecked, will always fail (i.e. evaluate to FALSE). The correct way to test for this condition is by using,

if ( dfblank( @T ) )...


5.6.2. dfmissing

The dfmissing function is used to determine if a variable is missing, either because the data record containing it does not exist, the data record containing it is missed, the value is blank, or the field contains a missing value code.

Table 5.3. dfmissing usage

Syntax:dfmissing(var)
Input Parameters:var is any variable
Return Value:TRUE if variable is missing. TRUE if variable references keys belonging to a missed record. TRUE if the variable type is date, and the value is a nonsensical date. FALSE in all other cases.
Example:
if (dfmissing(DOB))
    dfmessage("DOB is missing");

# returns TRUE if keys match an entry identified as missed
if (dfmissing(DFSTUDY[,0,1]))
    dfmessage("Plate 1, visit 0 is a missed record");
Notes:

Testing for a missing record can also be performed with the dfmissingrecord() function.


5.6.3. dfmissval

The dfmissval function returns the missing value label equivalent to the variable's value.

Table 5.4. dfmissval usage

Syntax:dfmissval(var)
Input Parameters:var is any variable
Return Value: The missing value label equivalent to the variable's value.

A Blank string in all other cases.

Example:
label = dfmissval(DOB);

5.6.4. dfmisscode

The dfmisscode function returns the missing value code equivalent to the variable's value.

Table 5.5. dfmisscode usage

Syntax:dfmisscode(var)
Input Parameters:var is any variable
Return Value: The missing value code equivalent to the variable's value.

A blank string in all other cases.

Example:
code = dfmisscode(DOB);

5.6.5. dfmissingrecord

The dfmissingrecord function returns information about a missing record.

Table 5.6. dfmissingrecord usage

Syntax:dfmissingrecord(id, visit, plate)
Input Parameters:id is the subject ID of the missing record

visit is the visit number of the missing record

plate is the plate number of the missing record

Return Value: Information about a missing record for the specified keys. Return values are as follows:
  • 0 if the data record is in the database, that is, not missing.

  • 1 if a missed record matching the specified keys is in the database.

  • 2 if there is no record in the database matching the specified keys.

  • 3 if the record in the database matching the specified keys has a pending status.

Example:
if( dfmissingrecord(99001,0,1) == 1 )
    dfmessage( "Plate 1 at visit 0 is missed for this subject." );

5.6.6. dflostcode

dflostcode returns information about a missed record entered in DFexplore, specifically the actual reason code assigned to the missed record.

Table 5.7. dflostcode usage

Syntax:dflostcode(id, visit, plate)
Input Parameters:id is the subject ID belonging to the missed record

visit is the visit number belonging to the missed record

plate is the plate number belonging to the missed record

Return Value: The numeric reason code that has been assigned to the missed record having the specified keys.
Example:
code = dflostcode(99001,0,1);

5.6.7. dflosttext

dflosttext returns the actual text reason assigned to a missed record in DFexplore.

Table 5.8. dflosttext usage

Syntax:dflosttext(id, visit, plate)
Input Parameters:id is the subject ID belonging to the missed record

visit is the visit number belonging to the missed record

plate is the plate number belonging to the missed record

Return Value: The reason text (explanation) that has been assigned to the missed record having the specified keys.
Example:
dfmessage( "The missed reason is ", dflosttext(99001,0,1) );

5.6.8. Missing Records

The edit check language supports two functions that can be used to identify data records that do not exist in the database - dfmissing and dfmissingrecord.

Using dfmissing requires testing for the existence of any one of the required fields on the data record of interest. For example, we could determine whether plate 1 at visit 0 is missing for the current subject by testing to see if the DFdiscover study number is blank for this record using dfmissing.

Example 5.7. Testing for a missing record with dfmissing

if( dfmissing( DFSTUDY[,0,1] )
    dferror("Plate 1 at Visit 0 is missing");

DFSTUDY is the name for the DFdiscover study number field that is present on all plates in the database. This field can never be left blank or contain a missing value code because it is maintained by the system and users cannot edit it. Thus the above test will only return true when the data record is missing or if the specified keys match an entry marked as missed.


dfmissingrecord removes the need for the 'trick' of testing for a missing record by evaluating the value of a known, fixed variable. dfmissingrecord takes as arguments the subject ID, visit and plate keys of the data record of interest and returns the following:

  • 0 if the data record is in the database

  • 1 if a missed record matching the keys is in the database

  • 2 if there is no record in the database matching the keys

Example 5.8. Testing for a missing record with dfmissingrecord

if( dfmissingrecord(,0,1) == 2 )
    dfmessage( "Plate 1 visit 0 is not in the database" );


5.6.9. Examples

Example 5.9. Calculate the subject's age provided both birth date (bdate) and the study entry date (edate) are available.

if( !dfmissing( bdate ) && !dfmissing( edate ) )
    age = ( edate - bdate ) / 365.25;

Example 5.10. Complain if the subject was hospitalized (hospital has the value 2) but the hospitalization date has been left blank

if( hospital==2 && dfblank(hospdate) )
    dferror("Hospitalization date is required.");

Example 5.11. Check for a specific missing value label

if(dfmissval(hosp)=="not hospitalized" && !dfmissing(hospdate))
    dferror("Reason for hospitalization has missing value
    code = \"not hospitalized\", but a hospitalization date
    has been entered");

This example shows an edit check that might be placed on a field used to record hospitalization date. It checks the reason for hospitalization field (hosp) for the existence of the missing value reason "not hospitalized" and if it finds this reason pops up an error message.


Example 5.12. Check for a specific missing value code

If "N" was the missing value code for the above missing value reason, the same edit check could be written using dfmisscode.

if(dfmisscode(hosp)=="N" && !dfmissing(hospdate))
    dferror("Reason for hospitalization has missing value " +
    "code = \"not hospitalized\", but a hospitalization " +
    "date has been entered");

5.7. Arithmetic Operators

This section summarizes the operators available and their results. In these tables a blank box indicates that the operation is not permitted and an error message will be produced.

The order of operations (highest to lowest precedence) is as follows:

  • unary minus

  • exponentiation

  • multiplication, division, modulus

  • addition, subtraction

You can override the default order of operations by using the grouping operators, ( and ). Expressions within parentheses are always evaluated first, and nested parenthetical expressions are always evaluated from the lowest nesting point to the highest. For example,

2 + 3 * 5

evaluates to 17, because normal precedence evaluates the * before the +, whereas

( 2 + 3 ) * 5

evaluates to 25, and

( ( 2 + 3 ) * 5 - 2 ) * 2

evaluates to 46. In this latter case, ( 2 + 3 ) is evaluated first (5), followed by ( 2 + 3 ) * 5 (25), and then ( ( 2 + 3 ) * 5 - 2 ) (23), and finally ( ( 2 + 3 ) * 5 - 2 ) * 2.

5.7.1. Addition

The addition operator, +, adds two operands and returns the result as summarized by this table. A missing variable may be a blank field, a missing value code or a missing data record. Returned missing values evaluate TRUE with dfblank and dfmissing

Table 5.9. Result table for Addition operator

 MissingNumberChoiceCheckStringDateTimeVAS
MissingMissingMissingMissingMissingString [a]MissingMissingMissing
NumberMissingNumberNumberNumber Date [b]Time [c]Number
ChoiceMissingNumberNumberNumber DateTimeNumber
CheckMissingNumberNumberNumber DateTimeNumber
StringString   String [d]   
DateMissingDateDateDate   Date
TimeMissingTimeTimeTime   Time
VASMissingNumberNumberNumber DateTimeNumber

[a] Unlike other data types, string concatenation makes a distinction between blank fields which are concatenated as an empty string, and fields that contain missing value codes or where the data record is missing, which abort the concatenation and return a null string.

[b] Date/Number addition returns the date number days in the future for a positive number or days in the past for a negative number.

[c] Time/Number addition returns the time number seconds in the future for a positive number or seconds in the past for a negative number.

[d] String/String addition returns the concatenated string, unless one of the strings is missing, in which case the result for Missing/String applies.


5.7.2. Subtraction

The subtraction operator, -, subtracts two operands and returns the result as summarized by the table. A missing variable may be a blank field, a missing value code or a missing data record. Returned missing values evaluate TRUE with dfblank and dfmissing.

Table 5.10. Result table for Subtraction operator

 MissingNumberChoiceCheckStringDateTimeVAS
MissingMissingMissingMissingMissing MissingMissingMissing
NumberMissingNumberNumberNumber Date [a]Time [b]Number
ChoiceMissingNumberNumberNumber DateTimeNumber
CheckMissingNumberNumberNumber DateTimeNumber
StringString       
DateMissingDate [a] DateDate Number [c] Date
TimeMissingTime [d] TimeTime  Number [e]Time
VASMissingNumberNumberNumber DateTimeNumber

[a] Subtracting a number from a date returns the date number days in the past for a positive number of days or in the future for a negative number of days.

[b] Subtracting a number from a time returns the time number seconds in the past for a positive number of seconds or in the future for a negative number of seconds.

[c] Date/Date subtraction returns the number of days between the dates.

[d] Subtracting a number from a time returns the time number seconds in the past for a positive number of seconds or in the future for a negative number of seconds.

[e] Time/Time subtraction returns the number of seconds between the times.


5.7.3. Multiplication/Division/Modulus

The multiplication, *, division, /, and modulus, %, operators return the results as summarized by the table. A missing variable may be a blank field, a missing value code or a missing data record. Returned missing values evaluate TRUE with dfblank and dfmissing.

Table 5.11. Result table for Multiplication/Division/Modulus operators

 MissingNumberChoiceCheckStringDateTimeVAS
MissingMissingMissingMissingMissing   Missing
NumberMissingNumber[a]NumberNumber  TimeNumber
ChoiceMissingNumberNumberNumber  TimeNumber
CheckMissingNumberNumberNumber  TimeNumber
String        
Date        
Time TimeTimeTime   Time
VASMissingNumberNumberNumber  TimeNumber

[a] For division in which both operands are integers (no decimal or decimal places), the returned value will be truncated to an integer, e.g., 100/60 will return 1. To obtain a floating point result at least one of the operands must be a floating-point number, e.g., 100/60.0 returns 1.666667.

Local and database fields may be cast to a floating-point number before the operation is performed by adding 0.0. For example, if A and B are 2 numeric fields which contain the integers 100 and 60 respectively, any of the following expressions: (A+0.0)/B, A/(B+0.0), (A+0.0)/(B+0.0) will return 1.666667.

Division or modulus by zero results in an error message.


5.7.4. Exponentiation

The exponentiation operator, ^, raises a number (the base) to the power of an exponent, and is written as:

base ^ exponent

where both the base and the exponent are numbers (integer, fixed point, or floating point). For example,

4 ^ 3 = 4 * 4 * 4 = 64
4 ^ 0.5 = sqrt( 4 ) = 2

The result of raising any number (including 0) to the exponent 0 (or 0.0) is 1. The result of raising 0 (or 0.0) to any positive exponent is 0. The result of raising 0 (or 0.0) to any negative exponent is illegal. The base can be negative only if the exponent is an integer value; that is, it is illegal to raise a negative number to a fractional exponent. In such a case, and in any case where the calculation is illegal, the returned result is a blank/empty string.

Results of exponentiation are summarized in the table. A missing variable may be a blank field, a missing value code or a missing data record. Returned blank/empty values evaluate TRUE with dfblank and dfmissing.

Table 5.12. Result table for Exponentiation

 Missing[a] NumberChoiceCheckStringDateTimeVAS
MissingMissingMissingMissingMissing   Missing
NumberMissingNumberNumberNumber   Number
ChoiceMissingNumberNumberNumber   Number
CheckMissingNumberNumberNumber   Number
String        
Date        
Time        
VASMissingNumberNumberNumber   Number

[a] The rows or the columns may be the base or the exponent - the table results are symmetric.


Example 5.13. Calculation of Body Surface Area

The calculation of an individual's body surface area in meters square, given their weight in kilograms and height in centimeters, is:

((kg)^0.425 x (cm)^0.725 x 71.84) / 10,000

An average adult has a body surface area of 1.73 meters square.


[Note]Note

With a sufficiently large base and exponent, the calculated values may be extremely large. Performing large number computations on systems that are capable of accurately handling only numbers of 32 or 64 bits, means that the math libraries internal to the computer's operating system must use algorithms and compromises to handle such large numbers. As a result, with sufficiently large base and exponents, the calculated values may not be reliable. It is unlikely however, that these types of numbers will be used in edit checks.

The same also holds true for calculations performed using very small fractional numbers.

5.7.5. Assignment

The assignment operator, =, is used to store a value in a database or local variable. The assignment operator will cast (i.e. change) values from one type to another, and follows these rules:

  • The assignment of a date to a string type causes the string to contain the date value formatted according to the edit check date format, which is YY/MM/DD unless otherwise defined by the date format statement in the DFedits file.

  • The assignment of a string to a local date variable causes the string to be converted to a date using the default date format. If the resulting date is stored in the database it is stored according to the date format associated with the database field.

  • The assignment of a string to a local time variable causes the string to be converted to a time. If the resulting time is stored in the database it is stored according to the time format associated with the database field.

  • The assignment of a time to a string type causes the string to contain the time value formatted hh:mm:ss

  • When assigning numeric values to data fields the whole number part of the value is honored, but decimal places may be truncated to the number of decimal places provided by the format. For example, assigning the value 95.1234 to a numeric field with a format of nn.n would store the value as 95.1, and if the format was nn the stored value would be 95.

  • If the assigned value is too large to fit in the specified data field the result in DFexplore is a blank field, and a dialog appears describing the problem. The dialog identifies the field and the value that could not be assigned. In DFbatch the field is left unchanged and an error message is written to the log. For example, this would occur on attempting to assign the value 1000 to a numeric field with a format of nn, nnn, nn.nn, etc., or to an unformatted field with a store length less than 4.

  • Missing values are stored in database fields according to their missing value codes. Legal missing value codes are defined in DFmissing_map. If DFmissing_map has not been defined the default DFdiscover missing value code is an *. It is legal to assign a missing value code to any field, regardless of type, e.g. bdate = "*".

  • A database field of any type can be blanked by assigning it a blank string, e.g. bdate="".

  • Attempting to assign a value to DFdiscover protected fields (e.g. fields 1 to the end of the barcode and the last 3 record status fields), results in the edit check being aborted with an error message.

  • Attempting to store a | character in a database field causes the character to be converted to a space.

  • Attempting to store an illegal value in a database check or choice field (i.e. a value which does not correspond to any of the defined boxes on the CRF) results in assignment of the no choice code to the database field.

  • It is possible to store values into a data field on the current data record by assigning the value to the variable name of the desired field. For example, if the current record has three variables named dispensed, returned and compliance to track how compliant a subject is at taking their medication, the following example would calculate compliance and store it in the compliance variable of the current record:

    compliance = 100 * ( dispensed - returned ) / dispensed;

5.8. Conditional Execution

5.8.1. Comparison Operators

The edit check language provides the standard set of comparison operators:

  • less than, <

  • less than or equal, <=

  • equal, ==

  • greater or equal, >=

  • greater than, >, and

  • not equal, !=

Be careful to differentiate between the assignment, =, and the equality, ==, operators.

Example 5.14. Probable erroneous use of the = operator

number i=5,j=6;
if (i = j) {
        dferror("i equals j");
} else {
        dferror("i is not equal to j");
}

Erroneous use of = results in i being assigned the value of j and then being checked for a non-zero value, which is probably not what was intended. The correct syntax for checking the equality of i and j is to use the equality, ==, operator.


String comparisons are done on a character-by-character basis using the ASCII sort order. This means that all uppercase letters come before all lowercase letters.

Date comparisons are done by julian date, regardless of the date format associated with the date.

Comparing two missing values returns TRUE for <=, ==, and >= and FALSE for all others. Comparing a missing value with any other value returns FALSE except for the != operator.

When performing a simple comparison, (e.g. if( @[15]==@[16] ) ), the edit check language does not distinguish between different missing value codes or between blank fields and fields containing missing value codes. Thus a simple comparison of 2 fields will return TRUE when one field is blank and the other contains a missing value code, or when the 2 fields contain different missing value codes. If such distinctions are important the dfblank and dfmisscode functions should be used.

The edit check language represents FALSE as zero and TRUE as any non-zero number. Do not count on TRUE being any specific non-zero value (such as one).

5.8.2. Logical Operators

The DFdiscover edit check language also provides the logical operators && (AND), || (OR), and ! (NOT). Each of these operators will cast their operands to TRUE/FALSE values as required based on the following rules:

Table 5.13. Casting rules for logical operators

Input TypeTRUEFALSE
Numeric/Datenon-zerozero
Stringnon-zero lengthzero-length
Missing/Blankneveralways


The && (AND) operator returns TRUE only if both operands are TRUE. Otherwise it returns FALSE.

The || (OR) operator returns TRUE if at least one of the two operands is TRUE. If both operands are FALSE, it returns FALSE.

The ! (NOT) operator takes one operand and returns TRUE if the operand is FALSE and FALSE if the operand is TRUE.

5.8.3. if/else

The if statement provides a conditional execution mechanism. The two basic if constructs are:

if (condition) statement_1
if (condition) statement_1 else statement_2

statement in the above examples can be a single statement or a group of statements enclosed in a pair of { and }. condition is an expression that evaluates to TRUE or FALSE. If the condition evaluates to TRUE then statement_1 is executed, otherwise statement_2 is executed.

if statements can be nested and the most deeply nested else clause always goes with the most deeply nested if clause. These two examples are identical:

if (age < 50)
    if (sex == 1)
        dferror("<50 year old male");
    else
        if (sex == 2)
            dferror("<50 year old female");
        else
            dferror("<50 year old, no sex info");

and

if (age < 50)
if (sex == 1)
    dferror("<50 year old male");
else
if (sex == 2)
    dferror("<50 year old female");
else
    dferror("<50 year old, no sex info");

The only differences are cosmetic.

The addition of braces makes the code even easier to read and understand without changing the meaning:

if (age < 50){
    if (sex == 1){
        dferror("<50 year old male");
    } else {
        if (sex == 2){
            dferror("<50 year old female");
        } else {
            dferror("<50 year old, no sex info");
        }
    }
}

Indentation, while useful for visually indicating logic flow to the programmer, does not influence program execution. When writing edit checks, use indentation to show the logic flow. It will make maintaining the checks much easier. The addition of braces around blocks of code also helps, and makes your intentions clear to the language compiler and any subsequent reader(s).

5.9. Built-in Functions and Statements

The edit checks language includes a robust set of built-in functions for use in any edit check.

5.9.1. Edit check Function Compatibility Notes

Generally speaking, the built-in functions behave identically across DFexplore, DFcollect and DFweb. However there are some differences dictated by the purpose of, and features available in, each application. The following table details where those differences are.

Table 5.14. Edit check Function Implementation

FunctionImplemented inNotes
DFexploreDFcollectDFweb
DFopen_study and DFopen_patient_binder 
dfaccess 
dfaccessinfo 
dfaddqc 
dfaddmpqc   
dfaddreason 
dfanympqc   
dfanyqc 
dfanyqc2 
dfanyreason 
dfask 
dfautoreason 
dfbatch 
dfcapture 
dfcenter 
dfclosestudy 
dfdate2str 
dfday 
dfdelmpqc   
dfdirectionIn DFcollect, the return value is always 0.
dfeditqcIn DFweb, there is no confirmation dialog if the query status is changed to delete.
dfentrypoint 
dfexecute In DFcollect, dfexecute does nothing if called while offline.
dfgetfield 
dfgetlevel/dflevelIn DFweb and DFcollect, the return value is always 1.
dfgetseq 
dfhelp In DFweb, there is no visible effect of dfhelp until the user clicks the field help icon.
dfillegal 
dfimageinfo 
dflegalThere is no concept of missing plates in DFweb and DFcollect. Hence a dflegal test of fields on missing plates always returns FALSE. This is opposite from DFexplore which returns TRUE for dflegal test references to fields on missing plates.
dflength 
dflogoutAfter logout and re-login, DFcollect returns the user to the site list - it does not resume where the user was before logout.
dflookup   
dflostcode   
dflosttext   
dfmailIn DFcollect, dfmail does nothing if called while offline.
dfmatch 
dfmessage/dfdisplay/dferror/dfwarningIn DFweb and DFcollect, the message area is static and does not accept user input.
dfmetastatus 
dfmissingThere is no concept of missing plates in DFweb and DFcollect.
dfmissingrecordThere is no concept of missing plates in DFweb and DFcollect. In DFexplore, a missed record will return 1, and in DFweb and DFcollect, it will return 2.
dfmodeDFweb and DFcollect only has 2 modes: modify and locked, so these are the only two return values possible from dfmode in DFweb and DFcollect.
dfmonth 
dfmovetoDFweb stops infinite loops after 20 iterations. DFcollect only works on plate enter. When dfmoveto(DFSCREEN) is used on plate enter in DFcollect, the cursor moves to the last field.
dfneed 
dfpageinfo 
dfpasswordDFcollect does not remember the previously entered Username. In DFcollect the user must always enter both the Username and Password.
dfpasswdxDFcollect does not remember the previously entered Username. In DFcollect the user must always enter both the Username and Password.
dfplateinfo  
dfpref In DFcollect, only auto logout timer can be set.
dfprefinfo In DFcollect, only auto logout timer can be queried, returning a numeric value; otherwise an empty string is returned.
dfprotocol 
dfqcinfo 
dfqcinfo2 
dfreasoninfo 
dfreplyqc 
dfresqc/dfunresqc 
dfrole 
dfsiteinfo 
sqrt 
dfstay 
dfstr2date 
dfstudyinfo 
dfsubstr 
dftask  
dftime 
dftoday 
dftoolRespective arguments to test for are: "DFexplore", "DFcollect" and "DFweb".
dftriggerActions 3 and 4 behave like actions 1 and 2 respectively in DFcollect and DFweb.
dfvarinfo 
dfvarname 
dfview 
dfvisitinfo 
dfwhoami 
dfyear 
int 

5.9.2. dfaccess

Function dfaccess can be used to change the user's access to data fields on the the current data entry screen in Data View. This is its only purpose. Changes made by dfaccess do not persist after leaving the current data screen and cannot be used to prevent users from seeing or printing data values in List View. For this purpose define Hidden Fields in DFsetup and specify whether users are allowed to see them when defining study roles in DFadmin.

When dfaccess sets a field to 'VIEWONLY', 'MASKED' or 'IMMUTABLE' users can not change the field manually, but an edit check can still change the field; thus some automated change may still occur or users might be prompted using dfask or dfcapture to consent or provide input for some change that is controlled by an edit check.

When a field is set to 'VIEWONLY', users are not prevented from adding or modifying metadata on that field. This remains subject to the metadata permissions defined in the user's study role.

When a field is set to 'MASKED', users cannot view or change the metadata. But the user can tell whether the field has metadata attached by observing the color of the masked field.

Table 5.15. dfaccess usage

Syntax:dfaccess(variable, mode)
Input Parameters:variable is a field on the current page.

mode is one of:

  • DFACCESS_IMMUTABLE - allow user to see but not change the data field or metadata.

  • DFACCESS_VIEWONLY - allow user to see but not change the data field. Allows user to make changes to the metadata.

  • DFACCESS_MASKED - hide the field value beneath a mask and do not allow the user to change it.

  • DFACCESS_HIDDEN - hide the data entry widget so that it does not appear on screen.

  • DFACCESS_NORMAL - return to normal field access

Return Value:None
Example:
dfaccess(@T, DFACCESS_VIEWONLY); # Set current field to viewonly
Example:
# Use: make all data fields view only for users with specified roles
# Run: on plate entry
# e.g. VIEWONLY("monitors,statisticians") - applies to only 2 roles
# e.g. VIEWONLY("ALL") - prevent all users from changing data fields
# note -assumes role names are single words
edit VIEWONLY(string roles)
{
   number fn=6;
   number max1=dfvarinfo(DFSCREEN,DFVAR_FLDNUM);
   number max=max1-1;
   string role=dfrole();
   if( roles=="ALL" || dfmatch(role,roles,"W") ) {
     while( fn <= max ) {
       dfaccess(@[fn],DFACCESS_VIEWONLY);
       fn=fn+1;
     }
   }
}
Notes: dfaccess can be used to prevent users from changing fields manually but it does not prevent fields from being changed by edit checks.

When a field is masked or hidden any queries or reasons are also hidden from view.

For two kinds of special fields, DFCREATE and DFMODIFY, dfaccess cannot change the access modes of them and dfaccessinfo will always return Normal.

dfaccess is implemented only in Data and Fax views in DFexplore. It has no effect in List view or in DFbatch.


5.9.3. dfaccessinfo

dfaccessinfo takes one argument, the name of a field on the current page and returns the current access mode for the requested field. The return mode also depends on 'Show Hidden Fields' role permission.

In DFbatch only, dfaccessinfo will always return 'Normal'.

Table 5.16. dfaccessinfo usage

Syntax:dfaccessinfo(variable)
Input Parameters:variable is a field on the current page.
Return Value:Returns one of the following modes:

  • normal - User is able to see and change the field values/metadata.

  • hidden - The data entry widget is hidden. No change to the field value or metadata is allowed.

  • masked - The field value is hidden beneath a mask. No change to the field value or metadata is allowed.

  • viewonly - User is able to see but not change the data field. Changes to the metadata is allowed.

  • immutable - User is able to see the field values. No change to data field or metadata is allowed.

Example:
dfmessage(dfaccessinfo(@T)); # Print the access mode of the current field
Notes: This is a brief summary of dfaccessinfo:

In DFexplore:

  • If the property 'Hidden' is set to 'No' in DFsetup, the permission 'View Hidden Fields' is unchecked in DFadmin, dfaccessinfo will return 'normal'.

  • If the property 'Hidden' is set to 'Masked' in DFsetup, the permission 'View Hidden Fields' is unchecked in DFadmin, dfaccessinfo will return 'masked'.

  • If the access mode of the next field is set to 'Hidden' using dfaccess, the permission 'View Hidden Fields' is unchecked in DFadmin, dfaccessinfo will return 'hidden'.

  • If the permission 'View Hidden Fields' is checked in DFadmin, the access mode can only be changed using dfaccess. dfaccessinfo will return the current access mode according to the setting of dfaccess. If the Hidden property is changed using DFsetup instead of dfaccess, dfaccessinfo will always return 'normal'.

In DFbatch, dfaccessinfo will always return 'normal'.

As implemented, dfaccessinfo is a synonym for
dfvarinfo(var,DFVAR_ACCESS)
 

5.9.4. dfask

The dfask function is used to pose to the user a question with two possible answers and wait for the user to respond by choosing one of the answers. dfask returns 1 or 2 depending on which response is selected by the user.

Table 5.17. dfask usage

Syntax:dfask(query, default-option, option1, option2)
Input Parameters:query is the question to be asked (string)

default-option is the default response (1 or 2)

option1 is the label to appear on button 1 (string, typically Yes)

option2 is the label to appear on button 2 (string, typically No)

Return Value:1 if button 1 is selected

2 if button 2 is selected

Example:
if (dfask("Add Query?", 1, "Yes", "No") == 1)
Notes: If dfask is executed in batch, there is no opportunity for the question dialog to appear and so default-option is always returned.

5.9.5. dfbatch

The dfbatch function returns TRUE/FALSE depending on whether the edit check is executing in batch mode or not.

Table 5.18. dfbatch usage

Syntax:dfbatch()
Input Parameters:None
Return Value: TRUE if edit check is executing in batch, FALSE otherwise
Example:
if (dfbatch())
    dfmessage("See this when in batch");
else
    dfmessage("See this when not in batch");
Notes: Executing dfbatch in DFexplore always returns FALSE.

5.9.6. dfcapture

The dfcapture function is used to display a dialog for user input.

Table 5.19. dfcapture usage

Syntax:dfcapture(instructions,fieldlist,defaultvalues)
Input Parameters:Your instructions will appear at the top of the dfcapture dialog. They can be in plain text with '\n' used for line breaks, or in html.

The field list is a '|' delimited string comprised of field name and field length pairs.

The default values are a '|' delimited string of initial field values or an empty string if there are no initial values.

Return Value:The return value depends on whether the user presses OK or Cancel in the dfcapture dialog.

If the user presses Cancel dfcapture returns an empty string.

If the user presses OK dfcapture returns a '|' delimited string containing the values entered in the fields presented by the dialog.

Example 1: Here's a very simple example.
string m="Please enter your name and address in the fields below.";
string f="Name|50|Street|50|City|30|State|2|Zip|10";
string v=dfcapture(m,f,"");
Example 2: Here's a more complicated example.
string mycapture(string a)
{
  string msg= "<HTML><HEAD><STYLE>\n"
          + ".g {background-color: #999; color: #fff; padding: 2px; text-align: right;}\n"
          + ".h {background-color: #fff; padding: 2px; text-align: right;}\n"
          + "</STYLE></HEAD>\n"
          + "<BODY><DIV>\n"
          + "<P>CHECK FOR DUPLICATES</P>\n"
          + "<P>Enter the following values:</P>\n"
          + "<TABLE BORDER=0 MARGIN=0>"
          + "<TR><TD CLASS=g>First Name:</TD><TD CLASS=h>Participant's first name</TD></TR>"
          + "<TR><TD CLASS=g>Last Name:</TD><TD CLASS=h>omit titles: Sr., Jr, II, etc.</TD></TR>"
          + "<TR><TD CLASS=g>Birth Month:</TD><TD CLASS=h>1-12</TD></TR>"
          + "<TR><TD CLASS=g>Birth Year:</TD><TD CLASS=h>yyyy, e.g. 1950</TD></TR>"
          + "</TABLE>\n"
          + "<P>DFdiscover will check for a match in the database.\n"
          + "If one is found you will be asked for confirmation.\n\n"
          + "If name or birth date is missing select cancel to enter\n"
          + "this subject without checking for duplicates.</P>\n"
          + "</DIV></BODY></HTML>\n";
  string x="First Name|30|Last Name|30|Birth Month|2|Birth Year|4";
  string s;
  s=dfcapture(msg,x,a);
  return(s);
}
Notes:dfcapture has the following restrictions:
  • The maximum number of fields that can be specified is 10.

  • No field can contain more than 2000 characters.

  • The maximum length of the return string (including all field values and delimiters) is 16384 (4096 if the string contains UNICODE characters).

  • If a default value is specified for a field, but exceeds the field length, it will be used but truncated.

  • The field name does not allow to be empty.

If the dfcapture dialog is moved by the user to a new screen location it will continue to open in that location each time dfcapture is called.


5.9.7. dfcenter

This function returns the study site number for a specified subject ID.

Table 5.20. dfcenter usage

Syntax:

dfcenter(ID)

Input Parameters:ID, subject ID key field identified by variable name or position
Return Value:Site ID to which the subject ID belongs.
Example:
number cid;
cid = dfcenter(SUBJECT); # called using subject ID variable name 'SUBJECT'
cid = dfcenter(@T); # works if edit check is on the subject ID field
cid = dfcenter(@[7]); # always works since the subject ID is always field 7
Notes:

If the input subject ID is not associated with any of the defined sites, returns the ERROR MONITOR site ID.

For historical reasons and backwards compatibility, the function continues to be named dfcenter(), not dfsite().


5.9.8. dfclosestudy

This function can be used to close the current study and return the user to the DFexplore login dialog and list of studies.

Table 5.21. dfclosestudy usage

Syntax:

dfclosestudy(message)

Input Parameters:message - a string containing a message to be displayed in the close study dialog.
Return Value:None
Example:
{
if(dfpasswdx(msg,3) == 0)
    dfclosestudy("Incorrect password. The study will be closed!");
}
Notes:

When dfclosestudy closes a study, all remaining edit checks and pending events will be aborted/canceled.

dfclosestudy will be ignored and do nothing when executed within DFopen_patient_binder and DFopen_study edit checks, and when run in batch mode.


5.9.9. dfdate2str

This function is used to convert a date to a string using a specified date format.

Table 5.22. dfdate2str usage

Syntax:dfdate2str(date,"format")
Input Parameters:date is a date variable

format is the date format to convert to

Return Value: dfdate2str converts the edit check language's internal representation of a date to a string using the specified date format.
Example:
# Convert Visit date variable VDATE to a string in yy/mm/dd format
string d;
d = dfdate2str(VDATE, "yy/mm/dd");

5.9.10. dfday

This function returns the day component, as a number, from a date.

Table 5.23. dfday usage

Syntax:dfday( date )
Input Parameters:date is a date field, a local/global variable or a literal string containing a date
Return Value:A number equal to the day component of the parameter date
Example:
number day;
day = dfday( @T );
if ( day == 1 )
    dfmessage( "This is the first day of the month. Reset pill counter." );
Notes: The date parameter may be a data field defined as a date, or a local/global date variable. If the parameter cannot be interpreted as a date, the return value is -1. If the parameter contains a partial date in the data field, the day component of the parameter is "00", and partial date imputation is set to 'None', the return value is 0. Otherwise, the day component from the date is returned.

5.9.11. dfdirection

The dfdirection function is used when an edit check program needs to know the user's field traversal direction while working in DFexplore.

Table 5.24. dfdirection usage

Syntax:dfdirection()
Input Parameters:None.
Return Value:>0 if the keyboard traversal direction is forward
 <0 if the keyboard traversal direction is backward
 0 if mouse traversal was used
Example:
number whichway;
whichway = dfdirection();
Notes: In plate enter and plate exit edit checks dfdirection returns a number greater than 0. This ensures that these edit checks fire when traversal direction is being used to abort an edit check during backward traversal and/or mouse jumps.

In a field enter edit check dfdirection returns <0,0 or >0, depending on the met