Variables that behave

Every variable in SAS has a name, a type and a length. The type is character (text) or numeric. A variable can also have a label and a format. Most silent errors in clinical programs come from a wrong length or a wrong type. In real production programs, LENGTH and LABEL appear in about half of all programs, and ATTRIB in more than 4 of 10. This chapter shows why. Then it shows the functions you use every day for text, dates and numbers.

3.1 The first use sets the length

SAS decides the length of a character variable when it first meets the variable at compile time. If the first value is short, every later value is cut to that length.

/* Chapter 3: the first use of a variable sets its length */
DATA trap;
  SET refadm.adsl(OBS=3 KEEP=usubjid age);
  IF age < 65 THEN agegrp = "<65";
  ELSE             agegrp = "65 or older";
RUN;

PROC PRINT DATA=trap NOOBS; RUN;

What you should see:

USUBJID	AGE	agegrp
01-701-1015	63	<65
01-701-1023	64	<65
01-701-1028	71	65

The third subject is 71, so agegrp should be “65 or older”. It shows “65”. The first assignment, "<65", has 3 characters, so agegrp got a length of 3. SAS cuts the longer text without any message in the log. No ERROR, no WARNING, no NOTE.

DataStep Studio after F10 on the ELSE line for subject 01-701-1028 (AGE 71): agegrp holds 65, not “65 or older”. The first assignment, “<65”, made agegrp 3 characters long.

Exercise 3-1. In 03_length.sas, swap the two assignments, so that the ELSE text comes first. Predict the result before you run it.

3.2 ATTRIB: length and label first

Declare the variable before any other statement uses it. ATTRIB sets the length, the label and the format in one statement.

/* Chapter 3: ATTRIB sets length and label before the first use */
DATA fixed;
  ATTRIB agegrp LENGTH=$11 LABEL="Age group";
  SET refadm.adsl(OBS=3 KEEP=usubjid age);
  IF age < 65 THEN agegrp = "<65";
  ELSE             agegrp = "65 or older";
RUN;

PROC PRINT DATA=fixed NOOBS LABEL; RUN;
PROC CONTENTS DATA=fixed VARNUM; RUN;

What you should see: “65 or older” in full for subject 01-701-1028, and in PROC CONTENTS:

#	Variable	Type	Len	Label
1	agegrp	Char	11	Age group
2	USUBJID	Char	11	Unique Subject Identifier
3	AGE	Num	8	Age

In ADaM, the specification gives every variable its length and label. Write them with ATTRIB at the top of the step. Then the data set matches the specification exactly.

Exercise 3-2. Add LABEL agegrp = "Age group (years)"; to 03_attrib.sas after the ATTRIB statement. Which label wins?

3.3 PUT makes text, INPUT makes numbers

A variable has one type for its whole life. To change the type, create a new variable:

PUT(number, format.) returns text.
INPUT(text, informat.) returns a number (or a date, with a date informat).

/* Chapter 3: PUT makes text, INPUT makes numbers */
DATA conv;
  SET sdtm.dm(OBS=3 KEEP=usubjid siteid age);
  LENGTH age_c $3 subj $4 txt $30;
  site_n = INPUT(siteid, 8.);
  age_c  = PUT(age, 3.);
  subj   = SCAN(usubjid, 3, "-");
  txt    = CATX(" / ", usubjid, age_c);
RUN;

PROC PRINT DATA=conv NOOBS; RUN;
PROC CONTENTS DATA=conv VARNUM SHORT; RUN;

What you should see:

USUBJID	SITEID	AGE	age_c	subj	txt	site_n
01-701-1015	701	63	63	1015	01-701-1015 / 63	701
01-701-1023	701	64	64	1023	01-701-1023 / 64	701
01-701-1028	701	71	71	1028	01-701-1028 / 71	701

SITEID is text in SDTM. site_n is its numeric copy.
SCAN(usubjid, 3, "-") takes the third word. Here a - separates the words.
CATX(" / ", ...) joins values with a separator. It removes the blanks before and after each value.

The log stays clean. There is no “converted” NOTE, because every conversion is on purpose.

Exercise 3-3. Create age_months = INPUT(age_c, 3.) * 12; in 03_put_input.sas. Is the log clean?

3.4 Text functions you use every day

/* Chapter 3: the text functions you use every day, on the adverse events of SDTM */
DATA ae_text;
  LENGTH site $3 subj $4 short $200 label $250;
  SET sdtm.ae(KEEP=usubjid aeterm aedecod);
  site     = SCAN(usubjid, 2, "-");
  subj     = SUBSTR(usubjid, 8, 4);
  has_pain = (FIND(aedecod, "pain", "i") > 0);
  short    = TRANWRD(aedecod, "APPLICATION SITE", "APPL. SITE");
  digits   = INPUT(COMPRESS(usubjid, , "kd"), 9.);
  label    = CATX(" | ", usubjid, aedecod);
RUN;

PROC FREQ DATA=ae_text;
  TABLES has_pain / NOCUM;
RUN;

PROC PRINT DATA=ae_text(OBS=3) NOOBS;
  VAR usubjid site subj digits short label;
RUN;

What you should see: 33 of the 1191 adverse events have “pain” in AEDECOD. The first three records:

USUBJID	site	subj	digits	short	label
01-701-1015	701	1015	17011015	APPL. SITE ERYTHEMA	01-701-1015 \| APPLICATION SITE ERYTHEMA
01-701-1015	701	1015	17011015	APPL. SITE PRURITUS	01-701-1015 \| APPLICATION SITE PRURITUS
01-701-1015	701	1015	17011015	DIARRHOEA	01-701-1015 \| DIARRHOEA

Function	What it does here
`SCAN(usubjid, 2, "-")`	the second word: the site
`SUBSTR(usubjid, 8, 4)`	4 characters from position 8
`FIND(aedecod, "pain", "i")`	the position of “pain”, 0 if not found; `"i"` ignores upper and lower case
`TRANWRD(aedecod, "APPLICATION SITE", "APPL. SITE")`	replaces a word with another
`COMPRESS(usubjid, , "kd")`	keeps (`k`) only the digits (`d`)
`CATX(" \| ", ...)`	joins the values with a separator

(FIND(...) > 0) is a comparison. It gives 1 when it is true and 0 when it is false. That makes a flag in one line.

Exercise 3-4. Search AETERM (the verbatim term) instead of AEDECOD. How many records have “pain”?

3.5 Clean text before you use it

Free text in SDTM can hold line breaks from the data entry system: the characters carriage return and line feed, written in SAS as '0D0A'x. You cannot see them, but they change the sort order, break merges and break the layout of a listing.

/* Chapter 3: clean text before you sort, merge or print it */
DATA text;
  LENGTH raw clean1 clean2 $40;
  raw    = "  Mild headache" || '0D0A'x || "after dosing ";
  clean1 = COMPRESS(raw, '0D0A'x);
  clean2 = COMPBL(STRIP(TRANSLATE(raw, "  ", '0D0A'x)));
  len_raw    = LENGTH(raw);
  len_clean2 = LENGTH(clean2);
  upper      = UPCASE(clean2);
  first8     = SUBSTR(clean2, 1, 8);
RUN;

PROC PRINT DATA=text NOOBS; VAR clean2 len_raw len_clean2 upper first8; RUN;

What you should see:

clean2	len_raw	len_clean2	upper	first8
Mild headache after dosing	29	26	MILD HEADACHE AFTER DOSING	Mild hea

COMPRESS(raw, '0D0A'x) removes the two characters. The words around them are then joined.
TRANSLATE(raw, " ", '0D0A'x) changes each one to a space, COMPBL changes several blanks to one, and STRIP removes the leading and trailing blanks. The result has 26 characters instead of 29.
UPCASE and SUBSTR(text, start, length) complete the set of text functions you use most.

Clean a text variable in the step that reads it from SDTM, before you use it as a sort key or BY variable.

3.6 Patterns: regular expressions

A regular expression describes a pattern of text. PRXMATCH(pattern, text) gives the position where the pattern starts, or 0. Here it sorts the AE start dates by how complete they are.

/* Chapter 3: a regular expression sorts ISO 8601 dates by how complete they are */
DATA ae_dtc;
  LENGTH dtc_type $10;
  SET sdtm.ae(KEEP=usubjid aeseq aestdtc);
  IF      PRXMATCH("/^\d{4}-\d{2}-\d{2}/", aestdtc)    THEN dtc_type = "complete";
  ELSE IF PRXMATCH("/^\d{4}-\d{2}$/", STRIP(aestdtc))  THEN dtc_type = "year-month";
  ELSE IF PRXMATCH("/^\d{4}$/", STRIP(aestdtc))        THEN dtc_type = "year";
  ELSE                                                      dtc_type = "missing";
RUN;

PROC FREQ DATA=ae_dtc;
  TABLES dtc_type / NOCUM;
RUN;

What you should see:

dtc_type	Frequency	Percent
complete	1165	97.82
year	11	0.92
year-month	15	1.26

How to read the pattern /^\d{4}-\d{2}$/:

^ is the start of the text and $ is the end.
\d is one digit. \d{4} is four digits.
- is the character itself.

So the pattern means: four digits, a dash, two digits, and nothing more. STRIP removes the blanks at the end first, because $ must see the real end. Chapter 11 imputes the 15 and the 11 partial dates.

Exercise 3-5. How many subjects have a start date with only the year?

3.7 Date and time functions

A SAS date counts days, so you can subtract two dates. The functions below do the rest.

/* Chapter 3: date and time functions on real dates */

/* Step 010: a date and a time from ISO 8601 text: the first laboratory sample */
DATA lb_dtm;
  SET sdtm.lb(KEEP=usubjid lbtestcd lbdtc OBS=1);
  lbdtm = INPUT(lbdtc, E8601DT.);
  lbdt  = DATEPART(lbdtm);
  lbtm  = TIMEPART(lbdtm);
  FORMAT lbdtm E8601DT. lbdt DATE9. lbtm TIME5.;
RUN;

PROC PRINT DATA=lb_dtm NOOBS;
RUN;

/* Step 020: AE duration in days, and the months since the first dose */
DATA ae_dur;
  SET refadm.adae(KEEP=usubjid aedecod astdt aendt trtsdt adurn);
  WHERE NOT MISSING(astdt) AND NOT MISSING(aendt);
  aedur   = aendt - astdt + 1;
  months  = INTCK("MONTH", trtsdt, astdt);
  mon_beg = INTNX("MONTH", astdt, 0, "B");
  FORMAT mon_beg DATE9.;
RUN;

PROC PRINT DATA=ae_dur(OBS=3) NOOBS;
  VAR usubjid aedecod trtsdt astdt aendt aedur adurn months mon_beg;
RUN;

PROC SQL;
  SELECT COUNT(*) AS records, SUM(aedur NE adurn) AS differ
    FROM ae_dur;
QUIT;

/* Step 030: INTCK counts boundaries, not full months */
DATA _NULL_;
  one_day   = INTCK("MONTH", "31JAN2014"d, "01FEB2014"d);
  full_mons = INTCK("MONTH", "31JAN2014"d, "01FEB2014"d, "C");
  PUTLOG one_day= full_mons=;
RUN;

/* Step 040: years on treatment */
DATA trt_years;
  SET refadm.adsl(KEEP=usubjid trtsdt trtedt trtdur);
  WHERE NOT MISSING(trtedt);
  years = YRDIF(trtsdt, trtedt, "ACT/ACT");
RUN;

PROC MEANS DATA=trt_years N MEAN MAX MAXDEC=2;
  VAR trtdur years;
RUN;

What you should see:

Step	Result
010	`2013-12-26T14:45` becomes the datetime `2013-12-26T14:45:00`, the date `26DEC2013` and the time `14:45`
020	718 events have a start and an end date. For 01-701-1015, DIARRHOEA from 09JAN2014 to 11JAN2014 lasts 3 days. `months` is 0 and `mon_beg` is 01JAN2014.
020	`aedur` equals `ADURN` in 714 records. 4 records differ.
030	`one_day=1 full_mons=0`
040	254 subjects: on average 116.09 days or 0.32 years on treatment, at most 212 days or 0.58 years

E8601DT. reads an ISO 8601 date and time. The result is a datetime: the seconds since 1 January 1960.
DATEPART and TIMEPART take the date and the time from a datetime.
aendt - astdt + 1 counts both the first and the last day.
INTCK("MONTH", a, b) counts the month boundaries between two dates. From 31JAN to 01FEB there is one boundary, but no full month. With "C" (continuous), INTCK counts full months.
INTNX("MONTH", date, 0, "B") moves to the beginning of the month.
YRDIF(a, b, "ACT/ACT") gives the years between two dates, with leap years.

Exercise 3-6. Print the 4 records where aedur and ADURN differ. Look at ASTDTF. Why does the reference leave ADURN empty?

3.8 Numbers: two traps

/* EXPECT: NOTE Missing values were generated */
/* Chapter 3: numbers. Two traps: decimal fractions, and + with a missing value */
DATA num;
  x           = 0.1 * 3;
  equal_raw   = (x = 0.3);
  equal_round = (ROUND(x, 1E-9) = 0.3);
  mean        = 147.32;
  mean_1      = ROUND(mean, 0.1);
  up          = ROUND(2.5);
  down        = ROUND(-2.5);
  p           = .;
  q           = 4;
  plus        = p + q;
  total       = SUM(p, q);
  PUTLOG x= equal_raw= equal_round= mean_1= up= down= plus= total=;
RUN;

What you should see in the log:

x=0.3 equal_raw=0 equal_round=1 mean_1=147.3 up=3 down=-3 plus=. total=4
NOTE: Missing values were generated as a result of performing an operation on missing values.

Decimal fractions. A computer stores 0.1 as a binary fraction that is a little wrong. So 0.1 * 3 is not exactly 0.3, although the log shows 0.3. Never compare computed decimals with =. Round them first: ROUND(x, 1E-9).
ROUND rounds to the unit you give: ROUND(147.32, 0.1) is 147.3. A value exactly halfway goes away from zero: 2.5 becomes 3, and -2.5 becomes -3.
A missing value in +. p + q is missing when one value is missing, and the log gets a NOTE. SUM(p, q) ignores the missing value and gives 4.

Choose on purpose. A total score must often be missing when one item is missing. Then + is right. A count of events should ignore the missing ones. Then SUM is right.

Exercise 3-7. a = 1; b = .; c = 3;. What do (a + b + c) / 3 and MEAN(a, b, c) give?

3.9 Take the text of a code from its format

ADaM often holds a code and its text: TRT01PN and TRT01P. Do not type the text a second time in IF statements. Attach a format to the code and let SAS give you the text.

/* Chapter 3: take the text of a code from its format */
PROC FORMAT;
  VALUE trtn 0  = "Placebo"
             54 = "Xanomeline Low Dose"
             81 = "Xanomeline High Dose";
RUN;

DATA decode;
  SET refadm.adsl(KEEP=usubjid trt01pn WHERE=(usubjid IN ("01-701-1015" "01-701-1028" "01-701-1033")));
  LENGTH trt_txt $40;
  FORMAT trt01pn trtn.;
  IF NOT MISSING(trt01pn) THEN trt_txt = VVALUE(trt01pn);
RUN;

PROC PRINT DATA=decode NOOBS; RUN;

What you should see:

USUBJID	TRT01PN	trt_txt
01-701-1015	Placebo	Placebo
01-701-1028	Xanomeline High Dose	Xanomeline High Dose
01-701-1033	Xanomeline Low Dose	Xanomeline Low Dose

VVALUE(trt01pn) returns the value as the attached format shows it. When a label changes, you change the format once, and every program follows. Two rules:

Make sure that a format is attached. The FORMAT or ATTRIB statement may stand anywhere in the step, because SAS attaches formats when it compiles the step.
Test for a missing code first. A missing number shows as ..

Exercise 3-8. A code 99 is not in the trtn. format. What does VVALUE return for it? How can a program detect this?

The answers to exercises 3-1 to 3-8 are in Answers, Part I.