PLScsi Home

Cdb Complete

This web page was http://members.aol.com/plscsi/cdbcomplete.html sometime after Sunday, April 21, 2002.

( PageUp Contents PageDown )

Why don't the public standards of Appendix B, Bibliography, accurately describe how Scsi hosts & Scsi devices actually do interoperate?

Here we offer the shortest credible analysis we've seen (four chapters, three appendices, less than 75KiB total).

In the loose term "Scsi", we include all of Scsi-over-whatever. The links of Appendix B point to actual specifications or else final public drafts for all of: classic Scsi in two flavours: eight-bit parallel and sixteen-bit parallel; Scsi-over-Ide in four flavours: Atapi Pio, Atapi SwDma, Atapi MwDma, Atapi UDma; Scsi-over-Usb; Scsi-over-1394

The exercise of treating Ata as a form of Scsi with a renumbered command set intrigues us, but we have not yet gone far down that road.

 


0. Contents

( PageUp Contents PageDown )

 

Chapters:

 

0. Contents

0.1 Preface

 

1. A Cdb Is Not Enough

1.1 All Of The Command

1.2 All Of The Status

 

2. The Thirteen Possible Cases

2.1 (D = H) The Thin Diagonal

2.2 (D < H) Positive Residue

2.3 (H < D) Negative Residue

2.4 (I != O) Wrong Way

 

3. Scsi Over Whatever

3.1 All Of The Pipe

3.2 The Visible End of The Pipe

3.3 Reporting Of D, Surveyed

3.4 So What If D Is Secret

3.5 Indeterminacy, Surveyed

 

4. The Five Common Results

 

Appendices:

 

A. Examples

B. Bibliography

B.1 Scsi Web Links

B.2 Scsi Web Trails

C. On Reproducing Our Results

Finally

 


0.1 Preface

( PageUp Contents PageDown )

Got a storage peripheral? Bought it retail? Odds are good your device talks Scsi-over-whatever. Scsi-over-whatever includes the Scsi over Ide also known as Atapi, Scsi over Usb, Scsi over 1394, .... Even Ata hard drives talk a protocol that can usefully be viewed as a creatively renumbered subset of Scsi.

Ever wonder what the last moments are like, just before the first version of a Scsi-over-whatever device ships retail? We think http://www.dilbert.com/ correctly suggests those last moments often go something like this:

[ Boss ] Have you seen a version of Microsoft Windows copy some files Out to and In from your device?

[ Engineer ] Yes, but ...

[ Boss interrupts.] Got any known problems?

[ Engineer ] No, but ...

[ Boss interrupts.] Ship it.

The consequences of a development process like this are:

What occurs commonly works well; what occurs less commonly works less well.

Not surprising, eh? Ok then, what does occur commonly when talking Scsi-over-whatever to a boot device: a hard drive, a floppy drive, a Cd-rom drive?

Here we offer the shortest accurate vocabulary we know for talking about Scsi really means there.


1. A Cdb Is Not Enough

( PageUp Contents PageDown )

In the terms of the C programming language, we say that how Scsi is spoken across platforms is:

int plscsiCall(void const * cdb, int cdbLength, void * data, int dataLength, int pleaseIn);

Ask yourself now, why is this common interface not just:

boolean plscsiCall(void const * cdb, void * data);

We answer this below:

1.1 All of The Command (why specify dataLength, pleaseIn)

1.2 All of The Status (why return int)


1.1 All of The Command

( PageUp Contents PageDown )

Given:

int plscsiCall(void const * cdb, int cdbLength, void * data, int dataLength, int pleaseIn);

we ask: why not pass less parameters? Why redundantly specify dataLength and pleaseIn?

Because the Cdb is not complete. The bytes of a Cdb do not plainly tell the device the host's idea of:

  1. cdbLength = how many bytes the Cdb contains
  2. dataLength = how many Data bytes to copy
  3. pleaseIn = whether to copy Data bytes In to the host or to copy bytes Out from the host

Now remember that many Scsi-over-whatever protocols pass nothing but the Cdb to the device. Therefore, necessarily:

The host and the device choose independently how many Data bytes to copy which way.

We classify the consequences of this potential for disagreement at 2.0 The Thirteen Possible Cases.

The Scsi host & device also choose independently how many bytes of the Cdb to copy out.

 


1.2 All of The Status

( PageUp Contents PageDown )

Given:

int plscsiCall(void const * cdb, int cdbLength, void * data, int dataLength, int pleaseIn);

we ask: why not return less bits?

Because after the host asks to copy dataLength bytes In or Out, how the device responds varies. Host interfaces that ask to copy data bytes to/from devices commonly report not only if a copy occurred, but also specifically how many data bytes were copied.

Contrast these three standard C services:

void * memmove(void * to, void * from size_t length);

void * memcpy(void * to, void * from size_t length);

int fread(void * to, void * from size_t length);

The designers of memmove and memcpy were so sure of complete success that they chose to return a copy of the to parameter for your convenience. The designers of fread knew they had to make more careful distinctions for I/O.

For fread we have:

(int == length) // complete success

(0 <= int <= length) // less bytes actually copied than requested

((int < 0) || (length < int)) // some other form of failure

See where we say (0 <= int) rather than (0 < int)? The fact that I/O may succeed without copying any bytes frequently astonishes novice I/O programmers. To guarantee that a loop of fread terminates within a finite count of iterations, the host has to count how often no bytes were copied at all despite its polite request. Naive application code often works only because the local implementation of fread never does return zero twice in a row.

Many I/O folk prefer an equivalent, but different, set of distinctions:

(int == 0) // complete success

(0 < int <= length) // residue

((int < 0) || (length < int)) // some other form of failure

Here we introduce the term "residue", also known as the "residual byte count":

The "residue" is the difference between the count of bytes the host asked to copy and the count of bytes that the host did copy.

This alternative convention makes complete success easier to recognise: zero returned means complete success, nothing else does.

 

 


2.The Thirteen Cases

( PageUp Contents PageDown )

Back in 1.1 All of The Command we explained: "the host and the device choose independently how many Data bytes to copy which way". More specifically now:

A Scsi host can ask to copy no Data bytes, some Data bytes In, or some Data bytes Out. Then, symmetrically ...

A Scsi device can ask to copy no Data bytes, some Data bytes In, or some Data bytes Out.

Two players, each with three independent choices, together make 3x3 = 9 possible combinations. Furthermore, when the host and the device do agree to copy some bytes In or Out, then still either one may ask to copy less bytes than the other. That adds 4 more possible cases, for a total of 9 + 4 = 13 cases to discuss.

The committee that specified how Scsi-over-Usb works published a diagram of these 13 cases:

You see here above the 10,178 bytes of our 13cases.gif diagram. The committee's actual diagram appears as "Table 6.1 - Host/Device Data Transfer Matrix" in the section titled "6.7 The Thirteen Cases" of the 103,609 bytes of http://www.usb.org/developers/devclass_docs/usbmassbulk_10.pdf. A smaller copy of a late draft of their diagram appears at 20000125/13cases.gif .

We made our diagram black & white and we used font size to clearly divide the thirteen cases into four groups.

We now discuss those four groups in order, from what occurs most to least commonly:

2.1 (D = H) The Thin Diagonal

2.2 (D < H) Positive Residue

2.3 (H < D) Negative Residue

2.4 (I != O) Wrong Way

 


2.1 (D = H) The Thin Diagonal

( PageUp Contents PageDown )

Cases 1, 6, and 12 occur most commonly.

In these three cases, the device actually agrees to copy as many bytes as the host asked to copy, and in the same direction.

People say "the thin diagonal" to mean this group of cases, because together these three cases form a thin diagonal of the matrix of the 13 possible cases, if you draw the matrix like the Scsi-over-Usb folk do.

 


2.2 (D < H) Positive Residue

( PageUp Contents PageDown )

Cases 4, 5, 9, and 11 occur next most commonly.

In these four cases, the host asks to copy some bytes, but then the device actually asks to copy less bytes than the host asked, though yes in the same direction.

People say "residue", or "residual count", to mean the count of bytes the host asked to copy that the device did not agree to copy. People like counting residue rather than counting bytes copied because by counting residue the thin diagonal becomes easy to distinguish. The thin diagonal is where the residue is zero. In these four cases, the residue is positive.

Positive residue can be harmless if reported. Only the Scsi host can know if it meant to be imprecise in forecasting how many bytes to copy.

Imprecision has small benefits. Some systems can only copy N * 8 bytes. To copy just N * 8 - 7 bytes, a system like that has to copy an extra 7 pad bytes. A host can allow for this by always allocating an extra few bytes. We recommend allocating an extra physical page (often 4KiB).

Imprecision has large costs. Some systems hang if residue is positive. Others dramatically slow down.

No cases but the thin diagonal and the positive residue cases Should ever occur. The device and the host should at least agree over the max count of bytes to copy which way. Meanwhile, back in the real world ...

 


2.3 (H < D) Negative Residue

( PageUp Contents PageDown )

Cases 2, 3, 7, and 13 occur next most commonly.

In these four cases, the host asks to copy some bytes, but then the device actually asks to copy more bytes than the host asked, though yes in the same direction. A host that agrees to this has agreed to evil: such a host either has to discard bytes copied In or else fabricate bytes copied Out.

To say "negative residue" here, we have to expand our definition of "residue". We say the "signed residue" is the difference between the count of bytes the host asked to copy and the count of bytes the device agreed to copy. In these cases, that signed residue is negative.

Some systems hang if the signed residue is negative. Others tolerate small negative residues: for example, as many as 7 extra bytes when the count of bytes to be copied is not a multiple of 8. Other systems really do discard and fabricate data without complaint.

 


2.4 (I != O) Wrong Way

( PageUp Contents PageDown )

Cases 8 and 10 occur next most commonly.

In these final two cases, the device actually asks to copy data in the opposite direction from the host.

If we were to separately calculate a residue Out and a residue In, then one residue would be zero but the other would be negative.

Many systems hang when this occurs. Others really do discard and fabricate all the data without complaint.

 

 


 

3. Scsi Over Whatever

( PageUp Contents PageDown )

Back in chapter 2, The Thirteen Cases, we passed quickly over how we discover what D and H is.

We focus on that issue now, with a particular emphasis on D, since the host knows H by definition.

 


3.1 All of The Pipe

( PageUp Contents PageDown )

Can we talk intelligibly about just one host and one device?

No, not above the link layer. People actually implement hosts & devices in layers.

From elsewhere, maybe incorrectly from the ISO Networking model, we borrow the terms:

host

 

device

...

 

...

transport

 

transport

link

--- bus ---

link

 

Life is simplest when A = H = B = D, but when Scsi-over-whatever folk choose a "word" size K other than 1 for a bus, they constrain the higher layers perceptibly:

Given the constraint B = N * K,

we lose A = H = B = D when D != N * K.

Our variables here are:

H   the count of bytes the host asks to copy, at the highest level
     
B   the count of bytes the host & the device agree to copy, down at the link layer
N   the count of data word clocks, down at the link layer
K   the size in bytes of each data word, down at the link layer
     
D   the count of bytes the device asks to copy, at the highest level

and also:

A   the count of bytes the host actually copies to/from its own virtual memory, at the highest level

People talk mostly about life as it is when A = H = B = D. But what about when we do have D != N * K, perhaps because H != N * K. Well then:

To guarantee D <= B, we have to accept B - D = N * K - D.

Here B - D = N * K - D is the count of pad bytes that we clock across the bus. That is, to copy all of D sometimes we have to start with B = D but then round up B to the next bus word boundary: we have to accept D < B. We have to clock one or more extra pad bytes across the bus any time D != N * K. These pad bytes are input to the recipient, but by convention we suppose they are utterly meaningless all the same.

 


3.2 The Visible End of The Pipe

( PageUp Contents PageDown )

Merely by writing an application on the host, we can easily measure:

P = A - D = the count of pad bytes actually copied

The count of copied pad bytes correlates no more than loosely with the count of pad bytes on the bus. We see real live people commonly choose to implement one of:

P = A - D = 0

(e.g. K = 2 and P = 0, no matter if B - D = 1, for Microsoft Win98/WinMe Atapi Pio)

P = A - D = B - D

(e.g. K = 2 and P = B - D for other Atapi Pio & often for Microsoft Atapi Dma)

P = A - D = B' - D where B' = D rounded up to some word size K' > K

(e.g. K = 2 and K' = 4 for Microsoft Atapi Dma on some motherboards)

 


3.3 Reporting of D, Surveyed

( PageUp Contents PageDown )

As of April, 2002, ...

If we survey the popular forms of Scsi over whatever, we see they vary in whether they permit the device to report D to the host, either directly or by having the host observe B or H and then reporting B - D or H - D.

Having lost A = H = B = D, we can still grasp at the workable compromise A= D < B if and only if some layer reports D. If we see D < B, then we "double-buffer". We copy all of the B bytes into one memory buffer, but then we copy just A = D bytes from that buffer to the buffer of the application. We only have to bother with this when D != N * K is possible - but that's always.

No single one of us can testify to all of the results in this survey, but on the 'net we hear:

     

April, 2002

   
           
K     Scsi Over WHAT   Reports D (*)
           
1     Classic Scsi
i.e. parallel eight-bit, fully handshaked
  YES: Reports D via D = B
           
2     Sixteen-bit Scsi   optional, not much used:
IgnoreWideResidue can report B - D
2     Atapi Pio
i.e. Scsi over Ide below 17e+6 byte/s
  YES: Reports D via B - D
2     Atapi SwDma MwDma UDma
i.e. Scsi over Ide above 17e+6 byte/s
  possible, but not a standard option
           
8/16/32/64     Scsi over (serial) Usb   YES: Reports D via H - D
4     Scsi over (serial) IEEE 1394
i.e. Apple FireWire i.e. Sony iLink
  yes and no (**)
(unknown)     Scsi over serial Ide (aka serial Ata)   (unknown)

We've bolded the names of the Scsi over whatever protocols that do report D.

(*) Some hosts try to limit I/O to accessing only the allocated bytes of memory. With a host designed to guarantee A <= H, D may be reported precise to the byte only when D <= H, because otherwise the host stops the copy before copying D bytes.

(**) IEEE 1394 gets a "yes and no" because IEEE 1394 counts pad bytes, and Ansi Sbp2 can count pad bytes, but then the next layer up, the Scsi over 1394 that is Ansi Sbp2 Normative Annex B, that layer leaves D and H - D unmentioned. The robust lower layers do let the device choose between A = D and A = B, but the device has no standard way to report its choice. In particular, at the level of Sbp2 Normative Annex B, the device does not report D, nor does the device report H - D.

 


3.4 So What If D Is Secret

( PageUp Contents PageDown )

When a bus does not bother to report D, we say the indeterminacy in the device byte count is:

I = B - D

Now remember that commonly we know people design hosts and devices to try to guarantee:

0 <= B - D <= K - 1
0 <= B - H <= K - 1

Therefore when D = H we have:

max I = max(B - D) = K - 1

But when we have D < H or H < D, then N * K may rise as far above D as H does, or as far above H as D does:

max I = max(B - D) = X * K + K - 1

Here we have introduced a further observation at the bus:

X is the count of extra data clocks

X is the count of data clocks that did cross the bus after the byte boundary at which just the device or just the host did first ask to stop the copy.

So in summary, when we do not know if D = H, we have:

max I = max(B - D) = X * K - 1

 


3.5 Indeterminacy in D, Surveyed

( PageUp Contents PageDown )

As of April, 2002, ...

If we survey the popular forms of Scsi over whatever, we see they vary in how large they permit X to be, thus how large they permit max I, "the indeterminacy in the device byte count", to be.

With hosts designed to guarantee A <= H and A <= D, we see:

     

April, 2002

   
           
K     Scsi Over WHAT   max I = max( B - D) (*)
           
1     Classic Scsi
i.e. parallel eight-bit, fully handshaked
  0
           
2     Sixteen-bit Scsi   1
2     Scsi over Ide - Atapi Pio   0
2     Scsi over Ide - Atapi SwDma   1
2     Scsi over Ide - Atapi MwDma   3
2     Scsi over Ide - Atapi UDma   5 for UDma33,
rises with burst rate
           
8/16/32/64     Scsi over serial Usb   0
4     Scsi over serial IEEE 1394
i.e. Apple FireWire i.e. Sony iLink
  H - D
(unknown)     Scsi over serial Ide (aka serial Ata)   (unknown)

We've bolded the names of the Scsi over whatever protocols that do guarantee I = 0: all these report D.

(*) These results change if recklessly we assume D <= H.

For Atapi UDma In, max I falls to K - 1 = 1 with any device designed to guarantee B - D < K

For Atapi UDma Out, max I remains 5 at UDma33, increasing as X increases with burst rate.

 


4.The Five Common Results

( PageUp Contents PageDown )

In the terms of the C programming language, we say that how Scsi is spoken across platforms is:

int plscsiCall(void const * cdb, int cdbLength, void * data, int dataLength, int pleaseIn);

Diagrammed as a pair of communicating state machines, our model is:

In our model, the Scsi host programs the host machine on the left, the Scsi device programs the device machine on the right. The programs for these machines say to copy the CommandOut from the host and to copy the StatusIn to the host. Between CommandOut and StatusIn, the programs say to copy no data else DataIn to the host else DataOut from the host. Variations on Scsi designed to tolerate large latencies often move the host machine into the device, but our model remains valid: still the Scsi host programs the host machine, still the Scsi host collects results from the host machine.

We have consciously, sharply, restricted our model to accurately represent the limitations commonly found in the actual machines common to laptops, desktops, and their peripherals. In comparison with the arbitrary model that Ansi publishes ("Figure 16 - Phase Sequences" on page 51 of the 6,394,357 bytes of ftp://ftp.t10.org/t10/drafts/s2/s2-r10l.pdf), we always copy only one Command, we copy the Command only before Data or Status, we copy Data only one way, we always copy Status only once, and we copy Status only after Command or Data.

 

 

 

[ CAUTION: THIS PAGE IS INCOMPLETE AS YET - ABOUT HERE YOU CAN BEGIN TO SEE THAT CLEARLY. ]

 

 

[ !!! THIS TEXT IS GARBLED !!! ]

 

The two machines copy three kinds of bits: Command, Data, and Status. Therefore, yes, in theory, we have to consider The Thirteen Cases above for each of those copies. That makes 13x13x13 = 2,197 possible cases. But in practice, we can divide these two thousand plus cases according to what their visible results should be, again from most to least commonly occurring:

  1. Passed Without Demur (zero return code)
  2. Passed Except For Positive Residue (positive return code)
  3. Failed With Positive Residue (-x102)
  4. Failed Without Residue (-2)
  5. Other (-1, other)

As we discuss what the results should be, again we will mention what actual results commonly are, in the hopes of persuading people over time to correct them. We first present our claims in summary, then we return to defend each claim with reproducible examples.

 

Passed Without Demur (zero return code)

The int result should be zero if and only if:

  1. Both machines copied all of the CommandOut.
  2. Both machines copied in the expected direction precisely the expected count of Data bytes: either no bytes, else all of DataIn, else all of DataOut.
  3. Both machines copied all of the StatusIn, and that StatusIn meant PassedWithoutDemur.

Some actual Scsi hosts wrongly report some of the following cases as Passed.

Some actual Scsi APIs let some bits of the StatusIn say the remaining bits probably don't matter. We find guesses like that wrong often enough that we prefer zeroed bits.

 

Passed Except Positive Residue (positive return code)

 

[ !!! THIS TEXT IS GARBLED !!! ]

 

The int result should be positive if and only if both machines copied less than the expected count of Data bytes but otherwise the result would have been PassedWithoutDemur.

Again, by the "residue" we mean the difference between the count of bytes the host asked to copy and the count of bytes the device agreed to copy. If the residue is positive, then the residue should be the int result.

1394 Sbp2 protocol by definition cannot distinguish PassedExceptPositiveResidue from PassedWithoutDemur.

The Dma subset of the Ata and Atapi protocols force the device to copy In an extra byte when D is odd. And they force the device to discard up to X * 2 + 1 extra bytes Out when Do < Ho. X is 0 for SwDma. X is 1 for MwDma as specified by Ansi, but often 0 for MwDma in practice. X is 2 for UDma33. X rises with burst rate.

Some actual Scsi APIs make the counting of bytes optional. To get correct results, the client must work to discover if that option is supported or not. Some Scsi hosts report falsely that they do not support the option. When the option is not supported, the client cannot correctly distinguish PassedWithoutDemur from PassedExceptPositiveResidue.

Some actual Scsi APIs count bytes rather than counting residue. To get correct results, the client must calculate the residue. Again, the client cannot correctly distinguish PassedWithoutDemur from PassedExceptPositiveResidue unless the client does calculate residue.

Some actual Scsi APIs separate the residue from other Status copied In. To get correct results, the client must always work to collect the residue, no matter that commonly the residue is zero.

Some actual Scsi hosts recklessly provoke positive residue, for eminently practical reasons like wanting to talk as much as possible like Microsoft talks. For example, many Scsi hosts send, as the first command, -x 12 0 0 0 24 0 -i x24 // Inquiry for up to x24 bytes. If the device exercises the freedom Ansi ingenuously offers to make less than x24 bytes of Inquiry data available, then we have Di < Hi i.e. positive residue.

When residue is positive, some actual Scsi host hardware, especially Dma hardware, waits indefinitely for a timeout and reset.

By definition, a positive residue in excess of <limits.h> INT_MAX can't be returned as a positive int, so then the int result should be negative. See Other.

 

Failed With Positive Residue (-x102)

 

[ !!! THIS TEXT IS GARBLED !!! ]

 

Suppose the Status copied In did not mean PassedWithoutDemur and both machines copied less than the expected count of Data bytes but otherwise the result would have been PassedWithoutDemur. Then the int result should be -x102. -2 says Failed, -x100 says positive residue.

This occurs relatively often. A failure to read a block often looks like this.

 

Failed Without Residue (-2)

 

[ !!! THIS TEXT IS GARBLED !!! ]

 

Suppose the Status copied In did not mean PassedWithoutDemur but otherwise the result would have been PassedWithoutDemur. Then the int result should be -2. -2 says Failed, the lack of -x100 says not positive residue.

Ata Pio protocol by definition cannot distinguish FailedWithoutResidue from PassedWithoutDemur. (Atapi Pio protocol can.)

This occurs commonly via Hn = Dn, reasonably commonly via Ho = Do, but less commonly via Hi = Di.

An failure to read a block looks like this (Hi = Di) only if the device detects the failure too late to stop from copying the block. Scsi mode page 1 [tbd name] bit TB [tbd value] (Transfer Block In Error) makes a failure to read a block look like this only if the block in error was the last block the host asked to copy.

 

Other (-1, other)

 

 

[ !!! THIS TEXT IS GARBLED !!! ]

 

And then we have everything else Bad that happens. Notable here are:

CommandOut omitted or stuttered

Rumour (slander?) says some Scsi devices don't always ask to copy one CommandOut.

The Scsi host should notice that it failed to copy precisely one copy of the CommandOut, and accordingly report Other.

Bad examples include:

Rumour (slander?) says some Atapi devices sometimes ask twice for the bytes of the CommandOut.

StatusIn omitted

Entirely within spec, the Scsi device machine may "go unexpectedly" to BusFree. This occurs most commonly because of an uncoordinated reset, like when the selected device gets reset because some other thread (or interrupt or power managing Bios or ...) reset some other device nearby.

The Scsi host should notice that it failed to copy some bits of StatusIn, and accordingly report Other.

Bad examples include:

Rumour (slander?) says Microsoft Atapi ignores final C/D I/O - thus misinterpreting an "I've been reset" signature to mean PassedWithoutDemur or PassedExceptPositiveResidue, simply because BSY = DRQ = ERR = 0.

D != H for the copy of the CommandOut

Scsi hosts commonly copy Cdb's only Out, in byte lengths of x 0A 06 0C 10 (decimal 10 6 12 16). To decide the length of the Cdb, Scsi devices commonly lookup the opcode (the first byte) in a table, so if a host gets the length right once, it gets it right always. Devices differ over how meaningful the last byte of the Cdb is - placing x00 there is safest.

Bad examples include:

Rumour (slander?) says Linux [tbd which obsolete version] arrogantly guessed the CdbLength was ... for ....

Rumour (slander?) says some Atapi hosts write x0C (decimal 12) "word"s i.e. x18 (decimal 24) bytes of CommandOut rather than the standard x0C (decimal 12) bytes. Supposing that writing extra bytes doesn't actually cause a page fault, then an Atapi device can tolerate this abuse, if the device runs slow enough to keep BSY set while the host was writing the extra bytes and the device takes care to ignore writes to the x1F0 Data i/o port while BSY is set.

Rumour (slander?) says some Atapi hosts hard-code the CdbLength x0C (decimal 12), without checking to see if the Ata op xA1 IdentifyPacketDevice "word" 0 data says the CdbLength is x10 (decimal 16).

D != H for the copy of the StatusIn

Status per se is fixed length. Interpretation can be an art [ tbd elaborate].

Hosts machines vary in how configurable auto sense length is.

x03 Request Sense for x00 makes no sense (clear all, read none) but happens.

Scsi hosts commonly copy in a fixed number of Status bits, sometimes including "auto sense data" in byte lengths of x 12 0E 10 (18 14 16).

I != O for the DataIn or DataOut

Some Scsi API's guess the value of IsIn by looking up lookup the opcode (the first byte) in a table.

Bad examples include:

Rumour (slander?) says some Atapi hosts ignore C/D I/O rather than requiring them to be consisteny with BSY DRQ. [ tbd elaborate ]

Rumour (slander?) says Linux [tbd which obsolete version] arrogantly guessed IsIn was ... for ....

H < D for the DataIn or DataOut

Bad examples include:

Ansi Ata and Atapi protocols force the host to fabricate one extra byte if Ho < Do and Ho is odd.

Ansi Ata and Atapi protocols force the host to discard up to X * 2 + 1 extra bytes In when Hi < Di. X is 0 for Pio and SwDma. X is 1 for MwDma as specified by Ansi, but often 0 for MwDma in practice. X is 2 for UDma33. X rises with burst rate.

Platform-specific constraints

Data address space

A Scsi host should just slow down, not choke, when the client rudely places a buffer at an address locally unsuitable for I/O.

Good examples include:

Rumour (slander?) says code written for Microsoft Windows copies Data to memory down below the legacy 1MiB barrier of 20 bit addressing, before attempting I/O.

Bad examples include:

Rumour (slander?) says some Dos Aspi servers copy the wrong bytes if the segment:offset form of the Data address is not normalised and the DataLength exceeds (64KiB - segment).

Data address alignment

A Scsi host should just slow down, not choke, when the client rudely misaligns a buffer address.

Good examples include:

The sample code from Apple for a classic MacOS 1394 host double-buffers if the Data address is not aligned to a multiple of x10 (decimal 16) bytes.

DataLength alignment (alignment of H)

Some Scsi hosts choke over particular buffer lengths.

Some hosts choke over a particular kind of length only when the Data address is not aligned. For example, a system that accesses one extra byte when H is odd won't thereby page fault unless the extra byte is placed just beyond a physical page boundary (often a 4KiB boundary).

Scsi hosts commonly most rapidly copy "blocks" of Data bytes. Common block lengths include x 200 400 800 930 (decimal 512 1024 2048 2352). During plug 'n play, Scsi hosts copy In Data bytes in lengths of x 08 24 (decimal 8 36).

A device should choose, as Ansi secretly permits, to implement Scsi in a way that avoids making available lengths which are not quad-aligned.

Bad examples include:

Microsoft Win2K Ioctl sptd [ or was it spt? ] chokes over the length 1. [tbd link to Microsoft KB, spell out spt or sptd like Microsoft does]. Rumour (slander?) says Scsi device hardware has been seen to choke over the length 1 as well.

Microsoft Windows 95B Atapi chokes if H is odd. (Microsoft Win98 Atapi works fine.)

Rumour (slander?) says some Atapi Pio hosts that (needlessly) round up H if odd, thus possibly accessing at least one unallocated byte.

Ansi Ata explicitly limits its support to only copies of multiples of x200 (decimal 512) bytes.

DataLength maximum (max H)

Some Scsi hosts choke when the DataLength is large. Any length above about 4KiB is suspect.

Some hosts choke over a large length only if the placement of Data in virtual memory happens to be discontiguous. A euphemism for this is "no support yet for scatter/gather lists".

Examples include:

Some forms of Linux Scsi are limited to one physical page of Data per Cdb [tbd link].

Rumour (slander?) says some Dos Aspi servers choke if the DataLength exceeds 64KiB.

Code that counts bytes with a signed int will choke at 32KiB and up if locally <limits.h> INT_MAX is 32767, as it is in 16-bit Dos.

 

[ Tbd: fold in test cases, discuss bus by bus. ]

But the interoperability principle of rfc222 [DATE THAT and link it] has led


> "Be liberal in what you accept, > and conservative in what you send" Oldest source I know for this is the January 1980 http://www.ietf.org/rfc/rfc761.txt

 

 

 

 

 

 

 

Hosts and devices commonly agree to copy the bytes of a Cdb Out from the host, not In.

Copying too few bytes commonly substitutes garbage for the parameters not copied, with clearly bad effects.

Devices commonly decide the length of a Cdb by looking up the first byte of a Cdb in a table.

 

People debate what Status means, but agree it should be copied in.

 

 

it is the answers that Microsoft makes vague that matter the most.

Microsoft hasn't made vague the question of how many command bytes to copy. There is debate over how meaningful the last byte is (zero there is safest), but everyone agrees we should copy all the command bytes out, and copying significantly less than all the command bytes makes the wrong thing happen, so people notice.

 

 

(((We have never ourselves seen the obscure variation of Scsi that copies some data Out and some more data In. All the same, we imagine we'd cope with it by requiring the host to make two calls, one to copy Out, one to copy In, but we'd zero the cdbLength of the first or the second call.)))

 


A.Examples

( PageUp Contents PageDown )

At 3.1 All of The Pipe we explain what we mean by the surprise of seeing something other than A = H = B = D.

At 3.2 The Visible End of The Pipe we explain how we measure P.

We offer short, concrete, trivially reproduceable examples of measuring P at:

plscsi examples

See also Appendix C, On Reproducing Our Results.

 


B.Bibliography

( PageUp Contents PageDown )

B.1 Scsi Web Links

B.1.1 The PLScsi Folk

( PageUp Contents PageDown )

As of April, 2002, the Google search:

http://www.google.com/search?q=plscsi

finds us at:

http://members.aol.com/plscsi/

by way of:

http://www.bswd.com/cornucop.htm

B.1.2 Scsi Specifications

( PageUp Contents PageDown )

We maintain a collection of links to actual specifications or else final public drafts at:

http://members.aol.com/plscsi/#whatever

B.2 Scsi Web Trails

( PageUp Contents PageDown )

We show how we search the web for specifications at:

http://members.aol.com/plscsi/trails.html

 


C.On Reproducing Our Results

( PageUp Contents PageDown )

At:

http://members.aol.com/plscsi/

we offer binaries by platform, complete source code in C, and we point you to free C compilers that work.

The Windows source code is C++, but we use just standard C where possible.

If you recompile our source code without change, you can trivially reproduce our binaries precisely. With our original or your reproduced binaries, you can trivially reproduce most of the results of Appendix A, Examples. Some of the stranger results require particular equipment, such as a host whose K' = 4 > K = 2 or a device that reports D incorrectly.

If you fail to find reproducing our binaries or a particular result trivial, please do send us email so we can begin to guess why ...

 


Finally:

( PageUp Contents PageDown )

2007-06-15 Thanks to the volunteer who pointed out the tupo of "a residue Out and a residue Out" typed where "a residue Out and a residue In" was meant.

2007-06-05 Thanks to the volunteer who found the Usbmassbulk_10 link here had rotted. Thanks to Google for finding the newly standard link now shown here.

2007-06-05 Yes hello, I'm baaaack, as ppaatt plforth plscsi etc. ~2004-10 I started paying for broadband Internet. ~2005 I stopped paying for Aol service. ~2006 free Aol mail service reached me. 2007-06-05 free Aol Ftp service reached me. For "free" Aol Ftp, seems like I have to install Aol, log in that way, and then feed `user anonymous $me@aol.com` into ftp -n members.aol.com.

At mailto:p.lavarre@ieee.org we await your comments and questions.

PLScsi Home