std/net/httpbytes
std/net/httpbytes — the byte primitives every HTTP reader is built on,
and the bounds every one of them is held to.
Entry: a raw buffer, or a field value a peer wrote.
Exit: an index, a slice, or a number that has been checked before it is
believed.
Two rules run through this whole group of modules, and they start here.
Everything is measured in bytes, the head included. A field value
may carry obs-text (RFC 9110), so the head is not reliably ASCII, andstring's substring / indexOf count characters. Every index in
these modules therefore comes from byteAt or a raw-buffer scan, and a
slice is taken with [byteSlice] / [sliceToString]. Mixing the two
units shifts every field after the first non-ASCII byte. A body is
arbitrary octets and never becomes a string on this path at all —
that conversion is the caller's choice, at the point where it knows the
media type.
Everything is bounded, by a named constant. A peer we did not write
is on the other end of the socket, and every count it sends is a request
to allocate. A length field is a claim, not a fact: it is checked before
it is believed. The bounds are deliberately generous enough that no
honest server trips them and small enough that a hostile one cannot
exhaust memory.
Free functions
| Type | Method and description |
|---|---|
| maxStartLine() : i64 Longest start line accepted (request line or status line). |
| maxHeadBytes() : i64 Largest complete header block, start line included. |
| maxHeaderCount() : i64 Most header fields accepted in one message. |
| maxBodyBytes() : i64 Largest body accepted, whether framed by length, chunks or EOF. |
| maxChunkBytes() : i64 Largest single chunk accepted in a chunked body. |
| readChunk() : i64 Read granularity. One socket read never asks for more than this. |
| grow(own buf : i8[], used : i64, want : i64, cap : i64) : i8[] Grow buf past used bytes, copying them into the result.Doubling rather than exact-fit: a head or a body arrives in an unknown number of reads, and growing by the read size would make assembling one a quadratic copy. Capacity is read back off the answer with .length —a caller that tracked its own figure would be free to disagree with the allocation, and the direction that disagreement runs is a write past the end. |
| capacityOf(buf : i8[]) : i64 Capacity of a buffer, as the i64 every count in these modules is. |
| findHeadEnd(buf : i8[], used : i64, from : i64) : i64 Index of the first \r\n\r\n at or after from in buf[0..used),or -1. Scans the raw bytes rather than a string, because the bodythat follows the head may hold any octet and must not have been through a UTF-8 decode to be found. |
| findCrLf(buf : i8[], used : i64, from : i64) : i64 Index of the first \r\n at or after from in buf[0..used), or -1. |
| sliceToString(buf : i8[], from : i64, to : i64) : string The bytes [from, to) of buf, decoded as text.bytesToString only ever reads from index 0, so a slice that does notstart there has to be copied first. Bounds are clamped rather than raised on: every caller here derives them from a scan of this same buffer, so an out-of-range pair is a bug in these modules, and an empty answer is what makes it visible as a missing field. |
| byteSlice(s : string, from : i64, to : i64) : string The bytes [from, to) of s, as a string.Bytes, and that is the whole point. substring counts characterswhile every index in these modules comes from byteAt, so mixing thetwo shifts every field after the first non-ASCII byte — which RFC 9110 permits in a field value (obs-text) and which deployed servers send in Server, Set-Cookie and Content-Disposition. The shift reads as awrong Content-Length, or as an IndexOutOfBoundsException raised outof a function that declares only ! IOException.Clamped for the same reason as [ sliceToString]: the caller scannedthis very string to obtain the pair. |
| noSpill() : i8[] An empty spill — the answer of a reader that read nothing past its message. Sized to one byte because new i8[0] has no address tohand a copy loop that will not run. |
| spillFrom(buf : i8[], from : i64, len : i64) : i8[] Copy [from, from + len) out of buf into a buffer of its own.The one shape a spill is ever built in: every reader hands back what it read past its message, and every one of those hand-offs is this copy. A copy and not the array itself, because a spill is owned by the result that carries it — aliasing the source would give one allocation two owners and the second teardown would free what the first released. |
| parseDecimal(s : string) : i64 Interpret s as a non-negative decimal integer, or -1 when it is notone. Used for Content-Length, where a value that is not a plaindecimal is a framing error rather than a number to guess at. |
| parseChunkSize(s : string) : i64 Interpret s as a chunk-size line: hexadecimal digits, optionallyfollowed by ; and chunk extensions. Returns -1 when it is not one. |
| hasControlByte(s : string) : bool Whether s carries a byte no field value or request target may:CR, LF, or NUL. The check that keeps a caller's data out of the framing. A header value or a URL path holding \r\n does not become a malformed request— it becomes a well-formed one the caller did not write, with whatever fields, and whatever second request, the byte sequence spells. That is request splitting, and the only place to stop it is before the bytes reach the wire. |
| hasRequestLineDelimiter(s : string) : bool Whether s carries a byte the request line cannot hold: a control byte,or the SP / HTAB that delimit its three fields. Distinct from [ hasControlByte] because a header value may hold aspace and a request target may not: GET /q?a b HTTP/1.1 is read by onehop as a target /q?a and a version b HTTP/1.1, and by a lenient hopbehind it as something else — two readers of one line disagreeing about what was asked for, which is the desync the CR/LF check exists to close. |
| hasSpaceOrTab(s : string) : bool Whether s carries a space or a horizontal tab. |
Method detail
#maxStartLine
Longest start line accepted (request line or status line).
#maxHeadBytes
Largest complete header block, start line included.
#maxHeaderCount
Most header fields accepted in one message.
#maxBodyBytes
Largest body accepted, whether framed by length, chunks or EOF.
#maxChunkBytes
Largest single chunk accepted in a chunked body.
#readChunk
Read granularity. One socket read never asks for more than this.
#grow
Grow buf past used bytes, copying them into the result.
Doubling rather than exact-fit: a head or a body arrives in an unknown
number of reads, and growing by the read size would make assembling one
a quadratic copy. Capacity is read back off the answer with .length —
a caller that tracked its own figure would be free to disagree with the
allocation, and the direction that disagreement runs is a write past the
end.
buf current bufferused bytes of buf that carry datawant minimum capacity requiredcap hard ceiling; the result never exceeds itmin(want, cap) bytes, holdingbuf[0..used)#capacityOf
Capacity of a buffer, as the i64 every count in these modules is.
#findHeadEnd
Index of the first \r\n\r\n at or after from in buf[0..used),
or -1. Scans the raw bytes rather than a string, because the body
that follows the head may hold any octet and must not have been
through a UTF-8 decode to be found.
buf buffer to scanused valid byte count in buffrom first index to consider#findCrLf
Index of the first \r\n at or after from in buf[0..used), or -1.
buf buffer to scanused valid byte count in buffrom first index to consider#sliceToString
The bytes [from, to) of buf, decoded as text.bytesToString only ever reads from index 0, so a slice that does not
start there has to be copied first. Bounds are clamped rather than
raised on: every caller here derives them from a scan of this same
buffer, so an out-of-range pair is a bug in these modules, and an empty
answer is what makes it visible as a missing field.
#byteSlice
The bytes [from, to) of s, as a string.
Bytes, and that is the whole point. substring counts characters
while every index in these modules comes from byteAt, so mixing the
two shifts every field after the first non-ASCII byte — which RFC 9110
permits in a field value (obs-text) and which deployed servers send inServer, Set-Cookie and Content-Disposition. The shift reads as a
wrong Content-Length, or as an IndexOutOfBoundsException raised out
of a function that declares only ! IOException.
Clamped for the same reason as [sliceToString]: the caller scanned
this very string to obtain the pair.
#noSpill
An empty spill — the answer of a reader that read nothing past its
message. Sized to one byte because new i8[0] has no address to
hand a copy loop that will not run.
#spillFrom
Copy [from, from + len) out of buf into a buffer of its own.
The one shape a spill is ever built in: every reader hands back what it
read past its message, and every one of those hand-offs is this copy.
A copy and not the array itself, because a spill is owned by the
result that carries it — aliasing the source would give one allocation
two owners and the second teardown would free what the first released.
buf source bufferfrom first byte to takelen how many; a non-positive count answers [noSpill]#parseDecimal
Interpret s as a non-negative decimal integer, or -1 when it is not
one. Used for Content-Length, where a value that is not a plain
decimal is a framing error rather than a number to guess at.
s candidate text#parseChunkSize
Interpret s as a chunk-size line: hexadecimal digits, optionally
followed by ; and chunk extensions. Returns -1 when it is not one.
s the chunk-size line, without its trailing CRLF#hasControlByte
Whether s carries a byte no field value or request target may:
CR, LF, or NUL.
The check that keeps a caller's data out of the framing. A header
value or a URL path holding \r\n does not become a malformed request
— it becomes a well-formed one the caller did not write, with whatever
fields, and whatever second request, the byte sequence spells. That is
request splitting, and the only place to stop it is before the bytes
reach the wire.
s candidate text#hasRequestLineDelimiter
Whether s carries a byte the request line cannot hold: a control byte,
or the SP / HTAB that delimit its three fields.
Distinct from [hasControlByte] because a header value may hold a
space and a request target may not: GET /q?a b HTTP/1.1 is read by one
hop as a target /q?a and a version b HTTP/1.1, and by a lenient hop
behind it as something else — two readers of one line disagreeing about
what was asked for, which is the desync the CR/LF check exists to close.
s a request target or an authority#hasSpaceOrTab
Whether s carries a space or a horizontal tab.