-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bug 5266: Do not parse the same TLS record fragments twice #240
base: master
Are you sure you want to change the base?
Changes from 10 commits
350f3a6
75f04a1
6ef59b6
f3650cf
095601c
328b704
6932606
2180c95
32c24e5
c5f70c1
9a96bc7
eba1b8e
5d4b9b5
e304647
3a88001
71802a6
dbf2481
6b12750
47eb627
4b525e4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -10,6 +10,7 @@ | |||||
|
||||||
#include "squid.h" | ||||||
#include "base/IoManip.h" | ||||||
#include "parser/forward.h" | ||||||
#include "sbuf/Stream.h" | ||||||
#include "security/Handshake.h" | ||||||
#if USE_OPENSSL | ||||||
|
@@ -274,24 +275,26 @@ Security::HandshakeParser::parseModernRecord() | |||||
Must(record.fragment.length() || record.type == ContentType::ctApplicationData); | ||||||
|
||||||
if (currentContentType != record.type) { | ||||||
parseMessages(); | ||||||
Must(tkMessages.atEnd()); // no currentContentType leftovers | ||||||
fragments = record.fragment; | ||||||
currentContentType = record.type; | ||||||
} else { | ||||||
fragments.append(record.fragment); | ||||||
} | ||||||
|
||||||
if (tkRecords.atEnd() && !done) | ||||||
rousskov marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
parseMessages(); | ||||||
const auto haveUnparsedRecordBytes = !tkRecords.atEnd(); | ||||||
const auto expectMoreRecordLayerBytes = tkRecords.expectMore(); | ||||||
// TODO: consider adding BinaryTokenizer::exhausted() instead | ||||||
const auto expectMoreMessageLayerBytes = haveUnparsedRecordBytes || expectMoreRecordLayerBytes; | ||||||
|
||||||
tkMessages.expectMore(expectMoreMessageLayerBytes); | ||||||
tkMessages.append(record.fragment); | ||||||
|
||||||
parseMessages(); | ||||||
} | ||||||
|
||||||
/// parses one or more "higher-level protocol" frames of currentContentType | ||||||
void | ||||||
Security::HandshakeParser::parseMessages() | ||||||
{ | ||||||
tkMessages.reset(fragments, false); | ||||||
for (; !tkMessages.atEnd(); tkMessages.commit()) { | ||||||
rousskov marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
for (tkMessages.rollback(); !done; tkMessages.commit()) { | ||||||
switch (currentContentType) { | ||||||
case ContentType::ctChangeCipherSpec: | ||||||
parseChangeCipherCpecMessage(); | ||||||
|
@@ -631,7 +634,10 @@ Security::HandshakeParser::parseSupportedVersionsExtension(const SBuf &extension | |||||
void | ||||||
Security::HandshakeParser::skipMessage(const char *description) | ||||||
{ | ||||||
// tkMessages/fragments can only contain messages of the same ContentType. | ||||||
// skip at least one byte | ||||||
if (tkMessages.atEnd()) | ||||||
throw Parser::InsufficientInput(); | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. A TLS record may have a zero-length fragment (e.g., RFC 8446 says that "Zero-length fragments of Application Data MAY be sent"). Thus, a valid TLS message may have zero length1, and our code must handle zero-length TLS fragments and empty messages (without getting stuck, etc.). This "at least one byte" check can be reasonably interpreted as effectively using the opposite invariant -- that every message has some bytes. PR's current parseMessages() loop also effectively assumes that successfully parsing any message allows us to make progress which is not technically true for empty messages. Moreover, if tkMessages is no longer expectingMore() and is atEnd(), then it may be wrong to throw InsufficientInput here because our input was sufficient to parse existing tkMessages -- more tkMessages bytes (with the same record type) are not expected and adding those bytes would lead to a parsing failure). Yet, parseMessages() loop should stop. To find the right solution, we can try to (re)define what parseMessages() means. Once we have that definition, and assuming we are not going to add more special exceptions, we will through InsufficientInput when tkMessages do not have enough bytes to satisfy the defined parseMessages() goal. After rejecting many candidates for various reasons, I can suggest only two reasonable parseMessages() definitions:
Official code may be seen as trying to implement the first bullet (which ignores The current PR parseMessages() quits parsing tkMessages when AFAICT, to implement the above parseMessages() definition(s), we have to do the following:
/// Incrementally parses all sequential currentContentType TLS fragments.
/// Successfully stops parsing earlier if `done` becomes set.
void
Security::HandshakeParser::parseMessages()
{
switch (currentContentType) {
case ContentType::ctChangeCipherSpec:
return parseNonEmptyMessages(parseChangeCipherSpecMessage);
case ContentType::ctAlert:
return parseNonEmptyMessages(parseAlertMessage);
case ContentType::ctHandshake:
return parseNonEmptyMessages(parseHandshakeMessage);
case ContentType::ctApplicationData:
return skipPossiblyEmptyMessages("app data [fragment]");
}
return skipPossiblyEmptyMessages("unknown ContentType msg [fragment]");
}
/// Incrementally parses all sequential currentContentType messages using the given TLS message parser.
/// Each message is assumed to be serialized using at least one byte.
/// At least one message is expected per sequence.
/// Successfully stops parsing earlier if `done` becomes set.
void
Security::HandshakeParser::parseNonEmptyMessages(... messageParser)
{
// XXX: We should not silently ignore empty message sequences
for (tkMessages.rollback(); !done && !tkMessages.exhausted(); tkMessages.commit())
messageParser();
} The existing rollback/commit loop will go inside the new parseNonEmptyMessages() method. AFAICT, to match the new definition, that loop should run until tkMessages are exhausted (i.e. atEnd() and not expectingMore()) unless In the above sketch, skipPossiblyEmptyMessages() is a skipMessage() replacement that does not need a loop. It should throw InsufficientInput unless tkMessages are exhausted. I am not sure whether it needs rollback() and/or commit(). The above XXX in sketched parseNonEmptyMessages() will need to be addressed as well, which will probably require more refactoring, but it may be easier to do later, as a separate commit/step. Footnotes
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I tried to implement this approach (up to 47eb627).
I added this XXX for now but I am not sure how 'empty message sequences' are possible here. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Do you mean that we are "protected" by a "MUST NOT" text in an RFC? Surely an attacker can violate that MUST -- we cannot rely on that kind of protection or guarantees in a parser... AFAICT, the sketched code will just ignore/skip such invalid messages (i.e. there are no infinite loops or similar dangers here). This is not the end of the world, but to the parseNonEmptyMessages() caller it would look like a non-empty message was successfully parsed and that feels wrong, especially when some future refactoring might unknowingly create an infinite loop when handling such an invalid input (because all the method name suggests that this code should not worry about empty messages).
Yes, but that messageParser() call inside the loop can make There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
No, I meant the protection in HandshakeParser::parseModernRecord():
With this protection, only ctApplicationData can be empty, which are handled by skipPossiblyEmptyMessages(). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Ah, I see. I agree that the above Must() protects parseNonEmptyMessages() from empty input. However, it is difficult to assert that protection using the current code structure. I think we should use that protection to avoid the XXX without introducing assertions. We are expecting at least one byte, so we should parse input before checking whether it is exhausted: tkMessages.rollback();
do {
parseMessage();
tkMessages.commit();
} while (!done && !tkMessages.exhausted()); There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok, adjusted at 4b525e4. |
||||||
// tkMessages can only contain messages of the same ContentType. | ||||||
// To skip a message, we can and should skip everything we have [left]. If | ||||||
// we have partial messages, debugging will mislead about their boundaries. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think that "debugging will mislead" comment is stale because all skipMessage() callers add
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done. |
||||||
tkMessages.skip(tkMessages.leftovers().length(), description); | ||||||
|
@@ -640,24 +646,26 @@ Security::HandshakeParser::skipMessage(const char *description) | |||||
bool | ||||||
Security::HandshakeParser::parseHello(const SBuf &data) | ||||||
{ | ||||||
try { | ||||||
{ | ||||||
if (!expectingModernRecords.configured()) | ||||||
expectingModernRecords.configure(!isSslv2Record(data)); | ||||||
|
||||||
// data contains everything read so far, but we may read more later | ||||||
tkRecords.reinput(data, true); | ||||||
tkRecords.rollback(); | ||||||
while (!done) | ||||||
while (!done) { | ||||||
try{ | ||||||
if (tkRecords.atEnd()) | ||||||
return false; | ||||||
|
||||||
parseRecord(); | ||||||
debugs(83, 7, "success; got: " << done); | ||||||
// we are done; tkRecords may have leftovers we are not interested in | ||||||
return true; | ||||||
} | ||||||
catch (const Parser::BinaryTokenizer::InsufficientInput &) { | ||||||
debugs(83, 5, "need more data"); | ||||||
return false; | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This "return false" for InsufficientInput case was correct, and the caller relies on it AFAICT: We have to keep it or move InsufficientInput processing to the caller. The added atEnd() check above is another red flag. Overall, I am not sure that this method refactoring improved things, but perhaps I am missing some essential changes in this diff noise. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Unfortunately, these changes failed one of my per-byte test - the tkRecords.atEnd() check does not work when tkRecords have some unparsed bytes insufficient for the next record. We need to know somehow who threw InsufficientInput() in the catch() below - and, depending on that, either return false (insufficient tkRecords) or continue to the next record (insufficient tkMessages). I tried to add some synced() method (whether all parsed bytes were comitted) into BinaryTokenizer and calculate this state as (!synced() || atEnd()) for tkMessages. But it looks a bit complicated/unreliable - we should not calculate the state again with the exception at hand. Unfortunately, InsufficientInput() does not have any attribute about its initiator. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
To treat InsufficientInput emitted by message layer differently, catch InsufficientInput emitted by the that layer at that layer (i.e. when we parse messages). InsufficientInput caught while making parseRecord() calls will be specific to the records layer. AFAICT, InsufficientInput emitted while parsing messages should be "caught, reported, and ignored" -- the control will then bubble up to the next parseRecord() loop iteration. Alternatively, we could parse all records we have read before parsing the messages in the accumulated fragments. If message layer always sees all available same-type records, then InsufficientInput thrown by any layer will have the same meaning -- need to read more bytes (record bytes and message bytes). However, it is not clear to me whether such refactoring will make the code better or worse overall. Please try to fix the current (i.e. "parse one record fragment at a time") code before we attempt such a significant refactoring. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Ok. I hesitated doing this because recently we rejected tkMessages try/catch for a reason. |
||||||
} | ||||||
catch (const Parser::BinaryTokenizer::InsufficientInput &) { | ||||||
debugs(83, 5, "need more data"); | ||||||
} | ||||||
} | ||||||
return false; // unreached | ||||||
// we are done; tkRecords may have leftovers we are not interested in | ||||||
return true; | ||||||
} | ||||||
|
||||||
/// A helper function to create a set of all supported TLS extensions | ||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.