Repository navigation
Closed
Description
For decoding mime content, in TS we use this method:
public static decode = async (mimeMsg: Uint8Array): Promise<MimeContent> => {
let mimeContent: MimeContent = { attachments: [], headers: {}, subject: undefined, text: undefined, html: undefined, signature: undefined, from: undefined, to: [], cc: [], bcc: [] };
const parser = new MimeParser();
const leafNodes: { [key: string]: MimeParserNode } = {};
parser.onbody = (node: MimeParserNode) => {
const path = String(node.path.join('.'));
if (typeof leafNodes[path] === 'undefined') {
leafNodes[path] = node;
}
};
return await new Promise((resolve, reject) => {
try {
parser.onend = () => {
try {
for (const name of Object.keys(parser.node.headers)) {
mimeContent.headers[name] = parser.node.headers[name][0].value;
}
mimeContent.rawSignedContent = Mime.retrieveRawSignedContent([parser.node]);
for (const node of Object.values(leafNodes)) {
if (Mime.getNodeType(node) === 'application/pgp-signature') {
mimeContent.signature = node.rawContent;
} else if (Mime.getNodeType(node) === 'text/html' && !Mime.getNodeFilename(node)) {
// html content may be broken up into smaller pieces by attachments in between
// AppleMail does this with inline attachments
mimeContent.html = (mimeContent.html || '') + Mime.getNodeContentAsUtfStr(node);
} else if (Mime.getNodeType(node) === 'text/plain' && (!Mime.getNodeFilename(node) || Mime.isNodeInline(node))) {
mimeContent.text = (mimeContent.text ? `${mimeContent.text}\n\n` : '') + Mime.getNodeContentAsUtfStr(node);
} else if (Mime.getNodeType(node) === 'text/rfc822-headers') {
if (node._parentNode && node._parentNode.headers.subject) {
mimeContent.subject = node._parentNode.headers.subject[0].value;
}
} else {
mimeContent.attachments.push(Mime.getNodeAsAttachment(node));
}
}
const headers = Mime.headerGetAddress(mimeContent, ['from', 'to', 'cc', 'bcc']);
mimeContent.subject = String(mimeContent.subject || mimeContent.headers.subject || '');
mimeContent = Object.assign(mimeContent, headers);
resolve(mimeContent);
} catch (e) {
reject(e);
}
};
parser.write(mimeMsg);
parser.end();
} catch (e) { // todo - on Android we may want to fail when this happens, evaluate effect on browser extension
Catch.reportErr(e);
resolve(mimeContent);
}
});
}But @DenBond7 probably already has some method for decoding mime. Therefore I don't want to reimplement the above, but instead use Den's methods (and its outputs) for the above purpose (if possible)
Then you take outputs from the MIME decode, and you feed them into a new method that needs implementing, similar to the one below:
public static processDecoded = (decoded: MimeContent): MimeProccesedMsg => {
const blocks: MsgBlock[] = [];
if (decoded.text) {
const blocksFromTextPart = MsgBlockParser.detectBlocks(Str.normalize(decoded.text)).blocks;
// if there are some encryption-related blocks found in the text section, which we can use, and not look at the html section
if (blocksFromTextPart.find(b => b.type === 'encryptedMsg' || b.type === 'signedMsg' || b.type === 'publicKey' || b.type === 'privateKey')) {
blocks.push(...blocksFromTextPart); // because the html most likely containt the same thing, just harder to parse pgp sections cause it's html
} else if (decoded.html) { // if no pgp blocks found in text part and there is html part, prefer html
blocks.push(MsgBlock.fromContent('plainHtml', decoded.html));
} else { // else if no html and just a plain text message, use that
blocks.push(...blocksFromTextPart);
}
} else if (decoded.html) {
blocks.push(MsgBlock.fromContent('plainHtml', decoded.html));
}
for (const file of decoded.attachments) {
const treatAs = file.treatAs();
if (treatAs === 'encryptedMsg') {
const armored = PgpArmor.clip(file.getData().toUtfStr());
if (armored) {
blocks.push(MsgBlock.fromContent('encryptedMsg', armored));
}
} else if (treatAs === 'signature') {
decoded.signature = decoded.signature || file.getData().toUtfStr();
} else if (treatAs === 'publicKey') {
blocks.push(...MsgBlockParser.detectBlocks(file.getData().toUtfStr()).blocks);
} else if (treatAs === 'privateKey') {
blocks.push(...MsgBlockParser.detectBlocks(file.getData().toUtfStr()).blocks);
} else if (treatAs === 'encryptedFile') {
blocks.push(MsgBlock.fromAttachment('encryptedAttachment', '', { name: file.name, type: file.type, length: file.getData().length, data: file.getData() }));
} else if (treatAs === 'plainFile') {
blocks.push(MsgBlock.fromAttachment('plainAttachment', '', {
name: file.name, type: file.type, length: file.getData().length, data: file.getData(), inline: file.inline, cid: file.cid
}));
}
}
if (decoded.signature) {
for (const block of blocks) {
if (block.type === 'plainText') {
block.type = 'signedText';
block.signature = decoded.signature;
} else if (block.type === 'plainHtml') {
block.type = 'signedHtml';
block.signature = decoded.signature;
}
}
if (!blocks.find(block => ['plainText', 'plainHtml', 'signedMsg', 'signedHtml', 'signedText'].includes(block.type))) { // signed an empty message
blocks.push(new MsgBlock("signedMsg", "", true, decoded.signature));
}
}
return { headers: decoded.headers, blocks, from: decoded.from, to: decoded.to, rawSignedContent: decoded.rawSignedContent };
}
Activity
tomholub commentedon May 10, 2021
This is an interesting method that needs re-implementing, which will help the app decide how to treat each attachment. Here
thisis some sort of attachment object that has name, size, mime type and similar meta info on it.DenBond7 commentedon May 10, 2021
@IvanPizhenko Have you worked with
JavaMailbefore?DenBond7 commentedon May 10, 2021
@tomholub I've looked at TS code. It seems that your logic is clear and can be transferred to Kotlin. There is a lot of work with
JavaMail. How I see thatflowcrypt-android/FlowCrypt/src/main/java/com/flowcrypt/email/jetpack/viewmodel/MsgDetailsViewModel.kt
Line 389 in 1cb3583
here you will find how I iterate over the parts. Please feel free to look at other examples over the app.
IvanPizhenko commentedon May 11, 2021
a bit, I've implemented some sending emails in the FES.
IvanPizhenko commentedon May 11, 2021
@tomholub So which one I should continue with now - this one or #1057 ?
tomholub commentedon May 11, 2021
This one please. It's a prerequisite for the other one.
IvanPizhenko commentedon May 11, 2021
ok
IvanPizhenko commentedon May 23, 2021
@tomholub where can I get some Typescript unit tests for this?
tomholub commentedon May 23, 2021
I'll extract some test cases for you
IvanPizhenko commentedon May 23, 2021
And question in advance to @DenBond7 - creating
MimeMessageobject requiresSessionobject. How can I obtain it for unit tests?11 remaining items
DenBond7 commentedon May 24, 2021
@IvanPizhenko I think
MimeContentis redundant.JavaMailcovers it withMimeMessage. You don't need to parse MIME message manually, just useJavaMailandMimeMessageas a parsed object.tomholub commentedon May 24, 2021
@DenBond7 how about signedText and signedHtml? This may also be returned by current node - what would happen to them?
tomholub commentedon May 24, 2021
Or maybe you don't even send them to node, since they are not encrypted?
DenBond7 commentedon May 24, 2021
@tomholub Could you clarify a question? Too many 'node' words, I'm not sure that I understand right according to the context.
tomholub commentedon May 24, 2021
We also have MsgBlock types
signedHtmlandsignedTextthat may be returned by NodeJS (I saw it in some of the test cases above). There's alsopublicKeyfor example. How are these handled on Android?DenBond7 commentedon May 24, 2021
Thank you. Got it.
Please look at the follwoing code:
flowcrypt-android/FlowCrypt/src/main/java/com/flowcrypt/email/ui/activity/fragment/MessageDetailsFragment.kt
Line 719 in 015b392
This method contains a logic where we handle
MsgBlocktomholub commentedon May 24, 2021
I see - you treat signedText and signedHtml as "other" and render them as text (which would be a bug in the case of signedHtml).
If that is the case, Ivan's still has to be able to process most of the test cases that I submitted above, else these blocks would not show up, even to be rendered as "other".
@IvanPizhenko see what level of compatibility can you achieve easily. Then we can discuss if we need to support the remaining tests or not yet.
IvanPizhenko commentedon May 30, 2021
@tomholub @DenBond7 Do we need the
rawSignedContent? In the Typescript (in thedecode()listed above) it is extracted as follows:I did not port this, because I am not sure if this is needed. Otherwise I almost have it ready and can post a PR soon.
issue #1223 a port of Mime.processDecoded()
issue #1223 a port of Mime.processDecoded()
DenBond7 commentedon May 31, 2021
As we don't support handling the signed content yet I think it's redundant for now. it can be done as improvements
IvanPizhenko commentedon May 31, 2021
ok
issue #1223 a port of Mime.processDecoded()
issue #1223 a port of Mime.processDecoded() (#1267)