webmachine_multipart:get_all_parts adds extra \r\n to body?

Benjamin Nortier bjnortier at gmail.com
Tue Aug 10 12:48:44 EDT 2010


Hi

Unless I'm missing something, webmachine_multipart:get_all_parts/2 seems to
not correctly remove the \r\n at the end of the body of a file that's being
uploaded.

I've uploaded files from Firefox and Chrome, and I'm seeing an extra \r\n at
the end of the body, which should not be there. For example, this test that
I've created from an upload via Chrome fails because of the extra \r\n (the
file contents are "01234567890123456789012345678901234567890123456789");

chrome_test() ->
    Body =
<<"------WebKitFormBoundaryIHB9Xyi7ZCNKJusP\r\nContent-Disposition:
form-data; name=\"upload-test\"; filename=\"abcdef.txt\"\r\nContent-Type:
text/plain\r\n\r\n01234567890123456789012345678901234567890123456789\r\n------WebKitFormBoundaryIHB9Xyi7ZCNKJusP--\r\n">>,
    Boundary = "----WebKitFormBoundaryIHB9Xyi7ZCNKJusP",
    ?assertEqual(
       [{"upload-test",
         {[{<<"name">>,<<"upload-test">>},
           {<<"filename">>,<<"abcdef.txt">>}],
          [{<<"Content-Type">>,<<"text/plain">>}]},
         <<"01234567890123456789012345678901234567890123456789">>}],
       get_all_parts(Body,Boundary)).

So I've patched my webmachine to remove the \r\n from the end of the body:

 make_part(PartData) ->
-    [HeadData, Body] = re:split(PartData, "\\r\\n\\r\\n", [{parts,2}]),
+    [HeadData, BodyWithCRLF] = re:split(PartData, "\\r\\n\\r\\n",
[{parts,2}]),
+    %% Remove the trailing "\r\n" from the body
+    {ok, Re} = re:compile("^(.*)\r\n$", [dotall]),
+    {match, [_, Body]} = re:run(BodyWithCRLF, Re, [{capture, all,
binary}]),
     HeadList = [list_to_binary(X) ||


which also fixes what I believe are mistakes in the existing test cases, for
example there should be no \r\n at the end of the filename and "Submit
Query":

@@ -149,17 +151,17 @@
     ?assertEqual(
        [{"Filename",
          {[{<<"name">>,<<"Filename">>}],[]},
-         <<"testfile.txt\r\n">>},
+         <<"testfile.txt">>},
         {"Filedata",
          {[{<<"name">>,<<"Filedata">>},
            {<<"filename">>,<<"testfile.txt">>}],
           [{<<"Content-Type">>,<<"application/octet-stream">>}]},
-         <<"%%% The contents of this file are a test,\n%%% do not be
alarmed.\n\r\n">>},
+         <<"%%% The contents of this file are a test,\n%%% do not be
alarmed.\n">>},
         {"Upload",
          {[{<<"name">>,<<"Upload">>}],[]},
-         <<"Submit Query\r\n">>}],
+         <<"Submit Query">>}],
        get_all_parts(Body, Boundary)).

Like I said, unless I'm missing something. Full patch with extra tests
attached...

Thanks
Ben


-- 
Benjamin Nortier
e: bjnortier at gmail.com
c: +44 (0)778 946 1959
gtalk: bjnortier at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.therestfulway.com/pipermail/webmachine_lists.therestfulway.com/attachments/20100810/51330e52/attachment.html>
-------------- next part --------------
diff -r f103deda4daf src/webmachine_multipart.erl
--- a/src/webmachine_multipart.erl	Sat Aug 07 17:54:47 2010 -0400
+++ b/src/webmachine_multipart.erl	Tue Aug 10 17:43:50 2010 +0100
@@ -98,7 +98,10 @@
 get_more_data(Fun) -> Fun().
    
 make_part(PartData) ->
-    [HeadData, Body] = re:split(PartData, "\\r\\n\\r\\n", [{parts,2}]),
+    [HeadData, BodyWithCRLF] = re:split(PartData, "\\r\\n\\r\\n", [{parts,2}]),
+    %% Remove the trailing "\r\n" from the body
+    {ok, Re} = re:compile("^(.*)\r\n$", [dotall]),
+    {match, [_, Body]} = re:run(BodyWithCRLF, Re, [{capture, all, binary}]),
     HeadList = [list_to_binary(X) ||
                    X <- string:tokens(binary_to_list(HeadData), "\r\n")],
     {Name, Params, Headers} = make_headers(HeadList),
@@ -149,17 +152,17 @@
     ?assertEqual(
        [{"Filename",
          {[{<<"name">>,<<"Filename">>}],[]},
-         <<"testfile.txt\r\n">>},
+         <<"testfile.txt">>},
         {"Filedata",
          {[{<<"name">>,<<"Filedata">>},
            {<<"filename">>,<<"testfile.txt">>}],
           [{<<"Content-Type">>,<<"application/octet-stream">>}]},
-         <<"%%% The contents of this file are a test,\n%%% do not be alarmed.\n\r\n">>},
+         <<"%%% The contents of this file are a test,\n%%% do not be alarmed.\n">>},
         {"Upload",
          {[{<<"name">>,<<"Upload">>}],[]},
-         <<"Submit Query\r\n">>}],
+         <<"Submit Query">>}],
        get_all_parts(Body, Boundary)).
-    
+
 body2_test() ->
     Body = <<"-----------------------------89205314411538515011004844897\r\nContent-Disposition: form-data; name=\"Filedata\"; filename=\"akamai.txt\"\r\nContent-Type: text/plain\r\n\r\nCAMBRIDGE, MA - February 18, 2009 - Akamai Technologies, Inc. (NASDAQ: AKAM), the leader in powering rich media, dynamic transactions and enterprise applications online, today announced that its Service & Support organization was awarded top honors for Innovation in Customer Service at the 3rd Annual Stevie Awards for Sales & Customer Service, an international competition recognizing excellence in disciplines that are crucial to business success.\n\n\"We have always set incredibly high standards with respect to the service and support we provide our customers,\" said Sanjay Singh, vice president of Global Service & Support at Akamai. \"Our support team provides highly responsive service around the clock to our global customer base and, as a result, has become an extension of our customers' online businesses. This prestigious award is validation of Akamai's commitment to customer service and technical support.\"\n\nAkamai Service & Support professionals are dedicated to working with customers on a daily basis to fine tune, optimize, and support their Internet initiatives. Akamai's winning submission highlighted the key pillars of its service and support offering, as well as the initiatives established to meet customer requirements for proactive communication, simplification, and faster response times.\n\n\"This year's honorees demonstrate that even in challenging economic times, it's possible for organizations to continue to shine in sales and customer service, the two most important functions in business: acquiring and keeping customers,\" said Michael Gallagher, president of the Stevie Awards.\n\nThe awards are presented by the Stevie Awards, which organizes several of the world's leading business awards shows, including the prestigious American Business Awards. Nicknamed the Stevies for the Greek word \"crowned,\" winners were announced during a gala banquet on Monday, February 9 at Caesars Palace in Las Vegas. Nominated customer service and sales executives from the U.S.A. and several other countries attended. More than 500 entries from companies of all sizes and in virtually every industry were submitted to this year's competition. There are 27 categories for customer service professionals, as well as 41 categories for sales professionals.\n\nDetails about the Stevie Awards for Sales & Customer Service and the list of honorees in all categories are available at www.stevieawards.com/sales. \n\r\n-----------------------------89205314411538515011004844897--\r\n">>,
     Boundary = "---------------------------89205314411538515011004844897",
@@ -168,6 +171,29 @@
          {[{<<"name">>,<<"Filedata">>},
            {<<"filename">>,<<"akamai.txt">>}],
           [{<<"Content-Type">>,<<"text/plain">>}]},
-         <<"CAMBRIDGE, MA - February 18, 2009 - Akamai Technologies, Inc. (NASDAQ: AKAM), the leader in powering rich media, dynamic transactions and enterprise applications online, today announced that its Service & Support organization was awarded top honors for Innovation in Customer Service at the 3rd Annual Stevie Awards for Sales & Customer Service, an international competition recognizing excellence in disciplines that are crucial to business success.\n\n\"We have always set incredibly high standards with respect to the service and support we provide our customers,\" said Sanjay Singh, vice president of Global Service & Support at Akamai. \"Our support team provides highly responsive service around the clock to our global customer base and, as a result, has become an extension of our customers' online businesses. This prestigious award is validation of Akamai's commitment to customer service and technical support.\"\n\nAkamai Service & Support professionals are dedicated to working with customers on a daily basis to fine tune, optimize, and support their Internet initiatives. Akamai's winning submission highlighted the key pillars of its service and support offering, as well as the initiatives established to meet customer requirements for proactive communication, simplification, and faster response times.\n\n\"This year's honorees demonstrate that even in challenging economic times, it's possible for organizations to continue to shine in sales and customer service, the two most important functions in business: acquiring and keeping customers,\" said Michael Gallagher, president of the Stevie Awards.\n\nThe awards are presented by the Stevie Awards, which organizes several of the world's leading business awards shows, including the prestigious American Business Awards. Nicknamed the Stevies for the Greek word \"crowned,\" winners were announced during a gala banquet on Monday, February 9 at Caesars Palace in Las Vegas. Nominated customer service and sales executives from the U.S.A. and several other countries attended. More than 500 entries from companies of all sizes and in virtually every industry were submitted to this year's competition. There are 27 categories for customer service professionals, as well as 41 categories for sales professionals.\n\nDetails about the Stevie Awards for Sales & Customer Service and the list of honorees in all categories are available at www.stevieawards.com/sales. \n\r\n">>
+         <<"CAMBRIDGE, MA - February 18, 2009 - Akamai Technologies, Inc. (NASDAQ: AKAM), the leader in powering rich media, dynamic transactions and enterprise applications online, today announced that its Service & Support organization was awarded top honors for Innovation in Customer Service at the 3rd Annual Stevie Awards for Sales & Customer Service, an international competition recognizing excellence in disciplines that are crucial to business success.\n\n\"We have always set incredibly high standards with respect to the service and support we provide our customers,\" said Sanjay Singh, vice president of Global Service & Support at Akamai. \"Our support team provides highly responsive service around the clock to our global customer base and, as a result, has become an extension of our customers' online businesses. This prestigious award is validation of Akamai's commitment to customer service and technical support.\"\n\nAkamai Service & Support professionals are dedicated to working with customers on a daily basis to fine tune, optimize, and support their Internet initiatives. Akamai's winning submission highlighted the key pillars of its service and support offering, as well as the initiatives established to meet customer requirements for proactive communication, simplification, and faster response times.\n\n\"This year's honorees demonstrate that even in challenging economic times, it's possible for organizations to continue to shine in sales and customer service, the two most important functions in business: acquiring and keeping customers,\" said Michael Gallagher, president of the Stevie Awards.\n\nThe awards are presented by the Stevie Awards, which organizes several of the world's leading business awards shows, including the prestigious American Business Awards. Nicknamed the Stevies for the Greek word \"crowned,\" winners were announced during a gala banquet on Monday, February 9 at Caesars Palace in Las Vegas. Nominated customer service and sales executives from the U.S.A. and several other countries attended. More than 500 entries from companies of all sizes and in virtually every industry were submitted to this year's competition. There are 27 categories for customer service professionals, as well as 41 categories for sales professionals.\n\nDetails about the Stevie Awards for Sales & Customer Service and the list of honorees in all categories are available at www.stevieawards.com/sales. \n">>
         }],
        get_all_parts(Body,Boundary)).
+
+firefox_test() ->
+    Body = <<"-----------------------------823378840143542612896544303\r\nContent-Disposition: form-data; name=\"upload-test\"; filename=\"abcdef.txt\"\r\nContent-Type: text/plain\r\n\r\n01234567890123456789012345678901234567890123456789\r\n-----------------------------823378840143542612896544303--\r\n">>,
+    Boundary = "---------------------------823378840143542612896544303",
+    ?assertEqual(
+       [{"upload-test",
+         {[{<<"name">>,<<"upload-test">>},
+           {<<"filename">>,<<"abcdef.txt">>}],
+          [{<<"Content-Type">>,<<"text/plain">>}]},
+         <<"01234567890123456789012345678901234567890123456789">>}],
+       get_all_parts(Body,Boundary)).
+
+chrome_test() ->
+    Body = <<"------WebKitFormBoundaryIHB9Xyi7ZCNKJusP\r\nContent-Disposition: form-data; name=\"upload-test\"; filename=\"abcdef.txt\"\r\nContent-Type: text/plain\r\n\r\n01234567890123456789012345678901234567890123456789\r\n------WebKitFormBoundaryIHB9Xyi7ZCNKJusP--\r\n">>,
+    Boundary = "----WebKitFormBoundaryIHB9Xyi7ZCNKJusP",
+    ?assertEqual(
+       [{"upload-test",
+         {[{<<"name">>,<<"upload-test">>},
+           {<<"filename">>,<<"abcdef.txt">>}],
+          [{<<"Content-Type">>,<<"text/plain">>}]},
+         <<"01234567890123456789012345678901234567890123456789">>}],
+       get_all_parts(Body,Boundary)).
+       


More information about the webmachine mailing list