multipart icon indicating copy to clipboard operation
multipart copied to clipboard

StringError("text was not valid unicode")

Open oshinowo-stephen opened this issue 6 years ago • 12 comments

So I was trying to use the multipart Iron example from the source... But the thing is when I upload a file specifically an image file it responds back with an error and doesn't respond back with the data fields... the error is stating this

Field "myfile" has 1 entries:
0: Some("1!.jpg") (Some(Mime(Image, Jpeg, []))):
ERROR 2018-08-12T20:32:00Z: iron::iron: Error handling:
Request {
    url: Url { generic_url: "http://localhost:3000/req/send" }
    method: Post
    remote_addr: V4(127.0.0.1:52494)
    local_addr: V4(127.0.0.1:3000)
}
Error was: Custom { kind: InvalidData, error: StringError("text was not valid unicode") }

I don't fully understand what this error means and why i'm getting it... but I feel like i'm doing something wrong but i'm not sure what though.

oshinowo-stephen avatar Aug 12 '18 20:08 oshinowo-stephen

What are you using to submit the request? Can you capture the raw request body? It's trying to interpret some binary field in the request as text but the image file seems to be uploaded successfully, based on the output you gave (its filename and media-type are printed).

abonander avatar Aug 13 '18 00:08 abonander

Yes that's why I'm confused why it's not printing the data on screen... also i'm still confused why it's printing with that error. Also i'm using a HTML form to submit the request, that looks something like this.

<form action="/req/send" enctype="multipart/form-data" method="post">
  <input type="file" name="myfile" id="file">
  <input type="submit" value="Submit">
</form>

oshinowo-stephen avatar Aug 13 '18 08:08 oshinowo-stephen

Looking at the code, the error is occurring when trying to write the file contents to the response, but I haven't found which component is actually trying to parse the file contents as a string.

abonander avatar Aug 17 '18 18:08 abonander

@GitStonic nevermind capturing the request body, I just need to find where it's trying to parse the file as a string. It's somewhere in the call tree of process_entries(). I want to say it might be Hyper since the default content type for Iron responses is text/plain and Hyper might be trying to verify that.

abonander avatar Aug 17 '18 18:08 abonander

What version of Iron are you using, anyway?

abonander avatar Aug 17 '18 18:08 abonander

I believe i'm using the latest version of Iron 0.6.0 to be exact.

oshinowo-stephen avatar Aug 18 '18 10:08 oshinowo-stephen

@GitStonic just to test an assumption, does the error still occur if you set the content type of the response to something like application/octet-stream?

abonander avatar Aug 21 '18 10:08 abonander

yes the error still occurs after setting the content-type to application/octet-stream

oshinowo-stephen avatar Aug 23 '18 16:08 oshinowo-stephen

@GitStonic I started to get frustrated because I could not find that error string anywhere within multipart, Iron or Hyper. That's because it originates in the Windows stdio handling code in the Rust stdlib. You see, Windows doesn't like non-Unicode sequences being printed to the console, even though std exposes it as a byte-oriented interface. Unfortunately, that's exactly what StdoutTee is doing here.

I see a few possible solutions:

  • Don't print non-text file fields to stdout (text files still not guaranteed to be valid Unicode)
  • Don't print any files
  • For file fields, print some text representation of their contents instead (hash, hexadecimal, base-64?)

abonander avatar Aug 28 '18 03:08 abonander

I see... I will try those options, Thank you for your time and effort for me. Really appreciate it.

oshinowo-stephen avatar Aug 28 '18 14:08 oshinowo-stephen

I need to fix these examples as well so I'm looking for opinions on the best approach. I wanted it to print the fields to stdout somehow but I dunno if I want to pull in another crate just for hashing/base64 conversion.

abonander avatar Aug 28 '18 23:08 abonander

I've submitted a PR to rust-lang/rust to make the error message more specific and to document the portability hazard. I still have to fixup the examples, I'll get to that sometime.

abonander avatar Feb 04 '19 06:02 abonander