Amazon's SQS (Simple Queue Service) uses XML formatted payloads to push and pop messages to and from an SQS queue. In other words, the underlying request and response bodies are XML payloads, containing among other things, a message ID and a message receipt handle. This means that the message body itself (the thing you're pushing onto the queue) ultimately has to be properly XML escaped to work right. Well it turns out, Amazon's own AWS library is very flaky here, and does not do the right thing when unmarshalling an XML SQS response back into an Object. Essentially, it gets confused when it encounters an XML escaped ampersand in the message body. For example, this popped message (in pseudo XML) fails miserably to unmarshall:
Note that the ampersand in the message body is properly XML escaped, however, Amazon's own AWS library returns this as the message body once unmarshalled:
In other words, the message body ends abruptly at the ampersand, and needless to say, any reasonable JSON library will fail to parse this malformed block of nonsense.
Solution: if you are going to use SQS, and there's a possibility the messages you're going to push onto a queue will have issues with XML escaping, it appears you should always base-64 encode the message body, then base-64 decode it when popping. Fortunately, the Apache Commons Codec library has a wonderful Base64 class to handle this encoding and decoding mess for you.
Bottom line, if you're encountering encoding issues with Amazon's SQS, better check if your messages are able to make it through the XML marshalling and unmarshalling process. If not, you'll need to base-64 encode and decode your messages to dance around bugs in Amazon's AWS library.
Good luck, and hope this helps.
<sqs><message>{"music":"rock & roll"}</message></sqs>
Note that the ampersand in the message body is properly XML escaped, however, Amazon's own AWS library returns this as the message body once unmarshalled:
{"music":"rock &In other words, the message body ends abruptly at the ampersand, and needless to say, any reasonable JSON library will fail to parse this malformed block of nonsense.
Solution: if you are going to use SQS, and there's a possibility the messages you're going to push onto a queue will have issues with XML escaping, it appears you should always base-64 encode the message body, then base-64 decode it when popping. Fortunately, the Apache Commons Codec library has a wonderful Base64 class to handle this encoding and decoding mess for you.
Bottom line, if you're encountering encoding issues with Amazon's SQS, better check if your messages are able to make it through the XML marshalling and unmarshalling process. If not, you'll need to base-64 encode and decode your messages to dance around bugs in Amazon's AWS library.
Good luck, and hope this helps.
Did you find this post helpful, or at least, interesting?