maximum URL length?

Is there a limit to the size of urls that an RoUrlXfer will digest?

xfer = CreateObject(“roURLTransfer”)

xfer.seturl(“http://www.google.com”)

?xfer.geturl()

http://www.google.com

url=“http://api.soundcloud.com/tracks.xml?consumer_key=----------------------&limit=21&offset=0&order=hotness&created_at[from]=2010-8-24

xfer.seturl(url)

?xfer.GetUrl()

http://www.google.com

xfer.SetUrl(“http://www.ibm.com”)

?xfer.GetURl()

http://www.ibm.com

xfer.setURL(“http://api.soundcloud.com/tracks.xml?consumer_key=----------------------&limit=21&offset=0&order=hotness&created_at[from]=2010-8-24”)

?xfer.GetURl()

http://www.ibm.com

So that is why I’ve been tearing my hair out… Is this a bug or is there a reason for this? Any useful workarounds? now I know why my app broke last night when I added “&created_at[from]=2010-8-24” to the end of the URL, and why it broke in a way I couldn’t trace easily (poster items showing up in the wrong category… because the URL didn’t change)

I think the problem is actually with your URL itself, not the length. You should URL encode your square brackets…

http://api.soundcloud.com/tracks.xml?consumer_key=----------------------&limit=21&offset=0&order=hotness&created_at%5bfrom%5d=2010-8-24

The problem is the square brackets. Replace them with %1B and %1D respectively.

Endless beat me to it!

-JT

“renojim” wrote:
The problem is the square brackets. Replace them with %1B and %1D respectively.

Endless beat me to it!

-JT
%5B and %5D… not 1…

Would there be any benefit / problem with running the whole URL through URLEncode?

I’ve had mixed results with UrlEncode. I’d try it with whatever site you’re trying to connect to and see if it works. In my case, I wanted some things encoded but not others and encoding everything didn’t work. I believe it should work, but you never know until you try.

-JT

“TheEndless” wrote:

“renojim” wrote:
The problem is the square brackets. Replace them with %1B and %1D respectively.

Endless beat me to it!

-JT
%5B and %5D… not 1…

Oops! That’s what I get for trying to beat you to the punch!

-JT

You can’t do the whole URL, because it will encode the slashes, ampersands, and equals. You only want to URL encode the querystring parameters and values.

“TheEndless” wrote:
You can’t do the whole URL, because it will encode the slashes, ampersands, and equals. You only want to URL encode the querystring parameters and values.

And just so it’s completely obvious to anyone who’s just learning this, you encode the parameters and values not the whole query string. That is, you can’t encode “foo=bar&this=that” as the “=” and “&” will be encoded (that’s how you pass them in as variables). You need to encode each parameter name and value separately.

Ok, so just to make sure I get it, what is the criteria for whether a character should be URL encoded? one website I looked at said “all non-alphanumerics”, but I think I really only want the “weird” characters like spaces and brackets?

  • Joel

“jbrave” wrote:
Ok, so just to make sure I get it, what is the criteria for whether a character should be URL encoded? one website I looked at said “all non-alphanumerics”, but I think I really only want the “weird” characters like spaces and brackets?

That site encodes everything, slashes and all. I know this was all very confusing to me when I was trying to put a caret in a URL. I never did solve that problem.

-JT

“renojim” wrote:
That site encodes everything, slashes and all. I know this was all very confusing to me when I was trying to put a caret in a URL. I never did solve that problem.

-JT
Right, you’d need to enter the individual parameters and values separately. The point was it’s an easy way to encode instead of trying to figure it out manually yourself.

I guess it would be nice to have one function to do everything for you. It doesn’t seem like it would be too hard to do. Maybe the next time I need the functionality I’ll do it. Doing one parameter at a time just seems like a pain.

-Jt

“renojim” wrote:
I guess it would be nice to have one function to do everything for you. It doesn’t seem like it would be too hard to do. Maybe the next time I need the functionality I’ll do it. Doing one parameter at a time just seems like a pain.

Hm, I’m not sure what you mean by “do everything for you”, but I don’t see how it would be possible to pass a URL to a function and have it automatically pick out the parameters and encode them. Suppose you were passing an arithmetic expression to a URL, like
www.calculate.com?expr=x=2+3&0xff&precision=10
How could it figure out that the first and third equals signs shouldn’t be encoded, but the second one should be? And that the first ampersand should be encoded, but the second one shouldn’t? If it were possible to figure this out automatically in all cases, the web site itself could do it and there would be no need to encode anything.

–Mark

Well I’m sure there will always be cases that would have to have special handling. I meant for the generic case like:
http://somehost.com/path1/path2/query.p … &q3=param3
where the parameters don’t contain ampersands or equal signs, which would probably cover the vast majority of cases.

Even in your case it should be possible to figure out where the parameters begin and end. It might not be trivial, but not impossible. Sounds like a challenge.

Of course, there’s always the argument to be made that how tough is it to encode each parameter as the URL is being built, so maybe I’m just being lazy. It just seems to be a misnomer to have a function called UrlEncode that doesn’t actually encode a full URL.

-JT

URLEncode means “encode this string for use in a url”, not “encode this url”…
Another thing to consider with a function that does it for you is that you could end up encoding multiple times if you’re not careful. All that said, though, if you’re building the URL programmatically with string concatenation anyway, why’s it any more difficult to wrap the bits in a UrlEncode() call?

“renojim” wrote:

Even in your case it should be possible to figure out where the parameters begin and end. It might not be trivial, but not impossible. Sounds like a challenge.

The problem is that there are cases where it absolutely isn’t possible. For example, encoding a partial query string as a parameter…

To answer jbrave’s earlier question about knowing what to encode, just encode every query param and value if there’s any possibility you might run into a character that’s illegal. In most cases you can get by without it, but if the params or values are in any way generated from user input, you’ll almost definitely run into a problem at some point.