You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
URI::FTP#normalize returns incorrect (?) value if original path is empty #185
I don't know how common this case is, but I did run into it accidentally while testing some code.
u=URI("ftp://ftp.example.com")# Note that there's no slash at the endu.normalize.to_s# => "ftp://ftp.example.com/%2F"
I would expect the result to be "ftp://ftp.example.com/" or possibly the same as the original string, "ftp://ftp.example.com". As best as I can tell from RFC 1738, Section 3.2, "ftp://ftp.example.com" and "ftp://ftp.example.com/" are equivalent and refer to whatever the "default" directory is for FTP access, while "ftp://ftp.example.com/%2F" refers to the root directory.
This seems to be caused by a combination of two things. First, this code in URI::Generic#normalize!calls set_path('/') if the path is empty:
The function of %2F at the start of the path is clearly defined in the paragraph starting with "For example, the URL URL:ftp://myname@host.dom/%2Fetc/motd" in Section 3.2.2 of RFC 1738.
I can't say that the behavior is wrong. For the ftp: scheme, it looks as if there is an inherent conflict between personal access and generic access. Personal access means that FTP is used to access a user's personal files, starting in a user's home directory. Generic access means that it should be possible to move a set of files from FTP to HTTP or back and only change 'ftp' <-> 'http' in the URI.
I think this conflict can be resolved by noting that personal access implies giving an userid and password, whereas generic access uses an userid of anonymous. If that's true, then the correct start directory would be set in both cases without the need for %2F. Given that RFC 1738 is very old, the best thing is probably to check out other FTP client implementations (e.g. browsers) and see what they are doing. Then aligning with that behavior (if there's a reasonably uniform behavior) seems best. @maxfelsher: Can you set up a small testing ftp server (or use an existing server) and check the behavior of a few browser,... ftp implementations, and report back?
I have had a look at https://url.spec.whatwg.org (as of 18 August 2025). This kind of spec is usually quite difficult to understand, but the string ftp appears only twice, in both cases in parallel with other schemes such as http and https. That is one data point that would suggest that the special treatment with "%2F" in Ruby is an exception and probably should be removed.
And here is the pointer to the commit that introduced the special behavior about 15 years ago, by @mame: 0212ce0 @mame: Can you look at this issue? The absolute/relative distinction in your commit may make sense in general, but I don't think "ftp://ftp.example.com" can be called relative.
As Naruse-san pointed out in that discussion, there was no living specification for FTP URIs at that time, and it seems this is still the case today. An attempt was made to define FTP URIs in the following draft, but it has unfortunately expired:
Section 2.2 of this draft, "FTP uri-path", contains an interesting passage:
Historical note: Most FTP client implementations precede the
with a "/" before sending the CWD command. This is in conflict with
RFC 1738, although the practice is quite widespread. Thus, a client
that is presented with the URL URL:ftp://myname@example.com/abc/def
might send the two commands "CWD /abc" and "RETR def" or it might
send the two commands "CWD abc" and "RETR def". Server implementers
should be aware of these two different interpretations of the same
URL.
Therefore, @maxfelsher's interpretation of RFC 1738 is correct, but according to the draft note, most FTP client implementations do not follow RFC 1738.
Since there's no definitive FTP URI specification and common practice conflicts with the old RFCs, I think maintaining the current behavior is the best approach unless it's causing real-world issues.
Yeah, I think the current behavior is almost certainly inconsistent with RFC 1738, but I recognize that it's an extremely old RFC and that clients don't necessarily follow it. On the other hand, that historical note from the draft RFC suggests to me that the leading %2F was superfluous in the case of "Most FTP client implementations" at the time, since they preceded the first path component with a slash before sending the CWD command anyway.
Unfortunately, I don't have the bandwidth to do testing with actual FTP clients or servers; I just happened to be using #normalize on various URLs and noticed this behavior that surprised me. I think that ideally normalization in this case wouldn't add the %2F, but I also understand that it might not be worth the effort to determine and implement a better approach. At least now it's written up somewhere that'll show up in a web search!
Activity
duerst commentedon Oct 20, 2025
The function of %2F at the start of the path is clearly defined in the paragraph starting with "For example, the URL URL:ftp://myname@host.dom/%2Fetc/motd" in Section 3.2.2 of RFC 1738.
I can't say that the behavior is wrong. For the ftp: scheme, it looks as if there is an inherent conflict between personal access and generic access. Personal access means that FTP is used to access a user's personal files, starting in a user's home directory. Generic access means that it should be possible to move a set of files from FTP to HTTP or back and only change 'ftp' <-> 'http' in the URI.
I think this conflict can be resolved by noting that personal access implies giving an userid and password, whereas generic access uses an userid of anonymous. If that's true, then the correct start directory would be set in both cases without the need for %2F. Given that RFC 1738 is very old, the best thing is probably to check out other FTP client implementations (e.g. browsers) and see what they are doing. Then aligning with that behavior (if there's a reasonably uniform behavior) seems best.
@maxfelsher: Can you set up a small testing ftp server (or use an existing server) and check the behavior of a few browser,... ftp implementations, and report back?
duerst commentedon Oct 20, 2025
I have had a look at https://url.spec.whatwg.org (as of 18 August 2025). This kind of spec is usually quite difficult to understand, but the string ftp appears only twice, in both cases in parallel with other schemes such as http and https. That is one data point that would suggest that the special treatment with "%2F" in Ruby is an exception and probably should be removed.
And here is the pointer to the commit that introduced the special behavior about 15 years ago, by @mame: 0212ce0
@mame: Can you look at this issue? The absolute/relative distinction in your commit may make sense in general, but I don't think "ftp://ftp.example.com" can be called relative.
mame commentedon Oct 20, 2025
The commit was introduced based on the discussion in this ticket:
https://bugs.ruby-lang.org/issues/1699
As Naruse-san pointed out in that discussion, there was no living specification for FTP URIs at that time, and it seems this is still the case today. An attempt was made to define FTP URIs in the following draft, but it has unfortunately expired:
https://datatracker.ietf.org/doc/html/draft-hoffman-ftp-uri-04
Section 2.2 of this draft, "FTP uri-path", contains an interesting passage:
Therefore, @maxfelsher's interpretation of RFC 1738 is correct, but according to the draft note, most FTP client implementations do not follow RFC 1738.
Since there's no definitive FTP URI specification and common practice conflicts with the old RFCs, I think maintaining the current behavior is the best approach unless it's causing real-world issues.
maxfelsher commentedon Oct 21, 2025
Yeah, I think the current behavior is almost certainly inconsistent with RFC 1738, but I recognize that it's an extremely old RFC and that clients don't necessarily follow it. On the other hand, that historical note from the draft RFC suggests to me that the leading
%2Fwas superfluous in the case of "Most FTP client implementations" at the time, since they preceded the first path component with a slash before sending the CWD command anyway.Unfortunately, I don't have the bandwidth to do testing with actual FTP clients or servers; I just happened to be using
#normalizeon various URLs and noticed this behavior that surprised me. I think that ideally normalization in this case wouldn't add the%2F, but I also understand that it might not be worth the effort to determine and implement a better approach. At least now it's written up somewhere that'll show up in a web search!