Skip to content

URI::FTP#normalize returns incorrect (?) value if original path is empty #185

Open

Description

@maxfelsher

I don't know how common this case is, but I did run into it accidentally while testing some code.

u = URI("ftp://ftp.example.com") # Note that there's no slash at the end
u.normalize.to_s # => "ftp://ftp.example.com/%2F"

I would expect the result to be "ftp://ftp.example.com/" or possibly the same as the original string, "ftp://ftp.example.com". As best as I can tell from RFC 1738, Section 3.2, "ftp://ftp.example.com" and "ftp://ftp.example.com/" are equivalent and refer to whatever the "default" directory is for FTP access, while "ftp://ftp.example.com/%2F" refers to the root directory.

This seems to be caused by a combination of two things. First, this code in URI::Generic#normalize!calls set_path('/') if the path is empty:

uri/lib/uri/generic.rb

Lines 1341 to 1343 in 52077e9

if path&.empty?
set_path('/')
end

Second, URI::FTP overrides #set_path so that a leading slash in the argument is percent-encoded and appended to an unencoded slash:

uri/lib/uri/ftp.rb

Lines 245 to 247 in 52077e9

def set_path(v)
super("/" + v.sub(/^\//, "%2F"))
end

I didn't see any tests for normalizing FTP URIs, so I'm not sure if this behavior is intended.

Activity

  1. duerst commented on Oct 20, 2025

    @duerst
    Member

    The function of %2F at the start of the path is clearly defined in the paragraph starting with "For example, the URL URL:ftp://myname@host.dom/%2Fetc/motd" in Section 3.2.2 of RFC 1738.
    I can't say that the behavior is wrong. For the ftp: scheme, it looks as if there is an inherent conflict between personal access and generic access. Personal access means that FTP is used to access a user's personal files, starting in a user's home directory. Generic access means that it should be possible to move a set of files from FTP to HTTP or back and only change 'ftp' <-> 'http' in the URI.
    I think this conflict can be resolved by noting that personal access implies giving an userid and password, whereas generic access uses an userid of anonymous. If that's true, then the correct start directory would be set in both cases without the need for %2F. Given that RFC 1738 is very old, the best thing is probably to check out other FTP client implementations (e.g. browsers) and see what they are doing. Then aligning with that behavior (if there's a reasonably uniform behavior) seems best.
    @maxfelsher: Can you set up a small testing ftp server (or use an existing server) and check the behavior of a few browser,... ftp implementations, and report back?

  2. duerst commented on Oct 20, 2025

    @duerst
    Member

    I have had a look at https://url.spec.whatwg.org (as of 18 August 2025). This kind of spec is usually quite difficult to understand, but the string ftp appears only twice, in both cases in parallel with other schemes such as http and https. That is one data point that would suggest that the special treatment with "%2F" in Ruby is an exception and probably should be removed.

    And here is the pointer to the commit that introduced the special behavior about 15 years ago, by @mame: 0212ce0
    @mame: Can you look at this issue? The absolute/relative distinction in your commit may make sense in general, but I don't think "ftp://ftp.example.com" can be called relative.

  3. mame commented on Oct 20, 2025

    @mame
    Member

    The commit was introduced based on the discussion in this ticket:

    https://bugs.ruby-lang.org/issues/1699

    As Naruse-san pointed out in that discussion, there was no living specification for FTP URIs at that time, and it seems this is still the case today. An attempt was made to define FTP URIs in the following draft, but it has unfortunately expired:

    https://datatracker.ietf.org/doc/html/draft-hoffman-ftp-uri-04

    Section 2.2 of this draft, "FTP uri-path", contains an interesting passage:

    Historical note: Most FTP client implementations precede the
    with a "/" before sending the CWD command. This is in conflict with
    RFC 1738, although the practice is quite widespread. Thus, a client
    that is presented with the URL URL:ftp://myname@example.com/abc/def
    might send the two commands "CWD /abc" and "RETR def" or it might
    send the two commands "CWD abc" and "RETR def". Server implementers
    should be aware of these two different interpretations of the same
    URL.

    Therefore, @maxfelsher's interpretation of RFC 1738 is correct, but according to the draft note, most FTP client implementations do not follow RFC 1738.

    Since there's no definitive FTP URI specification and common practice conflicts with the old RFCs, I think maintaining the current behavior is the best approach unless it's causing real-world issues.

  4. maxfelsher commented on Oct 21, 2025

    @maxfelsher
    Author

    Yeah, I think the current behavior is almost certainly inconsistent with RFC 1738, but I recognize that it's an extremely old RFC and that clients don't necessarily follow it. On the other hand, that historical note from the draft RFC suggests to me that the leading %2F was superfluous in the case of "Most FTP client implementations" at the time, since they preceded the first path component with a slash before sending the CWD command anyway.

    Unfortunately, I don't have the bandwidth to do testing with actual FTP clients or servers; I just happened to be using #normalize on various URLs and noticed this behavior that surprised me. I think that ideally normalization in this case wouldn't add the %2F, but I also understand that it might not be worth the effort to determine and implement a better approach. At least now it's written up somewhere that'll show up in a web search!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions