Just a short, simple blog for Bob to share his thoughts.
18 December 2008 • by Bob • FTP
In my "FTP Clients - Part 1: Web Browser Support" blog post, I mentioned creating a secured Global Listener FTP Site when you're working with FTP virtual hosts, but I didn't really explain what I meant by that or why you would want to do this. With that in mind, today's blog post is to describe how and why you might want to create a Global Listener FTP Site.
To start things off, the concept is really simple - a Global Listener FTP Site is an FTP site with no virtual host binding and anonymous access disabled. It's kind of like having a "Default FTP Site" with restricted access. Here's why this is a good idea when you're working with FTP virtual hosts - some clients default to anonymous, like web browsers, and if anonymous succeeds then the FTP client doesn't have the opportunity to enter the FTP virtual host name, so you can't get to the virtual host site.
To refresh everyone's memory, there are two different methods for binding multiple FTP host names to IP addresses in FTP7:
Unless your FTP client allows sending custom FTP commands, you won't be able to use FTP True Host Names, so if you want to host several FTP sites on the same IP address then your only option is to use FTP Virtual Host Names. The trouble is, as I mentioned earlier, that some FTP clients (like web browsers) try to log in using anonymous first. If all of your FTP sites are bound to a virtual host name, the FTP client will get a "550-No such host is known" error from the FTP server, because the anonymous user did not specify a virtual host name as part of the USER command. On some clients you could fix that by specifying "ftp.example.com|anonymous" as your anonymous user name, but in most cases the login attempt will just fail.
If you create an FTP site that has no virtual host name, then the FTP service will have some place to send these default anonymous requests. When this FTP site does not have anonymous access enabled, the client will be prompted for their username, which will allow you to enter the "ftp.example.com|username" syntax to specify the virtual host name.
Please note that creating a Global Listener FTP site is really more of a workaround for the way that some FTP clients behave - it's certainly not required, and it only applies to situations where you are using FTP Virtual Host Names. For example, if you are using user isolation to restrict users to specific paths on a single FTP site, the Global Listener FTP site would be completely unnecessary.
Note: See my Virtual Hosts and Host Names in FTP7 blog post for more information about FTP Virtual Host Names and FTP True Host Names, and see https://datatracker.ietf.org/drafts/draft-hethmon-mcmurray-ftp-hosts/ for more information about status of the FTP HOST command.
Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
10 November 2008 • by Bob • FTP
In part 2 of my series on FTP clients, I thought it would be best to have a discussion about the differences between Implicit FTPS and Explicit FTPS. In my recent "FTP Clients - Part 1: Web Browser Support" blog post, I referenced Implicit and Explicit FTPS with a link to my Using FTP Over SSL walkthrough. But it occurred to me that some people may not understand the difference between the two, and my upcoming blog posts are going to build upon that knowledge, so I thought that a quick discussion of these two technologies would be prudent.
One of the many limitations of the File Transfer Protocol (FTP) is a general lack of security; e.g. user names and passwords are transmitted in clear text, data is transferred with no encryption, etc. In order to address this situation, FTP over SSL (FTPS) was introduced in Requests for Comments (RFC) article 2228 - FTP Security Extensions, and expanded in RFC 4217 - Securing FTP with TLS to address Transport Layer Security (TLS).
Following up on these RFC articles, the FTP service for Windows Server 2008 added support for FTPS, and the FTP SSL Settings Feature in the IIS Manager allows you to configure your FTPS settings to allow or require SSL, enforce 128-bit SSL, or customize your control/data channel SSL settings.
Explicit FTPS is really what RFCs 2228 and 4217 envisioned; basically the way this works is an FTP client connects over the control/command channel (usually on port 21), and then the client can negotiate SSL for either the command/control channel or the data channel using new FTP commands like AUTH, PROT, CCC, etc.
The FTP service for Windows Server 2008 allows customized settings for both the command/control channel and the data channel through the Advanced SSL Policy dialog:
There are several ways that Explicit FTPS might be implemented depending on your business needs:
Control Channel Data Channel Notes Allow Allow This configuration allows the client to decide whether any part of the FTP session should be encrypted. Require only
for credentialsAllow This configuration protects your FTP client credentials from electronic eavesdropping, and allows the client to decide whether data transfers should be encrypted. Require only
for credentialsRequire This configuration requires that the client's credentials must be secure, and then allows the client to decide whether FTP commands should be encrypted. However, all data transfers must be encrypted. Require Require This configuration is the most secure - the client must negotiate SSL using the FTPS-related commands before other FTP commands are allowed, and all data transfers must be encrypted.
Implicit FTPS takes SSL one step further than simply requiring that SSL-related commands must be sent first like you can with Explicit SSL; with Implicit FTPS, an SSL handshake must be negotiated before any FTP commands can be sent by the client. In addition, even though Explicit FTPS allows the client to arbitrarily decide whether to use SSL, Implicit FTPS requires that the entire FTP session must be encrypted. Basically the way that Implicit FTPS works is that an FTP client connects to the command/control channel, in this case using port 990, and immediately performs an SSL handshake; after SSL has been negotiated, additional FTP commands for the session can be sent by the FTP client.
Using FTPS in FTP service for Windows Server 2008 follows the Internet Assigned Numbers Authority (IANA) specification that the Implicit FTPS command/control channel is on port 990 and the Implicit FTPS data channel is on port 989.
Here's the way that you specify which type of FTP over SSL (FTPS) that you are using in Windows Server 2008:
Note: If you are using FTP on any ports other than the defaults of 21/20 and 990/989, you must make sure that those ports are not already assigned by IANA to another protocol. For more information, see the list of assigned port numbers on IANA's web site.
Choosing whether to use Explicit FTPS over Implicit FTPS is a personal choice, and generally this choice may depend on your business needs or your FTP client. In several FTP clients that I've tested, the FTP client chooses one form of FTPS over another as the default method, and the FTP client may require some manual configuration to use the other.
Shortly after shipping the FTP service for Windows Server 2008 we discovered an issue where the FTP service was not cleaning up Implicit SSL connections properly, and we issued a hotfix rollup package for the FTP service that is discussed in Microsoft Knowledge Base article 955136.
I hope this helps to clear things up a bit. ;-]
Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
25 September 2008 • by Bob • FTP
Since we've been testing a lot of FTP clients with our new FTP server for IIS 7, I thought that it would be a good idea to discuss some of the highlights and pitfalls that we have run into when testing various clients.
I thought that I'd begin this series with an examination of several web browsers, which are really not the best FTP clients around - web browsers are mostly just "putting a pretty face" on an FTP site rather than functioning as an FTP client. In any event, here's a summary table of different features that I tested with a few web browsers:
Client Name | Directory Browsing | Explicit FTPS | Implicit FTPS | Virtual Hosts | True HOSTs |
---|---|---|---|---|---|
FireFox 3.0.2 (Mozilla) | Rich | N | N | Y | N |
Google Chrome 0.2.149 (Beta) | Basic | N | N | Y | N |
Internet Explorer 7.0 | Basic | N | N | Y | N |
Opera 9.5.2 | Rich | N | N | Y | N |
See the individual client notes below for more information on each client.
As far as web browsers are concerned, they're not great FTP clients. That being said, here's my thoughts on their respective experiences:
In the next part of this series, I'll start taking a look at some specific FTP clients.
Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
27 September 2007 • by Bob • FTP, IIS, IIS News Item
Today Microsoft released the newest version of our FTP server for Windows Server 2008 Release Candidate 0 (RC0)!
Listed below are the links for the download pages for each of the individual installation packages:
I added the following text to my "Installing and Troubleshooting FTP7" walkthrough, but I’m going to repost that here because it sums up nicely what our new FTP server has to offer:
This new FTP service incorporates many new features that enable web authors to publish content better than before, and offers web administrators more security and deployment options.
- Integration with IIS 7.0: IIS 7.0 has a brand-new administration interface and configuration store, and the new FTP service is tightly integrated with this new design. The old IIS 6 metabase is gone, and a new configuration store that is based on the .NET XML-based *.config format has taken its place. In addition, IIS 7.0 has a new administration tool, and the new FTP server plugs seamlessly into that paradigm.
- Support for new Internet standards: One of the most significant features in the new FTP server is support for FTP over SSL. The new FTP server also supports other Internet improvements such as UTF8 and IPv6.
- Shared hosting improvements: By fully integrating into IIS 7.0, the new FTP server makes it possible to host FTP and Web content from the same site by simply adding an FTP binding to an existing Web site. In addition, the FTP server now has virtual host name support, making it possible to host multiple FTP sites on the same IP address. The new FTP server also has improved user isolation, now making it possible to isolate users through per-user virtual directories.
- Extensibility and custom authentication: The new FTP server supports developer extensibility, making it possible for software vendors to write custom providers for FTP authentication. Microsoft is using this extensibility feature to implement two new methods for using non-Windows accounts for FTP authentication for IIS Managers and .NET Membership.
- Improved logging support: FTP logging has been enhanced to include all FTP-related traffic, unique tracking for FTP sessions, FTP sub-statuses, additional detail fields in FTP logs, and much more.
- New supportability features: IIS 7.0 has a new option to display detailed error messages for local users, and the FTP server supports this by providing detailed error responses when logging on locally to an FTP server. The FTP server also logs detailed information using Event Tracing for Windows (ETW), which provides additional detailed information for troubleshooting.
Additional information about new features in FTP7 is available in the "What's New for Microsoft and FTP?" topic on Microsoft's http://www.iis.net/ web site.
The following prerequisites are required to install this new version:
To help you get started using the new FTP server, the following walkthroughs have been published on the http://www.iis.net/ web site:
Special thanks go to:
Thanks!
28 August 2007 • by Bob • FTP, IIS
I received the following question from Mickey Binder in response to my blog entries about the new FTP service for IIS7:
"When using virtual hosts will it always be necessary to provide DOMAIN|Username or is it me doing something wrong. I can see in your documentation how it should be setup, I just wonder why it still needs the DOMAIN in the user login as this should already be specified by the host I connect to (Like with the http virtual hosts)."
I thought that was a great question, and I think that it deserves a detailed response. The short answer is that you need to provide the host name somehow when connecting to an FTP site, and FTP simply does not work the way that HTTP does. I'll explain why in detail with this blog post.
HTTP provides a way to pass the host name in the headers that are passed between the client and server, but FTP currently does not provide this same functionality. Looking at one of my older blog posts, I pointed out that there are three ways that you can create unique bindings for a Web/HTTP site: IP address, port, or host header. Based on its current design, FTP can create unique bindings by IP address or port, but the FTP protocol currently does not allow for something like host headers.
Here's why - I'm sure most everyone is aware that HTTP packets consist of a set of headers and possibly a block of data. Here's an example of a simple GET request:
GET /default.aspx HTTP/1.0 [crlf]
Accept: */* [crlf]
[crlf]
When HTTP 1.1 was published in RFC 2068 and RFC 2616 it defined a header for specifying a "host" name in a separate name/value pair:
GET /default.aspx HTTP/1.1 [crlf]
Host: example.com [crlf]
Accept: */* [crlf]
[crlf]
This allows multiple virtual servers ("hosts") on the same IP address and port that are differentiated by host name. While this works great for the HTTP protocol, the FTP protocol currently has no comparable functionality. As such, the FTP protocol would have to be updated to allow multiple hosts on the same IP address and port, then IIS and all FTP clients would need to be updated to accommodate the changes to FTP.
I realize that most everyone is aware that when you attempt to connect to an HTTP or FTP server from a client, the client looks up the IP address using a name server and then creates a connection to the server by IP address. What many people may not realize is that the server is basically unaware of the host name that the client used - at connection time the server is really only concerned with inbound data on the IP address.
That being said, the connection protocol could provide a mechanism for specifying the host name. For example, as I mentioned above the HTTP/1.1 protocol provides a mechanism for specifying host names using a host header and FTP does not. In fact, if you were to open Network Monitor or similar tool and capture an FTP connection between a client and a server, you would be able to see that the host name is not passed as part of the FTP conversation between the client and server.
For the new FTP service for IIS7 we wanted to find a way to have host names in FTP, so we approached the situation with two possible ideas:
In the end, we chose to do both.
First of all we have "Virtual Host" names; this is the "ftp.example.com|username" syntax that led to the original question. Here's the way that this can be used in a hosting environment. Let's say that a web hoster has a customer with several Web/FTP sites that he needs to update. The web hoster gives all of his customers the same ftp address, like "ftp.example.com". When setting up the bindings for the Web/FTP sites the web hoster configures each site for the appropriate HTTP/FTP bindings, using host names like "www.contoso.com" & "www.fabrikam.com" for the HTTP bindings and "ftp.contoso.com" & "ftp.fabrikam.com" for FTP bindings. (All of the Web names would be set up in DNS, of course, but the FTP names are somewhat optional as I'll explain later.)
Since HTTP provides host headers to pass the host name, the Web client & Web server will both know which site they're working with. Since FTP doesn't have a built-in way to pass a host name, the customer will connect to the "ftp.example.com" name that his web hoster gave him and log in using the appropriate "ftp.contoso.com|customer" or "ftp.fabrikam.com|customer" syntax. The FTP service for IIS7 will then internally route the FTP activity to the correct site based on the FTP bindings. If the FTP names "ftp.contoso.com" & "ftp.fabrikam.com" were registered in DNS, the client would still need to specify the appropriate "ftp.contoso.com|customer" or "ftp.fabrikam.com|customer" syntax when logging in because the FTP activity did not actually pass the host name in any way.
The great thing about the "Virtual Host" names solution is that it is backwards-compatible because any client should be able to send the "ftp.example.com|username" syntax.
We also wanted to see if the FTP protocol could be updated to allow sending a host name as part of the FTP session like HTTP does. After some research I discovered that Robert Elz and Paul Hethmon had provided a detailed discussion of a "HOST" command for FTP in their Internet draft titled "Extensions to FTP" during their work with the FTPEXT Working Group at the IETF. I contacted the two of them and with their approval Paul and I submitted a new IETF draft detailing a HOST command for FTP, which is posted at the following URL:
http://www.ietf.org/internet-drafts/draft-hethmon-mcmurray-ftp-hosts-00.txt
Here's the way that the HOST command works: the FTP server and FTP client both need to know that the HOST command is supported. The FTP client connects to an FTP server using either a DNS name or IP address and sends a FEAT command and sees that HOST is supported, so the client sends "HOST ftp.example.com" before sending USER and PASS, which allows the FTP server to route the request to the correct FTP site based on the bindings. (An FTP client could skip the FEAT command completely and simply attempt a HOST command and process the FTP reply, but that's not the best approach.)
The FTP HOST command solution is not backwards-compatible, however, because an FTP client needs to be able to send the "HOST ftp.example.com" syntax. Some FTP clients allow sending custom commands, which enables this functionality. For example, if you are using the FTP.EXE command-line tool that comes in Windows, you can type "quote HOST ftp.example.com" when connecting to the new FTP server for IIS7 and it will route the request to the appropriate site. In an ideal world, FTP clients will start negotiating the HOST feature behind the scenes and you should never know that this is occurring, which is how almost all Web browsers currently work. (e.g. When you enter "www.example.com" in the address bar of a Web browser it will automatically add the host header to the HTTP request.)
So the long answer to the original question is that you have two ways of specifying a host with the new FTP service for IIS7:
So my thanks to Mickey Binder for his great question, and I hope this helps other people understand this concept a little better.
06 July 2007 • by Bob • FTP, Scripting
A few weeks ago my friend Jaroslav posted a blog entry about viewing the current FTP7 sessions using Javascript, and I followed that up with a blog post about viewing the current FTP7 sessions using VBScript.
This blog entry follows up on those postings by showing you how to view the current FTP7 sessions using C#. To do so, start a new Windows Console Application project using C# in Visual Studio 2005 on a computer running Windows Server 2008 with the new FTP7 server installed. You will need to add a reference to the AppHostAdminLibrary by manually browsing to the nativerd.dll file that's located in the %WinDir%\System32\InetSrv folder. After you've added the reference, replace all of the C# code from the project template with the following C# code:
using System; using System.Collections.Generic; using System.Text; using AppHostAdminLibrary; namespace FtpDumpSessions { class FtpDumpSessions { static void Main(string[] args) { AppHostWritableAdminManager objAdminManager = new AppHostWritableAdminManager(); // get the collection of sites IAppHostElement objSitesElement = objAdminManager.GetAdminSection( "system.applicationHost/sites", "MACHINE/WEBROOT/APPHOST"); uint intSiteCount = objSitesElement.Collection.Count; Console.WriteLine( "Site count: {0}", intSiteCount); try { // loop through the sites collection for (int intSite = 0; intSite < intSiteCount; ++intSite) { // get a site IAppHostElement objFtpSite = objSitesElement.Collection[intSite]; // get the FTP section IAppHostElement objFtpSiteElement = objFtpSite.ChildElements["ftpServer"]; // get the sessions collection IAppHostElement objFtpSessions = objFtpSiteElement.ChildElements["sessions"]; uint intSessionCount = objFtpSessions.Collection.Count; Console.WriteLine( "\tFTP sessions for {0}: {1}", objFtpSite.Properties["name"].Value, intSessionCount); // loop through the sessions for (int intSession = 0; intSession < intSessionCount; ++intSession) { IAppHostElement objFtpSession = objFtpSessions.Collection[intSession]; // loop through each session's properties for (int intProperty = 0; intProperty < objFtpSession.Properties.Count; ++intProperty) { Console.WriteLine( "\t\t{0}: {1}", objFtpSession.Properties[intProperty].Name, objFtpSession.Properties[intProperty].Value); } } } } catch (System.Exception ex) { Console.WriteLine( "\r\nError: {0}", ex.Message); } } } }
When you compile and run the project, you should see a listing of all users connected to your FTP7 sites.
That's about it for this post - have fun!
06 July 2007 • by Bob • FTP, Scripting
A few weeks ago my friend Jaroslav posted a blog entry about viewing the current FTP7 sessions using Javascript, and I followed that up with a blog post about viewing the current FTP7 sessions using C#.
This blog entry follows up on those postings by showing you how to view the current FTP7 sessions using VBScript. To do so, copy the following VBScript code to Windows Notepad and save the file as "ftp_sessions.vbs" on a computer running Windows Server 2008 with the new FTP7 server installed:
Option Explicit Dim objAdminManager, objSiteCollection, objFtpSiteElement Dim objSite, objFtpSession, objFtpSessions, objFtpProperty Dim intSite, intFtpSession, intFtpProperty Dim intSiteCount, intFtpSessionCount, intFtpPropertyCount Set objAdminManager = WScript.CreateObject("Microsoft.ApplicationHost.AdminManager") ' get the collection of sites Set objSiteCollection = objAdminManager.GetAdminSection( _ "system.applicationHost/sites", "MACHINE/WEBROOT/APPHOST" ) intSiteCount = CInt(objSiteCollection.Collection.Count) WScript.Echo String(40,"*") WScript.Echo "Site count: " & intSiteCount WScript.Echo String(40,"*") ' loop through the sites collection For intSite = 0 To intSiteCount-1 ' get a site Set objSite = objSiteCollection.Collection.Item(intSite) ' get the FTP section Set objFtpSiteElement = objSite.ChildElements.Item("ftpServer") ' get the sessions collection Set objFtpSessions = objFtpSiteElement.ChildElements.Item("sessions") intFtpSessionCount = CInt(objFtpSessions.Collection.Count) WScript.Echo String(40,"=") WScript.Echo "FTP sessions for " & _ objSite.Properties.Item("name").Value & _ ": " & intFtpSessionCount WScript.Echo String(40,"=") ' loop through the sessions For intFtpSession = 0 To intFtpSessionCount - 1 Set objFtpSession = objFtpSessions.Collection.Item(intFtpSession) intFtpPropertyCount = CInt(objFtpSession.Properties.Count) ' loop through each session's properties For intFtpProperty = 0 To intFtpPropertyCount - 1 Set objFtpProperty = objFtpSession.Properties.Item(intFtpProperty) WScript.Echo CStr(objFtpProperty.Name) & ": " & CStr(objFtpProperty.Value) Next WScript.Echo String(40,"-") Next Next
To make sure that you don't see any message box pop-ups, run the script from the command-line using the following syntax:
cscript.exe ftp_sessions.vbs
That's about it for this post - have fun!
26 April 2007 • by Bob • FTP, IIS
It's been a long time in development, but Microsoft has released a beta version of a new FTP service that we have completely rewritten for Windows Server code name "Longhorn". This new FTP service contains many great new features, such as:
There are way too many features to list in such a short space, so a good place for addtional information is the "What’s New for Microsoft and FTP?" article. I encourage you to download the beta version of the new FTP service today, and listed below are the links for the download pages for each of the individual installation packages:
After installing the new FTP server, the following walkthroughs on the www.iis.net web site should help get you started:
In closing, the beta version of this new FTP service has a lot to offer, and we've put a lot of time and effort into making what we think is a great start for the future of FTP for IIS. About the only piece of bad news that I have for anyone is that this new FTP service will not work on Windows Server 2003 with IIS 6.0.
All in all, our team is excited to see people start testing with this beta version. And lest I forget, my special thanks go to:
Thanks everyone!
09 May 2006 • by Bob • FTP, IIS
In IIS 4.0 and IIS 5.0, if you created a virtual directory that had a name that was identical to a user name, when the user logged in to the FTP site they would automatically be changed to their folder. When multiple users will access the same FTP content, you could create another virtual directory that is identical to the other user name and point it to the same content.
This allowed sharing a single FTP site across several users and content sets without advertising user names or content folders. Even though a user could type "CD /
" from an FTP prompt, they would not be able to see the virtual directories from other user accounts on that server because virtual directories are not listed when a user types "ls -l
" or "dir
" from an FTP prompt at the root. That being said, this security feature still doesn't go far enough from a security perspective.
One of the great IIS 6.0 features is User Isolation, which is discussed in the Hosting Multiple FTP Sites with FTP User Isolation (IIS 6.0) topic on MSDN. As a quick review, there are three different isolation modes that you can choose when creating an IIS 6.0 FTP site:
To configure the "Isolate Users" mode, you first need to create your FTP site and choose the "Isolate Users" option when prompted for FTP User Isolation. Once the FTP site has been created, you need to create a physical folder named "LocalUser" for local user accounts or named after your domain under your FTP server's root folder. To isolate users to a specific folder, you use these rules that I copied from the MSDN topic that I listed earlier in this post:
This is very easy to configure, and when a user logs in to your FTP server they will be restricted to their physical folder under the FTP root. Typing "CD /
" from an FTP prompt will always restrict the user within their own site.
That being said, because physical directories are required for this configuration it may seem like a step backward when you consider that you used to be able to create multiple virtual directories that pointed to content in varying locations and for multiple user accounts. Not to worry, however, because Windows provides a way around this limitation using NTFS junctions.
For those of you that are not familiar with NTFS junctions, there are several topics that discuss this. (For example, see Inside Win2K NTFS, Part 1.) A junction is somewhat like a symbolic directory link in the UNIX world, where a junction looks like a folder but points to content that is physically located somewhere else. There are two tools that you can use to create junctions, LINKD from the Windows Resource Kit, and JUNCTION from www.sysinternals.com. Using these tools with IIS 6.0 can allow you the freedom to deploy FTP folder structures much like you did with IIS 4/5 while utilizing the user isolation features in IIS 6.
Here's an example - when configuring an IIS 6.0 FTP site, I used the following steps:
When a user logs in to my FTP site using their user account, they are automatically dropped in their content folder via the junction. Since you can create multiple junctions that point to the same content folder, you can create junctions for every user account that will work with a set of content.
I hope this helps!
Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/