WSync user manual
Table of Contents
WSync is a web uploading tool, i.e. a kind of ``web mirroring''
program, but it operates by ``pushing'' (uploading) files rather than
downloading them. If you maintain a web site, you probably generate
HTML pages on a local repository on your computer, and then upload
them to the web server. WSync can help you a lot.
You give WSync a list of files and it keeps track of what versions
of what files have been uploaded, and can thus find out which files
have changed on your local repository, and upload them automatically.
It does that by maintaining locally a database of files ``signatures''
(made of a timestamp and a digest). Actual uploading is made through
a ``backend'', performing network operations. Currently, only the
File Transfer Protocol is available as backend (an FTP library is
included in WSync). In the future, I plan to add new backends
supportting other protocols, especially because FTP is known to be
insecure (see  for details).
When you run WSync it computes the signature for all files in the
local repository, and check if they match those stored in its
database. It does that by first comparing timestamps. If they match,
then the file hasn't changed since last upload. If they don't, it
computes the digest and compares it to the database digest. If they
differ, then the file has changed and needs to be uploaded, otherwise,
it means that the file has been regenerated, but with no changes, and
there is thus no need to upload it. This can often happen when you
generate the HTML (or whatever) files from some higher level
description language (such as LaTeX or XML, and in particular if
you generate several HTML files from one source file).
WSync also detects when you add or remove files in the list, and
uploads or remote-deletes them automatically. Thanks to the files
digests it maintains, it can even detect pairs of add/remove that
actually are a renaming of a file. So it can do the right thing, i.e.
rename the file on the remote server, rather than deleting it under
the old name and uploading it under the new name.
It also creates directories as needed (but never delete directories
after deleting files).
Two data files are necessary for WSync operation. A list
file containing the list of files that are to be synched with the
remote server, and a command file giving configuration
2.1 Command file
Here is a template command file, that is pretty
self-explanatory. Currently, the only available backend is ``FTP'',
and its parameters are host, login, and
password. The log file is automatically created if
it doesn't exist.
# template command file for wsync
# one directive per non blank non comment line
# '\' as last character (no trailing whitespaces) enables wrapping
# key : value
# key : par1 = val1, par2= val2 ,par3 = val3
# backend to perform upload
# connection parameters for the backend
params: host = my.host, login = mylogin, \
password = mypass
# path to the local repository
# name of file listing the synched files
# name of db file (where files status is stored)
# name of log file (defaults to 'wsync.log')
2.2 List file
The list file is basically a simple list, with one file per
line. Here is a sample. You can use the special file name '*' for
``all the files in this directory'', as well as including the content
of another list file by putting the file name on a line by itself,
prefixed by the `<' character. The parsing of this file is
very simple and stupid, so be sure to conform exactly to this!
#-- some files in top dir
#-- all files from a subdir
# include another list file
#-- files in various dirs
2.3 Operations and file states
You run WSync by giving it the name of the command file as
an argument. It reads the list file, its database, and
displays what the situation is.
It may warn you about a number of lost files. These are files
that are in some directory of the local repository (i.e. a directory
with files listed in the list file) but not in the list
file themselves. Maybe this is because you generated new files and
forgot to add them to the list file. You can give WSync the
-l option to get the list of these lost files displayed.
It then lists the files that need to be uploaded (for beeing new or
changed, if any), with their sizes and showing the total amount of data
that needs to be uploaded. It also list files to be deleted, renamed,
and directories to be created.
By default, it displays this information but doesn't call any backend
to perform any action on the remote server. It will if you give it
the -d (for doit) option. Of course, if any such
operation fails, the corresponding database entry is not updated, so
the operation will be tried again next time.
Some files may also be listed as ``touched''. These are files whose
timestamp has changed, but whose digest not. They won't be uploaded
and are listed just for information. Next time you run WSync they
won't be in the ``touched'' list again (except if their timestamp has
changed again). This is because the signature in the database is
updated with the new timestamp, so that if you don't touch them again,
WSync will known they haven't changed without needing to compute
Finally, files may be ``unknown'', meaning that they're present both
in the list file and in the database, but can't be found in the
local repository. You may thus delete some files in the repository
(e.g. to save disk space) without causing any trouble (but be sure
they've been uploaded before to do that). This is meaningfull since
if you generate HTML files from some other language, you shouln't have
to keep local copies of the HTML pages themselves, once they have been
uploaded. Next time you generate these files, WSync will realize
they're back and treat them normally (i.e. upload them if they have
A variant of this is ``added but not present'', showing files that are
new in the list file but can't be found in the local
repository. You probably need to fix something in this case.
It may happen that some files have changed, but you don't want all of
them to be uploaded, for example if you know you're going to change
them again very soon. In this case, you should delete (or move out)
the files you don't want WSync to upload from the local repository.
WSync will thus finds them to be ``unknown'', and will not attempt
to upload them. Do not delete them from the list file, or WSync
will think you want them to be removed from the remote server!
Another way to do that is to use the --prevent option,
2.4 Log file
All operations performed (or attempted) on the remote server are
logged in the log file that looks like that:
06/10/03 07:36:40 ( FTP my.host.com, mylogin
06/10/03 07:36:42 + prgm.html
06/10/03 07:36:43 + emacs.html
06/10/03 07:36:49 * Private
06/10/03 07:36:50 + Private/fp.tgz
06/10/03 07:36:50 - msx.html
06/10/03 07:36:51 > latxe.html --> latex.html
06/10/03 07:36:51 )
As you can see, each line shows a timestamp, an operation symbol
(`(' for opening the connection, `)' for closing it, `+'
for upload, `-' for delete, `*' for creating a directory,
`>' for renaming) followed by parameters of the operation. If the
operation has failed, the string `**FAILED**' ends the line.
2.5 Manual operations
WSync tries to act automatically as most as possible. But you can
also explicitally direct it to do certain things. For all the
following operations, there are two ways so specify a list of files:
- by passing a --option command line argument,
followed by a list of files (this must be the last thing on the
- by passing a --option-from file command
line argument specifying a list file containing the files
(this is precisely a list file as described in
2.2, with same syntax and files inclusion features)
2.5.1 Forcing/Avoiding uploads
You may want WSync to upload a given set of files, even if it
thinks they don't need to. Such files are called ``forced''. You
specify them using either --force file-list or
The reverse operation is to avoid uploading of files WSync thinks
need to be uploaded. Such files are called ``avoided''. You specify
them using either --avoid file-list or
2.5.2 Faking synchronization
You can direct WSync to perform a ``fake sync'': this is like a
sync, but the remote server is not contacted, WSync updates its DB
juste like if the operations on the remote server had been performed
successfully. This can be useful if the remote server state has been
modified independently of WSync. For example, if you have deleted
files ``by hand'' on the remote server and then remove them from the
list file, WSync will try (and fail) to delete theses files each
time you sync. To perform a fake sync, use the -f command
line option instead of the -d one.
Fake operations are logged with an initial ``!'' character in
the log file.
2.6 FTP options
- by default, files are uploaded under a temporary
name, which is their name plus the .tmp suffix. Only when
the file has been successfully uploaded is the file renamed to its
real name. This is so that if the upload is interrupted, the
previous version of the file will still be safe on the remote
server. This command line option allows to disable this feature:
files are then uploaded directly under their name.
- disable FTP passive mode, that is set by
default. Passive mode allows FTP to work even if you're behind a
NAT, and is a bit more secure (though FTP is never secure anyway) .
You should probably never need to disable it.
2.7.1 Files states
||file with no local copy available, so freshness is unknown
||file whose timestamp has changed, but not digest's
||file removed from the list file
||file new in the list file, with no local copy
||file new in the list file
||file whose signature has changed since last upload
||file whose name has changed
||file that shouldn't be uploaded, but so required by the user
||file that should be updated, but required not by the user
2.7.2 Command line options
||doit, sync (upload/delete/rename)
files on the remote server
||show the list of lost files
||show version information
||disable FTP safeput feature
||disable passive FTP
||force upload of listed files
||force upload of files listed in file
||avoid upload of listed files
||avoid upload of files listed in file
The core of WSync is written in Objective Caml, and linked with an FTP
library written in C, derived from the traditional Linux FTP client
netkit-ftp. The OCaml Unix module is used, so it
won't run on non-Unix platforms.
Sources and compiled files are available from the
top right ``Download'' links on this page. All are distributed under the
Public Licence (GPL).
3.1 From sources
Installation from sources requires the Objective Caml compiler
version 3.06 or above, a C compiler, and the FTPlib source package. Compilation must be performed
from ftplib source directory, as shown above:
If something goes wrong, make sure the caml bindings are enabled (the
NO_CAML macro should not be defined, it is commented out in
ftplib makefile by default), and do a make clean for ftplib
tar xvzf ftplib.tgz
- tar xvzf wsync.tgz
- cd ftplib
- cp ../wsync/wsync.ml .
- for the native code version: make libftp.a ftp.cmxa ;
make -f ../wsync/Makefile
for the bytecode version: make libftp.a ftp.cma ; make -f
A binary package for i386 Linux is also available for download. This
is a native code binary, and so does not need any OCaml tool installed
Send any bug report, comment, suggestion or anything to elviok AT free DOT fr.
Make sure to mention that you're talking about WSync and give
version information if applicable.
D. J. Bernstein.
FTP: File transfer protocol.
web site. [ url ]