# Plugin SyncMonotone by Thomas Keller (me@thomaskeller.biz) The SyncMonotone plugin allow the direct creation and synchronisation of monotone repositories with the InDefero database. It has been built to work together with monotone's "super server" usher, which is used to control several repositories at once, acts as proxy and single entrance. ## Prerequisites * a unixoid operating system * monotone >= 0.99.1 * for a proxy setup with usher: * boost headers (for usher compilation) * a current version of usher * a daemonizer, like supervise ## Installation of monotone If you install monotone from a distribution package, ensure you do not install and / or activate the server component. We just need a plain client installation which usually consists only of the `mtn` binary and a few docs. If you install monotone from source (), please follow the `INSTALL` document which comes with the software. It contains detailed instructions, including all needed dependencies. ## Choose your indefero (IDF) setup The monotone plugin can be used in several different ways: 1. One database for everything. This is the easiest setup and of possible use in case you do not want indefero to manage the access to your project. Your `idf.php` should look like this: $ cat idf.php ... $cfg['mtn_path'] = 'mtn'; $cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles'); $cfg['mtn_repositories'] = '/home/monotone/all_projects.mtn'; $cfg['mtn_remote_url'] = 'ssh://monotone@my.server.com:~all_projects.mtn'; $cfg['mtn_db_access'] = 'local'; ... Pro: * easy to setup and to manage Con: * you need to give committers SSH access to your machine * database lock problem: the database from which indefero reads its data might be locked in case a user syncs at the very moment via SSH 2. One database for every project. Similar to the above setup, but this time you use the '%s' placeholder which is replaced with the short name of the indefero project: $ cat idf.php ... $cfg['mtn_path'] = 'mtn'; $cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles'); $cfg['mtn_repositories'] = '/home/monotone/%s.mtn'; $cfg['mtn_remote_url'] = 'ssh://monotone@my.server.com:~%s.mtn'; $cfg['mtn_db_access'] = 'local'; ... The same pro's and con's apply. Additionally you have to be careful about not giving people physical read/write access of another project's database. Furthermore, if you do not want to use `ssh`, but `netsync` transport, each project's database must be served over a separate port. 3. One database for every project, all managed with usher. This is the recommended setup for a mid-size forge setup. The remaining part of this document will describe the process to set this up in detail. Pro: * access rights can be granted per project and are automatically managed by indefero, just like the user's public monotone keys * no database locking issues * one public server running on the one well-known port Con: * harder to setup ## Installation and configuration of usher 1. Clone usher's monotone repository: $ mtn clone "mtn://monotone.ca?net.venge.monotone.contrib.usher" 2. Compile usher: $ autoreconf -i $ ./configure && make $ sudo make install This installs the usher binary in $prefix/bin. 3. Create a new usher user: $ adduser --system --disabled-login --home /var/lib/usher usher 4. Create the basic usher setup: $ cd /var/lib/usher $ mkdir projects logs $ cat > usher.conf userpass "admin" "" adminaddr "127.0.0.1:12345" logdir "log" ^D $ chmod 600 usher.conf **ATTENTION:** Do _not_ configure a default monotone server key in `usher.conf`, otherwise the individual server key that IDF creates for each project is not used and this could cause problems. Your indefero www user needs later write access to `usher.conf` and `projects/`. There are two ways of setting this up: * Make the usher user the web user, for example via Apache's `suexec`. This is however a bit clumsy. * Preferred: Use Access Control Lists (ACLs), like this: # # Linux # $ setfacl -m u:www:rw usher.conf $ setfacl -m d:u:www:rwx projects/ $ setfacl -m d:u:usher:rwx projects/ # # FreeBSD # $ setfacl -m user:www:rw::allow usher.conf $ setfacl -m user:www:rwxp:fd:allow projects/ $ setfacl -m user:usher:rwxp:fd:allow projects/ # # Mac OS X # chmod +a '_www allow read,write' usher.conf chmod +a '_www allow read,write,delete,add_file,add_subdirectory,file_inherit,directory_inherit' projects/ chmod +a 'usher allow read,write,delete,add_file,add_subdirectory,file_inherit,directory_inherit' projects/ In each example's last line, `usher` is the user which is executing the usher instance. **It is very important to add this line, otherwise usher won't be able to read and write into the initial file system setup IDF creates!** 5. Wrap a daemonizer around usher, for example supervise from daemontools (): $ cat > run #!/bin/sh cd /var/lib/usher exec 2>&1 exec \ setuidgid usher \ usher usher.conf ^D The service can now be started through supervise: $ supervise /var/lib/usher ## Configuration of indefero Based on the above setup, the configuration in `src/IDF/conf/idf.php` should look like this: $ cat idf.php ... $cfg['mtn_path'] = 'mtn'; $cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles'); $cfg['mtn_repositories'] = '/var/lib/usher/projects/%s/'; $cfg['mtn_remote_url'] = 'mtn://my.server.com/%s'; $cfg['mtn_db_access'] = 'remote'; $cfg['mtn_usher_conf'] = '/var/lib/usher/usher.conf'; ... The `%s` placeholders are automatically replaced by the name of the indefero project. The plugin assumes that every project is separated by a distinct server name in the monotone URL (hence the use of `/%s`), so if a user calls $ mtn sync mtn://my.server.com/project1 then the database / repository of the indefero `project1` is used. Note that 'mtn_remote_url' is also used as internal URI to query the data for indefero's source view, so it *must* be a valid host! Usher also allows the identification of a project repository by hostname, which would allow an URL template like `mtn://%s.my.server.com`, however the plugin does not write out the configuration which is needed for this yet. For even more advanced setups, usher can also be used to forward sync requests to other remote servers for load balancing, please consult the README file for more information. ## Security and remote access Indefero distinguishs between public and private projects and so does the monotone plugin. Public projects can be pulled by everybody and pushed by team members or additional invited people. Remote command execution is enabled, but only for read-only commands. Remote commands can be helpful for a user or a 3rd party tool (like [mtn-browse](http://mtn-browse.sourceforge.net) or [guitone](http://guitone.thomaskeller.biz)) to browse the database contents remotely without having to pull everything in first instance. Private projects on the other hand can only be synced by team members or additional invited people. Remote command execution is disabled by default. If you want to enable that, simply put the keys of the users you want to give access to in your project's `remote-automate-permissions` file. In the future this plugin might handle this file just as it handles `read-permissions` and `write-permissions`. ## Notifications If you have successfully set up your monotone instance, you probably want to notify 3rd party systems for incoming changes or simply mirror them somewhere else for backup purposes. The monotone source tree already comes with [many example scripts and hooks](http://code.monotone.ca/p/monotone/source/tree/h:net.venge.monotone/contrib) which serve these purposes, after only little additional configuration. The usher/indefero-controlled setup automatically looks for *.lua files in a directory called `hooks.d` right under the project's base directory (configured via $cfg['mtn_repositories']) and this is the ideal place to put or link these additional lua sources. ## Custom project configurations and templates If a new project is created in IDF, the SyncMonotone plugin creates a new configuration tree for the project into the project's configuration directory, determined by `$cfg['mtn_repositories']`. IDF ships with the minimum set of files for this configuration tree and sets up everything automatically for you. Even more, most of the configuration files from the newly created tree are only symlinked to the original configuration directory which is configurable via `$cfg['mtn_confdir']` and defaults to `src/IDF/Plugin/SyncMonotone/`. This has the advantage that your standard IDF setup automatically receives updates to existing (symlinked) configuration files as soon as you update to a newer version. You could, however, also choose to place the directory tree somewhere else and adapt the contents of the individual files yourself, so these changes get automatically applied to all new projects you create. You could even go so far and add new files to the tree and let them be processed automatically just as the basic files! All you need to do is to copy your files and / or directories underknees your `$cfg['mtn_confdir']` and add their relative paths to `$cfg['mtn_confdir_extra']`. By convention, all entries which end with a slash are considered directories, so mkdir(1) is issued for these entries, all files which do not end up with ".in" are considered to be static script files which are just symlinked from the basic configuration dir and all entries ending on ".in" are considered configuration files or templates, which are copied over to the project's configuration tree and which get some basic project-specific values replaced. The following placeholders are currently recognized and replaced for these files: * %%PROJECT%% - the name of the created project * %%MTNPOSTPUSH%% - the absolute path to the `mtn-post-push` script * %%MTNCLIENTKEY%% - the public key hash of the key which is used by IDF to authenticate remote stdio access Thats it - I hope you find it useful :) ## Q&A ### After I created a new project, IDF throws an exception and tells me that it couldn't save the membership data with a cryptic error message. Whats wrong? Multiple issues could cause that. If you've set up usher, make sure the usher can fork your database at all and look out for specific errors in the log file of your project. If you stumble upon permission issues, ensure that the user who runs the usher can access all files in your project's configuration directory, including symlinked files. ### I pushed a branch to my server, but it does not show up in IDF. Whats wrong? Check if the heads of your branch are not suspended, i.e. do not carry a `suspend` certificate. This usually hides the branch and all of its history from monotone's eyes and therefor also from indefero. You can either choose to "unsuspend" the branch simply by committing and pushing another head or by letting monotone ignore all suspend certs. For the latter, its usually enough to add `--ignore-suspend-certs` to the list of options in `$cfg['mtn_opts']`. ### I want to display another default branch when I click the "Source" tab. How can I do that? Let the forge admin know the new master branch for your project. He is able to change that quickly. Depending on the backend / server setup this might also require some changes in the usher configuration, but only if usher recognizes and proxies your database on a branch name level.