diff --git a/doc/readme-monotone.mdtext b/doc/readme-monotone.mdtext deleted file mode 100644 index cbaf8ba..0000000 --- a/doc/readme-monotone.mdtext +++ /dev/null @@ -1,62 +0,0 @@ -# monotone implementation notes - -## general - - This version of indefero contains an implementation of the monotone - automation interface. It needs at least monotone version 0.99 - (interface version 13.0) or newer. - - To set up a new IDF project with monotone quickly, all you need to do - is to create a new monotone database with - - $ mtn db init -d project.mtn - - in the configured repository path `$cfg['mtn_repositories']` and - configure `$cfg['mtn_db_access']` to "local". - - To have a really workable setup, this database needs an initial commit - on the configured master branch of the project. This can be done easily - with - - $ mkdir tmp && touch tmp/remove_me - $ mtn import -d project.mtn -b master.branch.name \ - -m "initial commit" tmp - $ rm -rf tmp - - Its expected that more scripts arrive soon to automate this and other - tasks in the future for (multi)forge setups. - -## current state / internals - - The implementation should be fairly stable and fast, though some - information, such as individual file sizes or last change information, - won't scale well with the tree size. Its expected that the mtn - automation interface improves in this area in the future and that - these parts can then be rewritten with speed in mind. - - As the idf.conf-dist explains more in detail, different access patterns - are possible to retrieve changeset data from monotone. Please refer - to the documentation there for more information. - -## indefero critique: - - It was not always 100% clear what some of the abstract SCM API method - wanted in return. While it helped a lot to have prior art in form of the - SVN and git implementation, the documentation of the abstract IDF_Scm - should probably still be improved. - - Since branch and tag names can be of arbitrary size, it was not possible - to display them completely in the default layout. This might be a problem - in other SCMs as well, in particular for the monotone implementation I - introduced a special filter, called "IDF_Views_Source_ShortenString". - - The API methods getPathInfo() and getTree() return similar VCS "objects" - which unfortunately do not have a well-defined structure - this should - probably addressed in future indefero releases. - - While the returned objects from getTree() contain all the needed - information, indefero doesn't seem to use them to sort the output - f.e. alphabetically or in such a way that directories are outputted - before files. It was unclear if the SCM implementor should do this - task or not and what the admired default sorting should be. - diff --git a/doc/syncmonotone.mdtext b/doc/syncmonotone.mdtext new file mode 100644 index 0000000..89c1350 --- /dev/null +++ b/doc/syncmonotone.mdtext @@ -0,0 +1,175 @@ +# Plugin SyncMonotone by Thomas Keller (me@thomaskeller.biz) + +The SyncMonotone plugin allow the direct creation and synchronisation of +monotone repositories with the InDefero database. It has been built to +work together with monotone's "super server" usher, which is used to control +several repositories at once, acts as proxy and single entrance. + +## Prerequisites + +* a unixoid operating system +* monotone >= 0.99 +* for a proxy setup with usher: + * boost headers (for usher compilation) + * a current version of usher + * a daemonizer, like supervise + +## Installation of monotone + +If you install monotone from a distribution package, ensure you do not +install and / or activate the server component. We just need a plain +client installation which usually consists only of the `mtn` binary and +a few docs. + +If you install monotone from source (), +please follow the `INSTALL` document which comes with the software. +It contains detailed instructions, including all needed dependencies. + +## Choose your indefero setup + +The monotone plugin can be used in several different ways: + +1. One database for everything. This is the easiest setup and of possible + use in case you do not want indefero to manage the access to your project. + Your `idf.php` should look like this: + + $ cat idf.php + ... + $cfg['mtn_path'] = 'mtn'; + $cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles'); + $cfg['mtn_repositories'] = '/home/monotone/all_projects.mtn'; + $cfg['mtn_remote_url'] = 'ssh://monotone@my.server.com:~all_projects.mtn'; + $cfg['mtn_db_access'] = 'local'; + ... + + Pro: + * easy to setup and to manage + + Con: + * you need to give committers SSH access to your machine + * database lock problem: the database from which + indefero reads its data might be locked in case a user + syncs at the very moment via SSH + +2. One database for every project. Similar to the above setup, but this + time you use the '%s' placeholder which is replaced with the short name + of the indefero project: + + $ cat idf.php + ... + $cfg['mtn_path'] = 'mtn'; + $cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles'); + $cfg['mtn_repositories'] = '/home/monotone/%s.mtn'; + $cfg['mtn_remote_url'] = 'ssh://monotone@my.server.com:~%s.mtn'; + $cfg['mtn_db_access'] = 'local'; + ... + + The same pro's and con's apply. Additionally you have to be careful about + not giving people physical read/write access of another project's database. + + Furthermore, if you do not want to use `ssh`, but `netsync` transport, + each project's database must be served over a separate port. + +3. One database for every project, all managed with usher. This is the + recommended setup for a mid-size forge setup. The remaining part of this + document will describe the process to set this up in detail. + + Pro: + * access rights can be granted per project and are automatically + managed by indefero, just like the user's public monotone keys + * no database locking issues + * one public server running on the one well-known port + + Con: + * harder to setup + +## Installation and configuration of usher + +1. Clone usher's monotone repository: + + $ mtn clone "mtn://monotone.ca?net.venge.monotone.contrib.usher" + +2. Compile usher: + + $ autoreconf -i + $ ./configure && make + $ sudo make install + + This installs the usher binary in $prefix/bin. + +3. Create a new usher user: + + $ adduser --system --disabled-login --home /var/lib/usher usher + +4. Create the basic usher setup: + + $ cd /var/lib/usher + $ mkdir projects logs + $ cat > usher.conf + userpass "admin" "" + adminaddr "127.0.0.1:12345" + logdir "log" + ^D + $ chmod 600 usher.conf + + Your indefero www user needs later write access to `usher.conf` and + `projects/`. There are two ways of setting this up: + + * Make the usher user the web user, for example via Apache's `suexec` + * Use acls, like this: + + $ setfacl -m u:www:rw usher.conf + $ setfacl -m d:u:www:rwx projects/ + +5. Wrap a daemonizer around usher, for example supervise from daemontools + (): + + $ cat > run + #!/bin/sh + cd /var/lib/usher + exec 2>&1 + exec \ + setuidgid usher \ + usher usher.conf + ^D + + The service can now be started through supervise: + + $ supervise /var/lib/usher + +## Configuration of indefero + +Based on the above setup, the configuration in `src/IDF/conf/idf.php` should +look like this: + + $ cat idf.php + ... + $cfg['mtn_path'] = 'mtn'; + $cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles'); + $cfg['mtn_repositories'] = '/var/lib/usher/projects/%s/'; + $cfg['mtn_remote_url'] = 'mtn://my.server.com/%s'; + $cfg['mtn_db_access'] = 'remote'; + $cfg['mtn_remote_auth'] = true; + $cfg['mtn_usher_conf'] = '/var/lib/usher/usher.conf'; + ... + +The `%s` placeholders are automatically replaced by the name of the +indefero project. The plugin assumes that every project is separated +by a distinct server name in the monotone URL (hence the use of `/%s`), +so if a user calls + + $ mtn sync mtn://my.server.com/project1 + +then the database / repository of the indefero `project1` is used. +Note that 'mtn_remote_url' is also used as internal URI to query the data +for indefero's source view, so it *must* be a valid host! + +Usher also allows the identification of a project repository by hostname, +which would allow an URL template like `mtn://%s.my.server.com`, however +the plugin does not write out the configuration which is needed for this +yet. + +For even more advanced setups, usher can also be used to forward sync +requests to other remote servers for load balancing, please consult the +README file for more information. + diff --git a/src/IDF/conf/idf.php-dist b/src/IDF/conf/idf.php-dist index d4bcd5c..54c8ac8 100644 --- a/src/IDF/conf/idf.php-dist +++ b/src/IDF/conf/idf.php-dist @@ -73,90 +73,31 @@ $cfg['git_write_remote_url'] = 'git@localhost:%s.git'; $cfg['svn_repositories'] = 'file:///home/svn/repositories/%s'; $cfg['svn_remote_url'] = 'http://localhost/svn/%s'; -# Path to the monotone binary (you need mtn 0.99 or newer) +# +# You can setup monotone for use with indefero in several ways. +# Please look into doc/syncmonotone.mdtext for more information. +# + +# Path to the monotone binary $cfg['mtn_path'] = 'mtn'; + # Additional options for the started monotone process $cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles'); -# -# You can setup monotone for use with indefero in several ways. The -# two most-used should be: -# -# 1) One database for everything: -# -# Set 'mtn_repositories' below to a fixed database path, such as -# '/home/mtn/repositories/all_projects.mtn' -# -# Pro: - easy to setup and to manage -# Con: - while read access can be configured per-branch, -# granting write access rights to a user means that -# he can write anything in the global database -# - database lock problem: the database from which -# indefero reads its data cannot be used to serve the -# contents to the users, as the serve process locks -# the database -# -# 2) One database for every project with 'usher': -# -# Download and configure 'usher' -# (mtn clone mtn://monotone.ca?net.venge.monotone.contrib.usher) -# which acts as proxy in front of all single project databases. -# Create a basic configuration file for it and add a secret admin -# username and password. Finally, point the below variable -# 'mtn_usher_conf' to this configuration file. -# -# Then set 'mtn_remote_url' below to a string which matches your setup. -# Again, the '%s' placeholder will be expanded to the project's -# short name. Note that 'mtn_remote_url' is used as internal -# URI (to access the data for indefero) as well as external URI -# (for end users) at the same time. 'mtn_repositories' should then -# point to a directory where all project-related files (databases, -# keys, configurations) are kept, as these are automatically created -# on project creation by IDF. -# -# Example: 'mtn_repositories' is configured to be '/var/monotone/%s' -# -# - IDF tries to create /var/monotone/ as root directory -# - The database is placed in as /var/monotone//database.mtn -# - The server key is put into /var/monotone//keys and -# is named "-server@", where host is the host part -# of 'mtn_remote_url' -# -# therefor /var/monotone MUST be read/writable for the www user and all -# files which are created underknees MUST be read/writable by the user -# who is executing the usher instance! The best way to achieve this is with -# default (POSIX) ACLs on /var/monotone. -# -# -# You could also choose to setup usher by hand, i.e. with individual -# databases, in this case leave 'mtn_usher_conf' below commented out. -# -# Pro: - read and write access can be granted per project -# - no database locking issues -# - one public server running on the one well-known port -# Con: - harder to setup -# -# Usher can also be used to forward sync requests to remote servers, -# please consult its README file for more information. -# -# monotone also allows to use SSH as transport protocol, so if you do not plan -# to setup a netsync server as described above, then just enter a URI like -# 'ssh://my-host.biz/home/mtn/repositories/%s.mtn' in 'mtn_remote_url'. -# + +# The path to a specific database (local use) or a writable project +# directory (remote / usher use). %s is replaced with the project name $cfg['mtn_repositories'] = '/home/mtn/repositories/%s.mtn'; + +# The URL which is displayed as sync URL to the user and which is also +# used to connect to a remote usher $cfg['mtn_remote_url'] = 'mtn://my-host.biz/%s'; -# + # Whether the particular database(s) are accessed locally (via automate stdio) # or remotely (via automate remote_stdio). 'remote' is the default for -# netsync setups, while 'local' access should be choosed for ssh access. -# -# Note that you need to setup the hook 'get_remote_automate_permitted' for -# each remotely accessible database. A full HOWTO set this up is beyond this -# scope, please refer to the documentation of monotone and / or ask on the -# mailing list (monotone-users@nongnu.org) or IRC channel -# (irc.oftc.net/#monotone) -# -$cfg['mtn_db_access'] = 'remote'; -# +# use with usher and the SyncMonotone plugin, while 'local' access should be +# choosed for manual setups and / or ssh access. +$cfg['mtn_db_access'] = 'local'; + # If true, each access to the database is authenticated with an auto-generated # project key which is stored in the IDF project configuration # ('mtn_client_key_*') and written out to $cfg['tmp_folder']/mtn-client-keys @@ -167,14 +108,13 @@ $cfg['mtn_db_access'] = 'remote'; # the remote monotone server instance. In this case no project-specific # keys are generated and the server must be configured to allow at least # anonymous read access to the main functions. -# -$cfg['mtn_remote_auth'] = true; -# -# If configured, this allows basic control of a running usher process -# via the forge administration. The variable must point to the full (writable) +#$cfg['mtn_remote_auth'] = true; + +# Needs to be configured for remote / usher usage. +# This allows basic control of a running usher process via the forge +# administration. The variable must point to the full (writable) # path of the usher configuration file which gets updated when new projects # are added -# #$cfg['mtn_usher_conf'] = '/path/to/usher.conf'; # Mercurial repositories path