natalie/indefero
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Reorganize and expand the help of the monotone plugin.

Browse Source 
Make the commentary in idf.php-dist less verbose.
This commit is contained in:
Thomas Keller 2010-09-17 03:11:36 +02:00
parent 50638c768f
commit 617589f41b
3 changed files with 198 additions and 145 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

62
doc/readme-monotone.mdtext
View File

@ -1,62 +0,0 @@
# monotone implementation notes
## general
This version of indefero contains an implementation of the monotone
automation interface. It needs at least monotone version 0.99
(interface version 13.0) or newer.
To set up a new IDF project with monotone quickly, all you need to do
is to create a new monotone database with
$ mtn db init -d project.mtn
in the configured repository path `$cfg['mtn_repositories']` and
configure `$cfg['mtn_db_access']` to "local".
To have a really workable setup, this database needs an initial commit
on the configured master branch of the project. This can be done easily
with
$ mkdir tmp && touch tmp/remove_me
$ mtn import -d project.mtn -b master.branch.name \
-m "initial commit" tmp
$ rm -rf tmp
Its expected that more scripts arrive soon to automate this and other
tasks in the future for (multi)forge setups.
## current state / internals
The implementation should be fairly stable and fast, though some
information, such as individual file sizes or last change information,
won't scale well with the tree size. Its expected that the mtn
automation interface improves in this area in the future and that
these parts can then be rewritten with speed in mind.
As the idf.conf-dist explains more in detail, different access patterns
are possible to retrieve changeset data from monotone. Please refer
to the documentation there for more information.
## indefero critique:
It was not always 100% clear what some of the abstract SCM API method
wanted in return. While it helped a lot to have prior art in form of the
SVN and git implementation, the documentation of the abstract IDF_Scm
should probably still be improved.
Since branch and tag names can be of arbitrary size, it was not possible
to display them completely in the default layout. This might be a problem
in other SCMs as well, in particular for the monotone implementation I
introduced a special filter, called "IDF_Views_Source_ShortenString".
The API methods getPathInfo() and getTree() return similar VCS "objects"
which unfortunately do not have a well-defined structure - this should
probably addressed in future indefero releases.
While the returned objects from getTree() contain all the needed
information, indefero doesn't seem to use them to sort the output
f.e. alphabetically or in such a way that directories are outputted
before files. It was unclear if the SCM implementor should do this
task or not and what the admired default sorting should be.

175
doc/syncmonotone.mdtext Normal file
View File

@ -0,0 +1,175 @@
# Plugin SyncMonotone by Thomas Keller (me@thomaskeller.biz)
The SyncMonotone plugin allow the direct creation and synchronisation of
monotone repositories with the InDefero database. It has been built to
work together with monotone's "super server" usher, which is used to control
several repositories at once, acts as proxy and single entrance.
## Prerequisites
* a unixoid operating system
* monotone >= 0.99
* for a proxy setup with usher:
* boost headers (for usher compilation)
* a current version of usher
* a daemonizer, like supervise
## Installation of monotone
If you install monotone from a distribution package, ensure you do not
install and / or activate the server component. We just need a plain
client installation which usually consists only of the `mtn` binary and
a few docs.
If you install monotone from source (<http://monotone.ca/downloads.php>),
please follow the `INSTALL` document which comes with the software.
It contains detailed instructions, including all needed dependencies.
## Choose your indefero setup
The monotone plugin can be used in several different ways:
1. One database for everything. This is the easiest setup and of possible
use in case you do not want indefero to manage the access to your project.
Your `idf.php` should look like this:
$ cat idf.php
...
$cfg['mtn_path'] = 'mtn';
$cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles');
$cfg['mtn_repositories'] = '/home/monotone/all_projects.mtn';
$cfg['mtn_remote_url'] = 'ssh://monotone@my.server.com:~all_projects.mtn';
$cfg['mtn_db_access'] = 'local';
...
Pro:
* easy to setup and to manage
Con:
* you need to give committers SSH access to your machine
* database lock problem: the database from which
indefero reads its data might be locked in case a user
syncs at the very moment via SSH
2. One database for every project. Similar to the above setup, but this
time you use the '%s' placeholder which is replaced with the short name
of the indefero project:
$ cat idf.php
...
$cfg['mtn_path'] = 'mtn';
$cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles');
$cfg['mtn_repositories'] = '/home/monotone/%s.mtn';
$cfg['mtn_remote_url'] = 'ssh://monotone@my.server.com:~%s.mtn';
$cfg['mtn_db_access'] = 'local';
...
The same pro's and con's apply. Additionally you have to be careful about
not giving people physical read/write access of another project's database.
Furthermore, if you do not want to use `ssh`, but `netsync` transport,
each project's database must be served over a separate port.
3. One database for every project, all managed with usher. This is the
recommended setup for a mid-size forge setup. The remaining part of this
document will describe the process to set this up in detail.
Pro:
* access rights can be granted per project and are automatically
managed by indefero, just like the user's public monotone keys
* no database locking issues
* one public server running on the one well-known port
Con:
* harder to setup
## Installation and configuration of usher
1. Clone usher's monotone repository:
$ mtn clone "mtn://monotone.ca?net.venge.monotone.contrib.usher"
2. Compile usher:
$ autoreconf -i
$ ./configure && make
$ sudo make install
This installs the usher binary in $prefix/bin.
3. Create a new usher user:
$ adduser --system --disabled-login --home /var/lib/usher usher
4. Create the basic usher setup:
$ cd /var/lib/usher
$ mkdir projects logs
$ cat > usher.conf
userpass "admin" "<secret-password>"
adminaddr "127.0.0.1:12345"
logdir "log"
^D
$ chmod 600 usher.conf
Your indefero www user needs later write access to `usher.conf` and
`projects/`. There are two ways of setting this up:
* Make the usher user the web user, for example via Apache's `suexec`
* Use acls, like this:
$ setfacl -m u:www:rw usher.conf
$ setfacl -m d:u:www:rwx projects/
5. Wrap a daemonizer around usher, for example supervise from daemontools
(<http://cr.yp.to/damontools.html>):
$ cat > run
#!/bin/sh
cd /var/lib/usher
exec 2>&1
exec \
setuidgid usher \
usher usher.conf
^D
The service can now be started through supervise:
$ supervise /var/lib/usher
## Configuration of indefero
Based on the above setup, the configuration in `src/IDF/conf/idf.php` should
look like this:
$ cat idf.php
...
$cfg['mtn_path'] = 'mtn';
$cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles');
$cfg['mtn_repositories'] = '/var/lib/usher/projects/%s/';
$cfg['mtn_remote_url'] = 'mtn://my.server.com/%s';
$cfg['mtn_db_access'] = 'remote';
$cfg['mtn_remote_auth'] = true;
$cfg['mtn_usher_conf'] = '/var/lib/usher/usher.conf';
...
The `%s` placeholders are automatically replaced by the name of the
indefero project. The plugin assumes that every project is separated
by a distinct server name in the monotone URL (hence the use of `/%s`),
so if a user calls
$ mtn sync mtn://my.server.com/project1
then the database / repository of the indefero `project1` is used.
Note that 'mtn_remote_url' is also used as internal URI to query the data
for indefero's source view, so it *must* be a valid host!
Usher also allows the identification of a project repository by hostname,
which would allow an URL template like `mtn://%s.my.server.com`, however
the plugin does not write out the configuration which is needed for this
yet.
For even more advanced setups, usher can also be used to forward sync
requests to other remote servers for load balancing, please consult the
README file for more information.

106
src/IDF/conf/idf.php-dist
View File

@ -73,90 +73,31 @@ $cfg['git_write_remote_url'] = 'git@localhost:%s.git';
$cfg['svn_repositories'] = 'file:///home/svn/repositories/%s';
$cfg['svn_remote_url'] = 'http://localhost/svn/%s';
# Path to the monotone binary (you need mtn 0.99 or newer)
#
# You can setup monotone for use with indefero in several ways.
# Please look into doc/syncmonotone.mdtext for more information.
#
# Path to the monotone binary
$cfg['mtn_path'] = 'mtn';
# Additional options for the started monotone process
$cfg['mtn_opts'] = array('--no-workspace', '--no-standard-rcfiles');
#
# You can setup monotone for use with indefero in several ways. The
# two most-used should be:
#
# 1) One database for everything:
#
# Set 'mtn_repositories' below to a fixed database path, such as
# '/home/mtn/repositories/all_projects.mtn'
#
# Pro: - easy to setup and to manage
# Con: - while read access can be configured per-branch,
# granting write access rights to a user means that
# he can write anything in the global database
# - database lock problem: the database from which
# indefero reads its data cannot be used to serve the
# contents to the users, as the serve process locks
# the database
#
# 2) One database for every project with 'usher':
#
# Download and configure 'usher'
# (mtn clone mtn://monotone.ca?net.venge.monotone.contrib.usher)
# which acts as proxy in front of all single project databases.
# Create a basic configuration file for it and add a secret admin
# username and password. Finally, point the below variable
# 'mtn_usher_conf' to this configuration file.
#
# Then set 'mtn_remote_url' below to a string which matches your setup.
# Again, the '%s' placeholder will be expanded to the project's
# short name. Note that 'mtn_remote_url' is used as internal
# URI (to access the data for indefero) as well as external URI
# (for end users) at the same time. 'mtn_repositories' should then
# point to a directory where all project-related files (databases,
# keys, configurations) are kept, as these are automatically created
# on project creation by IDF.
#
# Example: 'mtn_repositories' is configured to be '/var/monotone/%s'
#
# - IDF tries to create /var/monotone/<projectname> as root directory
# - The database is placed in as /var/monotone/<projectname>/database.mtn
# - The server key is put into /var/monotone/<projectname>/keys and
# is named "<projectname>-server@<host>", where host is the host part
# of 'mtn_remote_url'
#
# therefor /var/monotone MUST be read/writable for the www user and all
# files which are created underknees MUST be read/writable by the user
# who is executing the usher instance! The best way to achieve this is with
# default (POSIX) ACLs on /var/monotone.
#
#
# You could also choose to setup usher by hand, i.e. with individual
# databases, in this case leave 'mtn_usher_conf' below commented out.
#
# Pro: - read and write access can be granted per project
# - no database locking issues
# - one public server running on the one well-known port
# Con: - harder to setup
#
# Usher can also be used to forward sync requests to remote servers,
# please consult its README file for more information.
#
# monotone also allows to use SSH as transport protocol, so if you do not plan
# to setup a netsync server as described above, then just enter a URI like
# 'ssh://my-host.biz/home/mtn/repositories/%s.mtn' in 'mtn_remote_url'.
#
# The path to a specific database (local use) or a writable project
# directory (remote / usher use). %s is replaced with the project name
$cfg['mtn_repositories'] = '/home/mtn/repositories/%s.mtn';
# The URL which is displayed as sync URL to the user and which is also
# used to connect to a remote usher
$cfg['mtn_remote_url'] = 'mtn://my-host.biz/%s';
#
# Whether the particular database(s) are accessed locally (via automate stdio)
# or remotely (via automate remote_stdio). 'remote' is the default for
# netsync setups, while 'local' access should be choosed for ssh access.
#
# Note that you need to setup the hook 'get_remote_automate_permitted' for
# each remotely accessible database. A full HOWTO set this up is beyond this
# scope, please refer to the documentation of monotone and / or ask on the
# mailing list (monotone-users@nongnu.org) or IRC channel
# (irc.oftc.net/#monotone)
#
$cfg['mtn_db_access'] = 'remote';
#
# use with usher and the SyncMonotone plugin, while 'local' access should be
# choosed for manual setups and / or ssh access.
$cfg['mtn_db_access'] = 'local';
# If true, each access to the database is authenticated with an auto-generated
# project key which is stored in the IDF project configuration
# ('mtn_client_key_*') and written out to $cfg['tmp_folder']/mtn-client-keys
@ -167,14 +108,13 @@ $cfg['mtn_db_access'] = 'remote';
# the remote monotone server instance. In this case no project-specific
# keys are generated and the server must be configured to allow at least
# anonymous read access to the main functions.
#
$cfg['mtn_remote_auth'] = true;
#
# If configured, this allows basic control of a running usher process
# via the forge administration. The variable must point to the full (writable)
#$cfg['mtn_remote_auth'] = true;
# Needs to be configured for remote / usher usage.
# This allows basic control of a running usher process via the forge
# administration. The variable must point to the full (writable)
# path of the usher configuration file which gets updated when new projects
# are added
#
#$cfg['mtn_usher_conf'] = '/path/to/usher.conf';
# Mercurial repositories path