Major configuration changes for SyncMonotone - we're now using a predefined

configuration tree as template for a new project and copy / symlink that on
project creation. To make this process a little more configurable, two new
configuration options, 'mtn_confdir' and 'mtn_confdir_extra', have been
added which allow the forge admin to adapt the directory structure and its
default hooks to his likings for all new projects. (More on that in
doc/syncmonotone.mdtext).
The 'mtn_remote_auth' configuration option was removed, because setting this
to false would have not worked for setups which did not allow write access
to remote automate commands for anonymous users and opening this would have
meant a huge security hole. Instead, for every project which is created a
corresponding client key is created as well which is used as authentication
in the IDF source frontend.
Finally the monolithic monotonerc file has been split up into individual,
easily configurable lua files which are linked / copied underknees hooks.d/
and which do not conflict with each other (for example by overwriting certain
main notification hooks).
This commit is contained in:
Thomas Keller 2011-01-06 01:44:41 +01:00
parent a437da6a4c
commit dd05a58c8c
12 changed files with 372 additions and 267 deletions

View File

@ -149,7 +149,6 @@ look like this:
$cfg['mtn_repositories'] = '/var/lib/usher/projects/%s/';
$cfg['mtn_remote_url'] = 'mtn://my.server.com/%s';
$cfg['mtn_db_access'] = 'remote';
$cfg['mtn_remote_auth'] = true;
$cfg['mtn_usher_conf'] = '/var/lib/usher/usher.conf';
...
@ -188,8 +187,10 @@ Remote commands can be helpful for a user or a 3rd party tool (like
contents remotely without having to pull everything in first instance.
Private projects on the other hand can only be synced by team members
or additional invited people. Also noo remote command execution is enabled
by default.
or additional invited people. Remote command execution is still enabled
by default - if you want to disable that, simply remove the symlink to
the file `indefero_authorize_remote_automate.conf` in your project's `hooks.d`
directory or copy the file from the original location and adapt it.
## Notifications
@ -204,8 +205,54 @@ in a directory called `hooks.d` right under the project's base directory
(configured via $cfg['mtn_repositories']) and this is the ideal place to
put or link these additional lua sources.
## Custom project configurations and templates
If a new project is created in IDF, the SyncMonotone plugin creates a new
configuration tree for the project into the project's configuration directory,
determined by `$cfg['mtn_repositories']`. IDF ships with the minimum set of
files for this configuration tree and sets up everything automatically for you.
Even more, most of the configuration files from the newly created tree are only
symlinked to the original configuration directory which is configurable via
`$cfg['mtn_confdir']` and defaults to `src/IDF/Plugin/SyncMonotone/`. This has
the advantage that your standard IDF setup automatically receives updates to
existing (symlinked) configuration files as soon as you update to a newer
version.
You could, however, also choose to place the directory tree somewhere else
and adapt the contents of the individual files yourself, so these changes get
automatically applied to all new projects you create. You could even go so far
and add new files to the tree and let them be processed automatically just
as the basic files! All you need to do is to copy your files and / or directories
underknees your `$cfg['mtn_confdir']` and add their relative paths to
`$cfg['mtn_confdir_extra']`.
By convention, all entries which end with a slash are considered directories,
so mkdir(1) is issued for these entries, all files which do not end up with
".in" are considered to be static script files which are just symlinked from
the basic configuration dir and all entries ending on ".in" are considered
configuration files or templates, which are copied over to the project's
configuration tree and which get some basic project-specific values replaced.
The following placeholders are currently recognized and replaced for these files:
* %%PROJECT%% - the name of the created project
* %%MTNPOSTPUSH%% - the absolute path to the `mtn-post-push` script
* %%MTNCLIENTKEY%% - the public key hash of the key which is used by IDF
to authenticate remote stdio access
Thats it - I hope you find it useful :)
## Q&A
### After I created a new project, IDF throws an exception and tells me that it couldn't save the membership data with a cryptic error message. Whats wrong?
Multiple issues could cause that. If you've set up usher, make sure the usher
can fork your database at all and look out for specific errors in the log file
of your project. If you stumble upon permission issues, ensure that the user
who runs the usher can access all files in your project's configuration directory,
including symlinked files.
### I pushed a branch to my server, but it does not show up in IDF. Whats wrong?
Check if the heads of your branch are not suspended, i.e. do not carry a

View File

@ -62,7 +62,7 @@ class IDF_Plugin_SyncMonotone
* 'mtn_repositories'
* 2) create a new server key in the same directory
* 3) create a new client key for IDF and store it in the project conf
* 4) write monotonerc
* 4) setup the configuration
* 5) add the database as new local server in the usher configuration
* 6) reload the running usher instance so it acknowledges the new server
*
@ -101,6 +101,36 @@ class IDF_Plugin_SyncMonotone
));
}
// check some static configuration files
$confdir = Pluf::f('mtn_confdir', false);
if ($confdir === false) {
$confdir = dirname(__FILE__).'/SyncMonotone/';
}
$confdir_contents = array(
'monotonerc.in',
'remote-automate-permissions.in',
'hooks.d/',
// this is linked and not copied to be able to update
// the list of read-only commands on upgrades
'hooks.d/indefero_authorize_remote_automate.conf',
'hooks.d/indefero_authorize_remote_automate.lua',
'hooks.d/indefero_post_push.conf.in',
'hooks.d/indefero_post_push.lua',
);
// check whether we should handle additional files in the config directory
$confdir_extra_contents = Pluf::f('mtn_confdir_extra', false);
if ($confdir_extra_contents !== false) {
$confdir_contents =
array_merge($confdir_contents, $confdir_extra_contents);
}
foreach ($confdir_contents as $content) {
if (!file_exists($confdir.$content)) {
throw new IDF_Scm_Exception(sprintf(
__('The configuration file %s is missing.'), $content
));
}
}
$shortname = $project->shortname;
$projectpath = sprintf($projecttempl, $shortname);
if (file_exists($projectpath)) {
@ -144,11 +174,6 @@ class IDF_Plugin_SyncMonotone
//
// step 3) create a client key, and save it in IDF
//
$clientkey_hash = '';
$monotonerc_tpl = 'monotonerc-noauth.tpl';
if (Pluf::f('mtn_remote_auth', true)) {
$monotonerc_tpl = 'monotonerc-auth.tpl';
$keydir = Pluf::f('tmp_folder').'/mtn-client-keys';
if (!file_exists($keydir)) {
if (!mkdir($keydir)) {
@ -195,27 +220,50 @@ class IDF_Plugin_SyncMonotone
escapeshellarg($clientkey_pubdata)
);
self::_mtn_exec($cmd);
//
// step 4) setup the configuration
//
// we assume that all confdir entries ending with a slash mean a
// directory that has to be created, that all files ending on ".in"
// have to be processed and copied in place and that all other files
// just need to be symlinked from the original location
foreach ($confdir_contents as $content) {
$filepath = $projectpath.'/'.$content;
if (substr($content, -1) == '/') {
if (!mkdir($filepath)) {
throw new IDF_Scm_Exception(sprintf(
__('Could not create configuration directory "%s"'), $filepath
));
}
continue;
}
//
// step 4) write monotonerc
//
$monotonerc = file_get_contents(
dirname(__FILE__).'/SyncMonotone/'.$monotonerc_tpl
);
$monotonerc = str_replace(
if (substr($content, -3) != '.in') {
if (!symlink($confdir.$content, $filepath)) {
IDF_Scm_Exception(sprintf(
__('Could not create symlink "%s"'), $filepath
));
}
continue;
}
$filecontents = file_get_contents($confdir.'/'.$content);
$filecontents = str_replace(
array('%%MTNPOSTPUSH%%', '%%PROJECT%%', '%%MTNCLIENTKEY%%'),
array($mtnpostpush, $shortname, $clientkey_hash),
$monotonerc
$filecontents
);
$rcfile = $projectpath.'/monotonerc';
if (file_put_contents($rcfile, $monotonerc, LOCK_EX) === false) {
// remove the .in
$filepath = substr($filepath, 0, -3);
if (file_put_contents($filepath, $filecontents, LOCK_EX) === false) {
throw new IDF_Scm_Exception(sprintf(
__('Could not write mtn configuration file "%s"'), $rcfile
__('Could not write configuration file "%s"'), $filepath
));
}
}
//
// step 5) read in and append the usher config with the new server
@ -378,7 +426,6 @@ class IDF_Plugin_SyncMonotone
}
}
if (Pluf::f('mtn_remote_auth', true)) {
$keydir = Pluf::f('tmp_folder').'/mtn-client-keys';
$keyname = $project->getConf()->getVal('mtn_client_key_name', false);
$keyhash = $project->getConf()->getVal('mtn_client_key_hash', false);
@ -390,7 +437,6 @@ class IDF_Plugin_SyncMonotone
));
}
}
}
$usher_rc = file_get_contents($usher_config);
$parsed_config = array();
@ -721,7 +767,7 @@ class IDF_Plugin_SyncMonotone
private static function _delete_recursive($path)
{
if (is_file($path)) {
if (is_file($path) || is_link($path)) {
return @unlink($path);
}

View File

@ -0,0 +1,10 @@
ARA_safe_commands = {
"get_corresponding_path", "get_content_changed", "tags", "branches",
"common_ancestors", "packet_for_fdelta", "packet_for_fdata",
"packets_for_certs", "packet_for_rdata", "get_manifest_of",
"get_revision", "select", "graph", "children", "parents", "roots",
"leaves", "ancestry_difference", "toposort", "erase_ancestors",
"descendents", "ancestors", "heads", "get_file_of", "get_file",
"interface_version", "get_attributes", "content_diff",
"file_merge", "show_conflicts", "certs", "keys", "get_extended_manifest_of"
}

View File

@ -0,0 +1,88 @@
-- ***** BEGIN LICENSE BLOCK *****
-- This file is part of InDefero, an open source project management application.
-- Copyright (C) 2011 Céondo Ltd and contributors.
-- Copyright (C) 2010 Thomas Keller <me@thomaskeller.biz>
-- Richard Levitte <richard@levitte.org>
--
-- InDefero is free software; you can redistribute it and/or modify
-- it under the terms of the GNU General Public License as published by
-- the Free Software Foundation; either version 2 of the License, or
-- (at your option) any later version.
--
-- InDefero is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU General Public License for more details.
--
-- You should have received a copy of the GNU General Public License
-- along with this program; if not, write to the Free Software
-- Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
--
-- ***** END LICENSE BLOCK *****
--
-- This script reads key identities from a file "remote-automate-permissions"
-- in the configuration directory and permits those authenticating with one
-- of those keys to perform dangerous (read/write) remote automate operations.
-- The format of the file is very simple, one key identity on every line.
-- Lines starting with # are ignore, as well as empty lines.
--
-- It's possible to configure this script to allow the performance of some
-- remote automate commands anonymously, through the variable
-- ARA_safe_commands, which has to be a table of commands as strings.
-- One example configuration, taken from the setup at code.monotone.ca, could
-- be this:
--
-- ARA_safe_commands = {
-- "get_corresponding_path", "get_content_changed", "tags", "branches",
-- "common_ancestors", "packet_for_fdelta", "packet_for_fdata",
-- "packets_for_certs", "packet_for_rdata", "get_manifest_of",
-- "get_revision", "select", "graph", "children", "parents", "roots",
-- "leaves", "ancestry_difference", "toposort", "erase_ancestors",
-- "descendents", "ancestors", "heads", "get_file_of", "get_file",
-- "interface_version", "get_attributes", "content_diff",
-- "file_merge", "show_conflicts", "certs", "keys", "get_extended_manifest_of"
-- }
--
do
local _safe_commands = {}
if ARA_safe_commands then
_safe_commands = ARA_safe_commands
end
local _save_get_remote_automate_permitted = get_remote_automate_permitted
function get_remote_automate_permitted(key_identity, command, options)
local permfile =
io.open(get_confdir() .. "/remote-automate-permissions", "r")
if (permfile == nil) then
return false
end
-- See if the incoming key matches any of the key identities or
-- patterns found in the permissions file.
local matches = false
local line = permfile:read()
while (not matches and line ~= nil) do
if not globish_match("#*", line) then
local _, _, ln = string.find(line, "%s*([^%s]*)%s*")
if ln == "*" then matches = true end
if ln == key_identity.id then matches = true end
if globish_match(ln, key_identity.name) then matches = true end
line = permfile:read()
end
end
io.close(permfile)
if matches then return true end
-- No matching key found, let's see if the command matches one the
-- admin allowed to be performed anonymously
for _,v in ipairs(_safe_commands) do
if (v == command[1]) then
return true
end
end
-- No matches found anywhere, then don't permit this operation
return false
end
end

View File

@ -0,0 +1,2 @@
IDF_project = "%%PROJECT%%"
IDF_push_script = "%%MTNPOSTPUSH%%"

View File

@ -0,0 +1,58 @@
-- ***** BEGIN LICENSE BLOCK *****
-- This file is part of InDefero, an open source project management application.
-- Copyright (C) 2011 Céondo Ltd and contributors.
--
-- InDefero is free software; you can redistribute it and/or modify
-- it under the terms of the GNU General Public License as published by
-- the Free Software Foundation; either version 2 of the License, or
-- (at your option) any later version.
--
-- InDefero is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU General Public License for more details.
--
-- You should have received a copy of the GNU General Public License
-- along with this program; if not, write to the Free Software
-- Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
--
-- ***** END LICENSE BLOCK *****
--
-- let IDF know of new arriving revisions to fill its timeline
--
_idf_revs = {}
push_hook_functions(
{
start =
function (session_id)
_idf_revs[session_id] = {}
return "continue",nil
end,
revision_received =
function (new_id, revision, certs, session_id)
table.insert(_idf_revs[session_id], new_id)
return "continue",nil
end,
["end"] =
function (session_id, ...)
if table.getn(_idf_revs[session_id]) == 0 then
return "continue",nil
end
local pin,pout,pid = spawn_pipe(IDF_push_script, IDF_project);
if pid == -1 then
print("could not execute " .. IDF_push_script)
return "continue",nil
end
for _,r in ipairs(_idf_revs[session_id]) do
pin:write(r .. "\n")
end
pin:close()
wait(pid)
return "continue",nil
end
})

View File

@ -1,79 +0,0 @@
-- ***** BEGIN LICENSE BLOCK *****
-- This file is part of InDefero, an open source project management application.
-- Copyright (C) 2010 Céondo Ltd and contributors.
--
-- InDefero is free software; you can redistribute it and/or modify
-- it under the terms of the GNU General Public License as published by
-- the Free Software Foundation; either version 2 of the License, or
-- (at your option) any later version.
--
-- InDefero is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU General Public License for more details.
--
-- You should have received a copy of the GNU General Public License
-- along with this program; if not, write to the Free Software
-- Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
--
-- ***** END LICENSE BLOCK *****
--
-- controls the access rights for remote_stdio which is used by IDFs frontend
-- and other interested parties
--
function get_remote_automate_permitted(key_identity, command, options)
if (key_identity.id == "%%MTNCLIENTKEY%%") then
return true
end
return false
end
--
-- let IDF know of new arriving revisions to fill its timeline
--
_idf_revs = {}
push_hook_functions({
["start"] = function (session_id)
_idf_revs[session_id] = {}
return "continue",nil
end,
["revision_received"] = function (new_id, revision, certs, session_id)
table.insert(_idf_revs[session_id], new_id)
return "continue",nil
end,
["end"] = function (session_id, ...)
if table.getn(_idf_revs[session_id]) == 0 then
return "continue",nil
end
local pin,pout,pid = spawn_pipe("%%MTNPOSTPUSH%%", "%%PROJECT%%");
if pid == -1 then
print("could not execute %%MTNPOSTPUSH%%")
return
end
for _,r in ipairs(_idf_revs[session_id]) do
pin:write(r .. "\n")
end
pin:close()
wait(pid)
return "continue",nil
end
})
--
-- Load local hooks if they exist.
--
-- The way this is supposed to work is that hooks.d can contain symbolic
-- links to lua scripts. These links MUST have the extension .lua
-- If the script needs some configuration, a corresponding file with
-- the extension .conf is the right spot.
--
-- First load the configuration of the hooks, if applicable
includedirpattern(get_confdir() .. "/hooks.d/", "*.conf")
-- Then load the hooks themselves
includedirpattern(get_confdir() .. "/hooks.d/", "*.lua")

View File

@ -1,92 +0,0 @@
-- ***** BEGIN LICENSE BLOCK *****
-- This file is part of InDefero, an open source project management application.
-- Copyright (C) 2010 Céondo Ltd and contributors.
--
-- InDefero is free software; you can redistribute it and/or modify
-- it under the terms of the GNU General Public License as published by
-- the Free Software Foundation; either version 2 of the License, or
-- (at your option) any later version.
--
-- InDefero is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU General Public License for more details.
--
-- You should have received a copy of the GNU General Public License
-- along with this program; if not, write to the Free Software
-- Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
--
-- ***** END LICENSE BLOCK *****
--
-- controls the access rights for remote_stdio which is used by IDFs frontend
-- and other interested parties
--
function get_remote_automate_permitted(key_identity, command, options)
local read_only_commands = {
"get_corresponding_path", "get_content_changed", "tags", "branches",
"common_ancestors", "packet_for_fdelta", "packet_for_fdata",
"packets_for_certs", "packet_for_rdata", "get_manifest_of",
"get_revision", "select", "graph", "children", "parents", "roots",
"leaves", "ancestry_difference", "toposort", "erase_ancestors",
"descendents", "ancestors", "heads", "get_file_of", "get_file",
"interface_version", "get_attributes", "content_diff",
"file_merge", "show_conflicts", "certs", "keys", "get_file_size",
"get_extended_manifest_of"
}
for _,v in ipairs(read_only_commands) do
if (v == command[1]) then
return true
end
end
return false
end
--
-- let IDF know of new arriving revisions to fill its timeline
--
_idf_revs = {}
push_hook_functions({
["start"] = function (session_id)
_idf_revs[session_id] = {}
return "continue",nil
end,
["revision_received"] = function (new_id, revision, certs, session_id)
table.insert(_idf_revs[session_id], new_id)
return "continue",nil
end,
["end"] = function (session_id, ...)
if table.getn(_idf_revs[session_id]) == 0 then
return "continue",nil
end
local pin,pout,pid = spawn_pipe("%%MTNPOSTPUSH%%", "%%PROJECT%%");
if pid == -1 then
print("could not execute %%MTNPOSTPUSH%%")
return
end
for _,r in ipairs(_idf_revs[session_id]) do
pin:write(r .. "\n")
end
pin:close()
wait(pid)
return "continue",nil
end
})
--
-- Load local hooks if they exist.
--
-- The way this is supposed to work is that hooks.d can contain symbolic
-- links to lua scripts. These links MUST have the extension .lua
-- If the script needs some configuration, a corresponding file with
-- the extension .conf is the right spot.
--
-- First load the configuration of the hooks, if applicable
includedirpattern(get_confdir() .. "/hooks.d/", "*.conf")
-- Then load the hooks themselves
includedirpattern(get_confdir() .. "/hooks.d/", "*.lua")

View File

@ -0,0 +1,30 @@
-- ***** BEGIN LICENSE BLOCK *****
-- This file is part of InDefero, an open source project management application.
-- Copyright (C) 2011 Céondo Ltd and contributors.
--
-- InDefero is free software; you can redistribute it and/or modify
-- it under the terms of the GNU General Public License as published by
-- the Free Software Foundation; either version 2 of the License, or
-- (at your option) any later version.
--
-- InDefero is distributed in the hope that it will be useful,
-- but WITHOUT ANY WARRANTY; without even the implied warranty of
-- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-- GNU General Public License for more details.
--
-- You should have received a copy of the GNU General Public License
-- along with this program; if not, write to the Free Software
-- Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
--
-- ***** END LICENSE BLOCK *****
---- Load local hooks if they exist.
-- The way this is supposed to work is that hooks.d can contain symbolic
-- links to lua scripts. These links MUST have the extension .lua
-- If the script needs some configuration, a corresponding file with
-- the extension .conf is the right spot.
----
-- First load the configuration of the hooks, if applicable
includedirpattern(get_confdir() .. "/hooks.d/","*.conf")
-- Then load the hooks themselves
includedirpattern(get_confdir() .. "/hooks.d/","*.lua")

View File

@ -0,0 +1 @@
%%MTNCLIENTKEY%%

View File

@ -70,11 +70,6 @@ class IDF_Scm_Monotone_Stdio
*/
public function _getAuthOptions()
{
// no remote authentication - the simple case
if (!Pluf::f('mtn_remote_auth', true)) {
return '--key= ';
}
$prjconf = $this->project->getConf();
$name = $prjconf->getVal('mtn_client_key_name', false);
$hash = $prjconf->getVal('mtn_client_key_hash', false);

View File

@ -98,17 +98,16 @@ $cfg['mtn_remote_url'] = 'mtn://my-host.biz/%s';
# choosed for manual setups and / or ssh access.
$cfg['mtn_db_access'] = 'local';
# If true, each access to the database is authenticated with an auto-generated
# project key which is stored in the IDF project configuration
# ('mtn_client_key_*') and written out to $cfg['tmp_folder']/mtn-client-keys
# for its actual use. This key is then configured on the server to have
# full read / write access to all functions, while anonymous access can be
# completely disabled.
# If false, IDF tries to connect anonymously, without authentication, to
# the remote monotone server instance. In this case no project-specific
# keys are generated and the server must be configured to allow at least
# anonymous read access to the main functions.
#$cfg['mtn_remote_auth'] = true;
# Full path to the directory tree which contains default configuration files
# that are automatically created for new projects. This is only needed
# if $cfg['mtn_db_access'] is set to remote, i.e. in case the SyncMonotone
# plugin should be used. If unset, it defaults to the tree underknees
# src/IDF/Plugin/SyncMonotone/. Don't forget the trailing slash!
#$cfg['mtn_confdir'] = '/path/to/dir/tree/';
# Additional configuration files you want to create / copy for new setups.
# All these file paths have to be relative to $cfg['mtn_confdir'].
#$cfg['mtn_confdir_extra'] = array('hooks.d/something.lua')
# Needs to be configured for remote / usher usage.
# This allows basic control of a running usher process via the forge