githubFork/gitblit.git - Solinfo Gitblit

Merge pull request #1073 from gitblit/1062-DocEditorUpdates

Paul Martin

2016-04-30 a502d96a860456ec5e8c96761db70f7cabb74751

commit \| author \| age
831469	1	## Federating Gitblit
JM	2
	3	SINCE 0.6.0
	4
ea936a	5	A Gitblit federation is a mechanism to clone repositories and keep them in sync from one Gitblit instance to another. Federation can be used to maintain a mirror of your Gitblit instance, to aggregate repositories from developer workstations, or to initially clone groups of repositories to developer workstations. If you are/were a Subversion user you might think of this as [svn-sync](http://svnbook.red-bean.com/en/1.5/svn.ref.svnsync.html), but better.
831469	6
d7fb20	7	If your Gitblit instance allows federation and it is properly registered with another Gitblit instance, each of the non-excluded repositories of your Gitblit instance can be mirrored, in their entirety, to the pulling Gitblit instance. You may optionally allow pulling of user accounts and backup of server settings.
JM	8
	9	The federation feature should be considered a security backdoor and enabled or disabled as appropriate for your installation.<br/>
	10	Please review all the documentation to understand how it works and its limitations.
ea936a	11
JM	12	![block diagram](fed_aggregation.png "Gitblit Federation Aggregation")
831469	13
d4c908	14	### Important Changes to Note
JM	15
df162c	16	The Gitblit 0.8.0 federation protocol adds retrieval of teams and referenced push scripts. Older clients will not know to request team or push script information.
997c16	17
JM	18	The Gitblit 0.7.0 federation protocol is incompatible with the 0.6.0 federation protocol because of a change in the way timestamps are formatted.
d4c908	19
JM	20	Gitblit 0.6.0 uses the default [google-gson](http://google-gson.googlecode.com) timestamp serializer which generates locally formatted timestamps. Unfortunately, this creates problems for distributed repositories and distributed developers. Gitblit 0.7.0 corrects this error by serializing dates to the [iso8601](http://en.wikipedia.org/wiki/ISO_8601) standard. As a result 0.7.0 is not compatible with 0.6.0. A partial backwards-compatibility fallback was considered but it would only work one direction and since the federation mechanism is bidirectional it was not implemented.
	21
2548a7	22	### Origin Gitblit Instance Requirements
831469	23
JM	24	- git.enableGitServlet must be true, all Git clone and pull requests are handled through Gitblit's JGit servlet
2c32fd	25	- federation.passphrase must be non-empty
2548a7	26	- The Gitblit origin instance must be http/https accessible by the pulling Gitblit instance.<br/>That may require configuring port-forwarding on your router and/or opening ports on your firewall.
831469	27
2c32fd	28	#### federation.passphrase
831469	29
2c32fd	30	The passphrase is used to generate permission tokens that can be shared with other Gitblit instances.
831469	31
2c32fd	32	The passphrase value never needs to be shared, although if you give another Gitblit instance the ALL federation token then your passphrase will be transferred/backed-up along with all other server settings.
831469	33
2c32fd	34	This value can be anything you want: an integer, a sentence, an haiku, etc. You should probably keep the passphrase simple and use standard Latin characters to prevent Java properties file encoding errors. The tokens generated from this value are affected by case, so consider this value CASE-SENSITIVE.
831469	35
2c32fd	36	The federation feature is completely disabled if your passphrase value is empty.
831469	37
3b5289	38	NOTE:
2c32fd	39	Changing your federation.passphrase will break any registrations you have established with other Gitblit instances.
831469	40
JM	41	### Pulling Gitblit Instance Requirements
	42
2548a7	43	- consider setting federation.allowProposals=true to facilitate the registration process from origin Gitblit instances
831469	44	- properly registered Gitblit instance including, at a minimum, url, federation token, and update frequency
JM	45
	46	### Controlling What Gets Pulled
	47
2548a7	48	If you want your repositories (and optionally users accounts and settings) to be pulled by another Gitblit instance, you need to register your origin Gitblit instance with a pulling Gitblit instance by providing the url of your Gitblit instance and a federation token.
831469	49
f6740d	50	Gitblit generates the following standard federation tokens:
6c4be1	51
JM	52	```java
2c32fd	53	String allToken = SHA1(passphrase + "-ALL");
JM	54	String usersAndRepositoriesToken = SHA1(passphrase + "-USERS_AND_REPOSITORIES");
	55	String repositoriesToken = SHA1(passphrase + "-REPOSITORIES");
6c4be1	56	```
JM	57
df162c	58	The ALL token allows another Gitblit instance to pull all your repositories, user accounts, server settings, and referenced push scripts.
3b5289	59	The USERS_AND_REPOSITORIES token allows another Gitblit instance to pull all your repositories and user accounts.
831469	60	The REPOSITORIES token only allows pulling of the repositories.
JM	61
	62	Individual Gitblit repository configurations such as description and accessRestriction are always mirrored.
	63
2c32fd	64	If federation.passphrase has a non-empty value, the federation tokens are displayed in the log file and are visible, to administrators, in the web ui.
831469	65
f6740d	66	The three standard tokens grant access to ALL your non-excluded repositories. However, if you only want to specify different groups of repositories to be federated then you need to define federation sets.
JM	67
8f73a7	68	#### Federation Sets
JM	69
f6740d	70	Federation Sets (federation.sets) are named groups of repositories. The Federation Sets are defined in `gitblit.properties` and are available for selection in the repository settings page. You can assign a repository to one or more sets and then distribute the federation token for the set. This allows you to grant federation pull access to a subset of your available repositories. Tokens for federation sets only grant pull access for the member repositories.
831469	71
2548a7	72	### Federation Proposals (Origin Gitblit Instance)
831469	73
2c32fd	74	Once you have properly setup your passphrase and can see your federation tokens, you are ready to share them with a pulling Gitblit instance.
831469	75
3b5289	76	The registration process can be partially automated by sending a federation proposal to the pulling Gitblit instance.
831469	77	To send a proposal:
JM	78
	79	1. Login to your Gitblit instance as an administrator
3b5289	80	2. Select and click the propose link for the appropriate token on the federation page
d7fb20	81	3. Confirm the publicly accessible url of your (origin) Gitblit instance
JM	82	4. Enter the url of the pulling Gitblit instance you want to federate with
	83	5. Optionally enter a message for the administrators
	84	6. Click propose
831469	85
JM	86	Not all Gitblit instances accept federation proposals, there is a setting which allows Gitblit to outright reject them. In this case an email or instant message to the administrator of the other Gitblit instance is required. :)
	87
d7fb20	88	If your proposal is accepted, the proposal is cached to disk on the pulling Gitblit server and, if properly configured, the administrators of that Gitblit server will receive an email notification of your proposal.
831469	89
JM	90	Your proposal includes:
	91
	92	1. the url of your Gitblit instance
	93	2. the federation token you selected and its type
	94	3. the list of your non-excluded repositories, and their configuration details, that you propose to share
	95
3b5289	96	Submitting a proposal does not automatically register your server with the pulling Gitblit instance.
831469	97	Registration is a manual process for an administrator.
JM	98
	99	### Federation Proposals (Pulling Gitblit Instance)
	100
	101	If your Giblit instance has received a federation proposal, you will be alerted to that information the next time you login to Gitblit as an administrator.
	102
d7fb20	103	You may view the details of a proposal by scrolling down to the bottom of the repositories page and selecting a proposal. Sample registration settings will be generated for you that you may copy & paste into either your `gitblit.properties` file or your `web.xml` file.
831469	104
2548a7	105	### Excluding Repositories (Origin Gitblit Instance)
831469	106
f6740d	107	You may exclude a repository from being pulled by any federated Gitblit instance by setting its federation strategy to EXCLUDE in the repository's settings page.
831469	108
JM	109	### Excluding Repositories (Pulling Gitblit Instance)
	110
	111	You may exclude repositories to pull in a federation registration. You may exclude all or you may exclude based on a simple fuzzy pattern. Only one wildcard character may be used within each pattern. Patterns are space-separated within the exclude and include fields.
	112
	113	federation.example.exclude = skipit.git
	114
	115	OR
	116
	117	federation.example.exclude = *
	118	federation.example.include = somerepo.git someotherrepo.git
	119
	120	OR
	121
	122	federation.example.exclude = *
	123	federation.example.include = common/* library/*
	124
	125	### Tracking Status (Pulling Gitblit Instance)
	126
2548a7	127	Below the repositories list on the repositories page you will find a section named federation registrations. This section enumerates the other gitblit servers you have configured to periodically pull. The status of the latest pull will be indicated on the left with a colored circle, similar to the status of an executed unit test or Hudson/Jenkins build. You can drill into the details of the registration to view the status of the last pull from each repository available from that origin Gitblit instance. Additionally, you can specify the federation.N.notifyOnError=true flag, to be alerted via email of regressive status changes to individual registrations.
831469	128
2548a7	129	### Tracking Status (Origin Gitblit Instance)
831469	130
2548a7	131	Origin Gitblit instances can not directly track the success or failure status of Pulling Gitblit instances. However, the Pulling Gitblit instance may elect to send a status acknowledgment (federation.N.sendStatus=true) to the origin Gitblit server that indicates the per-repository status of the pull operation. This is the same data that is displayed on the Pulling Gitblit instances ui.
831469	132
2548a7	133	### How does it work? (Origin Gitblit Instances)
831469	134
JM	135	A pulling Gitblit instance will periodically contact your Gitblit instance and will provide the token as proof that you have granted it federation access. Your Gitblit instance will decide, based on the supplied token, if the requested data should be returned to the pulling Gitblit instance. Gitblit data (user accounts, repository metadata, and server settings) are serialized as [JSON](http://json.org) using [google-gson](http://google-gson.googlecode.com) and returned to the pulling Gitblit instance. Standard Git clone and pull operations are used to transfer commits.
	136
	137	The federation process executes using an internal administrator account, $gitblit. All the normal authentication and authorization processes are used for federation requests. For example, Git commands are authenticated as $gitblit / token.
	138
2c32fd	139	While the $gitblit account has access to all repositories, server settings, and user accounts, it is prohibited from accessing the web ui and it is disabled if federation.passphrase is empty.
831469	140
JM	141	### How does it work? (Pulling Gitblit Instances)
	142
	143	Federated repositories defined in `gitblit.properties` are checked after Gitblit has been running for 1 minute. The next registration check is scheduled at the completion of the current registration check based on the registration's specified frequency.
	144
	145	- The shortest frequency allowed is every 5 minutes
	146	- Decimal frequency values are cast to integers
	147	- Frequency values may be specified in mins, hours, or days
	148	- Values that can not be parsed default to 60 minutes
	149
	150	After a repository has been cloned it is flagged as isFederated (which identifies it as being sourced from another Gitblit instance), isFrozen (which prevents Git pushes to this mirror) and federationStrategy=EXCLUDED (which prevents this repository from being pulled by another federated Gitblit instance).
	151
	152	#### Origin Verification
	153
3b5289	154	During a federated pull operation, Gitblit does check that the origin of the local repository starts with the url of the federation registration.
831469	155	If they do not match, the repository is skipped and this is indicated in the log.
JM	156
997c16	157	#### User Accounts & Teams
831469	158
997c16	159	By default all user accounts and teams (except the admin account) are automatically pulled when using the ALL token or the USERS_AND_REPOSITORIES token. You may exclude a user account from being pulled by a federated Gitblit instance by checking exclude from federation in the edit user page.
831469	160
93f472	161	The pulling Gitblit instance will store a registration-specific `users.conf` file for the pulled user accounts and their repository permissions. This file is stored in the federation.N.folder folder.
831469	162
997c16	163	If you specify federation.N.mergeAccounts=true, then the user accounts and team definitions from the origin Gitblit instance will be integrated into the `users.conf` file of your Gitblit instance and allow sign-on of those users.
831469	164
3b5289	165	NOTE:
JM	166	Upgrades from older Gitblit versions will not have the #notfederated role assigned to the admin account. Without that role, your admin account WILL be transferred with an ALL or USERS_AND_REPOSITORIES token.
d7fb20	167	Please consider adding the #notfederated role to your admin account!
831469	168
JM	169	#### Server Settings
	170
d7fb20	171	Server settings are only pulled when using the ALL token.
831469	172
JM	173	The pulling Gitblit instance will store a registration-specific `gitblit.properties` file for all pulled settings. This file is stored in the federation.N.folder folder.
	174
	175	These settings are unused by the pulling Gitblit instance.
	176
df162c	177	#### Push Scripts
JM	178
	179	Your Groovy push scripts are only pulled when using the ALL token.
	180
	181	The pulling Gitblit instance will retrieve any referenced (i.e. used) push script and store it locally as registration_scriptName.groovy in the federation.N.folder folder.
	182
	183	These scripts are unused by the pulling Gitblit instance.
	184
831469	185	### Collisions and Conflict Resolution
JM	186
df162c	187	Gitblit does not detect conflict and it does not offer conflict resolution of repositories, users, teams, or settings.
831469	188
JM	189	If an object exists locally that has the same name as the remote object, it is assumed they are the same and the contents of the remote object are merged into the local object. If you can not guarantee that this is the case, then you should not store any federated repositories directly in git.repositoriesFolder and you should not enable mergeAccounts.
	190
2548a7	191	By default, federated repositories can not be pushed to, they are read-only by the isFrozen flag. This flag is ONLY enforced by Gitblit's JGit servlet. If you push to a federated repository after resetting the isFrozen flag or via some other Git access technique then you may break Gitblit's ability to continue pulling from the origin repository. If you are only pushing to a local branch then you might be safe.
831469	192
2c32fd	193	## Federation Pull Registration Keys
JM	194
3cc6e2	195	<table class="table">
2c32fd	196	<tr><th>federation.N.url</th>
JM	197	<td>string</td>
3b5289	198	<td>the url of the origin Gitblit instance <em>(required)</em></td>
2c32fd	199	</tr>
JM	200
	201	<tr><th>federation.N.token</th>
	202	<td>string</td>
3b5289	203	<td>the token provided by the origin Gitblit instance <em>(required)</em></td>
2c32fd	204	</tr>
JM	205
	206	<tr><th>federation.N.frequency</th>
8f73a7	207	<td>x [mins/hours/days]</td>
2c32fd	208	<td>the period to wait between pull executions</td>
JM	209	</tr>
	210
	211	<tr><th>federation.N.folder</th>
	212	<td>string</td>
3b5289	213	<td>the destination folder, relative to <em>git.repositoriesFolder</em>, for these repositories.<br/>default is <em>git.repositoriesFolder</em></td>
d7fb20	214	</tr>
JM	215
	216	<tr><th>federation.N.bare</th>
	217	<td>boolean</td>
3b5289	218	<td>if <b>true</b> <em>(default)</em>, each repository is cloned as a bare repository (i.e. no working folder).</td>
2548a7	219	</tr>
JM	220
	221	<tr><th>federation.N.mirror</th>
	222	<td>boolean</td>
3b5289	223	<td>if <b>true</b> <em>(default)</em>, each repository HEAD is reset to <em>origin/master</em> after each pull. The repository is flagged <em>isFrozen</em> after the initial clone.<br/><br/>If <b>false</b>, each repository HEAD will point to the FETCH_HEAD of the initial clone from the origin until pushed to or otherwise manipulated.</td>
2c32fd	224	</tr>
JM	225
	226	<tr><th>federation.N.mergeAccounts</th>
	227	<td>boolean</td>
93f472	228	<td>if <b>true</b>, merge the retrieved accounts into the <code>users.conf</code> of <b>this</b> Gitblit instance.<br/><em>default is false</em></td>
2c32fd	229	</tr>
JM	230
	231	<tr><th>federation.N.sendStatus</th>
	232	<td>boolean</td>
3b5289	233	<td>if <b>true</b>, send the status of the federated pull to the origin Gitblit instance.<br/><em>default is false</em></td>
2c32fd	234	</tr>
JM	235
	236	<tr><th>federation.N.include</th>
8f73a7	237	<td>string array<br/>(space-delimited)</td>
3b5289	238	<td>list of included repositories <em>(wildcard and fuzzy matching supported)</em></td>
2c32fd	239	</tr>
JM	240
	241	<tr><th>federation.N.exclude</th>
8f73a7	242	<td>string array<br/>(space-delimited)</td>
3b5289	243	<td>list of excluded repositories <em>(wildcard and fuzzy matching supported)</em></td>
2c32fd	244	</tr>
JM	245
	246	<tr><th>federation.N.notifyOnError</th>
	247	<td>boolean</td>
3b5289	248	<td>if <b>true</b>, send an email to the administrators on an error.<br/><em>default is false</em></td>
2c32fd	249	</tr>
JM	250	</table>
	251
831469	252	## Example Federation Pull Registrations
JM	253
	254	These examples would be entered into the `gitblit.properties` file of the pulling gitblit instance.
	255
	256	#### (Nearly) Perfect Mirror Example
	257
ea936a	258	![block diagram](fed_mirror.png "Gitblit Mirror")
JM	259
d7fb20	260	This assumes that the token is the ALL token from the origin gitblit instance.
831469	261
df162c	262	The repositories, example1_users.conf, example1_gitblit.propertiesn and all example1_scripts.groovy will be put in git.repositoriesFolder and the origin user accounts will be merged into the local user accounts, including passwords and all roles. The Gitblit instance will also send a status acknowledgment to the origin Gitblit instance at the end of the pull operation. The status report will include the state of each repository pull (EXCLUDED, SKIPPED, NOCHANGE, PULLED, MIRRORED). This way the origin Gitblit instance can monitor the health of its mirrors.
d7fb20	263
df162c	264	This example is considered nearly perfect because while the origin Gitblit's server settings & push scripts are pulled and saved locally, they are not merged with your server settings so its not a true mirror.
831469	265
JM	266	federation.example1.url = https://go.gitblit.com
	267	federation.example1.token = 6f3b8a24bf970f17289b234284c94f43eb42f0e4
	268	federation.example1.frequency = 120 mins
d7fb20	269	federation.example1.folder =
JM	270	federation.example1.bare = true
2548a7	271	federation.example1.mirror = true
831469	272	federation.example1.mergeAccounts = true
JM	273	federation.example1.sendStatus = true
	274
	275	#### Just Repositories Example
	276
3b5289	277	This assumes that the token is the REPOSITORIES token from the origin gitblit instance.
831469	278	The repositories will be put in git.repositoriesFolder/example2.
JM	279
	280	federation.example2.url = https://tomcat.gitblit.com/gitblit
	281	federation.example2.token = 6f3b8a24bf970f17289b234284c94f43eb42f0e4
	282	federation.example2.frequency = 120 mins
	283	federation.example2.folder = example2
d7fb20	284	federation.example2.bare = true
2548a7	285	federation.example2.mirror = true
831469	286
JM	287	#### All-but-One Repository Example
	288
3b5289	289	This assumes that the token is the REPOSITORIES token from the origin gitblit instance.
831469	290	The repositories will be put in git.repositoriesFolder/example3.
JM	291
	292	federation.example3.url = https://tomcat.gitblit.com/gitblit
	293	federation.example3.token = 6f3b8a24bf970f17289b234284c94f43eb42f0e4
	294	federation.example3.frequency = 120 mins
	295	federation.example3.folder = example3
d7fb20	296	federation.example3.bare = true
2548a7	297	federation.example3.mirror = true
831469	298	federation.example3.exclude = somerepo.git
JM	299
	300	#### Just One Repository Example
	301
3b5289	302	This assumes that the token is the REPOSITORIES token from the origin gitblit instance.
831469	303	The repositories will be put in git.repositoriesFolder/example4.
JM	304
	305	federation.example4.url = https://tomcat.gitblit.com/gitblit
	306	federation.example4.token = 6f3b8a24bf970f17289b234284c94f43eb42f0e4
	307	federation.example4.frequency = 120 mins
	308	federation.example4.folder = example4
d7fb20	309	federation.example4.bare = true
2548a7	310	federation.example4.mirror = true
831469	311	federation.example4.exclude = *
f6740d	312	federation.example4.include = somerepo.git
JM	313
	314	## Federation Client
	315
	316	Instead of setting up a full-blown pulling Gitblit instance, you can also use the [federation client](http://code.google.com/p/gitblit/downloads/detail?name=%FEDCLIENT%) command-line utility. This is a packaged subset of the federation feature in a smaller, simpler command-line only tool.
	317
	318	The federation client relies on many of the same dependencies as Gitblit and will download them on first execution.
	319
	320	### federation.properties
	321	You may use the `federation.properties` file to configure one or more Gitblit instances that you want to pull from. This file is a subset of the standard `gitblit.properties` file.
	322
	323	By default this tool does not daemonize itself; it executes and then quits. This allows you to use the native scheduling feature of your OS. Of course, if you'd rather use Gitblit's scheduler you may use that by specifying the `--daemon` parameter.
	324
5955a1	325	### http.sslVerify
JM	326
	327	If you are pulling from a Gitblit with a self-signed SSL certificate you will need to configure Git/JGit to bypass certificate verification.
	328	([Git-Config Manual Page](http://www.kernel.org/pub/software/scm/git/docs/git-config.html))
	329
	330	<pre>git config --global --bool --add http.sslVerify false</pre>
	331
f6740d	332	### Command-Line Parameters
JM	333	Instead of using `federation.properties` you may directly specify a Gitblit instance to pull from with command-line parameters.
	334
	335	java -jar fedclient.jar --url https://go.gitblit.com --mirror --bare --token 123456789
	336	--repositoriesFolder c:/mymirror
	337
	338	java -jar fedclient.jar --url https://go.gitblit.com --mirror --bare --token 123456789
	339	--repositoriesFolder c:/mymirror --daemon --frequency "24 hours"
	340