Network definition

YaCy peer-to-peer network is completely decentralized and also does not require a single central server for the network to clamp up.

Network-Bootstrapping

Of course, a 'new' peer must know how to contact the other peers, for that is there so-called seed list. That, generated by a peer in the network, is used for the Network-Bootstrapping, but any participant of the network can generate seed-list. A peer that creates such a file may call itself a 'principal peer' and there can be several of them. The network is therefore defined by very specific peers, but these cannot be viewed as 'centers' because, in principle, every peer can be principal.

Now you still may ask yourself where a new peer finds the principal peers of the network: these are stored in the network definition file of the network, that you want to join. A new YaCy peer can basically connect to any network, but that doesn't mean you must set something complicated: every installation includes a network definition file of default network called 'freeworld'. That is located in defaults/yacy.network.freeworld.unit and is set by the attribute network.unit.definition in defaults/yacy.init. You don't have to adjust or change anything in that configuration file, everything neccessary for the "freeworld" network is already set.

However, every YaCy user can define their own network, and this article covers in detail how that works.

The processes that happen during bootstrapping are:

After the seed lists have been loaded once, a peer can even completely find its way back to the network without loading the peer lists.

Definition of the YaCy network with yacy.network.unit

In peer-to-peer mode, YaCy creates a network cluster of YaCy peers defined for a specific domain of web-index. By default, this search network is the public YaCy network, whose domain is the public Internet. The network is defined through the bootstrapping, and all peers within the network must have the same bootstrapping information with the properties of the network. These information are stored in yacy.network.unit.

The yacy.network.unit file is included in every YaCy instance from the default settings (yacy.init) with the property network.unit.definition, which defines the following properties, among others:

network.unit.name = freeworld
network.unit.description = Public YaCy Community
network.unit.domain = global
network.unit.dhtredundancy.junior = 1
network.unit.dhtredundancy.senior = 3
network.unit.bootstrap.seedlist0 = http://www.yacy.net/yacy/seed.txt
network.unit.update.location0 = https://download.yacy.net/

At the YaCy startup, the network is set up as follows:

  1. the yacy.init is loaded: the property network.unit.definition in yacy.init denotes yacy.network.unit as the network definition.
  2. yacy.network.unit is loaded: the property network.unit.bootstrap.seedlist0 in yacy.network.unit is set to "http://www.yacy.net/yacy/seed.txt", for a list of seeds from the network
  3. http://www.yacy.net/yacy/seed.txt is loaded and the seeds in it are loaded into the seed DB.
  4. The seeds contain information about the last known peer addresses.

The peers of this network all use the name mentioned in yacy.network.unit (network.unit.name) to identify themself as a participant in the same network. The property network.unit.description is just a free definable text that is displayed in the network graphic, for example.

A very important information is the web domain which the network indexes. The associated property is called network.unit.domain. The domain and can take the following values:

Another network-related setting is the "redundancy factor" or the number indicating how many copies of index are distributed within the DHT. In a public network, the availability of a peer cannot be ensured and therefore the redundancy factor is 3 (network.unit.dhtredundancy.senior). In a network with high availability, this factor can be set to 1.

If all peers in a network are administered by a single person you may want an automatic update of all the peers to take place. To do this, a download location must be defined, and a network operator can specify own location to be able to control the version for the automatic update. To do this, the property network.unit.update.location0 can contain the URL of page that contains links to releases. See autoupdate for details.

You can provide additional alternative addresses both to network.unit.bootstrap.seedlist0 and network.unit.update.location0; you can simply add additional properties named like that, with increased sequence number.

Creation of your own YaCy network

The network definition must be the same for all participants in a network, and this is achieved - in the standard case - by including that in the release.

A YaCy network operator may be interested in changing the network definition after the network has been set up for all peers, for example for advanced security settings for the network. Hence the setting the network.unit.definition in yacy.init is possible also via URL, which we use in the following example.

The construction of a new network consists of two major steps, first is the definition of the first peer of the new network and then in the deployment of the other peers assigned to the first peer.

Configuration of the first peer of a new network

The steps are:

network.unit.name = mynet
network.unit.description = My first very own YaCy network
network.unit.domain = any
network.unit.dhtredundancy.junior = 1
network.unit.dhtredundancy.senior = 1
network.unit.bootstrap.seedlist0 = http://www.meindomain.invalid/yacy.myseedlist
network.unit.update.location0 = https://download.yacy.net/
network.unit.definition = http://www.meindomain.invalid/yacy.mynetdef

This is only necessary for the initial installation, further updates of the peers must remain constant and cannot be set again.

Configuration of the participating peers of the new network

Once the first peer is running, additional peers can be added. They load the seedlist and contact the principal-peer, which provides the new information again through a seedlist-upload to the newly connected peer.

To ensure that new peers can automatically access the new network, you can make a special YaCy release with settings of the new network. To become a new peer, correctly configured to participate in the new network, it can also be configured with normal, unchanged release and updated without loosing network membership. The steps to define the Special releases are:

network.unit.definition = http://www.meindomain.invalid/yacy.mynetdef
update.process = auto
adminAccount=admin:myS3cr3tPa55w0rd
tar cf yacy_mynet.tar yacy

For a mass deployment, all you have to do to install YaCy on the network's computers, is to distribute and unpack the file yacy_mynet.tar.

To ensure a permanent availability of the YaCy installation, it's recommended to define a cron job that regularly restarts installed peer. For example, using following entries in /etc/crontab file:

0 0 * * * yacyuser /home/yacyuser/yacy/stopYACY.sh
2 0 * * * yacyuser /home/yacyuser/yacy/killYACY.sh
4 0 * * * yacyuser /home/yacyuser/yacy/startYACY.sh -l

or, simply:

0 3 * * * yacyuser /home/yacyuser/yacy/restartYACY.sh

provided that YaCy runs under the user 'yacyuser' and that YaCy directory is located in the yacyuser home directory.

For more network settings see also: yacy.network.readme.

Converted and translated from German from http://wiki.yacy.net/index.php?title=De:Netzdefinition. May be outdated.