Clustering Information

Owned by Christian Sprajc
Last updated: Mar 14, 2020 by Ahmad Abdullah

PowerFolder is able to run as clustered solution. This type of setup is highly recommended if you plan to serve more than several thousand users or you would like to run the installation high available. Please contact us if you plan such an installation - we offer professional consulting and installation services to tailor your PowerFolder installation to your needs. From an end-user perspective it is completely transparent if PowerFolder runs as clustered or non-clustered, single server solution. It is worth to mention, that PowerFolder clients are able to directly transfer files between each other via peer to peer connections. This grants the system a built-in high availability, even if a server (node) is not available.

As administrator you need to pay attention to four layers of the setup: Web (usually Apache), Storage/Filesystem, Database and PowerFolder Server (nodes).

Each of these layers need to be clustered to gain high availability, however if you aim for load balancing only, at least the PowerFolder Server layer needs to be clustered. The minimal clustered setup is two PowerFolder Server nodes and one external database. You can also start with an non-clustered setup and add cluster functionality later. In that case make sure to begin with one PowerFolder Server and an external database.

The following document outlines the concepts and gives an technical overview.

Schematic overview

Web layer

Central PowerFolder web services

On the web layer PowerFolder Server itself is working in the standard configuration on port 8080 for web access and provides the the below services to users and administrators:

  • Web interface for users and administrators
  • Mobile interface

As soon as a user logs in to one PowerFolder Server cluster node, PowerFolder Server forwards the request to the PowerFolder Server cluster node, which hosts the user's folders.

Encrypted sessions to the web interface

In order to ensure SSL encrypted traffic to those services we recommend to use the Apache HTTP Server to proxy the connections (from HTTPS port 443 to the standard port 8080) and SSL encrypt the sessions at the same time.

When operating in a public available network and using the same domain name for several PowerFolder cluster members, it is recommended to purchase a wildcard SSL certificate to keep the costs low and to be flexible, when subdomains are changing or the cluster is enhanced with the nodes.

To provide one unique URL to the user there should be one central host, which serves the URL and forwards requests to other PowerFolder Server cluster nodes to the corresponding host. Because this is a single point of failure, we recommend to setup at least two machines with an Apache HTTP Server serving the same URL (with the exact same configuration) and put a load balancer or round-robin DNS in front of it.

Distributed PowerFolder web services

There are other web services, which run on each PowerFolder Server node and don't need/support a central entry point:

  • Licensing interface
  • RPC interface for HTTP tunneled connections
  • WebDAV interface

The licensing service is handled by an internal PowerFolder Server failover routine, which polls several hosts after each other and therefore ensures high availability.

PowerFolder Server layer

Each PowerFolder Server node has to run on a separate machine (physical or virtual) and they need to share a common database (e.g. mySQL). By sharing the same database nodes recognize themselves as part of the same cluster. Each node serves a part of the userbase connected to the system. You can manually change the assignment of user accounts to server nodes under Accounts/Edit.

The client app keeps direct TCP/data connection to PowerFolder Server node, which serves the user logged in at the client. It also keeps backup connection to other PowerFolder Server nodes.

The most common way to setup the different PowerFolder Server instances is to start with first instance and make it work with all layer components (database, web, storage). Once it is up and running you can gradually extend your deployment by additional PowerFolder Server nodes. Please contact us for details.

Storage layer

For the storage, where the actual folders of the users are located, several types of underlying storage can be used

  • Local storage of the host
  • Network storage (NFS, SAN, ...) mounted on a directory or via UNC.

Depending on the technology used, network based storage has the advantage that it supports several types of high availability. 

Database layer

The recommended database technology for PowerFolder Server is MySQL. Each PowerFolder Server cluster node will connect to the same database to have the same information available about users and permissions available on each node.

The MySQL database can run on the same host as the PowerFolder Server, however we recommend to put it on a separate host, since depending on the number of users and folders it can use quite some CPU load. 

For larger installations the MySQL Cluster software can be used to make the database services highly available and load-balanced: http://dev.mysql.com/doc/refman/5.1/de/mysql-cluster-overview.html

In case one member of the MySQL Cluster fails, PowerFolder Server nodes will connect to another member of the MySQL Cluster.

Example setup



Cluster Policy 

Cluster policy defines that how should a cluster reacts and work in the case of new and active:

  • Users
  • Connections
  • Storage
  • Server Resources

Cluster policy can be defined in the PowerFolder.config and also in the Cluster.config that is present in the server_maintenance folder under cluster.

As an improvement we are introducing the default cluster policy in PowerFolder Server v15.1.100 as:

cluster.newconnects.policy=fewest_folders

After this improvement the new server connection is always connected to the server in cluster that contains the fewest folders and hence the resources can be managed automatically.


As an example of cluster.newconnects.policy=fewest_folders please consider the following diagram:

The PowerFolder Server 1 has the fewest folders so that all the new connections are automatically directed to it and the new user and their data can be successfully registered on the server that has the minimum resources in the cluster setup. 

Adopting the new Cluster Policy

To adopt the new cluster policy:

  • Upgrade your server to v15.1.100
  • Delete the configuration entry: cluster.newconnects.policy=external from PowerFolder.config and Cluster.config
  • Restart all the nodes of your cluster
  • Cross-check the cluster policy, it should be automatically taken as cluster.newconnects.policy=fewest_folders 

In the previous versions a default the cluster policy was given as:

cluster.newconnects.policy=external

That means the server should adopt the cluster policy that is defined in the external web server e.g. Nginx or Apache.

Conclusion

The actual setup you choose is depended on your requirements, budget and existing infrastructure.

At PowerFolder we offer full consultancy services and technical support for clustered and large scale installations.

Our experts will be happy to help you to setup your PowerFolder cluster.

Please contact us for more information. Please also check our further documentation on building a high-availability cluster.