About federated learning
Federated learning is a method of machine learning. Machine learning is a branch of artificial intelligence (AI) that involves the development of algorithms which “learn” by identifying patterns and connections in large datasets.
In principle, a machine learning system requires data to learn and solve problems. And, in general, the more data it is fed, the better it becomes at solving problems. However, developers can find it difficult to access sufficient data to develop good algorithms. This is especially true when the data contains personal data, the processing of which is strictly regulated.
If an organisation sees that it needs more data, it can join forces with other organisations. In general, the organisations upload their data to a shared central server or machine, which all the partner organisations can use to train machine learning models. However, it is not possible to share data if there is no legal basis for processing the underlying personal data. There is, therefore, great potential for artificial intelligence that can use large volumes of data to learn from but that does not require the sharing of personal data. And this is precisely the main purpose of federated learning – to achieve big data machine learning without data sharing.
History
Federated learning was developed by Google in 2016. Google used the method to train a machine learning model using data located on mobile phones without uploading the data to a centralised network. The objective was to build machine learning models that were updated on the basis of data stored on the users’ mobile phones. The technology was, for example, used in the keyboard application Gboard to predict which word was being typed in. Since then, the technology has been shared and used in other contexts.
Read Google’s own blog post about federated learning
Read the article “Federated Learning for Mobile Keyboard Prediction” on Google Research
In the past few years, various organisations have carried out research on federated learning, which has generated several types of alternative setups for the method. Nevertheless, federated learning remains a new tool and there are still few commercial or public applications involving large volumes of data.
How does federated learning take place?
Different models for federated learning
The most common architecture for federated learning is called “horizontal federated learning”. A less frequently used alternative is “vertical federated learning”. This method is more common if two entities share a dataset. Other architectures include “federated transfer learning”, “cross-silo federated learning” and “cross-device federated learning”. With vertical federated learning, entities have different columns/categories of data – in this situation, data standardisation is not required.
Federated learning can be used in different ways to train artificial intelligence. Below, we have described the step-by-step processes in a commonly used model for federated learning (based on the model Google developed in 2016).
- Each participant receives a machine learning algorithm.
- Each participant uses the local dataset to train the machine learning algorithm.
- Each participant encrypts its local “learning package” and sends it to an external, central server. The learning packages do not contain personal data.
- The server performs a secure aggregation of the packages.
- The aggregation of the packages is used to update the machine learning models, which are stored centrally, with learning from the participants. The machine learning model that is stored centrally is the same as the original model sent out to the participants for local training.
- Steps 1 to 5 are repeated until the machine learning model is fully trained.
- Each participant receives the fully trained machine learning model and is now able to make better local predictions.
Because only the model parameters are exchanged, local data – which often comprises personal data – does not in theory have to be transferred between participants or between participants and the central server. The built-in limitation on the sharing of local data means that federated learning is considered to be a more privacy-friendly approach to artificial intelligence.