What is claimed is:
1. A computer-implemented method of federated training that yields personalized machine-learning models on a plurality of mobile devices, the method comprising:
receiving, at each mobile device, from at least one parameter aggregation server, (i) a current set of shared model parameters that are subject to cross-device aggregation, (ii) an aggregation mask that distinguishes a shared parameter subset from a device-specific parameter subset, and (iii) one or more training hyperparameters;
on each mobile device, executing local training that jointly updates (a) the shared parameter subset and (b) the device-specific parameter subset by minimizing a composite objective that includes: (1) an empirical loss evaluated over local training data using a model instantiated with both the shared parameter subset and the device-specific parameter subset, (2) a proximal regularization term that penalizes deviation of the updated shared parameter subset from the received shared parameter subset, and (3) a personalization coupling term that penalizes deviation of the device-specific parameter subset from a function of the shared parameter subset;
forming, at each mobile device, an update message that excludes parameters identified by the aggregation mask as device-specific and that encodes a delta or gradient for at least part of the shared parameter subset;
privacy-preserving the update message at each mobile device by applying secure aggregation cryptographic encapsulation and optionally injecting calibrated random noise sufficient to satisfy a specified differential-privacy budget;
transmitting the privacy-preserved update messages from the plurality of mobile devices to the parameter aggregation server and aggregating, at the server without revealing individual device contributions, the update messages into an aggregated update for the shared parameter subset;
updating, at the server, the shared parameter subset based at least on a weighted average of the aggregated update, the weights reflecting per-device data quantities or quality metrics;
optionally partitioning, at the server, the plurality of mobile devices into two or more cohorts according to a similarity measure computed from device-provided update embeddings or statistics, and maintaining cohort-specific versions of the shared parameter subset;
transmitting, from the server to each mobile device, a shared parameter subset selected for that device, and repeating the foregoing steps for one or more additional federated rounds; and
deploying, at inference time on each mobile device, a personalized model instance that comprises the device’s device-specific parameter subset in combination with the most recently assigned shared parameter subset, wherein the device-specific parameter subset remains stored only on the mobile device and is not transmitted in the clear to the server.
2. The method of claim 1, wherein the proximal regularization term includes a norm-based penalty that constrains the updated shared parameter subset to remain within a bounded divergence from the received shared parameter subset to mitigate client drift.
3. The method of claim 1, wherein the personalization coupling term penalizes deviation of the device-specific parameter subset from a linear projection of the shared parameter subset, thereby encouraging alignment between representations learned by the shared and device-specific parameter subsets.
4. The method of claim 1, wherein the device-specific parameter subset comprises one or more adapter modules, low-rank factors, bias vectors, normalization statistics, or feature-wise modulation parameters that are excluded from server-side aggregation by the aggregation mask.
5. The method of claim 1, wherein the local training further comprises splitting local data into a training partition and a validation partition, determining a personalization weight coefficient by minimizing the empirical loss on the validation partition, and using the personalization weight coefficient within the composite objective.
6. The method of claim 1, wherein the forming of the update message further comprises compressing the update by at least one of sparsification, quantization, sketching, or low-rank approximation, while maintaining the exclusion of device-specific parameters specified by the aggregation mask.
7. The method of claim 1, wherein privacy-preserving the update message comprises using an additive secret-sharing or threshold cryptographic protocol such that the server obtains only an aggregated sum or average across devices and cannot reconstruct any individual device’s update.
8. The method of claim 1, wherein injecting calibrated random noise satisfies an (epsilon, delta) differential-privacy guarantee with respect to the local training data stored on each mobile device.
9. The method of claim 1, wherein the aggregating at the server is performed asynchronously with bounded staleness, and wherein the server down-weights stale updates according to an age-dependent factor.
10. The method of claim 1, wherein the similarity measure used for partitioning into cohorts comprises at least one of: cosine similarity of device update vectors, a divergence between device update statistics, or an affinity computed from a Fisher information approximation, and wherein the server maintains separate cohort-specific shared parameter subsets based on the partitioning.
11. The method of claim 1, wherein the aggregation mask is learned or adapted over federated rounds to select parameters for cross-device aggregation based on a utility metric estimated from observed reduction in validation loss or communication budget constraints.
12. The method of claim 1, further comprising selecting, by the server, a subset of the plurality of mobile devices to participate in a given federated round based on resource-awareness criteria including at least one of battery level, connectivity, compute capability, or data freshness, to meet a target training deadline or energy budget.
13. The method of claim 1, wherein the composite objective includes a term that penalizes the norm of the device-specific parameter subset to mitigate overfitting and to bound the device-side model capacity.
14. The method of claim 1, further comprising performing on-device knowledge distillation from the combined shared and device-specific parameter subsets to a compressed personalized student model for inference on the mobile device without sharing the student model off-device.
15. A system comprising:
one or more servers including at least one processor and memory storing program instructions that cause the servers to distribute shared model parameters and an aggregation mask to a plurality of mobile devices, to receive privacy-preserved update messages from the plurality of mobile devices, to aggregate the update messages without reconstructing any individual device’s update, to update the shared model parameters from the aggregated update, and to provide cohort-specific or global versions of the shared model parameters to the plurality of mobile devices; and
the plurality of mobile devices, each mobile device comprising a processor and a memory storing program instructions and a local dataset, the program instructions causing each mobile device to: receive the shared model parameters and aggregation mask; locally update a shared parameter subset and a device-specific parameter subset by minimizing a composite objective as recited in claim 1; form an update message that excludes the device-specific parameter subset; privacy-preserve the update message; and transmit the update message to the one or more servers;
wherein the device-specific parameter subset remains stored only on the corresponding mobile device and is not transmitted in the clear to the one or more servers.
16. The system of claim 15, wherein the one or more servers are further configured to cluster the mobile devices into cohorts using a similarity measure based on update embeddings, and to maintain cohort-specific shared model parameters that are broadcast to devices according to cohort membership.
17. The system of claim 15, wherein each mobile device is further configured to adjust a personalization weight of the composite objective via a validation-driven selection procedure on a held-out local partition.
18. The system of claim 15, wherein the one or more servers implement asynchronous aggregation with bounded staleness and staleness-aware weighting, and wherein the privacy-preserving update messages are formed using secure aggregation with secret sharing or threshold cryptography.
19. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a mobile device, cause the mobile device to:
receive, from a federated learning server, shared model parameters, an aggregation mask, and hyperparameters;
update, using local training data, both a shared parameter subset and a device-specific parameter subset by minimizing a composite objective including an empirical loss, a proximal regularization term on the shared parameter subset, and a personalization coupling term relating the device-specific parameter subset to the shared parameter subset;
form a privacy-preserved update message that includes an update to the shared parameter subset and excludes the device-specific parameter subset according to the aggregation mask; and
transmit the privacy-preserved update message to the federated learning server for aggregation, while retaining the device-specific parameter subset solely on the mobile device.
20. The non-transitory computer-readable medium of claim 19, wherein forming the privacy-preserved update message includes compressing the update via sparsification or quantization and applying differential-privacy noise calibrated to a target privacy budget.
