Abstract: Neural networks have shown remarkable performance across data domains, especially in regimes of increasing compute budgets. However, fundamental insights into how neural networks process information, share representations and traverse loss landscapes remain uncertain. In this work, we quantify the functional impact of distribution matching, facilitated by knowledge sharing mechanisms such as knowledge distillation, under student-teacher optimisation strategies.
This free event is open only to members of the University of Cambridge (and affiliated institutes). Please be aware that we are unable to offer consultations outside clinic hours.
If you would like to participate, please sign up as we will not be able to offer a consultation otherwise. Please sign up through the following link: https://forms.gle/5dHfs6vJrrvTbqst5. Sign-up is possible from May 21 midday (12pm) until May 25 midday or until we reach full capacity, whichever is earlier. If you successfully signed up, we will confirm your appointment by May 27 midday.
Computation and networking are ubiquitous. Many of us carry multiple networked computation devices almost constantly. Most of those devices spend much of their time exchanging data with external services via the Internet. But we still have to operate and manage them as independent devices, at best using cloud services to support limited integration within a closed ecosystem. So I believe it's time we thought more fundamentally about what a modern "personal computer" should be, and how the operating system can shape modern hardware to make one.
Abstract: Language models (LMs) are pre-trained on raw text datasets to generate text sequences token-by-token. While this approach facilitates the learning of world knowledge and reasoning, it does not explicitly optimise for linguistic competence. To bridge this gap, we propose L2T, a pre-training framework integrating Language Learning Tasks alongside standard next-token prediction.
AthenaZero is a bimanual manipulator designed to maximize control authority while minimizing inertia. By utilizing quasi-direct drive actuation and transmission remotization techniques, the system achieves an effective endpoint mass comparable to that of a human. Trading off trajectory tracking stiffness as compared to conventional high-impedance manipulators, this architecture reduces reflected inertia by an order of magnitude.
Dr Anna Breger, Project Leader, University of Cambridge
In this talk we discuss the potential of non-invasive imaging and machine learning techniques for the reconstruction of degraded medieval music notation. Our examples include manuscripts and fragments that suffer from different kinds of degradations rendering parts of the notation illegible. Such degradations may happen due to chemical or physical damage, for example from iron-gall acidity or from deliberate erasure.
Dr Charles Emogor, Dept of Computer Science and Technology
Are you an early career researcher (ECR) thinking about applying for your first grant or fellowship but are not sure where to start?
If you are interested in learning more about effective grant writing and what makes a strong application then please join us for this half day workshop.
The dominant paradigm in language modeling—scaling next-token prediction with parametric knowledge storage—delivers impressive capabilities but also fundamental limitations: brittle factual memory, inefficient parameters, and myopic reasoning. Progress requires a shift toward external memory and architectures that reason globally before committing to tokens.
Positional encodings are essential for transformer-based language models to understand sequence order, yet their influence extends far beyond simple position tracking. This talk explores the landscape of positional encoding methods in LLMs and reveals surprising insights about how these architectural choices shape model behavior.
We begin with the fundamental challenge: why attention mechanisms require explicit positional information.
Hamiltonian Monte Carlo (HMC) and its variants are among the most widely used algorithms for sampling from probability distributions. Despite their popularity, quantitative convergence guarantees for unadjusted HMC remain limited, especially in divergences that provide strong relative-density control such as KL divergence and Rényi divergence. In this talk, we establish regularization properties for unadjusted HMC via one-shot couplings, which enable Wasserstein convergence guarantees to be upgraded to guarantees in KL and Rényi divergence.
This free event is open only to members of the University of Cambridge (and affiliated institutes). Please be aware that we are unable to offer consultations outside clinic hours.
If you would like to participate, please sign up as we will not be able to offer a consultation otherwise. Please sign up through the following link: https://forms.gle/oKKFG78k4CrcE6JK6. Sign-up is possible from June 4 midday (12pm) until June 8 midday or until we reach full capacity, whichever is earlier. If you successfully signed up, we will confirm your appointment by June 10 midday.
