3D/4D Vision
Can a machine rebuild a moving scene, and everyone in it, from ordinary video?
Temporally coherent reconstruction of dynamic scenes from multi-view video and light fields, and of people and objects from a single image.
N4D Lab · University of Surrey
Armin Mustafa
Associate Professor in Computer Vision and AI
PhD, M.Tech, FHEA, DCMS AI Fellow
Centre for Vision, Speech and Signal Processing (CVSSP) · Surrey Institute for People-Centred AI (PAI)
I am an Associate Professor in Computer Vision and AI at the University of Surrey, where I direct the N4D Lab in CVSSP. My research asks how machines can understand not just what happens in video, but why, by combining 4D vision, which reconstructs people and scenes in 3D as they move, with narrative understanding of video, audio and language.
Over 15 years, working with partners including BBC R&D, Foundry and the CoSTAR National Lab, I have moved this research from the lab into use: audio-visual captioning in BBC pilots, light-field VR film production, and accessible video through SAIReco, the company I co-founded. As an AI Fellow at the Incubator for AI (i.AI), DCMS, I evaluate AI for UK public services. I previously held a five-year Royal Academy of Engineering Research Fellowship in 4D vision for perceptive machines, and my work appears at CVPR, ICCV, NeurIPS and ICLR.
N4D Lab · 4D Narrative AI
AI that understands what happened and why
The N4D Lab studies 4D Narrative AI: combining 4D vision with narrative understanding of video. I direct it at CVSSP, University of Surrey.
Research
Four themes, one question
How do you get from pixels and sound to an account of a scene that makes sense to a person?
Narrative Understanding
Can a model explain why one event led to the next, not just what happened?
Captioning that moves from describing what is on screen to explaining why it happened: causal and temporal links between events.
Video Understanding
Can we find every action and sound in long video, cheaply enough to use?
Dense action and event detection in long, untrimmed video, with audio-visual parsing and question answering.
Generative AI
Can we generate people and video that do exactly what we ask?
Controllable generation of people, video and 3D motion from text, pose and style, including text to 3D sign language.
Publications
Selected papers
Projects
Programmes since 2013
Application areas
Research that leaves the lab
01
Broadcast and media
Personalised captioning, chaptering and relighting for programmes at scale.
OutcomeOur audio-visual captioning research powers two BBC pilots: Moments, which lets BBC Radio 5 Live producers save, title and summarise moments from live broadcasts, and a search tool that helps BBC Creative find the right scenes for trailers.
BBC R&D · AI4ME02
Film, VR and creative industries
Light-field capture, 3D/4D assets and generative tools for production.
OutcomeKinch and the Double World, a light-field VR film nominated at Raindance 2018.
Foundry · Figment Productions · CoSTAR03
Accessibility
Audio description and video that adapts to the viewer.
OutcomeSAIReco’s Video Analysis Suite, shortlisted for Best Innovation in Accessible Tech 2025, and AI Can, launched with Active Prospects to make AI accessible for everyone.
Active Prospects · CDT in AI for Digital Media Inclusion04
Public sector
Evaluating multimodal video analysis for government, from transport to body-worn footage.
OutcomeA benchmark and evaluation framework for AI on body-worn video, reported by i.AI in 2026.
Incubator for AI05
Digital humans
Realistic, interactive people reconstructed from a single camera.
OutcomeThe first model-free method to reconstruct multiple people from one image (CVPR).
Humain Ltd06
Autonomous systems
Perception that holds up at night and in bad weather.
OutcomeThe A*3D dataset (ICRA) and a 13% accuracy gain in A*STAR’s self-driving car.
A*STAR, SingaporeSpin-out · SAIReco
From accessibility research to a product
I co-founded SAIReco to bring the lab’s video understanding and audio description research to broadcasters and creators. Upload a video once and get scenes, chapters, a transcript and fitted audio description back.
Finalist, Tech Impact Awards 2025 · Shortlisted, Best Innovation in Accessible Tech 2025
Visit saireco.orgNews
Latest from the lab
N4D Lab
Team and alumni
Team
Alumni
Recognition
Awards
Services
Serving the community
My editorial, conference, advisory and outreach roles.
Consultancy
Work with me
I advise companies and public bodies on video, audio and 3D AI, from first idea to tested system.
Feasibility studies
Can AI do this with your video, audio or 3D data? A rapid, honest assessment of what is possible and what it would take.
Evaluation and benchmarking
Independent testing of multimodal and video AI systems against your own data and requirements, including for the public sector.
Prototyping
Proof-of-concept systems for video understanding, 3D/4D reconstruction and generative media.
R&D strategy
Advice on AI roadmaps, technical choices and research partnerships with universities.
At a glance
Questions people ask
What is 4D Narrative AI?
4D Narrative AI is the research focus of my N4D Lab at the University of Surrey. It combines 4D vision, which reconstructs people and scenes in 3D as they move over time, with narrative understanding of video, so that machines can explain what happened, in what order, and why.
What is video understanding?
Video understanding is about getting machines to make sense of video the way people do: recognising who and what is in a scene, detecting actions and events as they happen, linking what is heard to what is seen, and describing it all in language. My lab works on dense action and event detection, audio-visual parsing and question answering, and captioning that explains cause and effect.
How can I join the lab?
PhD applicants can email me with a CV and a short note on their background and the problem they want to work on. I am happy to hear from candidates interested in 4D vision, video understanding, narrative AI or generative AI.
Can companies and public bodies work with the lab?
Yes. Organisations work with the lab through consultancy, Knowledge Transfer Partnerships, sponsored PhD studentships such as the BBC CASE studentship, and joint research grants. Partners have included BBC R&D, Foundry, Figment Productions, Humain, A*STAR, the Incubator for AI and the Alan Turing Institute. Email me with a short description of the problem.
Contact
Interested in a PhD in my lab, a collaboration or consultancy?
Centre for Vision, Speech and Signal Processing
University of Surrey, Guildford GU2 7XH, UK