ITML

We build mathematical frameworks that explain the behavior of foundation models — large language models, vision-language models, and the agents built on them — and use that understanding to design better algorithms. Our research follows the lifecycle of a foundation model — pre-training → post-training → inference — and at every stage we ask the same two questions: what is really happening inside the model? (theory) and how can we do it better? (algorithms). We are a research group led by Prof. Jy-yong Sohn in the Department of Statistics & Data Science at Yonsei University. We study topics across machine learning and AI using mathematical tools from information theory, optimization, learning theory, and probability & statistics. Our current focus is foundation models (FMs) — studied along their lifecycle, from pre-training through post-training to inference: at each stage we explain what the model is doing (theory) and use that understanding to build better algorithms. The theory of contrastive & multi-modal pre-training — what embeddings converge to and why — turned into methods that learn better representations. What is gained and lost when models are fine-tuned, continually updated, aligned, or made to forget — and adaptation methods with provable guarantees. The computations models can carry out at inference, statistically rigorous evaluation, and compression that preserves what matters.