Burak Varıcı

1

1. JMLR

   

   Score-based causal representation learning: Linear and General Transformations

   Burak Varıcı*, Emre Acartürk*, Karthikeyan Shanmugam, Abhishek Kumar, and Ali Tajer

   Journal of Machine Learning Research, 2025

   Abs Bib PDF Code

   This paper addresses intervention-based causal representation learning (CRL) under a general nonparametric latent causal model and an unknown transformation that maps the latent variables to the observed variables. Linear and general transformations are investigated. The paper addresses both the identifiability and achievability aspects. Identifiability refers to determining algorithm-agnostic conditions that ensure recovering the true latent causal variables and the latent causal graph underlying them. Achievability refers to the algorithmic aspects and addresses designing algorithms that achieve identifiability guarantees. By drawing novel connections between score functions (i.e., the gradients of the logarithm of density functions) and CRL, this paper designs a score-based class of algorithms that ensures both identifiability and achievability. First, the paper focuses on linear transformations and shows that one stochastic hard intervention per node suffices to guarantee identifiability. It also provides partial identifiability guarantees for soft interventions, including identifiability up to ancestors for general causal models and perfect latent graph recovery for sufficiently non-linear causal models. Secondly, it focuses on general transformations and shows that two stochastic hard interventions per node suffice for identifiability. Notably, one does not need to know which pair of interventional environments have the same node intervened. Finally, the theoretical results are empirically validated via experiments on structured synthetic data and image data.

   @article{varici2025score,
     title = {Score-based causal representation learning: Linear and General Transformations},
     author = {Var{\i}c{\i}*, Burak and Acart{\"u}rk*, Emre and Shanmugam, Karthikeyan and Kumar, Abhishek and Tajer, Ali},
     journal = {Journal of Machine Learning Research},
     year = {2025},
     volume = {26},
     number = {112},
     pages = {1--90},
     rank = {1}
   }

2

1. JMLR

   

   Causal Bandits for Linear Structural Equation Models

   Burak Varıcı, Karthikeyan Shanmugam, Prasanna Sattigeri, and Ali Tajer

   Journal of Machine Learning Research, 2023

   Abs Bib PDF Code Poster

   This paper studies the problem of designing an optimal sequence of interventions in a causal graphical model to minimize cumulative regret with respect to the best intervention in hindsight. This is, naturally, posed as a causal bandit problem. The focus is on causal bandits for linear structural equation models (SEMs) and soft interventions. It is assumed that the graph’s structure is known and has \(N\)nodes. Two linear mechanisms, one soft intervention and one observational, are assumed for each node, giving rise to \(2^N\)possible interventions. The majority of the existing causal bandit algorithms assume that at least the interventional distributions of the reward node’s parents are fully specified. However, there are \(2^N\)such distributions (one corresponding to each intervention), acquiring which becomes prohibitive even in moderate-sized graphs. This paper dispenses with the assumption of knowing these distributions or their marginals. Two algorithms are proposed for the frequentist (UCB-based) and Bayesian (Thompson sampling-based) settings. The key idea of these algorithms is to avoid directly estimating the \(2^N\)reward distributions and instead estimate the parameters that fully specify the SEMs (linear in \(N\)) and use them to compute the rewards. In both algorithms, under boundedness assumptions on noise and the parameter space, the cumulative regrets scale as \(\tilde{O}(d^L+\frac12 \sqrt{NT})\), where \(d\)is the graph’s maximum degree, and \(L\)is the length of its longest causal path. Additionally, a minimax lower of \(Ω(d^{L/2-2}\sqrt{T})\)is presented, which suggests that the achievable and lower bounds conform in their scaling behavior with respect to the horizon \(T\)and graph parameters \(d\)and \(L\).

   @article{varici2023causal,
     author = {Var{\i}c{\i}, Burak and Shanmugam, Karthikeyan and Sattigeri, Prasanna and Tajer, Ali},
     title = {Causal Bandits for Linear Structural Equation Models},
     journal = {Journal of Machine Learning Research},
     year = {2023},
     volume = {24},
     number = {297},
     pages = {1--59},
     url = {http://jmlr.org/papers/v24/22-0969.html},
     video_short = {https://neurips.cc/virtual/2024/poster/98317},
     rank = {2}
   }

3

1. ICML

   

   Contextures: Representations from Contexts

   Runtian Zhai, Kai Yang, Burak Varıcı, Che-Ping Tsai, J. Zico Kolter, and Pradeep Ravikumar

   In International Conference on Machine Learning, 2025

   Abs Bib PDF Code Poster

   Despite the empirical success of foundation models, we do not have a systematic characterization of the representations that these models learn. In this paper, we establish the contexture theory. It shows that a large class of representation learning methods can be characterized as learning from the association between the input and a context variable. Specifically, we show that many popular methods aim to approximate the top-d singular functions of the expectation operator induced by the context, in which case we say that the representation learns the contexture. We demonstrate the generality of the contexture theory by proving that representation learning within various learning paradigms – supervised, self-supervised, and manifold learning – can all be studied from such a perspective. We also prove that the representations that learn the contexture are optimal on those tasks that are compatible with the context. One important implication of the contexture theory is that once the model is large enough to approximate the top singular functions, further scaling up the model size yields diminishing returns. Therefore, scaling is not all we need, and further improvement requires better contexts. To this end, we study how to evaluate the usefulness of a context without knowing the downstream tasks. We propose a metric and show by experiments that it correlates well with the actual performance of the encoder on many real datasets.

   @inproceedings{zhai2025contextures,
     title = {Contextures: Representations from Contexts},
     author = {Zhai, Runtian and Yang, Kai and Var{\i}c{\i}, Burak and Tsai, Che-Ping and Kolter, J. Zico and Ravikumar, Pradeep},
     booktitle = {International Conference on Machine Learning},
     year = {2025},
     rank = {3}
   }

4

1. NeurIPS

   

   Interventional Causal Discovery in a Mixture of DAGs

   Burak Varıcı, Dmitriy A Katz, Dennis Wei, Prasanna Sattigeri, and Ali Tajer

   In Proc. Advances in Neural Information Processing Systems, 2024

   Abs Bib PDF Code Poster Slides

   Causal interactions among a group of variables are often modeled by a single causal graph. In some domains, however, these interactions are best described by multiple co-existing causal graphs, e.g., in dynamical systems or genomics. This paper addresses the hitherto unknown role of interventions in learning causal interactions among variables governed by a mixture of causal systems, each modeled by one directed acyclic graph (DAG). Causal discovery from mixtures is fundamentally more challenging than single-DAG causal discovery. Two major difficulties stem from (i) an inherent uncertainty about the skeletons of the component DAGs that constitute the mixture and (ii) possibly cyclic relationships across these component DAGs. This paper addresses these challenges and aims to identify edges that exist in at least one component DAG of the mixture, referred to as the true edges. First, it establishes matching necessary and sufficient conditions on the size of interventions required to identify the true edges. Next, guided by the necessity results, an adaptive algorithm is designed that learns all true edges using \(O(n^2)\)interventions, where \(n\)is the number of nodes. Remarkably, the size of the interventions is optimal if the underlying mixture model does not contain cycles across its components. More generally, the gap between the intervention size used by the algorithm and the optimal size is quantified. It is shown to be bounded by the cyclic complexity number of the mixture model, defined as the size of the minimal intervention that can break the cycles in the mixture, which is upper bounded by the number of cycles among the ancestors of a node.

   @inproceedings{varici2024interventional,
     title = {Interventional Causal Discovery in a Mixture of DAGs},
     author = {Var{\i}c{\i}, Burak and Katz, Dmitriy A and Wei, Dennis and Sattigeri, Prasanna and Tajer, Ali},
     booktitle = {Proc. Advances in Neural Information Processing Systems},
     year = {2024},
     video_short = {https://neurips.cc/virtual/2024/poster/93767},
     rank = {4}
   }

5

1. AISTATS (oral)

   

   General Identifiability and Achievability for Causal Representation Learning

   Burak Varıcı, Emre Acartürk, Karthikeyan Shanmugam, and Ali Tajer

   In Proc. International Conference on Artificial Intelligence and Statistics, 2024

   Abs Bib PDF Code Poster Slides

   This paper focuses on causal representation learning (CRL) under a general nonparametric latent causal model and a general transformation model that maps the latent data to the observational data. It establishes identifiability and achievability results using two hard uncoupled interventions per node in the latent causal graph. Notably, one does not know which pair of intervention environments have the same node intervened (hence, uncoupled). For identifiability, the paper establishes that perfect recovery of the latent causal model and variables is guaranteed under uncoupled interventions. For achievability, an algorithm is designed that uses observational and interventional data and recovers the latent causal model and variables with provable guarantees. This algorithm leverages score variations across different environments to estimate the inverse of the transformer and, subsequently, the latent variables. The analysis, additionally, recovers the identifiability result for two hard coupled interventions, that is when metadata about the pair of environments that have the same node intervened is known. This paper also shows that when observational data is available, additional faithfulness assumptions that are adopted by the existing literature are unnecessary.

   @inproceedings{varici2024general,
     title = {General Identifiability and Achievability for Causal Representation Learning},
     author = {Var{\i}c{\i}, Burak and Acart{\"u}rk, Emre and Shanmugam, Karthikeyan and Tajer, Ali},
     booktitle = {Proc. International Conference on Artificial Intelligence and Statistics},
     year = {2024},
     address = {Valencia, Spain},
     rank = {5}
   }