Formal Methods and Reasoning Group logo

We build systems that
interactively reason
alongside humans

About

The Formal Methods and Reasoning Group (FoRG) focuses on Automated Reasoning in Neurosymbolic AI and explores this at the intersection of humans and machines.

Fall 2026 recruiting: We look forward to recruiting Undergraduate and MS students in the Fall.

Projects

STEP-Bench puzzle grid

STEP-Bench

STEP-Bench (Symbolic Temporal Exploratory Puzzle Benchmark) is a series of interactive reasoning puzzles for studying causal reasoning, incorporating eye-tracking to monitor how participants solve temporal puzzle tasks.

A rack of FPGA boards

Manhattan Reasoning

Manhattan Reasoning is a platform for teaching AI systems to reason about hardware design, combining cloud-based FPGA access, sandbox environments, and open-source toolchains built on technologies like Yosys and nextpnr — bringing hardware reasoning to everyone.

A subset lattice linked to a transformer encoder block

Lattice Deduction Transformers

The Lattice Deduction Transformer (LDT) is a recurrent transformer that approximates logically sound deduction by projecting its latent state through a lattice between forward passes. This work is a collaboration with Axiom Math.

An automaton diagram from the TempoBench benchmark

TempoBench

TempoBench is a formal benchmark for evaluating counterfactual causal reasoning in large language models. While frontier LLMs can simulate systems forward with high accuracy, their performance drops sharply when asked to identify which inputs were necessary for a given output — a gap with important implications for debugging and root cause analysis.

The MaxPy fish mascot

MaxPyLang

MaxPyLang is a Python package for programmatically generating and editing MaxMSP patches — placing objects and building connections that would be tedious to create through the visual interface alone.

Archived Projects

Events

Talks and visitors hosted by the group. All are welcome.

Access: Events are open to anyone with a Barnard or Columbia ID. Coming from elsewhere? Email msantolu@barnard.edu at least a day ahead so we can arrange guest access.

Upcoming

Guest Lecture

A Hoare Logic for Continuous-Time Stochastic Processes #

Kei Imada, Cornell University

When
Where
LeFrak Seminar Meeting Room (101A), LeFrak Center, Barnard Hall, Barnard College

Abstract

Continuous-time stochastic processes like Brownian motion model noise in the physical world, from particles in a fluid to interest rates and stock prices. We have many tools for reasoning about probabilistic programs, but it remains unclear how to apply them to continuous-time stochastic processes. In this talk I will try to convince you that we can reason about these processes the way we reason about programs: by writing them as ordinary probabilistic programs with a small time step dt and taking dt to 0.

I will define a continuous semantics for pWhile-dt, a probabilistic imperative language with a time step variable dt, by collecting the limit points of its standard program semantics as dt approaches 0. I will present a Hoare logic over our continuous semantics in which properties proved on the standard semantics for small dt transfer to the continuous semantics. To demonstrate our logic, we will verify properties of Brownian motion and geometric Brownian motion with loop invariants. Finally, I will present a prototype implementation that applies off-the-shelf probabilistic program analysis tools to reason about continuous-time stochastic processes.

This is joint work with Justin Hsu.

Speaker

Portrait of Kei Imada

Kei Imada is a fourth-year Ph.D. candidate in Computer Science at Cornell University, advised by Justin Hsu. His research develops program logics for continuous-time systems, proving temporal and probabilistic properties of cyber-physical and stochastic systems as though they are programs. Before Cornell, he was a software engineer at Pure Storage (now Everpure), and he received his B.A. from Swarthmore College.

Guest Lecture

Non-Hallucinatory Legal AI: Neurosymbolic Reasoning over Rules and Regulation #

Sam Kouteili, CEO of Leibniz AI

When
Where
Seminar Room 119, Milstein Center, Barnard College

Abstract

Large language models read legal and regulatory text fluently, and are great interfaces to unstructured, natural language. However, LLMs don't apply the rules reliably, with hallucinations remaining a structural issue. This has limited adoption in rule-heavy industries like insurance and financial compliance, where the same facts must yield the same compliant answer every time, traced to the provisions that produce it.

In this talk, we explore how neurosymbolic AI techniques can be applied to correctly, consistently, and explainably reason about rules and regulations. Once rules are formalized in predicate logic, solvers can derive the verdict for a given scenario. We first highlight considerations for carefully formalizing rules, drawing on the work of linguists and analytical philosophers. We then show how such formalizations allow us to build trustworthy neurosymbolic AI architectures, with LLMs acting as the front-end interface and reasoning contained to formal solvers that cannot hallucinate. We present three downstream applications where we query the rules, compare the rules, and guard AI agents to the rules, maintaining the tenets of correctness, consistency, and explainability.

No background in logic or law is assumed.

Speaker

Portrait of Sam Kouteili

Sam Kouteili is the CEO of Leibniz AI, a regulatory AI startup founded by Yale Computer Science Chair Ruzica Piskac and Yale Law School Professor Scott Shapiro. The company builds neurosymbolic systems for healthcare, insurance, and compliance. Previously, Sam was a Computer Science PhD candidate at Yale in Piskac's ROSE group. His research covers the intersection of formal methods and artificial intelligence, exploring how formal techniques can improve the reliability of generative AI.

Publications

Selected publications from the group. For the complete list, see Google Scholar.

2027

Mining Beyond the Bools: Learning Data Transformations and Temporal Specifications
Sam Kouteili, William Fishell, Mark Santolucito, Christian Scaff, Ruzica Piskac
ESOP 2027 (to appear)

2026

Lattice Deduction Transformers
Liam Davis, Leopold Haller, Alberto Alfarano, Mark Santolucito
Measuring and Reducing Semantic Entropy in LTL Guardrail Generation
Maria Hwang, Rachel Papirmeister, Mark Santolucito
VSTTE 2026
Learning GR(1) Specifications from Traces
Sam Kouteili, William Fishell, Mark Santolucito, Ruzica Piskac
User Engagement With Intermediate Representations in LLM-Assisted Audio Programming
Chris Hyorok Lee, Miles Scharff, Katie Wang, Mark Santolucito
Cost-Efficient Theorem Proving via Agent Orchestration in Program Verification
Shuangjie Yao, Nikolaus Holzer, Mark Santolucito, Baishakhi Ray, Suman Jana, Dongdong She
VERICODEGEN workshop at NeurIPS 2026

2025

Embedding Alignment in Code Generation for Audio
Sam Kouteili, Hiren Madhu, George Typaldos, Mark Santolucito

2024

Statically Inferring Usage Bounds for Infrastructure as Code
Feitong Qiao, Aryana Mohammadi, Jürgen Cito, Mark Santolucito

2023

MaxPy: An Open-Source Python Package for Text-Based Generation of MaxMSP Patches
Ranger Liu, Satchel Peterson, Richard Lee, Mark Santolucito

2022

Can Reactive Synthesis and Syntax-Guided Synthesis Be Friends?
Wonhyuk Choi, Ruzica Piskac, Bernd Finkbeiner, Mark Santolucito
Learning CI Configuration Correctness for Early Build Feedback
Mark Santolucito, Jialu Zhang, Ennan Zhai, Jürgen Cito, Ruzica Piskac

2021

TSL Synthesis Synthesizer: Reconfigurable Signal Flows through Program Synthesis
Michel Vazirani, Wonhyuk Choi, Mark Santolucito
Analyzing Infrastructure as Code to Prevent Intra-update Sniping Vulnerabilities
Julien Lepiller, Ruzica Piskac, Martin Schaef, Mark Santolucito

2020

Grammar Filtering for Syntax Guided Synthesis
Kairo Morton, William Hallahan, Elven Shum, Ruzica Piskac, Mark Santolucito
AAAI 2020 (Oral Presentation)
Formal Methods and Computing Identity-based Mentorship for Early Stage Researchers
Mark Santolucito, Ruzica Piskac

2019

Temporal Stream Logic: Synthesis Beyond the Bools
Bernd Finkbeiner, Felix Klein, Ruzica Piskac, Mark Santolucito
Live Programming By Example
Mark Santolucito, William Hallahan, Ruzica Piskac

2017 and earlier

Synthesizing Configuration File Specifications with Association Rule Learning
Mark Santolucito, Ennan Zhai, Rahul Dhodapkar, Aaron Shim, Ruzica Piskac
Version Space Learning for Verification on Temporal Differentials
Mark Santolucito
Probabilistic Automated Language Learning for Configuration Files
Mark Santolucito, Ennan Zhai, Ruzica Piskac

Team

Faculty

Principal Investigator, Barnard College

Research Scientists

Research Scientist

Current Students

PhD Student
Masters Student
Chris Hyorok Lee
Masters Student
Undergraduate Student
Undergraduate Student

Alumni

Research Scientist (Post-bacc) → Turing
MS → PhD student, UC San Diego
BA/MS → PhD student, Carnegie Mellon University
BA → PhD student, UC San Diego (NSF GRFP)
MS → Reinforce Labs
MS → Meta
BS → PhD student, Princeton University

Contact

Interested in joining the group or collaborating? We'd love to hear from you — prospective students (see our Fall 2026 recruiting note on the About page), potential collaborators, and anyone curious about our work are welcome to reach out.

Prof. Mark Santolucito
Barnard College, Columbia University
Department of Computer Science

Email: msantolu@barnard.edu

GitHub: github.com/Barnard-PL-Labs