Post-Doctoral Research Visit F/M Benchmarks for Evaluating LLMs for Lean Elegance
*Le descriptif de l’offre ci\-dessous est en Anglais*
Type de contrat : CDD
Niveau de diplôme exigé : Thèse ou équivalent
Fonction : Post\-Doctorant
Contexte et atouts du poste
-------------------------------
The postdoc will be within the SIERRA team, which focuses on theoretical machine learning, statistics and optimization. There will be interactions with other teams within INRIA interested in AI for maths (ARGO, PICUBE, SCOOL) as well as at ENS (CSD).
Travel, equipment, and compute expenses will be covered within reasonable limits.
Mission confiée
-------------------
Proposed research subject :
Frontier models have demonstrated rapid progress in producing correct Lean code, saturating existing Lean benchmarks of advanced problems in both the IMO and Putnam competitions, and can produce Fields Medal\-level formalizations of research math. However, while Lean code that type checks might be reasonably declared "correct", for formalizations to be useful to humans we need to extend our assessment beyond mere correctness to code quality, such as concision, transparency, maintainability, human readability, elegance (e.g. “correct” abstractions), and execution efficiency. In this postdoc project, we aim to develop novel and multidimensional metrics of Lean code quality and establish new benchmarks and datasets to assess model performance. We believe that these benchmarks will expose new dimensions for improvement in frontier models, especially in long\-horizon autoformalizations of modern research mathematics. Potential metrics include code complexity, code volume, style adherence and idiomatic expression, passing linter checks, elaboration time. Evaluation methods can combine multiple techniques: including static analysis, resource profiling, LLM\-as\-a\-judge techniques, and optimally using feedback from human expert user studies. This project proposal fills a gap in the field because as frontier models generate increasingly correct Lean code, improving generated code quality to Mathlib standards becomes crucial to accelerate adoption in the mathematical community.
By shifting the evaluation of autoformalization from a single\-bit correctness check to a rich, multidimensional quality assessment, this project will:
- reveal underexplored weaknesses in current LLMs (e.g., long planning for abstraction),
- provide actionable training signals for future models by reinforcement learning,
- lower the barrier to adoption of AI\-generated formalizations in mathematical research.
The person recruited is responsible for defining, implementing and validating a multidimensional suite of Lean‑code‑quality metrics and will take initiatives for creating benchmark datasets, integrating metric suites into continuous‑evaluation pipelines, and organising human studies to calibrate automated scores against expert judgement.
Steering/Management :
The person recruited will be in charge of leading the Lean Quality Benchmark work package, interacting with other project members and interested third\-parties/partners labs, and possibly coordinating the contributions of interns.
Principales activités
-------------------------
- Develop novel, multidimensional quality metrics specifically tailored to proof\-assistant code (not generic software metrics).
- Curate new benchmarks and datasets that include long\-horizon, research\-level autoformalization tasks.
- Establish rigorous evaluation protocols combining automated analysis, resource profiling, and human judgment.
- Demonstrate that these benchmarks expose meaningful failure modes in state\-of\-the\-art frontier models, creating clear targets for improvement.
- Disseminate and promote research results among partner labs and in the community.
-------------
- Subsidized meals
- Partial reimbursement of public transport costs
- Leave: 7 weeks of annual leave \+ 10 extra days off due to RTT (statutory reduction in working hours) \+ possibility of exceptional leave (sick children, moving home, etc.)
- Possibility of teleworking (after 6 months of employment) and flexible organization of working hours
- Professional equipment available (videoconferencing, loan of computer equipment, etc.)
- Social, cultural and sports events and activities
- Health insurance scheme
--------------------------
- Thème/Domaine : Optimisation, apprentissage et méthodes statistiques
- Ville : Paris
- Centre Inria : Centre Inria de Paris
- Date de prise de fonction souhaitée : 2026\-11\-01
- Durée de contrat : 2 ans
- Date limite pour postuler : 2026\-07\-31
Consignes pour postuler
---------------------------
Sécurité défense :
Ce poste est susceptible d’être affecté dans une zone à régime restrictif (ZRR), telle que définie dans le décret n°2011\-1425 relatif à la protection du potentiel scientifique et technique de la nation (PPST). L’autorisation d’accès à une zone est délivrée par le chef d’établissement, après avis ministériel favorable, tel que défini dans l’arrêté du 03 juillet 2012, relatif à la PPST. Un avis ministériel défavorable pour un poste affecté dans une ZRR aurait pour conséquence l’annulation du recrutement.
Politique de recrutement :
Dans le cadre de sa politique diversité, tous les postes Inria sont accessibles aux personnes en situation de handicap.
Contacts
------------
- Équipe Inria : SIERRA
- Recruteur :
A propos d'Inria
--------------------
Inria est l’institut national de recherche dédié aux sciences et technologies du numérique. Il emploie 2600 personnes. Ses 215 équipes\-projets agiles, en général communes avec des partenaires académiques, impliquent plus de 3900 scientifiques pour relever les défis du numérique, souvent à l’interface d’autres disciplines. L’institut fait appel à de nombreux talents dans plus d’une quarantaine de métiers différents. 900 personnels d’appui à la recherche et à l’innovation contribuent à faire émerger et grandir des projets scientifiques ou entrepreneuriaux qui impactent le monde. Inria travaille avec de nombreuses entreprises et a accompagné la création de plus de 200 start\-up. L'institut s'efforce ainsi de répondre aux enjeux de la transformation numérique de la science, de la société et de l'économie.
Cette annonce provient de indeed. Voir l'annonce originale ↗