I have been working in AI for decades (1 2 3). In the end of 2023 I started talking about bootstrapping recursive self-improvement of LLM chatbots using two key ideas:
- Open-ended, non-imitative objectives, because imitative objectives are capped at the human level.
- Using automatic evaluation or ranking of performances by LLMs, but crucially evaluating the act of evaluation as well so that evaluation fidelity also improves and exceeds the human level.
At some point I decided that large corporate labs aren’t moving on this fast enough, so I started an open source project: Recursive Self-improvement Suite.
The project is just a proof-of-concept, describing the hypothesis and running the tasks, their evaluations and evaluations of the evaluations on OpenAI ChatGPT-3.5. It produces ranked data which can be used for reinforcement learning, DPO or even supervised continual training to improve the model performance on these kinds of open ended tasks.
In July 2024 Meta FAIR published a pre-print on Arxiv titled: Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge.
This pre-print describes the exact idea of evaluating the evaluations which I had described before, with a meta-judge.
Here is what I wrote in the project:
I had also implemented this already ages ago here.
This is what their article says, describing this as a “novel method”, proposed by them:
These are describing the same thing.
Here is the chatbot template they used:
Here is the template I used:
This is an understandable omission. People don’t often Google or search existing art on GitHub, although it could be argued GitHub projects are more peer reviewed (by stars) than Arxiv preprints are. I do expect to be cited though, that’s why the project has “Citing” section in the README.md, giving a convenient BibTex snippet for it.
I sent them an email to the email address who submitted the paper to Arxiv.
They responded very fast: “Thanks for the link.. ! Respectfully, not sure that’s quite enough to warrant a citation – but definitely good to brainstorm and try stuff!”:
This is not ok.
It is natural that they don’t want to cite existing art because it invalidates their claim of novelty. Furthermore, I would say my method is actually an evolution over theirs, because it ranks problem creation task as well.
It is however a severe breach of scientific ethics. They cannot claim proposing a novelty when someone else has proposed it before.
This is erasing independent researchers from the history of science and claiming credit for their inventions.
I am obviously not happy about this. I am myself trying to complete my Dr.Sc.(Tech.) degree with peer reviewed publications and it undermines my efforts if credit that properly belongs to the first inventor of a method is stolen.
Here are some posts where more discussion about this is happening. Feel free to share to science journalists or influencers, or other relevant parties!
Tero Keski-Valkama