this is a great suggestion for evaluation, thank you!

in general, detailed evaluation is an allocation question for us - with very limited funds currently available for compute, our general priority is to put all of them into datasets and models, to maximize the impact we can make on the open source ecosystem surviving and eventually winning. that is the whole point of our work, to achieve this result :) we can certainly provide more context by offering more benchmarks etc, but ultimately the user will always be better positioned to evaluate their own use cases with their own frameworks and areas of expertise. will still do it if we can!